Real-time monochrome camera input on Raspberry Pi Pico

Learn how to connect a Raspberry Pi Pico to a Himax HM01B0-based camera module and capture monochrome images using PIO and DMA in real time. This excellent guide is courtesy of our good friend Sandeep Mistry, Principal Software Engineer at Arm.

Raspberry Pi Pico is an extremely popular and capable MCU (microcontroller) development platform that is based on Raspberry Pi’s RP2040 MCU and costs only $4. RP2040 incorporates two 133MHz Arm Cortex-M0+ CPU cores, 264kB of SRAM, and a unique programmable I/O system. In past guides, I’ve covered how Raspberry Pi’s well documented open-source pico-sdk and the open-source software ecosystem can be used to:

Next up I’m pushing the platform further, and seeing if it is capable of running computer vision applications. The first step will be to capture image data from a camera module attached to the board.

This guide will walk through how to capture (monochrome) images on a Raspberry Pi Pico board using a Himax HM01B0-based camera module from Arducam. The Programmable I/O (PIO) and Direct Memory Access (DMA) features of the board’s Raspberry Pi RP2040 microcontroller (MCU) will be used to interface with the Himax HM01B0 image sensor.

Photo of an Arducam HM01B0 Monochrome QVGA SPI Camera Module for Raspberry Pi Pico

The Arducam HM01B0 Camera Module for Raspberry Pi Pico uses Himax’s HM01B0-MWA image sensor and is capable of capturing monochrome 320×320, 320×240 (QVGA), or 160×120 (QQVGA) images. This ultra-low-power image sensor consumes less than 2mW of power in QVGA mode and less than 1.1mW of power in QQVGA mode.

Once image data from the camera module is captured, the RP2040’s USB interface will be used to send the image to a PC for visualisation.

Hardware setup

If you are using a Raspberry Pi Pico board without pre-soldered headers, follow MagPi’s “How to solder GPIO pin headers to Raspberry Pi Pico” guide and solder headers to the board.

Use the cable included with the Arducam HM01B0 Camera Module and connect the camera module to the Raspberry Pi Pico board as follows:

Wiring diagram for connecting Arducam HM01B0 Camera Module to Raspberry Pi Pico

Arducam HM01B0	Raspberry Pi Pico
VCC	3V3
SCL	GPIO5
SDA	GPIO4
VSYNC	GPIO6
HREF	GPIO7
PCLK	GPIO8
D0	GPIO9
GND	GND

After the wires are connected, it will look like this:

Photo of an Arducam HM01B0 Camera Module connected to a Raspberry Pi Pico using the wires included with the camera module

If you’d like a more compact setup, you can use an Adafruit PiCowbell Proto for Pico to make an adapter board.

Compact setup of Arducam HM01B0 Camera Module connected to a Raspberry Pi Pico with an Adafruit PiCowbell Proto for Pico-based adapter board

HM01B0 Library for Pico

The HM01B0 Library for Pico provides a friendly interface to interact with Himax HM01B0-based cameras from Raspberry Pi RP2040-based boards. The library uses the I2C protocol to configure the image sensor, and then uses PIO and DMA to receive the image data from the sensor on GPIO pins.

Block diagram of RP2040 interfaced to Himax HM01B0 image sensor

To read an image frame, the PIO state machine in the library:

Waits for the VSYNC pin to toggle from LOW to HIGH
For each row in the image, waits for the HSYNC pin to toggle from LOW to HIGH
For each pixel in the row, waits for the PCLK pin to toggle from LOW to HIGH and read a data bit value from the D0 pin

RP2040’s DMA feature is used to transfer the image data directly from the PIO state machine to the RP2040’s memory. Once in memory, RP2040’s Arm Cortex-M0+ processor can be used process the image data. The image data can then be:

Sent over USB to a PC for visualization
Displayed on an LCD display
Saved to a microSD or SD card
Used as input to an on-board Machine Learning (ML) image classification model
Streamed over Wi-Fi, if using a Raspberry Pi Pico W board

The examples included in the library focus on visualizing the data on a PC to ensure the camera module is functioning properly.

Software setup

You’ll first need to set up your computer with Raspberry Pi’s Pico SDK and required toolchains.

See the Getting started with Raspberry Pi Pico guide for more information. Section 2.1 of the guide can be used for all operating systems, and is followed by sections with operating system-specific information:

Linux: Section 2.2
macOS: Section 9.1
Windows: Section 9.2

Make sure the PICO_SDK_PATH environment variable is set.

export PICO_SDK_PATH=/path/to/pico-sdk

In a terminal window, clone the git repository and change directories:

cd ~/

git clone https://github.com/ArmDeveloperEcosystem/hm01b0-library-for-pico.git

cd hm01b0-library-for-pico

Create a build directory and change directories to it:

mkdir build

cd build

Run cmake and make to compile:

cmake .. -DPICO_BOARD=pico

make

Terminal-based viewer example

The first example included with the library captures data from the camera module and converts the image pixels captured from the camera to a format that can be viewed in a VT100 terminal emulator like screen.

Each pixel on every second row is mapped to an ASCII character based on the mapping from the Character representation of grey scale images page. The set of ASCII characters is as follows:

$@B%8&WM#*oahkbdpqwmZO0QLCJUYXzcvunxrjft/|()1{}[]?-_+~i!lI;:,”^`’.

Darker pixel values map to characters at the start of the range, and lighter pixel values map to characters at the end of the range.

Running the example

Hold down the BOOTSEL button on the board, while plugging the board into your computer with a USB cable.

Copy the examples/serial_terminal/serial_terminal.uf2 file to the mounted Raspberry Pi Pico boot ROM disk:

cp ./examples/serial_terminal/serial_terminal.uf2 /Volumes/RPI-RP2/.

Now run the screen command with the path of the serial port of the Raspberry Pi Pico as an argument:

screen /dev/cu.usbmodem1101

NOTE: Replace the /dev/cu.usbmodem1101 value with the path of your Raspberry Pi Pico’s USB serial port on your PC.

Now you will see image data from the camera in your terminal! The example below shows me waving to the camera.

Screen recording of serial terminal image viewer

USB camera example

The second example included with the library uses the TinyUSB library to create a USB video class (UVC) device. The example will capture image data from the HM01B0 image sensor at a resolution of 160×120 and send the data in YUYV format over USB to a PC.

Block diagram of RP2040 interfaced to Himax HM01B0 image sensor with image data sent over USB to a PC

The Y value will be the monochrome image pixel value, while the U and V values of each pixel have a fixed value of 128; this will result in a grayscale image.

Running the example

At the time of writing this guide, the latest version of the Raspberry Pi Pico SDK was v1.5.1 and included v0.15.0 of the TinyUSB library. This example requires changes that were made in pull request #1985, after the v0.15.0 release of TinyUSB, so you will need to manually update the library:

cd $PICO_SDK_PATH/lib/tinyusb

git fetch

git checkout 5ce60c5d207b9e7de419be6956164aebde14c3f2

Change directories back to the hm01b0-library-for-pico/build directory and re-run the make command to rebuild the example with the updated version of TinyUSB:

cd /hm01b0-library-for-pico/build

make

Hold down the BOOTSEL button on the board, while plugging the board into your computer with a USB cable.

Copy the examples/usb_camera/usb_camera.uf2 file to the mounted Raspberry Pi Pico boot ROM disk:

cp ./examples/usb_camera/usb_camera.uf2 /Volumes/RPI-RP2/.

The Raspberry Pi Pico board will now appear like a regular USB web camera device on your PC! It will have a low 160×120 resolution and will only capture monochrome images.

Pico board taking a selfie in the mirror

If you are using macOS, open the QuickTime Player app, from the menu select File -> New Movie Recording, then click the v on the right side of the record button and select “TinyUSB UVC”. You will now get a live preview from the camera module connected to the Raspberry Pi Pico.

If you are using Linux, open a terminal and run v4l2-ctl --list-devices to get a list of video devices.

$ v4l2-ctl --list-devices

...

TinyUSB UVC: TinyUSB UVC (usb-0000:00:03.0-6):

    /dev/video2

    /dev/video3

    /dev/media1

On this PC the TinyUSB UVC device can be accessed via /dev/video2. The v4l2‑ctl ‑d /dev/video2 --list-formats-ext command can then be used to query the capture formats the video device supports:

$ v4l2-ctl -d /dev/video2 --list-formats-ext

ioctl: VIDIOC_ENUM_FMT

    Type: Video Capture


    [0]: 'YUYV' (YUYV 4:2:2)

        Size: Discrete 160x120

            Interval: Discrete 0.033s (30.000 fps)

You can then use VLC to get a live preview from the Raspberry Pi Pico board:

sudo apt-get install vlc

vlc v4l2:///dev/video2

The v4l2-ctl command can also be used to capture still images:

v4l2-ctl --device /dev/video2
--set-fmt-video=width=160,height=120,pixelformat=YUYV --stream-mmap
--stream-to=frame.raw --stream-count=1

Raw images will then need to be converted into JPEG format using ffmpeg:

sudo apt-get install ffmpeg

ffmpeg -y -s:v 160x120 -pix_fmt yuyv422 -i frame.raw frame.jpg

Next steps

RP2040’s PIO and DMA capabilities enabled us to capture (monochrome) images from a Himax HM01B0-based camera module on a Raspberry Pi Pico board using the HM01B0 Library for Pico.

The examples included in the library allowed us to visualize the camera input in both a serial terminal and as a USB UVC device on a PC.

As a next step, RP2040’s Cortex-M0+ processor can be used to process the image data on the board, including using it as input to a ML image classifier model. The USB camera example can also be used with a Raspberry Pi 4 to collect training data from the camera over USB via the Raspberry Pi Pico to train a custom ML image classification model.