Open Source Flipper Zero, Thread, Matter, and Zigbee on ESP32, Arduino go Zephyr, and more!

0:00:00 / -

Watch now on YouTube Watch now

Rp2040 Based Pi-day Bling

Made by sai-yamanoor / Wearables / IoT

About the project

RP2040-based bling that changes its LED color when it detects one of three keywords: "Pi", "3.14", or "Irrational"

Project info

Difficulty: Moderate

Platforms: Adafruit, Raspberry Pi, Edge Impulse

Estimated time: 1 hour

License: MIT license (MIT)

Items used in this project

Hardware components

	Machine Screw, M2.5	x 8
	M2.5 Spacer	x 8
	Custom PCB	x 1
	LED, RGB	x 1
	Adafruit WS2812B Nano	x 1
	Raspberry Pi RP2040	x 1

Software apps and online services

	Raspberry Pi Pico C SDK
	Edge Impulse Studio
	Arduino IDE

Story

Introduction

Happy Pi-Day! This project describes a bling designed around the RP2040 microcontroller. In the spirit of celebrating Pi-Day, it is designed to capable of detecting three words namely: Pi, 3.14, and Irrational. When it detects one of these keywords, the RGB LEDs on the bling change their color.

Pi Day Bling -Front View

Pi Day Bling -Front View

Components

The bling is designed around the RP2040 microcontroller. The following image shows the components of the bling.

Bling - Back side

Bling - Back side

The components of the bling on the back side (in numerical order) include:

RP2040 microcontroller
Goertek SD18OB261-060 PDM microphone
Flash memory
USB-C port
Li-Po battery port + charger
IS31FL3737 LED driver

The components on the front side of the bling include 12 WS2812B individually addressable RGB LEDs and 48 RGB LEDs.

Bling - Front Side

Bling - Front Side

The board was designed with Autodesk Eagle and assembled using JLCPCB. The bling's maximum dimensions are 144.9 mm * 165.9 mm.

Edge Impulse

The PDM microphone in the bling is used to detect three keywords associated with Pi namely: Pi, 3.14, and Irrational. The audio samples required for the classification was collected using a browser and a model was built using Edge Impulse. The Edge Pulse model is available from here.

Here is a demo of the model deployed on the RP2040 microcontroller:

Edge Impulse model deployed on the RP2040 microcontroller

Programming

The bling was programmed and tested in two ways:

Using Arduino IDE
Using Raspberry Pi Pico C/C++ SDK

Edge Impulse enables testing the code right away since it comes with code samples to test on different platforms. The Arduino code is made available below. Since the RP2040 is a dual core microcontroller, the code is structured as follows:

The first core collects audio from the PDM microphone and runs the inference using the Edge Impulse model.

bool m = microphone_inference_record();
    if (!m) {
        ei_printf("ERR: Failed to record audio...n");
        return;
    }

    ei_printf("Recording donen");

    signal_t signal;
    signal.total_length = EI_CLASSIFIER_RAW_SAMPLE_COUNT;
    signal.get_data = &microphone_audio_signal_get_data;
    ei_impulse_result_t result = { 0 };

    EI_IMPULSE_ERROR r = run_classifier(&signal, &result, debug_nn);
    if (r != EI_IMPULSE_OK) {
        ei_printf("ERR: Failed to run classifier (%d)n", r);
        return;
    }

If a keyword is detected with at least 90% confidence, we need to change the color. The first core of the microcontroller pushes the color to a queue.

void print_inference_result(ei_impulse_result_t result) {

    // Print how long it took to perform inference
    ei_printf("Timing: DSP %d ms, inference %d ms, anomaly %d msrn",
            result.timing.dsp,
            result.timing.classification,
            result.timing.anomaly);

    ei_printf("Predictions:rn");
    for (uint16_t i = 0; i < EI_CLASSIFIER_LABEL_COUNT; i++) {
        ei_printf("  %s: ", ei_classifier_inferencing_categories[i]);
        ei_printf("%.5frn", result.classification[i].value);
        if(i != 3 && result.classification[i].value > 0.90){
           rp2040.fifo.push(colors[i]);
        }
    }

}

On the other core, we change the color if new data is available in the queue by using the pop() method.

void loop1(){
    if(rp2040.fifo.available()){
      color = rp2040.fifo.pop();
    }
    colorWipe(color, 50); // Red
    colorWipe(strip.Color(0, 0, 0), 25); // Green
 }

Assembly

The Pi symbol was cut in brass while the background is cut is translucent white acrylic. They are assembled using M2.5 screws.

Assembled Bling

Assembled Bling

Demo

The following video shows the bling in action. It changes color every time a keyword is detected!

Pi Day Bling Video

Next Steps

The technical debt on this project include:

IS31FL3737 drivers: Currently, the bling was built using the Neopixel LEDs. There are 48 RGB LEDs. The drivers for this chipset needs to be completed.
Power Profiling: The power consumption of the bling needs to be profiled to determine the battery size.
Fine tuning Pico C SDK code performance: Currently, the Arduino IDE based deployment can deploy all three keywords correctly while the C SDK based code sample can only detect 2/3 keywords. This needs further investigation.

Schematics, diagrams and documents

Pi Badge Schematic

Pico Badge Schematic

Code

Github repository

Arduino Code

// If your target is limited in memory remove this macro to save 10K RAM #define EIDSP_QUANTIZE_FILTERBANK 0 /* ** NOTE: If you run into TFLite arena allocation issue. ** ** This may be due to may dynamic memory fragmentation. ** Try defining "-DEI_CLASSIFIER_ALLOCATION_STATIC" in boards.local.txt (create ** if it doesn't exist) and copy this file to ** `<ARDUINO_CORE_INSTALL_PATH>/arduino/hardware/<mbed_core>/<core_version>/`. ** ** See ** (https://support.arduino.cc/hc/en-us/articles/360012076960-Where-are-the-installed-cores-located-) ** to find where Arduino installs cores on your machine. ** ** If the problem persists then there's not enough memory for this model and application. */ /* Includes ---------------------------------------------------------------- */ #include <pi_day_audio_inferencing.h> #include <PDM.h> #include <Adafruit_NeoPixel.h> #define PIN 21 /** Audio buffers, pointers and selectors */ typedef struct { int16_t *buffer; uint8_t buf_ready; uint32_t buf_count; uint32_t n_samples; } inference_t; static inference_t inference; static signed short sampleBuffer[2048]; static bool debug_nn = false; // Set this to true to see e.g. features generated from the raw signal static volatile bool record_ready = false; Adafruit_NeoPixel strip = Adafruit_NeoPixel(12, PIN, NEO_GRB + NEO_KHZ800); uint32_t colors[4] = { strip.Color(127, 0, 0), strip.Color(0, 127, 0), strip.Color(0, 0, 127), strip.Color(127, 127, 127) }; uint32_t color = strip.Color(0, 0, 0); /** * @brief Arduino setup function */ void setup1(){ strip.begin(); strip.setBrightness(50); strip.show(); // Initialize all pixels to 'off' } void loop1(){ if(rp2040.fifo.available()){ color = rp2040.fifo.pop(); } colorWipe(color, 50); // Red colorWipe(strip.Color(0, 0, 0), 25); // Green } void setup() { // put your setup code here, to run once: Serial.begin(115200); // comment out the below line to cancel the wait for USB connection (needed for native USB) Serial.println("Edge Impulse Inferencing Demo"); // summary of inferencing settings (from model_metadata.h) ei_printf("Inferencing settings:\n"); ei_printf("\tInterval: "); ei_printf_float((float)EI_CLASSIFIER_INTERVAL_MS); ei_printf(" ms.\n"); ei_printf("\tFrame size: %d\n", EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE); ei_printf("\tSample length: %d ms.\n", EI_CLASSIFIER_RAW_SAMPLE_COUNT / 16); ei_printf("\tNo. of classes: %d\n", sizeof(ei_classifier_inferencing_categories) / sizeof(ei_classifier_inferencing_categories[0])); if (microphone_inference_start(EI_CLASSIFIER_RAW_SAMPLE_COUNT) == false) { ei_printf("ERR: Could not allocate audio buffer (size %d), this could be due to the window length of your model\r\n", EI_CLASSIFIER_RAW_SAMPLE_COUNT); return; } } /** * @brief Arduino main function. Runs the inferencing loop. */ void loop() { ei_printf("Starting inferencing in 2 seconds...\n"); delay(2000); ei_printf("Recording...\n"); bool m = microphone_inference_record(); if (!m) { ei_printf("ERR: Failed to record audio...\n"); return; } ei_printf("Recording done\n"); signal_t signal; signal.total_length = EI_CLASSIFIER_RAW_SAMPLE_COUNT; signal.get_data = &microphone_audio_signal_get_data; ei_impulse_result_t result = { 0 }; EI_IMPULSE_ERROR r = run_classifier(&signal, &result, debug_nn); if (r != EI_IMPULSE_OK) { ei_printf("ERR: Failed to run classifier (%d)\n", r); return; } // print inference return code ei_printf("run_classifier returned: %d\r\n", r); print_inference_result(result); } // Fill the dots one after the other with a color void colorWipe(uint32_t c, uint8_t wait) { for(uint16_t i=0; i<strip.numPixels(); i++) { strip.setPixelColor(i, c); strip.show(); delay(wait); } } /** * @brief PDM buffer full callback * Copy audio data to app buffers */ static void pdm_data_ready_inference_callback(void) { int bytesAvailable = PDM.available(); // read into the sample buffer int bytesRead = PDM.read((char *)&sampleBuffer[0], bytesAvailable); if ((inference.buf_ready == 0) && (record_ready == true)) { for(int i = 0; i < bytesRead>>1; i++) { inference.buffer[inference.buf_count++] = sampleBuffer[i]; if(inference.buf_count >= inference.n_samples) { inference.buf_count = 0; inference.buf_ready = 1; break; } } } } /** * @brief Init inferencing struct and setup/start PDM * * @param[in] n_samples The n samples * * @return { description_of_the_return_value } */ static bool microphone_inference_start(uint32_t n_samples) { inference.buffer = (int16_t *)malloc(n_samples * sizeof(int16_t)); if(inference.buffer == NULL) { return false; } inference.buf_count = 0; inference.n_samples = n_samples; inference.buf_ready = 0; // configure the data receive callback PDM.onReceive(pdm_data_ready_inference_callback); PDM.setBufferSize(2048); delay(250); // initialize PDM with: // - one channel (mono mode) if (!PDM.begin(1, EI_CLASSIFIER_FREQUENCY)) { ei_printf("ERR: Failed to start PDM!"); microphone_inference_end(); return false; } // optionally set the gain, defaults to 24 // Note: values >=52 not supported PDM.setGain(40); return true; } /** * @brief Wait on new data * * @return True when finished */ static bool microphone_inference_record(void) { bool ret = true; record_ready = true; while (inference.buf_ready == 0) { delay(10); } inference.buf_ready = 0; record_ready = false; return ret; } /** * Get raw audio signal data */ static int microphone_audio_signal_get_data(size_t offset, size_t length, float *out_ptr) { numpy::int16_to_float(&inference.buffer[offset], out_ptr, length); return 0; } /** * @brief Stop PDM and release buffers */ static void microphone_inference_end(void) { PDM.end(); ei_free(inference.buffer); } void print_inference_result(ei_impulse_result_t result) { // Print how long it took to perform inference ei_printf("Timing: DSP %d ms, inference %d ms, anomaly %d ms\r\n", result.timing.dsp, result.timing.classification, result.timing.anomaly); ei_printf("Predictions:\r\n"); for (uint16_t i = 0; i < EI_CLASSIFIER_LABEL_COUNT; i++) { ei_printf(" %s: ", ei_classifier_inferencing_categories[i]); ei_printf("%.5f\r\n", result.classification[i].value); if(i != 3 && result.classification[i].value > 0.90){ rp2040.fifo.push(colors[i]); } } // Print anomaly result (if it exists) #if EI_CLASSIFIER_HAS_ANOMALY == 1 ei_printf("Anomaly prediction: %.3f\r\n", result.anomaly); #endif } #if !defined(EI_CLASSIFIER_SENSOR) || EI_CLASSIFIER_SENSOR != EI_CLASSIFIER_SENSOR_MICROPHONE #error "Invalid model for current sensor." #endif

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

Rp2040 Based Pi-day Bling

About the project

Project info

Items used in this project

Hardware components

Software apps and online services

Story

Schematics, diagrams and documents

Pi Badge Schematic

Pico Badge Schematic

Code

Github repository

Arduino Code

Credits

sai-yamanoor

Leave your feedback...

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

Rp2040 Based Pi-day Bling

About the project

Project info

Items used in this project

Hardware components

Software apps and online services

Story

Schematics, diagrams and documents

Pi Badge Schematic

Pico Badge Schematic

Code

Github repository

Arduino Code

Credits

sai-yamanoor

Leave your feedback...

Related products