Open Source Flipper Zero, Thread, Matter, and Zigbee on ESP32, Arduino go Zephyr, and more!

0:00:00 / -

Watch now on YouTube Watch now

Phototrap With Object Detection Alarm

About the project

A Design Study to use ML with Edge Impluse to detect a object and use Blue Wireless hardware, notehub. io with Mqtt for notification

Project info

Difficulty: Easy

Platforms: Raspberry Pi, Edge Impulse, MQTT, Blues Wireless

Estimated time: 2 hours

License: GNU General Public License, version 3 or later (GPL3+)

Items used in this project

Hardware components

	Webcam , Cls Full Hd	x 1
	Blues Wireless Raspberry Pi Starter Kit	x 1
	Raspberry Pi 4 Model B	x 1

Software apps and online services

	MQTT
	Blues Wireless Notehub.io
	Edge Impulse Studio

Story

This is a design study to detect objects by means of edge pulses and to transmit them via Blue Wireless hardware to the cloud in order to transmit a signal or an alarm regardless of location or to evaluate this in a long-term analysis.

My settings with Raspberry Pi and Webcam

Conceivable scenarios here are a wildlife camera with detection of animals or detection of specific animals with alarm.

Of course this also works with people and an alarm or vice versa no alarm if the right person enters the entrance.

Hardware

Here we used a Raspberry Pi 4 2gb with a commercial webcam and the Blue Wireless Notecarrier Pi HAT.

The better the quality of the optics, the better the result, especially in difficult lighting conditions.

If the ML model is trained with the images from the camera, the recognition rate increases significantly.

For testing, I took two cuddly toys and trained them with some images from different distances and angles, the footage must then be split into learning and training images.

For programming I used Visual Studio Code with an extension (Remote SSH) for remote programming via SSH directly on the Raspberry Pi, which is very cool.

Working live on the Raspberry

Machine Learning with Edege Impulse

The program was implemented in Python, the camera image is evaluated with the ML model of Edge Impulse and in case of a hit it is transmitted to the Notehub.io in the cloud via the Blue Wireless module (Wlan or LTE modem). The project was created in Edege Impulse:

Here, the images need to be generated and classified accordingly to create and train a model.

For testing, the ML model of Edge Impulse can be used via.

1 / 2

Edge Impluse Trainigs output

More data (images) = higher accuracy.

What matters is the amount and quality of the data, which is displayed as a website by the Raspberry with a slight delay.

Deploying the model

The command

edge-impulse-linux-runner --download modelfile.eim

downloads the ML model locally to the Raspberry.

The command applies this in the Python script

/bin/python /home/manager/edge_impulse_cam.py modelfile.eim

In "edege_impulse_cam.py", the image is evaluated and if detected, it sent to Notehub.io via "send_to_notehub.py" with the Blue Wireless module.

In Notehub.io this is then displayed as follows under Events:

With a registered route, this can then be forwarded to a service. In my case to a Mqtt broker from HiveMQ.

with this JSON declaration

With MQTTlens I can see the message from my Phototrap.

Details of a message

This can also be used in a mobile app or with homeassistant to trigger an action here as a MQTT Client.

For example an alert -> Save my cookies from the bear :-)

Schematics, diagrams and documents

Signal schematic

Raspberry Pi detect with ML Modell from Edge Impluse an object an send via Blue Wirless Modul an alert via Notehub.io to a Mqtt Broker

Code

send_to_notehub.py

send the Data to Notehub.io

import json import notecard import notecard_pseudo_sensor from periphery import I2C import time productUID = "your_id" port = I2C("/dev/i2c-1") card = notecard.OpenI2C(port, 0, 0) req = {"req": "hub.set"} req["product"] = productUID req["mode"] = "continuous" print(json.dumps(req)) rsp = card.Transaction(req) print(rsp) object_name = "" last_send = None def send(object_name,object_number): print('Object: {} '.format(object_name)) req = {"req": "note.add"} req["file"] = "sensors.qo" req["start"] = True req["body"] = {"Object": object_name,"Alert_Number":object_number} rsp = card.Transaction(req) print(rsp)

edge_impulse_cam.py

search an Object and send it with send_to_notehub.py

#!/usr/bin/env python #import device_patches # Device specific patches for Jetson Nano (needs to be before importing cv2) import cv2 import os import sys, getopt import signal import time from edge_impulse_linux.image import ImageImpulseRunner import send_to_notehub as send_to runner = None # if you don't want to see a camera preview, set this to False show_camera = True if (sys.platform == 'linux' and not os.environ.get('DISPLAY')): show_camera = False def now(): return round(time.time() * 1000) def get_webcams(): port_ids = [] for port in range(5): print("Looking for a camera in port %s:" %port) camera = cv2.VideoCapture(port) if camera.isOpened(): ret = camera.read()[0] if ret: backendName =camera.getBackendName() w = camera.get(3) h = camera.get(4) print("Camera %s (%s x %s) found in port %s " %(backendName,h,w, port)) port_ids.append(port) camera.release() return port_ids def sigint_handler(sig, frame): print('Interrupted') if (runner): runner.stop() sys.exit(0) signal.signal(signal.SIGINT, sigint_handler) def help(): print('python classify.py <path_to_model.eim> <Camera port ID, only required when more than 1 camera is present>') def main(argv): try: opts, args = getopt.getopt(argv, "h", ["--help"]) except getopt.GetoptError: help() sys.exit(2) for opt, arg in opts: if opt in ('-h', '--help'): help() sys.exit() if len(args) == 0: help() sys.exit(2) model = args[0] dir_path = os.path.dirname(os.path.realpath(__file__)) modelfile = os.path.join(dir_path, model) print('MODEL: ' + modelfile) last_send = time.time() object_no = 0 with ImageImpulseRunner(modelfile) as runner: try: model_info = runner.init() print('Loaded runner for "' + model_info['project']['owner'] + ' / ' + model_info['project']['name'] + '"') labels = model_info['model_parameters']['labels'] if len(args)>= 2: videoCaptureDeviceId = int(args[1]) else: port_ids = get_webcams() if len(port_ids) == 0: raise Exception('Cannot find any webcams') if len(args)<= 1 and len(port_ids)> 1: raise Exception("Multiple cameras found. Add the camera port ID as a second argument to use to this script") videoCaptureDeviceId = int(port_ids[0]) camera = cv2.VideoCapture(videoCaptureDeviceId) ret = camera.read()[0] if ret: backendName = camera.getBackendName() w = camera.get(3) h = camera.get(4) print("Camera %s (%s x %s) in port %s selected." %(backendName,h,w, videoCaptureDeviceId)) camera.release() else: raise Exception("Couldn't initialize selected camera.") next_frame = 0 # limit to ~10 fps here for res, img in runner.classifier(videoCaptureDeviceId): if (next_frame > now()): time.sleep((next_frame - now()) / 1000) # print('classification runner response', res) if "classification" in res["result"].keys(): print('Result (%d ms.) ' % (res['timing']['dsp'] + res['timing']['classification']), end='') for label in labels: score = res['result']['classification'][label] print('%s: %.2f\t' % (label, score), end='') print('', flush=True) elif "bounding_boxes" in res["result"].keys(): print('Found %d bounding boxes (%d ms.)' % (len(res["result"]["bounding_boxes"]), res['timing']['dsp'] + res['timing']['classification'])) for bb in res["result"]["bounding_boxes"]: print('\t%s (%.2f): x=%d y=%d w=%d h=%d' % (bb['label'], bb['value'], bb['x'], bb['y'], bb['width'], bb['height'])) print("\tGot %s " %(bb['label']) ) time_now = time.time() if time_now - last_send > 10: print("Send to notehub") send_to.send(str(bb['label']),str(object_no)) object_no += 1 last_send = time_now img = cv2.rectangle(img, (bb['x'], bb['y']), (bb['x'] + bb['width'], bb['y'] + bb['height']), (255, 0, 0), 1) if (show_camera): cv2.imshow('edgeimpulse', cv2.cvtColor(img, cv2.COLOR_RGB2BGR)) if cv2.waitKey(1) == ord('q'): break next_frame = now() + 100 finally: if (runner): runner.stop() if __name__ == "__main__": main(sys.argv[1:])

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

Phototrap With Object Detection Alarm

About the project

Project info

Items used in this project

Hardware components

Software apps and online services

Story

Schematics, diagrams and documents

Signal schematic

Code

send_to_notehub.py

edge_impulse_cam.py

Credits

daniel-bradtmoeller

Leave your feedback...

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

Phototrap With Object Detection Alarm

About the project

Project info

Items used in this project

Hardware components

Software apps and online services

Story

Schematics, diagrams and documents

Signal schematic

Code

send_to_notehub.py

edge_impulse_cam.py

Credits

daniel-bradtmoeller

Leave your feedback...

Related products