Open Source Flipper Zero, Thread, Matter, and Zigbee on ESP32, Arduino go Zephyr, and more!

0:00:00 / -

Watch now on YouTube Watch now

Esp32cam Pan And Tilt Using Facial Tracking

Made by departedmessenger / 3D Printing / Artificial intelligence / Robotics / Security / Sensors

About the project

I wanted to post this project because its really easy and rewarding to make. I'll just tell you how to modify the Espressif example, by adding some lines of code in the right places. In the end you will be able to perform computer vision, facial tracking, and pan and tilt without computer support.

Project info

Difficulty: Moderate

Platforms: Arduino, Espressif

Estimated time: 4 hours

License: GNU General Public License, version 3 or later (GPL3+)

Items used in this project

Hardware components

	ESP32-CAM Development Board	x 1
	Micro Servo	x 2
	lm7805 5 volt regulators	x 2
	Mini Bread Board Self Adhesive	x 1
	2.54mm header pins	x 1
	jumperwires	x 10
	2s lipo battery or an 5 volt power supply	x 1

Software apps and online services

Arduino IDE

Hand tools and fabrication machines

	small drill	x 1
	glue gun	x 1
	small Phillips screwdriver	x 1

Story

I wanted to post this project because its really easy and rewarding to make. The ESP32Cam has an example project where it allows the user to detect faces, recognise faces, and stream the camera through a nice web page. You can find this library on your Arduino IDE under:

File-->examples-->esp32-->camera-->CameraWebServer

Its worth taking some time to get to know this example. You can adjust the video feed, detect faces, and train the program to recognise specific faces too. I think Espressif really pushed the boundaries of computer vision with this code. Most computer vision or machine learning programs require large operating systems, compilers, and libraries to process video as data. This is so stream-lined and embedded into a RISC-V instruction set, they must have pre-trained the tensors and used a quantisation/approximation. There aren't many edge products that can do computer vision so well on a tiny ESP32 processor.

I really went down a rabbit hole with this code, but I'll just tell you how to modify their program by adding some lines of code in the right places. In the end you will be able to perform computer vision, facial tracking, pan and tilt, and possible modify it for even self driving cars or robots. Lets start.

Schematics, diagrams and documents

Electronics

CAD, enclosures and custom parts

Pan and tilt .stl files for micro servos

This will link you to thingiverse and the pan and tilt design I used, but any others ones will also work fine.

Go to download

Code

Cut and paste this code into the Expressif ESP32cam example file shown below.

This code needs to be pasted into the 'app_httpd.cpp' file of the camera example code found at: File-->examples-->esp32-->camera-->CameraWebServer More detail on where to put the code, and some explanation, can be found on my blog: departedmessenger.blogspot.com

//put this in to line 22 of the app_httpd.cpp file
#include "ESP32Servo.h"
// 

//insert this code to line 75 of the app_httpd.cpp file:
int servoPin1 = 15;
int servoPin2 = 14;
int x_pos;
int y_pos;
int delta_x;
int delta_y;

Servo myservo1;
Servo myservo2;
//

////pantilt code
//insert this code to line 243 of the app_httpd.cpp file:
ESP32PWM::allocateTimer(0);
ESP32PWM::allocateTimer(1);
ESP32PWM::allocateTimer(2);
ESP32PWM::allocateTimer(3); 

if (!myservo1.attached()) {
	myservo1.setPeriodHertz(50); // standard 50 hz servo
	myservo1.attach(servoPin1, 1000, 2000); // Attach the servo after it has been detatched
}
if (!myservo2.attached()) {
	myservo2.setPeriodHertz(50); // standard 50 hz servo
	myservo2.attach(servoPin2, 1000, 2000); // Attach the servo after it has been detatched
}          

int x_cent = x + (w/2);   //find center point on the x axis 
int y_cent = y + (h/2);   //find center point on the y axis

int delta_x = ((x_cent - 120)/3)^2;//tune the response of the servos by varying the divisor and exponential values
int delta_y = ((y_cent - 120)/3)^2;
      
int x_pos_temp = x_pos;
int y_pos_temp = y_pos;
for (i = 1; i <= 10; i++) {                  	//use 10 steps for each adjustment
 	x_pos = x_pos_temp - (i*delta_x)/10; 	//pan
        if (x_pos > 180) {   			//upper x servo limit
         	x_pos = 180; }     		//
        if (x_pos < 0) {    			//lower x servo limit
        	x_pos = 0; } 
        myservo1.write(x_pos);			// tell servo to go to position   
	delay(15);           			// waits 15ms for the servo 
  
        y_pos = y_pos_temp - (i*delta_y)/10; 	//tilt
        if (y_pos > 140) {   			//upper y servo limit
        	y_pos = 140; }     		//
        if (y_pos < 40) {    			//lower y servo limit
        	y_pos = 40; }       		//
        myservo2.write(y_pos);			// tell servo to go to position
	delay(15);				// waits 15ms for the servo
         delay(50);				//varies the speed of the servos
}
////end pantilt code

Credits

departedmessenger

I am a maker who enjoys sharing with the community. My interests :Biometric robots, AI, wifi-vision, and fanart.

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

Esp32cam Pan And Tilt Using Facial Tracking

About the project

Project info