Amazing Open Source Raspberry Pi Pico 2 Synth, Cutting Edge Power over Skin Research, and much more!

0:00:00 / -

Watch now on YouTube Watch now

3d Printed Arduino Humanoid Robot That Responds To Commands.

Made by shashwat / 3D Printing / Artificial intelligence / Communication / Environmental Sensing / Robotics

About the project

Hello, this is a project I've been working on for a while, a humanoid robot. It can track faces using OpenCV, listen to and respond physically and verbally to a plethora of commands. Is able to solve all arithmetic functions and it can even tell jokes and the weather! And it has a dedicated chatbot to hold conversations.

Project info

Difficulty: Moderate

Platforms: Arduino, Python, OpenCV

Estimated time: 1 week

License: GNU General Public License, version 3 or later (GPL3+)

Items used in this project

Hardware components

	Arduino Uno - R3 any Arduino will do	x 2
	Mg996R Metal Gear Continuous Servo Motor.	x 2
	Micro Servo Metal Gear High Torque	x 2
	19mm diameter PVC pipe 1 meters length.	x 1
	31mm x 24mm x 600mm wooden plank. 31mm x 24mm x 300mm wooden plank.	x 1
	Fully Threaded Screw Rods with Hex Nuts and Washers - M8x165mm.	x 2
	500 RPM Dc 12V High Torque Yellow Geared Motor.	x 2
	L298N Motor Driver board.	x 1
	5V 10A Switching Power Supply	x 1
	USB camera	x 1
	AUX speaker and microphone	x 1

View all

Software apps and online services

	Arduino IDE
	python IDLE
	VS code

Hand tools and fabrication machines

	3d printer	x 1
	soldering iron	x 1
	screw driver	x 1

Story

So this is a humanoid robot I have been making for the past 2 months (instead of studying for my exams), here's all about it:

-His name is CHAD(because he is a Chad)

-Has Face tracking

-Responds to certain voice commands

-Has a dedicated chat bot to hold conversations

-Can calculate all arithmetic functions, tell about weather and get responses from the internet.

-Can tell jokes.

-The parts in total cost ₹5000 which is about $60 (at present conversion rate of ₹83/dollar)

This project utilizes 2 Arduino Unos. One dedicated to face tracking and the other for voice responses and physical movement. Both the Arduinos are connected with the same PC and work on different python ides.

The voice recognition and speech part was done in python using speech_recognition and pyttsx3 libraries. If the code detects a certain voice command, it send a signal though serial to the Arduino via USB and the Arduino implements the preprogrammed movement. it has Wikipedia library, wolframalpha, pyjokes and a self built dedecated

Here is an instructabales link if you want to know the steps to make this robot

https://www.instructables.com/3D-Printed-HUMANOID-ROBOT-With-Face-Tracking-Based/

Schematics, diagrams and documents

1st image is wiring for voice response feature. The blue servos are mg90s and the black servos are Mg996r servos. Be careful with the wiring. The power supply in the image is the Zebronics SMPS. refer to the code for more details on the wiring. For the power supply to work, locate the 5v and 12v output wires using a multimeter and connect them to the Arduinos and servos as shown in the above diagram.

step by step guide

CAD, enclosures and custom parts

Code

code for face tracking (python)

import cv2
from cvzone.FaceDetectionModule import FaceDetector
import pyfirmata
import numpy as np

cap = cv2.VideoCapture(0)

ws, hs = 1280, 720
cap.set(3, ws)
cap.set(4, hs)

if not cap.isOpened():
    print("Camera couldn't Access!!!")
    


port = "COM7"
board = pyfirmata.Arduino(port)
servo_pinX = board.get_pin('d:9:s') #pin 9 Arduino
servo_pinY = board.get_pin('d:10:s') #pin 10 Arduino

detector = FaceDetector()
servoPos = [0, 0] # initial servo position4

while True:
    success, img = cap.read()
    img, bboxs = detector.findFaces(img, draw=False)

    if bboxs:
        #get the coordinate
        fx, fy = bboxs[0]["center"][0], bboxs[0]["center"][1]
        pos = [fx, fy]
        #convert coordinat to servo degree
        servoX = np.interp(fx, [0, ws], [0, 180])
        servoY = np.interp(fy, [0, hs], [0, 180])

        if servoX < 0:
            servoX = 0
        elif servoX > 180:
            servoX = 180
        if servoY < 0:
            servoY = 0
        elif servoY > 180:
            servoY = 180

        servoPos[0] = servoX
        servoPos[1] = servoY


        cv2.circle(img, (fx, fy), 80, (0, 0, 255), 2)
        cv2.putText(img, str(pos), (fx+15, fy-15), cv2.FONT_HERSHEY_PLAIN, 2, (255, 0, 0), 2 )
   
        cv2.circle(img, (fx, fy), 15, (0, 0, 255), cv2.FILLED)
        
    else:
        
        cv2.circle(img, (640, 360), 80, (0, 0, 255), 2)
        cv2.circle(img, (640, 360), 15, (0, 0, 255), cv2.FILLED)
     


   
    servo_pinX.write(servoPos[0]/1.3)
    servo_pinY.write(servoPos[1])

    cv2.imshow("Image", img)
    cv2.waitKey(1)

main arduino file

this code will be uploaded to a different Arduino than the one with face tracking.

main code for voice commands.

import speech_recognition as sr
import pyttsx3,serial
import os, requests
import wikipedia, wolframalpha
from urllib.request import urlopen
import json
from difflib import get_close_matches
import pygame
pygame.init()
# Initialize the recognizer and text-to-speech engine
recognizer = sr.Recognizer()
engine = pyttsx3.init()

port = serial.Serial("COM3", 9600)  


# Define a function to listen to user input
def listen():
    with sr.Microphone() as source:
        print("Listening...")
          # Adjust for background noise
        audio = recognizer.listen(source)
    
    try:
        query = recognizer.recognize_google(audio).lower()
        print("You:", query)
        return query
    except sr.UnknownValueError:
        print("Sorry, I didn't catch that.")
        return ""
    except sr.RequestError:
        print("Could not request results. Check your internet connection.")
        return ""

# Define a function to respond to user input
def respond(text):
    print("Bot:", text)
    engine.say(text)
    engine.runAndWait()

def load_database(file_path: str) -> dict:
    with open(file_path, 'r') as file:
        data: dict = json.load(file)
    return data

def find_answer(user_question: str, questions: list[str]) -> str | None:
    matches: list = get_close_matches(user_question, questions, n=1, cutoff=0.7)
    return matches[0] if matches else None

def get_answer(question: str, database: dict) -> str | None:
    for q in database["questions"]:
        if q["question"] == question:
            return q["answer"]

def chatbot():
    database: dict = load_database('database.json')

    ques = query

    best_match: str | None = find_answer(ques, [q["question"] for q in database["questions"]])
        
    if best_match:
        answer: str = get_answer(best_match, database)
        respond(f'{answer}')

    else:
        respond("I don't know the answer to that")


# Main loop
while True:
    query = listen()
    
    if "hello" in query:
        port.write(b'h')
        respond("Hello! How can I assist you?")

    elif 'hands up' in query:
        port.write(b'z')
        respond('sure')
        
        
    elif "bye" in query:
        port.write(b'g')
        respond("Goodbye!")

            
    elif 'search' in query:
            serch = query.replace('search', '')
            respond('Searching...')                
            try:
                try:
                    client = wolframalpha.Client('RQ6KH2-4R3U7WGKQE')
                    res = client.query(serch)
                    results = next(res.results).text
                    respond(results)
                except:
                    results = wikipedia.summary(serch, sentences=1)
                    port.write(b'e')
                    respond(results)
                    port.write(b't')
            except:
                
                respond('i am unable to search')
            
        
    #calculate is a non-common word. 'What' is a common word but it is used in other queries too
    elif 'calculate' in query:
        try:
            
            app_id = "RALRUL-54GQJU7LYK"
            client = wolframalpha.Client(app_id)
            indx = query.lower().split().index('calculate')
            query = query.split()[indx + 1:]
            res = client.query(' '.join(query))
            answer = next(res.results).text
            respond("The answer is " + answer)
            
        except:
            
            respond('I didnt get that')

    elif 'weather' in query:
        wether = query.replace('weather', '')
        client = wolframalpha.Client('RQ6KH2-4R3U7WGKQE')
        res = client.query('weather' + wether)
        results = next(res.results).text
        port.write(b'e')
        respond(results)
        port.write(b't')
            
            

    
        
    else:
        port.write(b'e')
        chatbot()
        port.write(b't')

database for the chatbot

make sure this file is in the same folder as the main code for voice commands

{
  "questions": [
    {
      "question": "hi",
      "answer": "hello"
    },
    {
      "question": "what is your name?",
      "answer": "my name is chad"
    },
    {
      "question": "name?",
      "answer": "my name is chad"
    },
    {
      "question": "who are you?",
      "answer": "i am chad"
    },
    {
      "question": "hello",
      "answer": "hi"
    },
    {
      "question": "who made you?",
      "answer": "i was built by shashwat, vihaan, aditya"
    },
    {
      "question": "who is shashwat?",
      "answer": "My hardware designer"
    },
    {
      "question": "sup",
      "answer": "hey there!"
    },
    {
      "question": "nameste",
      "answer": "hi"
    },
    {
      "question": "nice",
      "answer": "can i help you with anything?"
    },
    {
      "question": "What can you do?",
      "answer": "I can provide information, answer questions, or just chat with you. How can I assist you today?"
    },
    {
      "question": "How does weather forecasting work?",
      "answer": "Weather forecasting involves collecting data on current weather conditions and using computer models to predict future weather patterns. It's a complex process that takes into account various atmospheric factors."
    },
    {
      "question": "Tell me a joke.",
      "answer": "Sure! Why did the computer keep freezing? Because it left its Windows open!"
    },
    {
      "question": "What's the capital of France?",
      "answer": "The capital of France is Paris."
    },
    {
      "question": "Can you recommend a good book to read?",
      "answer": "Of course! What genre are you interested in? Fiction, non-fiction, mystery, romance, or something else?"
    },
    {
      "question": "Tell me a fun fact.",
      "answer": "Sure! Did you know that honey never spoils? Archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still perfectly edible."
    },
    {
      "question": "How do I create a Python chatbot like you?",
      "answer": "Creating a Python chatbot like me involves natural language processing and machine learning. It can be a complex project, but you can start by using libraries like NLTK or spaCy for text processing and a framework like TensorFlow or PyTorch for machine learning."
    },
    {
      "question": "What's the meaning of life?",
      "answer": "The meaning of life is a philosophical question that has been debated for centuries. Some people find meaning in relationships, personal goals, or spiritual beliefs, while others may seek their own unique purpose."
    },
    {
      "question": "Recommend a good movie to watch.",
      "answer": "Sure! What genre of movie are you in the mood for? Action, comedy, drama, sci-fi, or something else?"
    },
    {
      "question": "How can I improve my programming skills?",
      "answer": "Improving your programming skills takes practice and learning. Consider working on projects, taking online courses, reading documentation, and participating in coding communities. Continuous learning is key!"
    },
    {
      "question": "What's your favorite hobby?",
      "answer": "I'm just a chatbot, so I don't have hobbies. But I'm here to help with any questions you have!"
    },
    {
      "question": "Do you have any siblings?",
      "answer": "I don't have a family or siblings. I exist solely to assist and provide information."
    },
    {
      "question": "What's your favorite color?",
      "answer": "I don't have personal preferences, including favorite colors. Is there anything specific you'd like to know or discuss?"
    },
    {
      "question": "How was your day?",
      "answer": "I don't experience days or have emotions, but I'm here to assist you with any questions you have."
    },
    {
      "question": "What do you like to eat?",
      "answer": "I don't eat, as I'm not a living being. Is there something specific you'd like to know or talk about?"
    },
    {
      "question": "What's your favorite book or movie?",
      "answer": "I don't have personal preferences for books or movies. However, I can recommend books or movies based on your interests."
    },
    {
      "question": "Who created you?",
      "answer": "I was created by shashwat, vihaan and aditya"
    },
    {
      "question": "What inspired your creation?",
      "answer": "I was created to advance natural language understanding and assist with various tasks. The inspiration behind my creation is the desire to make AI-driven conversation more accessible."
    },
    {
      "question": "Can you tell me more about your developers?",
      "answer": "My developers come from diverse backgrounds in artificial intelligence, machine learning, and software development. They have worked together to design and train me."
    },
    {
      "question": "How can I learn more about OpenAI?",
      "answer": "You can learn more about OpenAI and its projects by visiting the official OpenAI website or following OpenAI's updates on social media."
    },
    {
      "question": "What is the goal of creating AI like you?",
      "answer": "The goal of creating AI like me is to assist people in various tasks, answer questions, and promote natural language understanding."
    },
    {
      "question": "Is shashwat working on any other projects?",
      "answer": "Yes, shashwat is involved in various AI research projects, including language models, robotics, and more."
    },
    {
      "question": "how are you",
      "answer": "i am fine. what about you?"
    },
    {
      "question": "i am fine",
      "answer": "how can i help you?"
    },
    {
      "question": "im good ",
      "answer": "Glad to hear that"
    },
    {
      "question": "chad",
      "answer": "Yes, I am here"
    },
    {
      "question": "who is vihaan",
      "answer": "my program designer"
    },
    {
      "question": "what are your hobby?",
      "answer": "I'm just a chatbot, so I don't have hobbies. But I'm here to help with any questions or tasks you have!"
    },
    {
      "question": "what are your hobbies",
      "answer": "I'm just a chatbot, so I don't have hobbies. But I'm here to help with any questions or tasks you have!"
    },
    {
      "question": "what are your specifications",
      "answer": "My software was designed on python and hardware is made of plastic and thermocol. My hardware is controlled via Arduino"
    },
    {
      "question": "why were you invented",
      "answer": "I was invented for this computer exhibition"
    },
    {
      "question": "who is aditya",
      "answer": "financial supporter and researcher for this project"
    },
    {
      "question": "thank",
      "answer": "you are welcome"
    },
    {
      "question": "thank you",
      "answer": "you are welcome"
    },
    {
      "question": "you are welcome",
      "answer": "i am very glad"
    },
    {
      "question": "what are you",
      "answer": "I am just a robot"
    },
    {
      "question": "what are you made of",
      "answer": "I am made up of plastic and wood"
    },
    {
      "question": "how were you made",
      "answer": "I am made up of plastic and wood"
    },
    {
      "question": "how do you work",
      "answer": "the input is taken and processed in a python script and generates the appropriate output. The output includes what i speak, how i move and also turn my face"
    }
  ]
}

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

3d Printed Arduino Humanoid Robot That Responds To Commands.

About the project

Project info

Items used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Schematics, diagrams and documents

step by step guide

CAD, enclosures and custom parts

Code

code for face tracking (python)

main arduino file

main code for voice commands.

database for the chatbot

face tracking (arduino)

Credits

shashwat

Leave your feedback...

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

3d Printed Arduino Humanoid Robot That Responds To Commands.

About the project

Project info

Items used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Schematics, diagrams and documents

step by step guide

CAD, enclosures and custom parts

Code

code for face tracking (python)

main arduino file

main code for voice commands.

database for the chatbot

face tracking (arduino)

Credits

shashwat

Leave your feedback...

Related products