Open Source Flipper Zero, Thread, Matter, and Zigbee on ESP32, Arduino go Zephyr, and more!

0:00:00 / -

Watch now on YouTube Watch now

Barbara: The Talking Ai Camera

Made by 8BitsAndAByte / Artificial intelligence / Photos & Video / Retro Tech / Upcycling / Voice

About the project

A retro camera captioning every picture.

Project info

Difficulty: Moderate

Platforms: Raspberry Pi

Estimated time: 3 days

License: Creative Commons Attribution-NonCommercial-ShareAlike CC BY-NC-SA version 4.0 or later (CC BY-NC-SA 4+)

Items used in this project

Hardware components

	Raspberry Pi 3 Model B	x 1
	Cell-phone TRRS Headset - Earbud Headphones w/ Microphone	x 1
	Powerbank	x 1

Software apps and online services

	Tape, Electrical
	DeepAI
	OpenCV
	Raspberry Pi Raspbian

Hand tools and fabrication machines

scissors

x 1

Story

Overview

Below a quick overview of the content.

Introduction and showcase video
Camera
Electronics
Code
Result

Project Video

Introduction & Showcase Video

AI being the latest buzzword, this project is all about the slightly useless. After finding a neat API that captions images and an old camera, the goals is set: A camera that captions what it sees!

Camera

This camera is an AGFA Synchro Box from the 50's. Any camera/device will do, as long as a Raspberry Pi, Powerbank and Pi Camera fit inside.

AGFA Synchro Box

Electronics

The electronics are pretty straightforward, to start off setup your Pi and connect the Pi Camera. To power everything you connect the powerbank, the same way you would with your phone.

When everything is wired up, we start on the fiddly part, placing the camera. In this case I kept the shutter open and made sure it was aligned. I takes some time, but hang in there!

Placing and aligning the camera

All that remains is to plug in in our headphones and make sure the Pi plays our audio via the 3.5mm jack.

Plugin in the headphones

Code

The code is a slightly tricky bit, but to make things easy, here is a high level overview:

Take a picture every second
Check if the picture is mostly black (shutter is closed), if so, delete the picture
If not, send the picture to the AI API, receiving the caption back.
This caption is turned into a MP3 file, playing it via the headphones.

For anyone curious, the full code is attached to this article.

Result

Now we have a camera that doesn't take pictures, it only captions them!Some examples for your enjoyment.

A close up of a person holding a cell phone

A close up of a large window with a bird on it

A view of a UNK of a clock tower

Code

import requests
import calendar
import time
from picamera import PiCamera
from PIL import Image
import cv2
import os
from gtts import gTTS

camera = PiCamera()

def speak(text):

	file = "file.mp3"
	tts = gTTS(text, 'en')
	tts.save(file)
	os.system("mpg123 " + file)

def captionImage(imagePath):

    response = requests.post(
        "https://api.deepai.org/api/neuraltalk",
        files={
            'image':  open(imagePath, 'rb')
        },
        headers={'api-key': 'XXXXXXXXXXXXX'}
    )

    return response.json().get("output")

def takePicture(camera):
	
	imageName ='./images/raw/image_' + str(calendar.timegm(time.gmtime()))+'.jpg'
	rawImg = camera.capture(imageName)
	return imageName

while True:
	
	imgPath = takePicture(camera)
	img = Image.open(imgPath)
  
  #Check the min and max colour valeu
	valeus = img.convert("L").getextrema()
  
  #If the brighest colour is below this, shutter must be closed
	if valeus[1] < 160:
		os.remove(imgPath)
	else:
		speak("click!")
		resultCaption = captionImage(imgPath)
		speak(resultCaption)

	
	time.sleep(0.5)

Credits

8BitsAndAByte

We’re Dane & Nicole, two makers that create tremendously terrible tech, which we happily share with you on our channel! https://www.youtube.com/c/8bitsandabyte

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

Barbara: The Talking Ai Camera

About the project

Project info