Open Source Flipper Zero, Thread, Matter, and Zigbee on ESP32, Arduino go Zephyr, and more!

0:00:00 / -

Watch now on YouTube Watch now

Remake Of S. Francis Writing Machine

Made by TommyBiancoTM / Art / Lights / Music

About the project

Modern QWERTY keyboards are simply not challenging enough. Let's use a piano roll instead!

Project info

Difficulty: Easy

Platforms: Arduino

Estimated time: 1 hour

License: Creative Commons Attribution-NonCommercial CC BY-NC version 4.0 or later (CC BY-NC 4+)

Items used in this project

Hardware components

	Male/Female Jumper Wires	x 1
	Arduino UNO	x 1
	RGB LED Pixel Matrix, NeoPixel NeoMatrix	x 1
	MIDI Keyboard	x 1

Software apps and online services

	Hairless MIDI
	loopMIDI

Story

What and Why?

This is a Samuel Francis writing machine, nicknamed "Literary Piano" [0]. Patented in 1857, it is an early typing mechanism that combines a piano-like keyboard with a circular typewriter mechanism. It has nothing in common with a modern computer keyboard, which is is quite a shame. Many of the brain areas that process language also process music [1], only making the association of a piano with a typewriter more appealing.

So let's see if we can revive Samuel Francis heritage by turning a modern MIDI keyboard into a "perfect" writing machine, starring in the second image below:

A typewriter prototype patented by Dr. Samuel Ward Francis of Newport, Rhode Island in 1857. Image from National Museum of American History.

A typewriter prototype patented by Dr. Samuel Ward Francis of Newport, Rhode Island in 1857. Image from National Museum of American History.

An advanced prototype made with a marker pen, sticky notes and tape, circa 2023 (colourised).

An advanced prototype made with a marker pen, sticky notes and tape, circa 2023 (colourised).

Project on video

Project summary in an entertaining form & results showcase.

Capturing MIDI controller input

Once we have added letter stickers, it is time to learn about capturing and handling MIDI signals. Python has a number of libraries designed to control MIDI streams [2]; in this project we are using Pygame [3].

Once a MIDI controller is connected to the laptop, it will appear on the list of the available input [1, 0, 0] and output [0, 1, 0] devices. We can simply pick our keyboard, read the message and start deciphering.

MIDI input and output device selection: select your controller as input and loopMIDI as output.

MIDI input and output device selection: select your controller as input and loopMIDI as output.

A single MIDI message consists of a status byte followed by some data bytes. First, we isolate the status byte to tell if the key is pressed down or released. 1001 corresponds to a "Note On" message - the signal that we are looking for. From the first data byte, we can read the key number, adjust for changing octaves and convert it to an appropriate letter output using keyboard library [4].

MIDI message structure. MIDI CH = MIDI channel. Diagram by Bernardo Breve et al. 2020 / DOI:10.18293/DMSVIVA20-011

MIDI message structure. MIDI CH = MIDI channel. Diagram by Bernardo Breve et al. 2020 / DOI:10.18293/DMSVIVA20-011

Sending MIDI output

Typing messages with the piano is cool, but where is the music? Unfortunately, our little script "hijacks" the MIDI stream, which means that the message is never sent to your musical software. Let's fix that.

All it takes is adding a MIDI output and forwarding a copy of the received message back into the system. Or at least so I thought... To make this actually work on my Windows machine, I needed a little crutch - loopMIDI software [5] - that acted as an intermediate virtual MIDI port between my script and Ableton. Theoretically, Mac users shouldn't have this issue. Practically - let me know :)

Another neat trick we can do is use the keyboard library [4] to capture standard keyboard input and convert it to MIDI. Just in case you want to make your typing more musical without the hustle of sticking letters on a MIDI controller. Unlike with MIDI, keyboard hooks do not interrupt the signal, so there is no need to return the keys "into the wild".

MIDI visualisation with Arduino

Now, this is a project on Hackster, so we have to talk hardware and prototyping. The same MIDI capture and release code can be repurposed to be sent to two outputs: a Digital Audio Workstation (DAW) and to an Arduino.

If you use a simpler microcontroller that does not support MIDI over USB, you will need another small program - Hairless MIDI [6] - to send a receive MIDI messages with your microcontroller. Once the data stream is established, you can hook up your favourite visuals, like an 8x8 RGB matrix, and convert every press of a piano key into a cool animation.

The best part is that your visuals can be essentially anything. Live performers, here is an upgrade idea for your stage presence :)

Tell me what you think about this project!

Please share some feedback! I need your input to keep improving my articles and videos.

Thanks!

Code

Keyboard to MIDI signal

Guess what it does from the name...

import pygame import pygame.midi import keyboard import time # Initialise a MIDI instance def MIDI_init (): if pygame.midi.get_init() == 1: pygame.midi.quit() pygame.midi.init() def output_select(): for n in range(pygame.midi.get_count()): print (n,pygame.midi.get_device_info(n)) device_ID = int(input ("Enter your OUTPUT device number []\n")) return (pygame.midi.Output(device_ID)) def keyboard_read(): key = '' while (key == ''): key = keyboard.read_key() if keyboard.is_pressed(key) != True: key = 'Released' keyboard.unhook_all() if key != 'Released': return key myDict = {'a': 22, 'A': 22, 'b': 10, 'B': 10, 'c': 0, 'C': 0, 'd': 2, 'D': 2, 'e': 17, 'E': 17, 'f': 5, 'F': 5, 'g': 6, 'G': 6, 'h': 4, 'H': 4, 'i': 18, 'I': 18, 'j': 8, 'J': 8, 'k': 3, 'K': 3, 'l': 9, 'L': 9, 'm': 16, 'M': 16, 'n': 15, 'N': 15, 'o': 20, 'O': 20, 'p': 1, 'P': 1, 'q': 13, 'Q': 13, 'r': 11, 'R': 11, 's': 24, 'S': 24, 't': 12, 'T': 12, 'u': 7, 'U': 7, 'v': 23, 'V': 23, 'w': 19, 'W': 19, 'x': 21, 'X': 21, 'y': 14, 'Y': 14, 'z': 26, 'Z':26} signal_log = [False] * 27 octave = 2 def TXT_to_MIDI (midi_output, key): keynum = myDict.get (key) #use get() to avoid keyerror if (keynum == None): #redundant return midi_output.note_off (keynum + 24*octave, velocity=127, channel=0) midi_output.note_on (keynum + 24*octave, velocity=127, channel=0) time.sleep(0.1) # MIDI signal duration of 0.1 seconds midi_output.note_off (keynum + 24*octave, velocity=127, channel=0) # Main body MIDI_init () midi_output = output_select() key = None print ("Press ESC to stop the script") while (key != 'esc'): key = keyboard_read() TXT_to_MIDI (midi_output, key)

MIDI to Arduino

Run this code together with "Arduino Serial Visualization" to make a little lightshow :)

import time import pygame import pygame.midi def MIDI_init (): if pygame.midi.get_init() == 1: pygame.midi.quit() pygame.midi.init() def input_select(): for n in range(pygame.midi.get_count()): print (n,pygame.midi.get_device_info(n)) device_ID = int(input ("Enter your INPUT device number\n")) return (pygame.midi.Input(device_ID)) def output_select(): for n in range(pygame.midi.get_count()): print (n,pygame.midi.get_device_info(n)) device_ID = int(input ("Enter your OUTPUT device number []\n")) return (pygame.midi.Output(device_ID)) def MIDI_read (midi_input, midi_output): if pygame.midi.Input.poll(midi_input) == True: # Read the MIDI message and pass it on midi_data = midi_input.read (1) #or midi_data = pygame.midi.Input.read(midi_input,1) midi_output.write_short(midi_data[0][0][0],midi_data[0][0][1],127) # converts velocity to MAX # Dissect the midi message midi_note, timestamp = midi_data[0] note_status, keynum, velocity, unused = midi_note #print("Note Status: ", note_status, " Key Number: ", keynum," Velocity: " , velocity, "\n\tTime Stamp: ", timestamp) # https://www.cs.cmu.edu/~music/cmsip/readings/MIDI%20tutorial%20for%20programmers.html status_byte = bin(note_status)[2:6] channel = int(bin(note_status)[6:], 2) + 1 if status_byte == "1001": key_down = True else: key_down = False return (keynum, key_down, channel) else: time.sleep(.01) return False import serial ser = serial.Serial() ser.baudrate = 115200 ser.port = 'COM3' ser.open() # message = [keynum, key_down, channel] def MIDI_pass_to_Arduino(keynum): value_sent = bytearray([int(keynum)]) ser.write(value_sent) value_received = int.from_bytes(ser.read(1), 'big') print (value_received) return (value_sent) MIDI_init () midi_input = pygame.midi.Input(3) midi_output = pygame.midi.Output(6) while (True): message = MIDI_read(midi_input,midi_output) if message != False: if message[1] == True: MIDI_pass_to_Arduino(message[0]) # Run this line at the end to prevent odd errors ser.close()

Arduino Serial Visualization

Run this code together with "MIDI to Arduino" or any other script sending numeric Serial input to your Arduino to make a little lightshow :)

#include <Adafruit_NeoPixel.h> #define PIXEL_PIN D5 #define PIXEL_COUNT 64 Adafruit_NeoPixel RGB_matrix = Adafruit_NeoPixel(PIXEL_COUNT, PIXEL_PIN, NEO_GRB + NEO_KHZ800); void setup() { Serial.begin(115200); RGB_matrix.begin(); RGB_matrix.setBrightness(20); RGB_matrix.clear(); RGB_matrix.show(); } int colors [13][3] = { {255, 0, 0}, {0, 255, 0}, {0, 0, 255}, {255, 0, 255}, {255, 255, 50}, {50, 255, 255}, {255, 222, 30}, {0, 255, 120}, {0,255,255}, {255, 175, 0}, {255, 75, 0}, {0, 255, 40} }; int input = 0; void loop() { while (Serial.available()){ input = Serial.read(); Serial.write(input); } int note = input % 12; if (input > 0) { for (int X=0; X<PIXEL_COUNT; X++) { RGB_matrix.setPixelColor(X, colors[note][0], colors[note][1], colors[note][2]); RGB_matrix.show(); } input = 0; } }

MIDI to Keyboard input

Guess what it does from the name...

import time import pygame import pygame.midi import keyboard # Initialise a MIDI instance def MIDI_init (): if pygame.midi.get_init() == 1: pygame.midi.quit() pygame.midi.init() # Selecting input channel def input_select(): for n in range(pygame.midi.get_count()): print (n,pygame.midi.get_device_info(n)) device_ID = int(input ("Enter your INPUT device number\n")) return (pygame.midi.Input(device_ID)) # Selecting output channel for release def output_select(): for n in range(pygame.midi.get_count()): print (n,pygame.midi.get_device_info(n)) device_ID = int(input ("Enter your OUTPUT device number []\n")) return (pygame.midi.Output(device_ID)) # Read and release a MIDI message def MIDI_read (midi_input, midi_output): if pygame.midi.Input.poll(midi_input) == True: # Read the MIDI message and pass it on midi_data = midi_input.read (1) midi_output.write_short(midi_data[0][0][0],midi_data[0][0][1],127) # converts velocity to MAX # Dissect the midi message midi_note, timestamp = midi_data[0] note_status, keynum, velocity, unused = midi_note print("Note Status: ", note_status, " Key Number: ", keynum," Velocity: " , velocity, "\n\tTime Stamp: ", timestamp) status_byte = bin(note_status)[2:6] channel = int(bin(note_status)[6:], 2) + 1 if status_byte == "1001": key_down = True else: key_down = False return (keynum, key_down, channel) else: time.sleep(.01) return False # Map controller keys to letters letters_on_keys = ['c','p','d','k','h','f','g','u','j','l','b','r','t','y','n,'m','e','i','w','o','x','a','v','s'] letters_on_pads = ['"',':','!','?','','','','','','','','','#5','#6','#7','#8','#1','#2','#3','#4','#13',',','.',' '] shift = False # remember the SHIFT key state # Convert a MIDI key number to a letter def MIDI_to_TXT (message, shift): keynum = messsage[0] channel = message [2] shift = shift keynum = keynum % 24 if channel == 1: keyboard.write(letters_on_keys[keynum]) # Release SHIFT if it was pressed if shift == True: keyboard.release("shift") shift = False elif channel == 10: # Press shift if #13 is pressed if letters_on_pads[keynum] == '#13': keyboard.press("shift") shift = True else: keyboard.write(letters_on_pads[keynum]) # Main body MIDI_init () midi_input = input_select() midi_output = output_select() while (True): messsage = MIDI_read(midi_input,midi_output) if messsage != False: if messsage[1] == True: MIDI_to_TXT (messsage, shift)

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

Remake Of S. Francis Writing Machine

About the project

Project info

Items used in this project

Hardware components

Software apps and online services

Story

What and Why?

Project on video

Capturing MIDI controller input

Sending MIDI output

MIDI visualisation with Arduino

Tell me what you think about this project!

Code

Keyboard to MIDI signal

MIDI to Arduino

Arduino Serial Visualization

MIDI to Keyboard input

Credits

TommyBiancoTM

Leave your feedback...

Shop

Project Hub

Video

Blog

Shop home

3D printing

Single Board Computers

Development kits

Sensors

Internet of things

Robotics

Accessories

Product of the Week

Electromaker Educator

The Electromaker Show

The Electromaker Podcast

Remake Of S. Francis Writing Machine

About the project

Project info

Items used in this project

Hardware components

Software apps and online services

Story

What and Why?

Project on video

Capturing MIDI controller input

Sending MIDI output

MIDI visualisation with Arduino

Tell me what you think about this project!

Code

Keyboard to MIDI signal

MIDI to Arduino

Arduino Serial Visualization

MIDI to Keyboard input

Credits

TommyBiancoTM

Leave your feedback...

Related products