Raspberry Pi Media Player Controlled With Gesture

We have seen, in the past numbers, how the intelligent controller The Tactigon interfaces easily with different types of devices, from an Android smartphone to a single computer board like Raspberry Pi 3 equipped with a Bluetooth Low Energy transceiver. In this article we use The Tactigon Skin (T-Skin) in an application which allows interaction with the well-known multimedia VLC Media Player, in order to manage a playlist.

Before getting in the heart of the application we remind that The Tactigon is an intelligent device (and moreover Arduino-compatible) that allows to program controllers capable of detecting three-dimensional gesture and environmental parameters such as temperature and pressure, suitable to develop user interfaces for electronic games, management of industrial and service robots, machinery, drones and advanced model radiometers. The board is ideal in general for the capture of movements and also for applications within the augmented reality.

This little technological jewel is governed by a 32-bit microcontroller of ST Microelectronics, operating with a clock of 32 MHz, equipped with 80 kB of RAM, 512 kB of program memory and 16 kB of EEPROM. The Tactigon (https://thetactigon.com/) is also equipped with Bluetooth Low Energy wireless connection and has IMU on board with a 10 degrees of freedom, with accelerometer, gyroscope and magnetometer, temperature and barometric pressure sensors.

OUR APPLICATION

In this article we will use The Tactigon to interact with a computer application instead of controlling control hardware as we did, for example, in the last April file, where we have implemented a 4-channels lighting control via gesture. More precisely, The Tactigon will function as a gesture controller for VLC program for Linux installed on a Raspberry Pi 3 board, and interfaced via Bluetooth.

We have used Python 2.7.13 to manage the reception of Bluetooth Low Energy commands and their reinstatement to VLC.

The used library is bluepy, which uses the BlueZ module.

In the April  article we explained how to install and configure correctly these two software on Raspbian Stretch; we refer to it for the in-depth studies of the case, reminding you anyway to enable the experimental functions to be able to take advantage of the Bluetooth Low Energy.

The library for binding the VLC APIs, vlc.py, is available on GitHub at the following link: https://github.com/oaubert/python-vlc.

This library gives us an interface with the VLC application, we can use through python script commands that we usually give via the graphic  interface. It is possible to proceed with the installation via PiPy with the command:

pip install python-vlc

Keeping the same firmware for The Tactigon, we only changed the behavior of the python code, which interprets commands by following this protocol:

• 0x5A5A: Play / Pause; play starts the playlist, or pause if it’s already on;

• 0x0088: Stop; stops the playback;

• 0x00BB: Previous, ie the command allows to go to the previous track /video;

• 0x1515: Next, which is the command that passes to the net track /video.

It is possible to extend this protocol and to obtain more control, but we leave you the exercise to implement these functions. For example, by using a different gesture you can advance of a second in the song, set the whole screen or the window, and so on.

THE TACTIGON - ARDUINO IDE

The firmware, meaning the sketch of The Tactigon, remains substantially unchanged compared to that of the last April article and deals with board position detection by deducting it from the accelerometer acquisitions. The code that performs this function is shown below.

if (!lastSupX && accData.x > threshold) {

lastSupX = true;

lastInfX = false;

lastInfY = false;

lastSupY = false;

buffData[0] = 0x00;

buffData[1] = 0xBB;

flash();

bleChar.update(buffData);

justSent = true;

}

This code, which is repeated for the two used x-and y axis, allows to detect the exact position of the board. The Boolean variable last-SupX is used, in fact, to prevent the same configuration is sent repeatedly and then to lighten the load on the Raspberry Pi 3; this detail, although not very relevant in the application here described, becomes important when The Tactigon must be engaged in more demanding applications from the point of view of the calculation.

Once this control is passed, inequality is evaluated:

accData.x > threshold.

In this inequality the threshold parameter is an acceleration threshold, expressed in m / s², which allows us to establish the sensitivity of The Tactigon to the "gesture" that we are going to perform; in other words, the threshold allows you to detect movements  over a certain speed as a gesture  and ignore the slow movements of the hands, leaving us free to move them without  giving incorrect commands.

Instead, accData.x is the acceleration component recorded by The Tactigon on the X axis.

We have the opportunity to take advantage of the 9-freedom movement IMU to capture the gesture we need. Using the library supplied in The Tactigon installation package you can easily get information on inclinations and accelerations, in order to program the board and customize it to the maximum.

The integrated Bluetooth Low Energy module will think about the communication of these gestures in the form of commands as we have developed in our firmware.

RASPBERRY PI 3 - PYTHON 2.7.13

The Raspberry Pi runs a python script that:

• performs the initialization of the VLC player, the playlist upload;

• connects to the The Tactigon card, by searching the mac address and the UUID of the service and the feature;

• sets the notification on the Bluetooth Low Energy, to be able to use one callback when receiving new data;

• the callback, when activated, executes the parse of the commands received via Bluetooth Low Energy and, thanks to the vlc.py binding library, checks the actions of the VLC player.

These two lines of code let you initialize our VLC player:

mediaListPath = “file:///home/pi/vlc-tactigon/media/ playlist.xspf”

instance = vlc.Instance([“--sub-source=marq”] +[mediaListPath])

In order to use it, however, we need to get a new instance of media_list_player and associate a playlist, that’s our Media_list:

list_player = vlc.Instance.media_list_player_ new(instance)

list_player.set_media_list(Media_list)

Now we can interact with the player using the simple list_player.play () commands, list_player.pause (), list_player.next (), list_player, previous (), list_player.stop () used in the code.

Pic. 1 shows the position of The Tactigon corresponding to the playback of the video in VLC e Pic. 2 is the one with which you obtain pause of movie playback.

Progress can be made, but it is necessary get the instance of the MediaPlayer object through the method list_player.get_media_player ().

Once this object is obtained, it is possible use its functions, such as:

• obtain information on the media being played;

• jump to a point in the movie;

• skip chapters;

• set the mute and the volume.

If desired, with appropriate customizations it is possible act on other functions, associating them with complex gestures, or simply faster than expected (playing on the sensitivity threshold of The Tactigon).

CONCLUSIONS

Now that we've got control of our playlist via gesture, we can think to further development of the interaction between Raspberry Pi and The Tactigon via Bluetooth; in fact, the concept on which the control of VLC is based can be applied and adapted to many others applications.