Back in 2014, British artist Imogen Heap collaborated with the University of the West of England to develop the Mi.Mu musical gloves, which transform gestures into music.
A special system tracks the position of the hands and fingers and translates this data into signals for controlling various instruments. At first, only Imogen herself used such gloves, but soon the device gained popularity among various performers.
A set of software and the gloves themselves cost a lot – 2,500 British pounds. But now software called Glover can be purchased separately for just £ 119. True, in any case, it will need some kind of interface that can capture motion, but the platform supports the development of third-party companies – Leap Motion optical modules and micro: bit microcontrollers.
There is even a free program called Gliss for smartphones, designed specifically to track movements while working with Glover – so far it is only available on the iPhone, but will soon appear on Android.
In Glover, you can assign a specific motion to a specific action in the connected MIDI and OSC controllers. Moreover, there are no restrictions on gestures in the application – they are not taken from strictly configured libraries. Users can create their own “music sign language” using absolutely any movement that the interface recognizes. A machine learning algorithm is used for correct recognition.
Glover has a free 2 week trial period. The software runs on Windows and MacOS.