The Master Thesis in full: “TaGlove – a New Interface for Musical Expression (NIME)”

I’ve decided the best way of sharing the information on how I made the TaGlove would be to show you the full Master thesis I wrote in 2007. Please let me know if you plan to reference or base your NIME on my thesis.

TaGlove – a New Interface for Musical Expression (NIME)

Flow: Action leading to Sound

I’ve been trying to peel the skin of my system – and how it “flows” from Action leading to sound:

Action: Movement

Leading to

Gesture: Subcategories: Tilt (x,y,z – accelerometer), Long armmovement (velocity – accelerometer), Short armmovement (velocity – accelerometer), medium armmovement (velocity – accelerometer), bending fingers (Xi – Yi) (4), Using pressurepads (Xn – Yn) (2), movement in the Z plane (Xz – Yz) (accelerometer). All of the accelerometer velocity values are in relation to the law of gravity

Sensor: Subcategories: FSR 1 or 2, Bend 1-4, Accelerometer (X,Y,Z)

Vector / Size: Subcategories: Big, medium, small

Relation: What combination of vectors (sensors) are in use at any point

Address: Management of address labeling (vectornaming)

Matrix: Dynamic creation and subtraction of matrices when appropriate (Music is not Mapping, MnM from Ircam). Holds vector data.

Space: In relation to Euclidean Spaces. The spread of data(?)

Output: Fetching data in matrices

Sound

The Relation points back to vector, the addresses points back to relation and vector, the matrices points back to addresses, the relation and vectors. The space is an internal component of the matrices which is direct relation to fetching data – which in turn controls the relationship between the gesture / control data the digital sound processes.

I’ll add the visual flowchart on this later – its a bit easier to get accross my point! This is the chart I was inspired by – based on the Hidden Markov model

New Interfaces for Musical Expression, video examples

I’ve been browsing YouTube today for various videos relating to music technology and I found some interesting things – and I say interesting, not all of these are in my opinion succesful examples! Some of these examples I have written some critisism or praise about…
The tangible sequencer ring:

Sonic City, Wearable Design:

Eric Singer, Sonic Banana 2002:

Mapping, Sonic example:
“The Sonic Banana consists of four bend sensors in a row, running the length of a rubber tube, with a pushbutton switch at the end. Software in Max converts this data into musical functions, turning the Sonic Banana into a versatile performance instrument.

Arpeggiator: The bend sensors are used to play an arpeggiation algorithm in Max. The sensors control several parameters of the algorithm, including chord selection, tempo, note duration, tempo and volume.” (ericsinger.com 2006)

My main criticism of Singer’s Sonic Banana is that the sonic material he uses is too tonal. The experience for me is not then a new piece of music, it is just an arpiggiator constantly running, with a man taking himself very seriously, bending a tube to alter the (for me) very static sound image he creates. I think that there is nothing wrong with the way Singer performs, but for me it almost becomes comical that he looks so serious when the tonal material sounds so simplistic. Had he chosen the soundscape approach I think that the “banana” had been more successful / believable, as the mapping seems to work rather well. It looks responsive, and it offers some resistance to the performer, so that the force applied is proportional with the sonic result.

Recreation of Russolo’s Intonrumori:

Reactable, basic demo #1:

Petecube:

NeuroGlove:

Michael Waisvisz – the Hands 2003:

This example is for me personally one of the most sonically pleasing examples. That is because the material he uses is his own voice; it is clear what and where the material he uses comes from. It is also clear when he is adding new material and when he is just working on the previous input. He alters the sonic material with his hands, which seems like an obvious part of the performance. You can also clearly see where he gets his sounds from (his mouth) and where they are going (the microphone attached to the glove). What is not obvious is the gesture mapping relationship. In this case however, it works. What is clear is that many things are happening with less amount of action. That means that we have a one to many relationship between gesture and mapping.

Matthew Burtner – Metasax:

LEMUR Installtion, Beall Center for Art & Technology:

Kinetic Cube Interface:

JamFone:

Ed Snyder – Henri Matisse song 1982:

This is a pre-MIDI example of a one to one mapping. Snyder has taken components from an ordinary keyboard and re-attached them on his fingers. He starts off with a call-response, reciting a line of poem and then exposition of the main rhythmic / tonal “theme”, which can be said to be relatively simple – easily explained by the obvious simplicity of the layout of the keys on the fingers. What strikes me when I watch the video is why did he have to do this when he could easily have achieved the same sonic result (or perhaps a better) with using a conventional keyboard? I am sure Snyder could justify this at the time, but it is hard to find any evidence of this today when I looked for writing on the performance. For me this then stands as the prime example on how one to one mapping is, not perhaps pointless, but more annoying than sonically pleasing. The positive factor I can draw from this performance however is a very clear cognitive link between what Snyder is doing with his fingers and the sonic outcome, something I want to achieve as well.

The development of an intuitive gesture-based real time controller for synthesis

For my MA project, I’m going old school. NES oldschool to be more precise. I’m harvesting my bendsensors from the Mattel PowerGlove

I discovered, after paying $50 dollars on eBay (30 for the glove, 20 for postage) that I could have ordered a complete one (with the infrared sensor gear) from RetroZone for $70 + $6 postage, ready to plug straight in to my mac. But I think I’m going to stick with my original idea – to take out the bend sensors and then reattach them on another glove. Well, it’s not completely my idea, it has been done several times before, alas, this glove has apparently proven to be particulary suitable in previous experiments.

I also think that if I had all of the original features of the glove, it would be distractive, as I want to focus on making the glove intuitive I think it is good to use the less is more philosophy!

Update – 1/12/2006:

How to get the sensors from the glove:

Here’s how the glove looks initially

The inside
After the glove has been taken off, you can see the 4 sensors
Carefully peeling off the sensors

Taking off the front top to get the sensors loose without breaking them

Top of the glove, note the Piezo in the middle of the black thing.

I think I’m going to use this part for a different project

More to come later!

Note on mapping

In my Masters degree, I support my research with making a New Interface for Musical Expression (NIME).

This is currently on the writing and research stage, and I have recently been looking in to the MnM set of Externals for Max / MSP. I hope to make a simplified version of something similar. Even though I understand that there is a need to use maths to explain how things work, I do not think that this is necessary to be viewed by the end user.

MnM from IRCAM

I would be interested to see successful implementations of A.I. / Adaptive systems in Mapping. There does not seem to be too many examples around!