The Master Thesis in full: “TaGlove – a New Interface for Musical Expression (NIME)”

I’ve decided the best way of sharing the information on how I made the TaGlove would be to show you the full Master thesis I wrote in 2007. Please let me know if you plan to reference or base your NIME on my thesis.

TaGlove – a New Interface for Musical Expression (NIME)

About the TaGlove

I just wanted to post a quick link about what happened with the TaGlove as I have been getting some requests for how I made it lately.

Reading back on this blog I think I really didn’t do it justice Online. After all I spent one year developing it and the feedback was very positive.

I’ll start posting more about the project in the coming weeks so stay tuned for more information about the TaGlove!

I’ll start by posting a video of the finished TaGlove Performance Environment – this is a program / environment I made in Max / MSP to go with the glove which was connected wirelessly by Bluetooth. It was made to show some of the scope the user has when using the glove.

Larry Cotton’s Screechophone

I think this thing is fantastic!

Larry Cotton’s Screechophone Make contributor Larry Cotton says:

I built this bizarre musical instrument–based loosely on the playing-card-in-the-bicycle-spokes principle–a while back, shot a quick video, then cannibalized it for parts. As you’ll see, the bicycle spokes were scaled down to a small white plastic gear. The card remained.
I called it a screechophone for reasons which will become obvious as you watch (and listen) to the video.

The whole thing was synced to accompanying music from a mini-CD through an ancient Commodore 64 computer, programmed in BASIC. A transparent disc printed only with black arcs spun with the CD and was read by a photocell.

Flow: Action leading to Sound

I’ve been trying to peel the skin of my system – and how it “flows” from Action leading to sound:

Action: Movement

Leading to

Gesture: Subcategories: Tilt (x,y,z – accelerometer), Long armmovement (velocity – accelerometer), Short armmovement (velocity – accelerometer), medium armmovement (velocity – accelerometer), bending fingers (Xi – Yi) (4), Using pressurepads (Xn – Yn) (2), movement in the Z plane (Xz – Yz) (accelerometer). All of the accelerometer velocity values are in relation to the law of gravity

Sensor: Subcategories: FSR 1 or 2, Bend 1-4, Accelerometer (X,Y,Z)

Vector / Size: Subcategories: Big, medium, small

Relation: What combination of vectors (sensors) are in use at any point

Address: Management of address labeling (vectornaming)

Matrix: Dynamic creation and subtraction of matrices when appropriate (Music is not Mapping, MnM from Ircam). Holds vector data.

Space: In relation to Euclidean Spaces. The spread of data(?)

Output: Fetching data in matrices

Sound

The Relation points back to vector, the addresses points back to relation and vector, the matrices points back to addresses, the relation and vectors. The space is an internal component of the matrices which is direct relation to fetching data – which in turn controls the relationship between the gesture / control data the digital sound processes.

I’ll add the visual flowchart on this later – its a bit easier to get accross my point! This is the chart I was inspired by – based on the Hidden Markov model

The making of a data-glove, part III

This morning I managed to get the FSR sensor working!

It is not inside of the glove, because I have also been working on building a Plasti-Dip layer around the bend sensors on the actual glove so that it is protected when I start using it. Especially around the connector area is places I would say are critical, so I’ve been building a supportive layer underneath them to ‘lift’ them up from the actual glove itself, so they will not get pushed in to the palm of the glove when opening and closing the palm.

What I did was; found a connection on my Interface board, set up two leads, and then soldered the leads from the ready FSR sensor on to it, and then I opened up Max/ MSP – the program I will be using to make the program for interfacing and sound generation. And I got it to work! This is a major ‘breakthrough’, although in theory I always knew (and hoped) it would function As illustrated in the video, the sensor is quite sensitive, so I think I have to have some sort of ‘rubbish cutter’ on the low pressure areas.

Testing the FSR sensor…

A bit later in the afternoon I finished the cables on the glove itself and also, the FSR sensors are now in place inside the fingertips of the glove…

The making of a data-glove part II

I went to Maplin (The electronics store) today, to buy some more stuff for the glove, since all the parts pretty much have arrived now – and I’m pretty keen to get the glove up and running. I bought a soldering iron, wires, breadboard and some other bits and bobs. So now it’s assembly time! The Accelerometer (which measures the velocity of movement and tilt in three axis, x y and z) arrived yesterday. It is tiny… 4×4mm! I have to make my own pcb board, making an own circuit and then solder it on to the breadboard.  (correction 12.02)

Some pictures and explainations:

This is the accelerometer -

The FSR sensor, which measures pressure – with the cables soldered on. These need to be shielded before they go inside of the glove. that is because the sensor is extremely fragile around the connecting area. I’ve got some PlastiDip which will take care of that. PlastiDip is a liquid that dries to a flexible plastic-like material and is extremely handy for this type of use. Picture II here is the sensor after it has been coated:

This is where the sensor above is going – in to the fingertip of the glove. As you can see I have taken out the stiches in the seam of the glove, so that I can sow it toghether again when the sensor is in place!

This is the palm of the glove at the moment….!

Check back for more progression reports the coming days!

New Interfaces for Musical Expression, video examples

I’ve been browsing YouTube today for various videos relating to music technology and I found some interesting things – and I say interesting, not all of these are in my opinion succesful examples! Some of these examples I have written some critisism or praise about…
The tangible sequencer ring:

Sonic City, Wearable Design:

Eric Singer, Sonic Banana 2002:

Mapping, Sonic example:
“The Sonic Banana consists of four bend sensors in a row, running the length of a rubber tube, with a pushbutton switch at the end. Software in Max converts this data into musical functions, turning the Sonic Banana into a versatile performance instrument.

Arpeggiator: The bend sensors are used to play an arpeggiation algorithm in Max. The sensors control several parameters of the algorithm, including chord selection, tempo, note duration, tempo and volume.” (ericsinger.com 2006)

My main criticism of Singer’s Sonic Banana is that the sonic material he uses is too tonal. The experience for me is not then a new piece of music, it is just an arpiggiator constantly running, with a man taking himself very seriously, bending a tube to alter the (for me) very static sound image he creates. I think that there is nothing wrong with the way Singer performs, but for me it almost becomes comical that he looks so serious when the tonal material sounds so simplistic. Had he chosen the soundscape approach I think that the “banana” had been more successful / believable, as the mapping seems to work rather well. It looks responsive, and it offers some resistance to the performer, so that the force applied is proportional with the sonic result.

Recreation of Russolo’s Intonrumori:

Reactable, basic demo #1:

Petecube:

NeuroGlove:

Michael Waisvisz – the Hands 2003:

This example is for me personally one of the most sonically pleasing examples. That is because the material he uses is his own voice; it is clear what and where the material he uses comes from. It is also clear when he is adding new material and when he is just working on the previous input. He alters the sonic material with his hands, which seems like an obvious part of the performance. You can also clearly see where he gets his sounds from (his mouth) and where they are going (the microphone attached to the glove). What is not obvious is the gesture mapping relationship. In this case however, it works. What is clear is that many things are happening with less amount of action. That means that we have a one to many relationship between gesture and mapping.

Matthew Burtner – Metasax:

LEMUR Installtion, Beall Center for Art & Technology:

Kinetic Cube Interface:

JamFone:

Ed Snyder – Henri Matisse song 1982:

This is a pre-MIDI example of a one to one mapping. Snyder has taken components from an ordinary keyboard and re-attached them on his fingers. He starts off with a call-response, reciting a line of poem and then exposition of the main rhythmic / tonal “theme”, which can be said to be relatively simple – easily explained by the obvious simplicity of the layout of the keys on the fingers. What strikes me when I watch the video is why did he have to do this when he could easily have achieved the same sonic result (or perhaps a better) with using a conventional keyboard? I am sure Snyder could justify this at the time, but it is hard to find any evidence of this today when I looked for writing on the performance. For me this then stands as the prime example on how one to one mapping is, not perhaps pointless, but more annoying than sonically pleasing. The positive factor I can draw from this performance however is a very clear cognitive link between what Snyder is doing with his fingers and the sonic outcome, something I want to achieve as well.

Kaada Wrongroom

I just wanted to share this blog link. I just think that Kaada’s music is beautiful – and that he really does wonderful things with crappy old effects. And I really like his approach to DIY electronics as well, check his blog out:

Kaada Wrongroom 

The development of an intuitive gesture-based real time controller for synthesis

For my MA project, I’m going old school. NES oldschool to be more precise. I’m harvesting my bendsensors from the Mattel PowerGlove

I discovered, after paying $50 dollars on eBay (30 for the glove, 20 for postage) that I could have ordered a complete one (with the infrared sensor gear) from RetroZone for $70 + $6 postage, ready to plug straight in to my mac. But I think I’m going to stick with my original idea – to take out the bend sensors and then reattach them on another glove. Well, it’s not completely my idea, it has been done several times before, alas, this glove has apparently proven to be particulary suitable in previous experiments.

I also think that if I had all of the original features of the glove, it would be distractive, as I want to focus on making the glove intuitive I think it is good to use the less is more philosophy!

Update – 1/12/2006:

How to get the sensors from the glove:

Here’s how the glove looks initially

The inside
After the glove has been taken off, you can see the 4 sensors
Carefully peeling off the sensors

Taking off the front top to get the sensors loose without breaking them

Top of the glove, note the Piezo in the middle of the black thing.

I think I’m going to use this part for a different project

More to come later!

A new Interface don’t have to be all good…

Wow, that’s exactly what I don’t want to go and see at a concert. Just an excellent proof that not all new Interfaces are succesful!