Though computer music is at the edge of the avant-garde today, its roots go back to 1957, when Mathews wrote the first version of “Music,” a program that allowed an IBM 704 mainframe computer to play a 17-second composition. He quickly realized, as he put it in a 1963 article in Science, “There are no theoretical limits to the performance of the computer as a source of musical sounds.”

Rest in peace, Max.

UPDATE: I haven’t updated this blog in a while, and I realized after posting this that my previous post was about the 2010 Modulations concert. Max Mathews played at Modulations too, and that was the last time I saw him.

It’s called the Multi-Touch Light Table and it was created by East Bay-based artist/fidget-house DJ Gregory Kaufman. The video is beautifully put together, highlighting the importance of presentation when documenting new ideas.

I really like some of the interaction ideas presented in the video. Others, I’m not so sure about. But that’s all right: the significance of the MTLT is that it’s the first surface-based DJ tool that systematically accounts for the needs of an expert user.

Interestingly, even though it looks futuristic and expensive to us, interfaces like this will eventually be the most accessible artistic tools. Once multi-touch surface are ubiquitous, the easiest way to gain some capability will be to use inexpensive or open-source software. The physical interfaces created for DJing, such as Technics 1200s, are prosthetic objects (as are musical instruments), and will remain more expensive because mechanical contraptions will always be. Now, that isn’t to say that in the future our interfaces won’t evolve to become digital, networked, and multi-touch sensitive, or even that their physicality will be replaced with a digital haptic display. But one of the initial draws of the MTLT—the fact of its perfectly flat, clean interactive surface—seems exotic to us right now, and in the near future it will be default.

Wilson is a neurologist who works with musicians who have been afflicted with debilitating chronic hand pain. As he writes about his many interviews, a few themes emerge that are especially relevant to our interests here.

Incorporation
Incorporation is the phenomenon of internalizing external objects; it’s the feeling that we all get that a tool has become one with our body.

The idea of “becoming one” with a backhoe is no more exotic than the idea of a rider becoming one with a horse or a carpenter becoming one with a hammer, and this phenomenon itself may take its origin from countless monkeys who spent countless eons becoming one with tree branches. The mystical feel comes from the combination of a good mechanical marriage and something in the nervous system that can make an object external to the body feel as if it had sprouted from the hand, foot, or (rarely) some other place on the body where your skin makes contact with it…

The contexts in which this bonding occurs are so varied that there is no single word that adequately conveys either the process or the many variants of its final form. One term that might qualify is “incorporation”—bringing something into, or making it part of, the body. It is a commonplace experience, familiar to anyone who has ever played a musical instrument, eaten with a fork or chopsticks, ridden a bicycle, or driven a car. (p. 63)

Projection
Projection is the ability to use the hand as a bridge for projecting consciousness from one location to another. (Wilson did not use the word “projection” in the book.) In some ways, projection can be seen as the opposite of incorporation. Master puppeteer Anton Bachleitner:

It takes at least three years of work to say you are a puppeteer. The most difficult job technically is to be able to feel the foot contact the floor as it actually happens. The only way to make the puppet look as though it is actually walking is by feeling what is happening through your hands. The other thing which I think you cannot really train for, but only can discover with very long practice and experience, is a change in your own vision.

The best puppeteer after some years will actually see what is happening on the stage as if he himself was located in the head of the puppet, looking out through the puppet’s eyes—he must learn to be in the puppet. This is true not only in the traditional actor’s sense, but in an unusual perceptual sense. The puppeteer stands two meters above the puppet and must be able to see what is on the stage and to move from the puppet’s perspective. Moving is a special problem because of this distance, because the puppet does not move at the same time your hand does. Also, there can be several puppets on the stage at the same time, and to appear realistic they must react to each other as they would in real life. So again the puppeteer must himself be mentally on the stage and able to react as a stage actor would react. This is something I cannot explain, but it is very imprortant for a puppeteer to be able to do this. (pp. 92–93)

Serge Percelly, professional juggler:

[An act is successful] not because you put something in the act that’s really difficult, but because you put something in the act in exactly the right way—in a way that makes it more interesting, not only for me but for the audience as well. I’m just trying somehow to do the act that I would have loved to see. (p. 111)

Skill
Wilson is a musician and a doctor to musicians, so he has special insight into the neurology of musical skill—which he recognizes as special case of manual skill that involves gesture, communication, and emotion.

Musical skill provides the clearest example and the cleanest proof of the existence of a whole class of self-defined, personally distinctive motor skills with an extended training and experience base, strong ties to the individual’s emotional and cognitive development, strong communicative intent, and very high performance standards. Musical skill, in other words, is more than simply praxis, ordinary manual dexterity, or expertness in pantomime. (p. 207)

The upper-limb (or “output”) requirements for an instrumentalist are not unique either; they depend upon the possession of arms, fingers, and thumbs, specific but idiosyncratic limits on the rage of motion at the shoulder, elbow, wrist, hand, and finger joints, variable abilities to achieve repetition rates and forces with specific digital configurations in sequence at multiple contact points on a sound-making device, and so on. Peculiarities in the physical configuration and movement capabisities of the musician’s limbs can be an advantage or disadvantage but are reflected in (and in adverse cases can be overcome through) instrument design: How wide can you make the neck of a guitar? How far apart should the keys be on a piano? Where should the keys be placed on a flute—in general? and for Susan and Peter? (p. 225)

Awareness
Touch experience can be a gateway to awareness, which can in turn heal both the mind and the body. Moshe Feldenkrais invented a form of physical therapy that focuses on stimulating an awareness of touch and movement sensations in order to relieve pain.

Most people slouch, tilt, shuffle, twist, stumble, and hobble along. Why should that be? Was there something wrong with their brains? After considering what dancers and musicians go through to improve control of their movements, [Feldenkrais] guessed that people must either be ignorant of the possibilities or refuse to act on them. So they just heave themselves around, lurching from parking place to office to parking place, utterly oblivious to what they are doing, to their appearance, and even to the sensations that arise from bodily movement. He suspected that people just lose contact with their own bodies. If and when they do notice, it is because they are so stiff that they can’t get out of bed or are in so much pain that they can barely get out of a chair. Then they start noticing…

What [Feldenkrais] was doing did not seem complicated. The goal of the guided movements was not to learn how to move, in the sense of learning to do a new dance step. The goal was not to stretch ligaments or muscles. It was not to increase strength. The goal, as he saw it, was to get the messages moving again and to encourage the brain to pay attention to them. (p. 244)

And his student, Anat Baniel, on the deep psychological roots of movement disorders:

I think working with children has given me this idea, which isn’t often discussed in medicine: a lot of disease—medical disease and emotional “dis-ease”—is an outcome of a lack of full development. It’s not something we can get to just by removing a psychological block…

Of course there are problems due to traumatic events in childhood, or disease—you name it. Feldenkrais said that ideal development would happen if the child was not opposed by a force too big for its strength. When you say to a small child, “Don’t touch that, it’s dangerous!” you create such a forceful inhibition that you actually distort the child’s movement, and growth, in a certain way.

Feldenkrais taught us to look for what isn’t there. Why doesn’t movement happen in the way that it should, given gravity, given the structure of the body, given the brain? For all of us there is a sort of sphere, or range, of movement that should be possible. Some people get only five or ten percent of that sphere, and you have to ask, “What explains the difference between those who get very little and those who get a lot?” Feldenkrais said that the difference is that in the process of development, the body encountered forces that were disproportionate to what the nervous system could absorb without becoming overinhibited—or overly excited, which is a manifestation of the same thing. (p. 252)

Feldenkrais’s approach is fascinating, but there is scant discussion in Wilson’s book about the role of the therapist’s hand in this process. After all, this kind of therapy is wholly reliant on an accidental discovery: that the patient can be made aware of her own body through an external, expert hand radiating pressure and heat. How is this possible? The topic isn’t explored.

There are many, many wonderful things to learn from this book for anyone with an interest in biology, art, music, history, or sports. You can find Frank Wilson on the web at Handoc.com

(via Engadget)

Donald Norman might break it down like this:

• Pull out the “tuning knob,” which looks like a standard gain control and doesn’t look like it’s supposed to be pulled. In fact, when the knobs on normal guitars are pulled in this way they come off, so most guitarists have learned to avoid pulling on knobs.
• Make sure you’re not touching the neck or the tuning keys! You know, those things that look like they were made for human fingers to touch and that are an essential part of the manual interface to all other string instruments? Forget that, and instead keep this knowledge in your head: hands off.
• Strum the guitar several times. While you do this, crouch over your guitar so you can see the LEDs on the front of the tuning knob, which faces the audience.
• After a few strums, the LEDs change color. Now you’re “ready to rock.”

Meanwhile, Moog has extended the creative capabilities of electric guitars with innovative, elegant technology that may actually deserve the title “world’s first,” though Moog is confident enough in their product that they don’t bother with that kind of marketing nonsense. By feeding some of the output signal back into the strings, the instrument allows precise control of the envelope of the sound. “Infinite sustain” as well as banjo-like damping are both made possible. Powerful stuff.

Ray Kurzweil’s The Singularity Is Near is worth reading. It’s a great introduction to the “emerging technologies” meme. If you’ve already begun wandering down the diabolical path of contemplating the obsolescence of biology and the societal transformation that will result, Singularity doesn’t reveal much. However it’s an easy read and a cultural milestone, so pushing through it is still probably a good idea.

That said, I had a few gripes with the book. Firstly, the author’s tone doesn’t sound like the cool scientific theorist he claims to be. Virtually every discussion of progress in biotechnology lends particular weight to the synthesizing of artificial pancreas cells, which Kurzweil readily acknowledges he needs to reverse the course of his diabetes. He repeats over and over that people (like him) who take full advantage of today’s knowledge about longevity will live to see the Singularity (and by corollary, live indefinitely), and that those who do not will needlessly pay the price of annihilation. I don’t remember even one acknowledgment that his own efforts may fail (or be misguided!) despite his dogged adherence to health guidelines derived from cutting edge research. This doesn’t seem to reflect the attitude of a detached intellectual.

Maybe I’m being too harsh. It’s pretty clear that Kurzweil tries to make his subject singularly exciting, and the book seems to be written at least partially for the journalists who might like to capitalize on the author’s theory by writing articles to the effect of, “This man says you’ll socialize with robots one day. Isn’t that insane plus totally interesting?”

Still, I found his discussions of sociology and morality hugely lacking. Kurzweil falls far short in his brief examination of the coming collisions between the Singularity and virtually every modern social structure. He might counter that his goal for the book was to cultivate optimism and open minds to the grand potential of the coming transcendence. But with its foreboding title, one would expect the book to not only open minds but also put them on guard against perversions of his grand vision. And for his slight allowance that catastrophe may wipe us all out before (or after) the Singularity, he does not explicitly acknowledge that our place in the universe is so absurdly minute that, for instance, even after our entire solar system becomes a giant computer the collision of two unknown (or even known) neutron stars halfway across our galaxy could fry it all, and human technology might be powerless to stop it.

I extracted some passages that were relevant to haptics:

The current disadvantages of Web-based commerce (for example, limitations in the ability to directly interact with products and the frequent frustrations of interacting with inflexible menus and forms instead of human personnel) will gradually dissolve as the trends move robustly in favor of the electronic world. By the end of this decade, computers will disappear as distinct physical objects, with displays built in our eyeglasses and electronics woven in our clothing, providing full-immersion visual virtual reality. Thus, “going to a Web site” will mean entering a virtual-reality environment—at least for the visual and auditory senses—where we can directly interact with products and people, both real and simulated… Haptic (tactile) interfaces will enable us to touch products and people. It is difficult to identify any lasting advantage of the old brick-and-mortar world that will not ultimately be overcome by the rich interactive interfaces that are soon to come. (104–105)

I don’t like “haptic (tactile)” (they’re not equivalent), but the point is clear. There were also some references to musical interaction research:

Edward Taub at the University of Alabama studied the region of the cortex responsible for evaluating the tactile input from the fingers. Comparing non-musicians to experienced players of stringed instruments, he found no difference in the brain regions devoted to the fingers of the right hand but a huge difference for the fingers of the left hand. (174)

Dr. Alvaro Pascual-Leone at Harvard University scanned the brains of volunteers before and after they practiced a simple piano exercise. The brain motor cortex of the volunteers changed as a direct result of their practice. He then had a second group just think about doing the piano exercise but without actually moving any muscles. This produced an equally pronounced change in the motor-cortex network. (175)

Finally, I think the use of the word “mastering” here brings up an important point:

Machines have exacting memories. Contemporary computers can master billions of facts accurately, a capability that its doubling every year. (261)

Haptic metaphors like this are used generously throughout the book, no doubt to underscore Kurzweil’s intense belief that humanity is ingrained in nonbiological intelligent technology. However, as a terminology freak I would argue that computers memorize facts, but don’t master them. Mastery comes with an ability to manipulate freely. For mastery, you need a body.

The McGill University Digital Orchestra is putting on their first performance on March 5th at 7:30 p.m. in Montreal’s Pollack Hall. The lovely and talented Xenia Pestova will be playing the Rulers, an instrument I invented. The piece she will be playing, which I haven’t heard yet, was composed by D. Andrew Stewart and is the first music to be written for the instrument. There will be a live webcast of the show. To watch it, launch the QuickTime application (free download) a few minutes before the start of the show, select “Open URL in New Player” from the File menu, and enter: rtsp://132.206.142.8/pollackhall.sdp

What I find especially interesting is how the words “world’s first robot guitar” are tossed around in this press release. First of all, it seems as if the word “robot” is being used vaguely to refer to the presence of a servo system. If this guitar is robotic, then so is my laptop for its ability to read and eject optical media. I think we’re going to see more of this, similar to the way “net” was overused in the nineties. We are entering a robo-sheik era where any product that can possibly justify doing so will be incorporating the word “robot” into its name.

As for the “world’s first” claim, someone should tell Gibson about TransPerformance, the company that has already been selling automatic tuner retrofits since 2005, as well as the dozens of other music technology projects that are based on guitar interaction and involve motors. It’s old, but anyone who hasn’t yet seen the League of Musical Urban Robots (LEMUR) video of the LEMUR Guitar Bot should check it out:

It’s easier for me to accept calling the LEMUR Guitar Bot a “robot” than the Gibson self-tuner. What do you think?

I’m excited to announce that, as I write this post, a musical instrument that I designed and built is being shown at WIRED magazine’s NextFest conference and trade show in Los Angeles. The first version of the Rulers came out of a physical interaction design workshop hosted by Stanford University’s CCRMA music technology program. Later I gave it a full version upgrade and it was purchased by the Digital Orchestra at McGill University’s Schulich School of Music. Since the Digital Orchestra is at NextFest, so are the Rulers. Here’s the press release for the exhibition.

Update: I’m having issues with the video… apparently Sony’s MPG isn’t being recognized by any video playback application I’ve tried yet except for the Finder preview. YouTube and Motionbox both don’t recognize them either. I hope I will solve the problem soon.