Image Image Image Image Image

Presented at CHI 2012, Touché is a capacitive system for pervasive, continuous sensing. Among other amazing capabilities, it can accurately sense gestures a user makes on his own body. “It is conceivable that one day mobile devices could have no screens or buttons, and rely exclusively on the body as the input surface.” Touché.

Noticing that many of the same sensors, silicon, and batteries used in smartphones are being used to create smarter artificial limbs, Fast Company draws the conclusion that the market for smartphones is driving technology development useful for bionics. While interesting enough, the article doesn’t continue to the next logical and far more interesting possibility: that phones themselves are becoming parts of our bodies. To what extent are smartphones already bionic organs, and how could we tell if they were? I’m actively researching design in this area – stay tuned for more about the body-incorporated phone.

A study provides evidence that talking into a person’s right ear can affect behavior more effectively than talking into the left.

One of the best known asymmetries in humans is the right ear dominance for listening to verbal stimuli, which is believed to reflect the brain’s left hemisphere superiority for processing verbal information.

I heavily prefer my left ear for phone calls. So much so that I have trouble understanding people on the phone when I use my right ear. Should I be concerned that my brain seems to be inverted?

Read on and it becomes clear that going beyond perceptual psychology, the scientists are terrifically shrewd:

Tommasi and Marzoli’s three studies specifically observed ear preference during social interactions in noisy night club environments. In the first study, 286 clubbers were observed while they were talking, with loud music in the background. In total, 72 percent of interactions occurred on the right side of the listener. These results are consistent with the right ear preference found in both laboratory studies and questionnaires and they demonstrate that the side bias is spontaneously displayed outside the laboratory.

In the second study, the researchers approached 160 clubbers and mumbled an inaudible, meaningless utterance and waited for the subjects to turn their head and offer either their left of their right ear. They then asked them for a cigarette. Overall, 58 percent offered their right ear for listening and 42 percent their left. Only women showed a consistent right-ear preference. In this study, there was no link between the number of cigarettes obtained and the ear receiving the request.

In the third study, the researchers intentionally addressed 176 clubbers in either their right or their left ear when asking for a cigarette. They obtained significantly more cigarettes when they spoke to the clubbers’ right ear compared with their left.

I’m picturing the scientists using their grant money to pay cover at dance clubs and try to obtain as many cigarettes as possible – carefully collecting, then smoking, their data – with the added bonus that their experiment happens to require striking up conversation with clubbers of the opposite sex who are dancing alone. One assumes that, if the test subject happened to be attractive, once the cigarette was obtained (or not) the subject was invited out onto the terrace so the scientist could explain the experiment and his interesting line of work. Well played!

Another MRI study, this time investigating how we learn parts of speech:

The test consisted of working out the meaning of a new term based on the context provided in two sentences. For example, in the phrase “The girl got a jat for Christmas” and “The best man was so nervous he forgot the jat,” the noun jat means “ring.” Similarly, with “The student is nising noodles for breakfast” and “The man nised a delicious meal for her” the hidden verb is “cook.”

“This task simulates, at an experimental level, how we acquire part of our vocabulary over the course of our lives, by discovering the meaning of new words in written contexts,” explains Rodríguez-Fornells. “This kind of vocabulary acquisition based on verbal contexts is one of the most important mechanisms for learning new words during childhood and later as adults, because we are constantly learning new terms.”

The participants had to learn 80 new nouns and 80 new verbs. By doing this, the brain imaging showed that new nouns primarily activate the left fusiform gyrus (the underside of the temporal lobe associated with visual and object processing), while the new verbs activated part of the left posterior medial temporal gyrus (associated with semantic and conceptual aspects) and the left inferior frontal gyrus (involved in processing grammar).

This last bit was unexpected, at first. I would have guessed that verbs would be learned in regions of the brain associated with motor action. But according to this study, verbs seem to be learned only as grammatical concepts. In other words, knowledge of what it means to run is quite different than knowing how to run. Which makes sense if verb meaning is accessed by representational memory rather than declarative memory.

Researchers at the University of Tampere in Finland found that,

Interfaces that vibrate soon after we click a virtual button (on the order of tens of milliseconds) and whose vibrations have short durations are preferred. This combination simulates a button with a “light touch” – one that depresses right after we touch it and offers little resistance.

Users also liked virtual buttons that vibrated after a longer delay and then for a longer subsequent duration. These buttons behaved like ones that require more force to depress.

This is very interesting. When we think of multimodal feedback needing to make cognitive sense, synchronization first comes to mind. But there are many more synesthesias in our experience that can only be uncovered through careful reflection. To make an interface feel real, we must first examine reality.

Researchers at the Army Research Office developed a vibrating belt with eight mini actuators — “tactors” — that signify all the cardinal directions. The belt is hooked up to a GPS navigation system, a digital compass and an accelerometer, so the system knows which way a soldier is headed even if he’s lying on his side or on his back.

The tactors vibrate at 250 hertz, which equates to a gentle nudge around the middle. Researchers developed a sort of tactile morse code to signify each direction, helping a soldier determine which way to go, New Scientist explains. A soldier moving in the right direction will feel the proper pattern across the front of his torso. A buzz from the front, side and back tactors means “halt,” a pulsating movement from back to front means “move out,” and so on.

A t-shirt design by Derek Eads.

Recent research reveals some fun facts about aural-tactile synesthesia:

Both hearing and touch, the scientists pointed out, rely on nerves set atwitter by vibration. A cell phone set to vibrate can be sensed by the skin of the hand, and the phone’s ring tone generates sound waves — vibrations of air — that move the eardrum…

A vibration that has a higher or lower frequency than a sound… tends to skew pitch perception up or down. Sounds can also bias whether a vibration is perceived.

The ability of skin and ears to confuse each other also extends to volume… A car radio may sound louder to a driver than his passengers because of the shaking of the steering wheel. “As you make a vibration more intense, what people hear seems louder,” says Yau. Sound, on the other hand, doesn’t seem to change how intense vibrations feel.

Max Mathews, electronic music pioneer, has died.

Though computer music is at the edge of the avant-garde today, its roots go back to 1957, when Mathews wrote the first version of “Music,” a program that allowed an IBM 704 mainframe computer to play a 17-second composition. He quickly realized, as he put it in a 1963 article in Science, “There are no theoretical limits to the performance of the computer as a source of musical sounds.”

Rest in peace, Max.

UPDATE: I haven’t updated this blog in a while, and I realized after posting this that my previous post was about the 2010 Modulations concert. Max Mathews played at Modulations too, and that was the last time I saw him.

I finally got around to recording and mastering the set I played at the CCRMA Modulations show a few months back. Though I’ve been a drum and bass fan for many years, this year’s Modulations was the first time I’d mixed it for others. Hope you like it!

Modulations 2010
Drum & Bass | 40:00 | May 2010

Download (mp3, 82.7 MB)


1. Excision — System Check
2. Randomer — Synth Geek
3. Noisia — Deception
4. Bassnectar — Teleport Massive (Bassnectar Remix)
5. Moving Fusion, Shimon, Ant Miles — Underbelly
6. Brookes Brothers — Crackdown
7. The Ian Carey Project — Get Shaky (Matrix & Futurebound’s Nip & Tuck Mix)
8. Netsky — Eyes Closed
9. Camo & Krooked — Time Is Ticking Away feat. Shaz Sparks

Over the last few days this video has been so much bombshell to many of my music-prone friends.

It’s called the Multi-Touch Light Table and it was created by East Bay-based artist/fidget-house DJ Gregory Kaufman. The video is beautifully put together, highlighting the importance of presentation when documenting new ideas.

I really like some of the interaction ideas presented in the video. Others, I’m not so sure about. But that’s all right: the significance of the MTLT is that it’s the first surface-based DJ tool that systematically accounts for the needs of an expert user.

Interestingly, even though it looks futuristic and expensive to us, interfaces like this will eventually be the most accessible artistic tools. Once multi-touch surface are ubiquitous, the easiest way to gain some capability will be to use inexpensive or open-source software. The physical interfaces created for DJing, such as Technics 1200s, are prosthetic objects (as are musical instruments), and will remain more expensive because mechanical contraptions will always be. Now, that isn’t to say that in the future our interfaces won’t evolve to become digital, networked, and multi-touch sensitive, or even that their physicality will be replaced with a digital haptic display. But one of the initial draws of the MTLT—the fact of its perfectly flat, clean interactive surface—seems exotic to us right now, and in the near future it will be default.

Check out this flexible interface called impress. Flexible displays just look so organic and, well impressive. One day these kinds of surface materials will become viable displays and they’ll mark a milestone in touch computing.

It’s natural to stop dancing between songs. The beat changes, the sub-rhythms reorient themselves, a new hook is presented and a new statement is made. But stopping dancing between songs is undesirable. We wish to lose ourselves in as many consecutive moments as possible. The art of mixing music is to fulfill our desire to dance along to continuous excellent music, uninterrupted for many minutes (or, in the best case, many hours) at a time. (Even if we don’t explicitly move our bodies to the music, when we listen our minds are dancing; the same rules apply.)

I don’t remember what prompted me to take that note, but it was probably not that the mixing was especially smooth.



A tomato hailing from Capay, California.

LHCSound is a site where you can listen to sonified data from the Large Hadron Collider. Some thoughts:

  • That’s one untidy heap of a website. Is this how it feels to be inside the mind of a brilliant physicist?
  • The name “LHCSound” refers to “Csound”, a programming language for audio synthesis and music composition. But how many of their readers will make the connection?
  • If they are expecting their readers to know what Csound is, then their explanation of the process they used for sonification falls way short. I want to know the details of how they mapped their data to synthesis parameters.
  • What great sampling material this will make. I wonder how long before we hear electronic music incorporating these sounds.

The Immersive Pinball demo I created for Fortune’s Brainstorm:Tech conference was featured in a BBC special on haptics.

I keep watching the HTC Sense unveiling video from Mobile World Congress 2010. The content is pretty cool, but I’m more fascinated by the presentation itself. Chief marketing officer John Wang gives a simply electrifying performance. It almost feels like an Apple keynote.

The iFeel_IM haptic interface has been making rounds on the internet lately. I tried it at CHI 2010 a few weeks ago and liked it a lot. Affective (emotional haptic) interfaces are full of potential. IFeel_IM mashes together three separate innovations:

  • Touch feedback in several different places on the body: spine, tummy, waist.
  • Touch effects that are generated from emotional language.
  • Synchronization to visuals from Second Life

All are very interesting. The spine haptics seemed a stretch to me, but the butterfly-in-the-tummy was surprisingly effective. The hug was good, but a bit sterile. Hug interfaces need nuance to bring them to the next level of realism.

The fact that the feedback is generated from the emotional language of another person seemed to be one of the major challenges—the software is built to extract emotionally-charged sentences using linguistic models. For example, if someone writes “I love you” to you, your the haptic device on your tummy will react by creating a butterflies-like sensation. As an enaction devotee I would rather actuate a hug with a hug sensor. Something about the translation of words to haptics is difficult for me to accept. But it could certainly be a lot of fun in some scenarios!

I’ve re-recorded my techno mix Awake with significantly higher sound quality. So if you downloaded a copy be sure to replace it with the new file!

Awake

Awake
Techno | 46:01 | October 2009

Download (mp3, 92 MB)


1. District One (a.k.a. Bart Skils & Anton Pieete) — Dubcrystal
2. Saeed Younan — Kumbalha (Sergio Fernandez Remix)
3. Pete Grove — I Don’t Buy It
4. DBN — Asteroidz featuring Madita (D-Unity Remix)
5. Wehbba & Ryo Peres — El Masnou
6. Broombeck — The Clapper
7. Luca & Paul — Dinamicro (Karotte by Gregor Tresher Remix)
8. Martin Worner — Full Tilt
9. Joris Voorn — The Deep

I recently started using Eclipse on OS X and it was so unresponsive, it was almost unusable. Switching tabs was slow, switching perspectives was hella slow. I searched around the web for a solid hour for how to make it faster and finally found the solution. Maybe someone can use it.

My machine is running OS X 10.5, and I have 2 GB of RAM. (This is important because the solution requires messing with how Eclipse handles memory. If you have a different amount of RAM, these numbers may not work and you’ll need to fiddle with them.)

  • Save your work and quit Eclipse.
  • Open the Eclipse application package by right-clicking (or Control-clicking) on Eclipse.app and select “Show Package Contents.”
  • Navigate to Contents→MacOS→, and open “eclipse.ini” in your favorite text editor.
  • Edit the line that starts with -”XX:MaxPermSize” to say “-XX:MaxPermSize=128m”.
  • Before that line, add a line that says “-XX:PermSize=64m”.
  • Edit the line that starts with “-Xms” to say “-Xms40m”.
  • Edit the line that starts ith “-Xmx” to say “-Xmx768m”.
  • Save & relaunch Eclipse.

Worked like a charm for me.

Scroll to Top

To Top

Towards a new conceptual framework for digital musical instruments

Joseph Malloch, David M. Birnbaum, Elliot Sinyor, Marcelo M. Wanderley
Input Devices and Music Interaction Laboratory
McGill University
Montreal, Canada

Abstract

This paper describes the adaptation of an existing model of human information processing for the categorization of digital musical instruments in terms of performance context and behavior. It further presents a visualization intended to aid the analysis of existing DMIs and the design of new devices. Three new interfaces constructed by the authors are examined within this framework to illustrate its utility.

1. Introduction

When considering and categorizing devices that produce sound, it is common to become entangled in the differentiation of instruments, musical toys, and installations. Even within categories, confusion arises: conceptual models of musical instruments vary according to historical, cultural, and personal biases. New musical devices have varying degrees of success in penetrating the conceptual boundary between instrument and non-instrument, and frequently their path into the instrument domain is unexpected from the perspective of design intentionality. The issue is further confused by a layer of artistic interpretation, exploding the possible definitions of ‘instrument’ to virtually any conceivable artifact that can involve sound (including the absence of sound). ‘Instrument’ can thus refer to a traditional acoustic device, a controller with no specific mapping, a software program that maps control input to musical output, or can be synonymous with a musical piece itself, in which the interface (including its physical component) is integrated with musical sound output in the composer’s expressive intent [1]. However, a systematic investigation of the design space of a musical device (such as dimension space analysis [2]) promotes an understanding of musical devices that considers both design goals and constraints arising from human capability and environmental conditions.

For the purposes of this investigation, the definition of ‘musical instrument’ will be restricted to refer to a sound-producing device that can be controlled by a variety of physical gestures and is reactive to user actions [3]. A digital musical instrument (DMI) implies a musical instrument with a sound generator that is separable (but not necessarily separate) from its control interface, and with musical and control parameters related by a mapping strategy [4]. While computers are an essential part of a system such as this, the representation of the computer as a symbolic, metaphorical machine generating function-relationships to which we interface sensor and feedback systems does not adequately articulate its role in problem-posing task domains such as music composition and performance.

It has been said that, with respect to computer music, the differentiation between computer and musical instrument is a misconception, and this is a problem that has its solution in interface design [5]. Indeed, a computer may be used to contain structural components of an instrument, or many instruments, whose limits are only defined in terms of the computer’s ability to implement known sound synthesis, signal processing, and interfacing methods. But if the ‘computer = instrument’ paradigm is used, it is likely to leave the impression that digital instruments are also general purpose tools, and that the freedom to change mapping and feedback parameters arbitrarily provides the player with a better musical tool. Instead, the computer can be more aptly viewed as a semiotic, connotative machine that hypothesizes design criteria rather than exclusively representing a priori interaction metaphors based on the cultural and personal experience of the user [6]. Understanding the computer in this way endows it a constitutive role in performance behaviors that are not guided by explicit intention and evaluation of feedback, and directs scrutiny toward a variety other factors.

Fields of research that have been applied to instrument analysis and development range from human-computer interaction [7], theories of design [8], music cognition and perception [9], organology [10], and artistic/musicological approaches [11], to name only a few. We propose another possible approach, tying together ideas from human-machine interaction and music performance practice by emphasizing the context of a musical performance.

2. A Human-Machine Interaction approach

We have developed a paradigm of interaction and musical context based on Jens Rasmussen’s model of human information processing [12], previously used to aid DMI design in [13]. Rasmussen examines the functions of “man-made systems” and human interaction in terms of the user’s perception and the reasons (rather than causes) behind system design and human behavior. He describes interaction behaviors as being skill-, rule-, or knowledge-based. Rasmussen himself suggests that knowledge-based might be more appropriately called model-based, and we believe this term more clearly denotes this mode of behavior, particularly during performance of music, as “musical knowledge” can have various conflicting definitions.

Briefly, skill-based behavior is defined as a real-time, continuous response to a continuous signal, whereas rule-based behavior consists of the selection and execution of stored procedures in response to cues extracted from the system. Model-based behavior refers to a level yet more abstract, in which performance is directed towards a conceptual goal, and active reasoning must be used before an appropriate action (rule- or skill-based) is taken. Each of these modes is linked to a category of human information processing, distinguished by their human interpretation; that is to say, during various modes of behavior, environmental conditions are perceived as playing distinct roles, which can be categorized as signals, signs, and symbols. Figure 1 demonstrates our adaptation of Rasmussen’s framework, in which both performance behaviors and context are characterized as belonging to model/symbol, rule/sign, or skill/signal domains.


Figure 1. A visualization of the framework. Click to enlarge.

2.1. Skill-, Rule-, and Model-based musical performance

Skill-based behavior is identified by [8] as the mode most descriptive of musical interaction, in that it is typified by rapid, coordinated movements in response to continuous signals. Rasmussen’s own definition and usage is somewhat broader, noting that in many situations a person depends on the experience of previous attempts rather than real-time signal input, and that human behavior is very seldom restricted to the skill-based category. Usually an activity mixes rule- and skill-based behavior, and performance thus becomes a sequence of automated (skill-based) sensorimotor patterns. Instruments that belong to this mode of interaction have been compared more closely in several ways. The “entry-fee” of the device [5], allowance of continuous excitation of sound after an onset [9], and the number of musical parameters available for expressive nuance [14] may all be considered.

It is important to note that comparing these qualities does not determine the ‘expressivity’ of an instrument. ‘Expressivity’ is commonly used to discuss the virtue of an interaction design in absolute terms, yet expressive interfaces rely on the goals of the user and the context of output perception to generate information. Expression, a concept that is unquantifiable and dynamically subjective, cannot be viewed as an aspectual property of an interaction. Clarke, for example, is careful not to state that musical expressivity depends on the possession of a maximum or minimum number of expressive parameters; instead, he states that the range of choices available to a performer will affect performance practice [14]. A musician can perform expressively regardless of the choices presented, but must transfer her expressive nuance into different structural parameters and performance behavior modes. This relates to the HCI principle that an interface is not improved by simply adding more degrees of freedom (DOF); rather, at issue is the tight matching of the device’s control structure with the perceptual structure of the task [15].

During rule-based performance the musician’s attention is focused on controlling a process rather than a signal, responding to extracted cues and internal or external instructions. Behaviors that are considered to be quintessentially rule-based are typified by the control of higher-level processes and by situations in which the performer acts by selecting and ordering previously determined procedures, such as live sequencing, or using ‘dipping’ or ‘drag and drop’ metaphors [5]. Rasmussen describes rule-based behavior as goal-oriented, but observes that the performer may not be explicitly aware of the goal. Similar to the skill-based domain, interactions and interfaces in the rule-based area can be further distinguished by the rate at which a performer can effect change and by the number of task parameters available as control variables.

The model domain occupies the left side of the visualization, where the amount of control available to the performer (and its rate) is determined to below. It differs from the rule-based domain in its reliance on an internal representation of the task, thus making it not only goal-oriented but goal-controlled. Rather than performing with selections among previously stored routines, a musician exhibiting model-based behavior possesses only goals and a conceptual model of how to proceed. He must rationally formulate a useful plan to reach that goal, using active problem-solving to determine an effectual course of action. This approach is thus often used in unfamiliar situations, when a repertoire of rule-based responses does not already exist.

2.2. Signals, Signs and Symbols

By considering their relationship with the types of information described by Rasmussen, performance context can also be distributed among the interaction domains. The signal domain relates to most traditional instrumental performance, whether improvised or pre-composed, since its output is used at the signal-level for performance feedback. The sign domain relates to sequenced music, in which pre-recorded or predetermined sections are selected and ordered. Lastly, the symbol domain relates to conceptual music, which is not characterized by its literal presentation but rather the musical context in which it is experienced. In this case, problem-solving and planning are required—such as the in the case of conceptual scores, which may lack specific ‘micro-level’ musical instructions but instead consist of a series of broader directives or concepts that must be actively interpreted by the performer [16].

3. Using the visualization

Consider the drum machine, for instance the Roland TR-808. To create a rhythm, the user selects a drum type using a knob, and then places the drum in a 16-note sequence by pressing the corresponding button(s). When the ‘start’ button is pressed, the sequence is played automatically at the selected tempo. Using the diagram, this is clearly a rule-based way to perform a rhythm. A skill-based example in a similar vein would be using a drum machine controlled by trigger pads that require the performer to strike the pads in real-time. Of course, a drum kit would be another obvious skill-based example. Using the same musical idiom but on the opposite end of the diagram we can consider using the live coding tool Chuck [17] to create the same rhythm. Here the performer would take a model-based approach: playing a beat would require breaking the task into sub-tasks, namely creating a loop and deciding on an appropriate rest interval based on the desired tempo.

4. Applications and implications

4.1 The Rulers

The Rulers, an interface developed by one of the authors at the Center for Computer Research in Music and Acoustics 2004 Summer Workshop, was designed to evoke the gesture of plucking or striking a ruler (or ‘tine’) that is fixed at one end. Utilizing infrared reflect sensors, the tines play one of seven percussive samples, slices extracted from the Amen breakbeat [18]. Each sample consists of either a single drum or cymbal, or a sequence of drums comprising a short rhythm. Because the samples contain sub-rhythms, the instrument must be played in the context of a global tempo set in the Max/MSP patch that remains fixed during the course of the performance. When plucked, each tine oscillates for a different amount of time; the sample it plays back has been assigned strategically, so that the length of sound output and physical oscillation are correlated. This provides an element of visual and passive haptic feedback to the player, as perceptual characteristics of the sound are tightly coupled to the physical construction of the interface. Output amplitude is determined by the amplitude of the tine’s oscillation, leading to control over the amplitude of initial excitation and damping—characteristics that classify it as an instrument that outputs musical events with a non-excited middle [9].


Figure 2. The Rulers, by David M. Birnbaum

Playing the Rulers is principally a skill-based behavior, requiring constant performer input to sustain musical output. While it does not allow for continuous excitation, it does allow continuous modification after an onset, as the tines may be damped to affect the decay rate of musical events. Yet because the musical output contains fixed elements of rhythm over which the performer has no real-time control, the interaction is also directing short-time musical processes that do not originate from the player but are hard-wired into the instrument/system; it therefore incorporates elements of both the signal and sign domains.

4.2 The Celloboard

Another new interface, the Celloboard, was designed to tie sound output with continuous energy input from the performer. Using contact microphones and accelerometers to sense the amplitude, direction and pressure of bowing gestures, this controller allows the continuous excitation, as well as modification, of its sound. Pitch and timbral sound elements, created using scanned synthesis [19], are controlled by sensors on the controller’s neck, sensing position and pressure of touch on two channels, and also strain of the neck itself on one axis.


Figure 3. The Celloboard, by Joseph Malloch

With its many continuously-controlled parameters and integral mapping, the Celloboard controller easily fits into the skill/signal domain. Any interruption in performance will immediately be audible since sound output requires constant bowing of the interface. It possesses a high ‘entry-fee’ for both sound excitation and modification, and does not easily allow high-level control of musical processes. Adaptations suggested by the framework might be to map the physical controls to a synthesis technique even less process-based than the present scanned synthesis implementation, or to allow the selection of discrete pitches (effectively lowering the modification entry-fee), in order to make the instrument more quickly mastered if a larger user-base is desired.

The Gyrotyre

The Gyrotyre [20] is a handheld bicycle wheel-based controller that uses a gyroscope sensor and a two-axis accelerometer to provide information about the rotation and orientation of the wheel. It was designed as a controller around a small group of mappings that would make use of the continuous motion data. We will look at two Gyrotyre mappings in order to place them in the framework.


Figure 4. The Gyrotyre, by Elliot Sinyor

In the first mapping, the interface controls playback of a sound file scrubbed backwards and forwards by spinning the wheel, evoking a turntable interface. The wheel may be spun very fast and then damped to achieve a descending glissando effect, or it may be kept spinning at a constant speed. This DMI (i.e., this particular mapping of the Gyrotyre controller) fits in the skill-based domain of the framework.

In an arpeggiator mapping, spinning the wheel while pressing one of the keys on the handle repeatedly cycles through a three-note arpeggio whose playback speed is directly correlated to the speed of the wheel. The performer changes the root note and the octave by tilting the Gyrotyre. In this case, performance behavior is predominantly rule-based. The musician reacts to signs, such as the current root note and the speed of the playback. The skill-based aspect of performance is the sustaining of a constant speed of rotation while holding a steady root-note position. Ostensibly, a performer could practice to develop these skills, but this would offer little advantage as the instrument outputs discrete, predetermined pitches. Considering musical context could lead to two changes: the mapping could be altered to reflect the required skill in the musical output, and/or the root-note selection method could be mapped to a gesture more appropriate for a rule-based behavior.

5. Conclusions

Our framework is intended to clarify some of the issues surrounding music interface design in several ways. Firstly, it may be used to analyze, compare, and contrast interfaces and instruments that have already been built, in order to facilitate an understanding of their relationships to each other. Additionally, the design of new
instruments can benefit from this description, whether the designer intends to start with a particular interface concept or wishes to work within a specific musical context. Finally, it may be useful for adapting existing DMIs to different musics, or to increase their potential for performance within a specific musical context.

6. Acknowledgments

The authors would like to thank Ian Knopke, Doug Van Nort, and Bill Verplank. The first author received research funds from the Center for Interdisciplinary Research in Music Media and Technology, and the last author received funding from an NSERC discovery grant.

7. References

[1] Schnell, N., Battier, M. Introducing composed instruments, technical and musicological implications. In Proceedings of the International Conference on New Instruments for Musical Expression, pp. 138–142. Dublin, Ireland, 2002.

Full text (pdf, 203 KB)

[2] Birnbaum, D. M., Fiebrink, R., Malloch, J., Wanderley, M. M. Towards a dimension space for musical devices. In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 192–195. Vancouver, Canada, 2005.

Full text (pdf, 458 KB) | HTML version

[3] Bongers, B. Physical interaction in the electronic arts: Interaction theory and interfacing techniques for real-time performance. In Wanderley, M. M. and Battier, M., editors, Trends in Gestural Control of Music. Ircam, Centre Pompidou, France, 2000.

Full text (pdf, 6.0 MB)

[4] Wanderley, M. M., Depalle, P. Gestural control of sound synthesis. In Proceedings of the IEEE Special Issue on Engineering and Music — Supervisory Control and Auditory Communication, 92(4):632–644, 2004.

Full text (pdf, 361 KB)

[5] Wessel, D., Wright, M. Problems and prospects for intimate musical control of computers. Computer Music Journal, 26(3):11–22, 2002.

Full text (pdf, 36 KB)

[6] Hamman, M. From symbol to semiotic: Representation, signification, and the composition of music interaction. Journal of New Music Research, 28(2):90–104, 1999.

Full text (pdf, 1.4 MB)

[7] Wanderley, M. M., Orio, N. Evaluation of input devices for musical expression: Borrowing tools from HCI. Computer Music Journal, 26(3):62–76, 2002.

Full text (pdf, 115 KB)

[8] Cariou, B. Design of an alternative controller from an industrial design perspective. In Proceedings of the International Computer Music Conference, pp. 366–367. San Francisco, California, USA, 1992.

Full text (pdf, 100 KB)

[9] Levitin, D., McAdams, S., Adams, R. L. Control parameters for musical instruments: a foundation for new mappings of gesture to sound. Organized. Sound, 7(2):171–189, 2002.

Full text (pdf, 194 KB)

[10] Kvifte, T., Jensenius, A. R. Towards a coherent terminology and model of instrument description and design. In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 220–225. Paris, France, 2006.

Full text (pdf, 476 KB)

[11] Jordà, S. Digital lutherie: Crafting musical computers for new musics’ performance and improvisation. Ph.D. dissertation, Universitat Pompeu Fabra, 2005.

[12] Rasmussen, J. Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering. New York, NY, USA: Elsevier Science Inc., 1986.

[13] Cariou, B. “The aXiO midi controller. In Proceedings of the International Computer Music Conference, pp. 163–166. Århus, Denmark, 1994.

Full text (pdf, 456 KB)

[14] Clarke, E. F. Generative Processes in Music. Oxford: J. A. Sloboda (Ed), Clarendon Press, 1988, ch. Generative principles in music performance, pp. 1–26.

[15] Jacob, R. J. K., Sibert, L. E., McFarlane, D. C., Mullen, Jr., M. P. Integrality and separability of input devices. ACM Transactions on Human Computer Interaction, 1(1):3–26, 1994.

Full text (pdf, 159 KB)

[16] Cage, J. Silence: Lectures and Writings. Middletown: Wesleyan University Press, 1961.

[17] Wang, G., Cook, P. On-the-fly programming: Using code as an expressive musical instrument. In Proceedings of the International Conference on New Instruments for Musical Expression, Hamamatsu, Japan, 2004, pp. 138–143.

Full text (pdf, 476 KB)

[18] Fink, R. The story of ORCH5, or, the classical ghost in the hip-hop machine. Popular Music, 24(3):339–356, 2005.

[19] Mathews, M., Verplank, B. Scanned synthesis. In Proceedings of the International Computer Music Conference, pp. 368–371. Berlin, Germany, 2000.

Full text (pdf, 174 KB)

[20] Sinyor, E., Wanderley, M. M. Gyrotyre: A dynamic hand-held computer-music controller based on a spinning wheel. In Proceedings of the International Conference on New Instruments for Musical Expression, pp. 42–45. Vancouver, Canada, 2005.

Full text (pdf, 2.9 MB)

← Back to Publications