• Touch feedback in several different places on the body: spine, tummy, waist.
• Touch effects that are generated from emotional language.
• Synchronization to visuals from Second Life

All are very interesting. The spine haptics seemed a stretch to me, but the butterfly-in-the-tummy was surprisingly effective. The hug was good, but a bit sterile. Hug interfaces need nuance to bring them to the next level of realism.

The fact that the feedback is generated from the emotional language of another person seemed to be one of the major challenges—the software is built to extract emotionally-charged sentences using linguistic models. For example, if someone writes “I love you” to you, your the haptic device on your tummy will react by creating a butterflies-like sensation. As an enaction devotee I would rather actuate a hug with a hug sensor. Something about the translation of words to haptics is difficult for me to accept. But it could certainly be a lot of fun in some scenarios!

Interesting new research suggests that mechanical skin disturbances caused by pulsating blood vessels may significantly contribute to your perception of your own heartbeat. This is important because it means that skin may play a larger role in emotion than has been previously thought.

The researchers found that, in addition to a pathway involving the insular cortex of the brain — the target of most recent research on interoception — an additional pathway contributing to feeling your own heartbeat exists. The second pathway goes from fibers in the skin to most likely the somatosensory cortex, a part of the brain involved in mapping the outside of the body and the sense of posture.

This sounds surprising at first, but it makes perfect sense. There have been other instances where the functionality of perceptual systems overlap. For example, it’s been found that skin receptors contribute to kinesthesia: as the joints bend, sensations of skin stretch are used to perceive of joint angles. This was also somewhat surprising at the time, because it was thought that perception of one’s joint angles arose out of the receptors in the joints themselves, exclusively. The same phenomenon, of skin movement being incidentally involved in some other primary action, is at work here. We might be able to say that any time the skin is moved perceptibly, cutaneous signals are bound up with the percept itself.

In fact, I think this may be a good object lesson in how words about feelings can be very confusing. A few years ago, before the recent considerable progress in mapping the neural signature of interoception, the word ‘interoception’ was used to describe a class of perceptions—ones whose object was the perceiver. Interoception meant the perception of bodily processes: heartbeat, metabolic functioning, and so on. When scientists discovered a neural pathway that serves only this purpose, the word suddenly began to refer not to the perceptual modality, but exclusively to that neural pathway. Now that multiple pathways have been identified, the word will go back to its original meaning: a class of percepts, rather than a particular neural conduit.

“Our language is full of spatial metaphors, particularly when we attempt to explain or understand how other people think or feel. We often talk about putting ourselves in others’ shoes, seeing something from someone else’s point of view, or figuratively looking over someone’s shoulder,” Sohee Park, report co-author and professor of psychology, said. “Although future work is needed to elucidate the nature of the relationship between empathy, spatial abilities and their potentially overlapping neural underpinnings, this work provides initial evidence that empathy might be, in part, spatially represented.”

“We use spatial manipulations of mental representations all the time as we move through the physical world. As a result, we have readily available cognitive resources to deploy in our attempts to understand what we see. This may extend to our understanding of others’ mental states,” Katharine N. Thakkar, a psychology graduate student at Vanderbilt and the report’s lead author, said. “Separate lines of neuroimaging research have noted involvement of the same brain area, the parietal cortex, during tasks involving visuo-spatial processes and empathy.”

Wilson is a neurologist who works with musicians who have been afflicted with debilitating chronic hand pain. As he writes about his many interviews, a few themes emerge that are especially relevant to our interests here.

Incorporation
Incorporation is the phenomenon of internalizing external objects; it’s the feeling that we all get that a tool has become one with our body.

The idea of “becoming one” with a backhoe is no more exotic than the idea of a rider becoming one with a horse or a carpenter becoming one with a hammer, and this phenomenon itself may take its origin from countless monkeys who spent countless eons becoming one with tree branches. The mystical feel comes from the combination of a good mechanical marriage and something in the nervous system that can make an object external to the body feel as if it had sprouted from the hand, foot, or (rarely) some other place on the body where your skin makes contact with it…

The contexts in which this bonding occurs are so varied that there is no single word that adequately conveys either the process or the many variants of its final form. One term that might qualify is “incorporation”—bringing something into, or making it part of, the body. It is a commonplace experience, familiar to anyone who has ever played a musical instrument, eaten with a fork or chopsticks, ridden a bicycle, or driven a car. (p. 63)

Projection
Projection is the ability to use the hand as a bridge for projecting consciousness from one location to another. (Wilson did not use the word “projection” in the book.) In some ways, projection can be seen as the opposite of incorporation. Master puppeteer Anton Bachleitner:

It takes at least three years of work to say you are a puppeteer. The most difficult job technically is to be able to feel the foot contact the floor as it actually happens. The only way to make the puppet look as though it is actually walking is by feeling what is happening through your hands. The other thing which I think you cannot really train for, but only can discover with very long practice and experience, is a change in your own vision.

The best puppeteer after some years will actually see what is happening on the stage as if he himself was located in the head of the puppet, looking out through the puppet’s eyes—he must learn to be in the puppet. This is true not only in the traditional actor’s sense, but in an unusual perceptual sense. The puppeteer stands two meters above the puppet and must be able to see what is on the stage and to move from the puppet’s perspective. Moving is a special problem because of this distance, because the puppet does not move at the same time your hand does. Also, there can be several puppets on the stage at the same time, and to appear realistic they must react to each other as they would in real life. So again the puppeteer must himself be mentally on the stage and able to react as a stage actor would react. This is something I cannot explain, but it is very imprortant for a puppeteer to be able to do this. (pp. 92–93)

Serge Percelly, professional juggler:

[An act is successful] not because you put something in the act that’s really difficult, but because you put something in the act in exactly the right way—in a way that makes it more interesting, not only for me but for the audience as well. I’m just trying somehow to do the act that I would have loved to see. (p. 111)

Skill
Wilson is a musician and a doctor to musicians, so he has special insight into the neurology of musical skill—which he recognizes as special case of manual skill that involves gesture, communication, and emotion.

Musical skill provides the clearest example and the cleanest proof of the existence of a whole class of self-defined, personally distinctive motor skills with an extended training and experience base, strong ties to the individual’s emotional and cognitive development, strong communicative intent, and very high performance standards. Musical skill, in other words, is more than simply praxis, ordinary manual dexterity, or expertness in pantomime. (p. 207)

The upper-limb (or “output”) requirements for an instrumentalist are not unique either; they depend upon the possession of arms, fingers, and thumbs, specific but idiosyncratic limits on the rage of motion at the shoulder, elbow, wrist, hand, and finger joints, variable abilities to achieve repetition rates and forces with specific digital configurations in sequence at multiple contact points on a sound-making device, and so on. Peculiarities in the physical configuration and movement capabisities of the musician’s limbs can be an advantage or disadvantage but are reflected in (and in adverse cases can be overcome through) instrument design: How wide can you make the neck of a guitar? How far apart should the keys be on a piano? Where should the keys be placed on a flute—in general? and for Susan and Peter? (p. 225)

Awareness
Touch experience can be a gateway to awareness, which can in turn heal both the mind and the body. Moshe Feldenkrais invented a form of physical therapy that focuses on stimulating an awareness of touch and movement sensations in order to relieve pain.

Most people slouch, tilt, shuffle, twist, stumble, and hobble along. Why should that be? Was there something wrong with their brains? After considering what dancers and musicians go through to improve control of their movements, [Feldenkrais] guessed that people must either be ignorant of the possibilities or refuse to act on them. So they just heave themselves around, lurching from parking place to office to parking place, utterly oblivious to what they are doing, to their appearance, and even to the sensations that arise from bodily movement. He suspected that people just lose contact with their own bodies. If and when they do notice, it is because they are so stiff that they can’t get out of bed or are in so much pain that they can barely get out of a chair. Then they start noticing…

What [Feldenkrais] was doing did not seem complicated. The goal of the guided movements was not to learn how to move, in the sense of learning to do a new dance step. The goal was not to stretch ligaments or muscles. It was not to increase strength. The goal, as he saw it, was to get the messages moving again and to encourage the brain to pay attention to them. (p. 244)

And his student, Anat Baniel, on the deep psychological roots of movement disorders:

I think working with children has given me this idea, which isn’t often discussed in medicine: a lot of disease—medical disease and emotional “dis-ease”—is an outcome of a lack of full development. It’s not something we can get to just by removing a psychological block…

Of course there are problems due to traumatic events in childhood, or disease—you name it. Feldenkrais said that ideal development would happen if the child was not opposed by a force too big for its strength. When you say to a small child, “Don’t touch that, it’s dangerous!” you create such a forceful inhibition that you actually distort the child’s movement, and growth, in a certain way.

Feldenkrais taught us to look for what isn’t there. Why doesn’t movement happen in the way that it should, given gravity, given the structure of the body, given the brain? For all of us there is a sort of sphere, or range, of movement that should be possible. Some people get only five or ten percent of that sphere, and you have to ask, “What explains the difference between those who get very little and those who get a lot?” Feldenkrais said that the difference is that in the process of development, the body encountered forces that were disproportionate to what the nervous system could absorb without becoming overinhibited—or overly excited, which is a manifestation of the same thing. (p. 252)

Feldenkrais’s approach is fascinating, but there is scant discussion in Wilson’s book about the role of the therapist’s hand in this process. After all, this kind of therapy is wholly reliant on an accidental discovery: that the patient can be made aware of her own body through an external, expert hand radiating pressure and heat. How is this possible? The topic isn’t explored.

There are many, many wonderful things to learn from this book for anyone with an interest in biology, art, music, history, or sports. You can find Frank Wilson on the web at Handoc.com

The “Emoti Chair” as they call it is built to bring musical pleasure to the deaf and the hearing impaired. The chair has a multitude of build-in speakers and vibrating devices delicately calibrated to “translate music and sound into movement. Whether it be rocking or vibrations, the music can be heard through the movement of the chair, expressing to the person sitting, the emotion heard in sound.”

(via Science Daily)

In the study, QRIO was introduced into a classroom of toddlers aged 18 months to 24 months. Children of this age group were chosen because they have no preconceived notions of robots and they communicate using touch as much as speech.

“The children accepted the presence of QRIO very well,” Movellan told LiveScience. “There were a few children who were very interested but maintained distance. Over time, the relationship between children and QRIO evolved positively.”

In phase I of the experiment, which lasted 27 sessions, QRIO was instructed to interact with the children using its full behavioral repertoire, which included head-turning, dancing and giggling. At first, the children would touch the robot on its face, but as they warmed to him, the majority of their touches were to its hands and arms — a pattern the children also displayed toward each other.

During phase II, which lasted 15 sessions, QRIO ignored the children’s touches and danced throughout the session. “At that point, the [children] quickly lost interest,” Movellan said.

When QRIO’s ability to respond to touch and giggle were returned for three sessions in phase III, the children became friendly with the robot again.

When robot’s batteries died and it laid on the floor, some of the children cried.

Others put a blanket over him and said, “nigh-nigh.”

And further down in the same article:

The ability to respond to touch is relatively easy to program into robots, Movellan said. “We had things like computer vision in the robot, and touch was the easiest thing,” he said. “And it turned out to be the most important to get things going.”

A lot of energy is focused on developing computer vision, neural networks, and other AI-type technology, which is great. But touch interfaces are inexpensive, relatively easy to engineer, and very effective at engaging human emotion. I think this study implies that, for the purposes of social acceptance, touch may be the most important human-robot interaction mode.