hapticity

Immersive Messaging, which I created in Immersion’s Advanced Research Lab, was unveiled at the Wall Street Journal’s All Things Digital conference. The demo was presented to Walt Mossberg on a stage in front of some of the world’s most influential players in high tech.

Immersive Messaging addresses an important question: Is it possible to enable people to trade emotional messages with gesture using their phones? Gesture and touch play a well known role in face-to-face communication. Much of emotional communication is pre-verbal, gestural, and haptic. Therefore, a handheld device with gestural input and haptic output should allow us to communicate emotion intuitively, just as we do everyday with other people in the same room. But we are going to need new interaction metaphors in order to make it happen.

Hapticons

User experience relies on interaction metaphor. For Immersive Messaging, this metaphor comes to life as a virtual object that we call a “hapticon.” Hapticons are about the size of icons, but have physical properties. A phone full of hapticons is like a treasure box full of small objects. When you move or tilt the phone, the hapticons move separately. When you shake the phone, they bounce of the walls and rattle around. Hapticons can be generated, edited, traded, and destroyed with gesture. Some examples follow.

Intimate hapticon

Useful for the sort of short, intimate messages that are often traded between close friends (e.g., SMS), the Intimate Hapticon looks like a heart. It’s created by moving the phone in a gentle heart-shaped motion in the air. You can then add a short text message to the heart hapticon. If the heart contains text, it beats—and you can both see and feel it beating in your phone. When you blow air at your phone gently, in a manner similar to blowing a kiss, the heart flutters off screen to the recipient.

Business hapticon

In a business setting, communication often consists of short blasts of bite-sized information. When you move the phone with a hammering motion, the Business Hapticon is created and shoots off the screen to the recipient. It’s shaped like a carpenter’s nail, and it contains the text, “You nailed it!” When the nail arrives in the recipient’s phone, it generates a strong “Clack!” that can be heard and felt.

Social hapticon

Social networks enable sharing of thoughts and emotions, but these are usually represented with text or visual icons. In contrast, the Social Hapticon is a virtual champagne glass, sent to your friends with a pouring gesture (turning the phone upside down, similar to pouring a bottle of champagne). When you perform this gesture, your friends hear and feel the champagne pouring into their phones in a way that imitates the sloshing of liquid. Once the glass is full, if the phones are raised in a toasting motion, they “Clink!”, with both sound and a realistic glass-on-glass haptic feedback effect. In this way it is possible to enact a toast over the internet.

Live touch

Sending hapticons back and forth has a lot of potential. But I wanted to take it a step further. Could we enable users to touch each other through their phone, in real time, with no intermediary object such as a hapticon?

Twiddling™

When two people are on a voice call, the Twiddling™ application turns their phone screens into blank canvases full of playful possibility. Just as people sometimes doodle on a piece of paper while they talk on the phone, the Twiddling™ environment lets them draw beautiful, colorful comet trails with their fingers. Both users draw on the same surface, so that they collaboratively create a light show for each other. When fingers cross, both users feel haptic feedback. In this way, they are touching each other’s fingertip in real time, just as they are hearing each other’s voices. The sense of emotional connection is enough to fill awkward silences and enable a new way to communicate over long distances.

Twiddling by Immersion Corporation. © 2009 Immersion Corporation

Here’s a video of the presentation:

Media

Here’s Clent Richardson, Immersion’s CEO at the time, discussing Immersive Messaging on the FOX Business Network:

You can find out more about Immersive Messaging on Immersion’s website, or in the press release.

← Back to Projects