But to me, synesthesia is the wrong word to describe overlapping, multimodal sensations. Overlapping means that there are seperable sensations, with different envelopes, that may co-occur. But the way I experience synesthesia is quite different. When I hear a loud, unexpected noise, it brings the sensation of bright color. Smells have texture. Flashing lights have an auditory rhythm. These combinations aren’t the result of exploration over time, and they don’t overflow. They aren’t like a coffee cup which can be seen, smelled, and touched at the same time. Synesthesia is self-contained. Synesthesia is integrated.

It’s pretty clear to me that Merleau-Ponty’s idea of synesthesia and the widely documented neurological condition by the same name are different phenomena. Therefore I think it’s important to use a different word to refer to each. So for the phenomenon that Merleau-Ponty is identifying, I say “intersthesia” is more apt. It better implies overlapping layers of sensation that come and go continuously, in multiple dimensions. With this new word, we can say that we are all intersthetic all the time, and some of us are synesthetic some of the time.

“Yes or no: do we have a body—that is, not a permanent object of thought, but a flesh that suffers when it is wounded, hands that touch?” — The Visible and the Invisible

Merleau-Ponty’s Philosophy by Lawrence Hass was the first full book I read on the great phenomenologist. If you’re fascinated by sensation, perception, synesthesia, metaphor, and flesh (and frankly, who isn’t?), please read it! It offers many wonderful revelations. I’ll briefly review the following topics from the book:

• Sensation/perception is a false dichotomy.
• Perception is “contact with otherness.”
• Synesthesia is a constant feature of experience.
• The concepts of “reversibility” and “flesh”

Sensation/perception is a false dichotomy

In the first section of the book, Hass lays out some pretty convincing evidence that Cartesian duality is patent nonsense. People have been putting down Descartes a lot lately, so it was helpful to get the nuts and bolts of these arguments all in one place.

In the course of this, Hass also shows that, for the same reasons that the mind/body dichotomy is false, so too is the sensation/perception dichotomy. This rocked my world. I was so sold on the idea that sensations were sense impressions which the brain used to build up knowledge of externalities that I didn’t even question who told it to me or why I thought it was true. But this theory quite simply does not survive Merleau-Ponty’s phenomenological analysis. Hass writes, “The pure, atomic sensation—the alleged building-block of complex perception—is not an empirical or experiential object.”

The implications of Merleau-Ponty’s argument are enormous. For one thing, it seems that the empiricist ontology of the sensation has it just backward: sensations do not build up to perceptions. On the contrary, empirical experience shows that a “sensation”… is always experienced amid perceptual complexity, always identified against a larger field… it is the same error that would be made if one cut a loaf of bread into slices and then assumed the loaf was originally composed of just those slices. (l. 509)

So we can define sensation in terms of perception. But if perception comes first, what is it?

Perception is “contact with otherness”

In perception I am opening onto things that transcend me, that go beyond me and my ideas. One aspect of this transcendence is that perception is a field of contact with otherness. I incessantly see and touch and smell coherent, holistic things that are distinct from me: tables, chairs, a beautiful piece of pottery, steaming hot food from the Chinese restaurant… And I open onto other perceivers perceiving me, touching me, and touching the things that I perceive. “Don’t think, look!” Look and see what you live and experience: below the intellectual concept of pure sense-data, basic perception is “knowledge of existences.”… It is not just strange, but contradictory to our knowledge by direct acquaintance to say that this contact, my caress, this sculpted piece you and I perceive together, my body as felt and experienced, your perceiving body as perceived, are “merely in my mind or brain.” Where precisely in my brain are you perceived and touched? Where precisely in my mind or brain is the chair, as I perceive it with my hands, body, and eyes? Fortunately we can be saved from these absurdities by remembering what we know from experience: perception is our opening onto things that are not oneself. Withdraw into our thoughts or imaginations as we might, withdraw into and reify our ideas or theoretical objects as we do, perception is our perpetual deliverance from narcissism. (l. 526)

There is an interesting paradox here which Merleau-Ponty never gets tired of trying to evoke: that perceptual experience is a field of contact with things, but it is a contact with things and a world that opens up, eludes, and limits our explorations. Further, it provides experiences of certain beings who demand a different kind of approach altogether: other perceiving creatures who perceive me and perceive things with me. Carnal contact, coherent things in an open context, radical otherness: once again, these are all experiences that emerge for one’s living body; they are all experiences we have of the world’s transcendence. (l. 910)

The passages above give you a taste of the argument against Descartes, but there’s much more.

Synesthesia is a constant feature of experience

Merleau-Ponty argues that, far from being a strange neurological condition, we all experience synesthesia when we use cross-sensory metaphors.

…Once we overturn the qualia view of sensation and return perceived things to their place in a symbiotic field, then we immediately open up to the inter-sensorial life of perception…. Seeing the coffee cup on the table before me brings with it the possibility of touching it and smelling its contents…. In virtually every experience of things in the world, there is an overlapping of at least some of our sense modalities, just as there is an overlapping of views in binocular vision. And just as binocular vision yields an experience of the world’s depth, the overlapping of sense modalities gives the things we perceive depth, consistency, and density… At this point it should be pretty clear how abstract it becomes to talk about “five separate senses.” This kind of talk derives from the analytic textbook, not life…. A great virtue of Merleau-Ponty’s account of inter-sensorial perception is that it coheres with phenomena we know perfectly well from the flow of life. For instance, it explains why seeing the blue water of the bay calms our bodies, and conversely, how red or pink brings arousal. It explains how one can see the brittleness of glass, the hardness of an airplane blade, or the dryness of linen from a certain fold in it…. Further, it explains why our everyday language is full of expressions where the senses crisscross: we say that sounds are “sharp” or “dull,” that tastes are “thick” or “dense”; we talk about music as “hard” or “soft,” “light” or “heavy.” There is no shortage of examples that demonstrate the overlapping of our sense-modalities: it is as common and constant in living experience as the beating of one’s heart. As as result, we need to embrace the fact that “synaesthesia” is not some weird laboratory condition, but quite the opposite: “Synaesthetic perception is the rule, and we are unaware of it only because scientific knowledge shifts the center of gravity of experience, so that we have unlearned how to see, hear, and generally speaking, feel.” (l. 1037)

This makes me wonder whether the neurological condition called synesthesia may be modeled as a spectrum that varies from person to person and experience to experience. My synesthesia certainly varies in intensity, but I don’t know why or what factors affect it. Other people I know experience strong synesthesia, all the time. Others do not, but they still use synesthetic metaphors. Maybe we are all using our innate capacity for synesthesia when we use these metaphors. In particular, spatial metaphors, because:

Spatial reasoning must have the body of a self at the center.

Merleau-Ponty offers the insight that spatial metaphor always assumes a perceiver at the center:

To be sure, one’s body is not experienced as “beside” things as, say, the chair is next to the couch. On the contrary, the body’s position on the room determines the “beside” relation in the first place… Up, down, beside, behind, inside, outside, left, right, concave, convex, vertical, horizontal, between, above, below: these are all spatial relations that draw their very meaning and intelligibility from my body lived as an organizing whole. (l. 1041)

This has some interesting implications for mathematics, Hass continues, because it indicates that geometry requires an embodied perceiver, and so does not exist before its truths are acquired by someone. Even algebra requires a body in time in order to make use of words such as “next,” “succession,” and “progression.” Rather than being heavenly and perfect, math is fundamentally perceptual.

Reversibility

Reversibility is a core part of Merleau-Ponty’s philosophy. My understanding of the word is that it refers to the feedback loops and singularities we are all familiar with. It’s the failure of causal attribution that’s just a part of life.

There is a relation of my body to itself that makes it the vinculum of the self and things. When my right hand touches my left, I am aware of it as a “physical thing.” But at the same moment, if I wish, an extraordinary event takes place: here is my left hand as well starting to perceive my right…. Thus I touch myself touching; my body accomplishes “a sort of reflection.” In it, through it, there is not just the unidirectional relationship of the one who perceives to what he perceives. The relationship is reversible, the touched hand becomes the touching hand, and I am obliged to say… that the body is a “perceiving thing,” a “subject-object.” (The Philosopher and His Shadow, quoted in Hass, l. 2008)

This passage makes clear how reversibility and synesthesia are closely tied:

We must habituate ourselves to think that every visible is cut out in the tangible, every tactile being in some manner promised to visibility, and that there is encroachment, infringement … between the tangible and the visible…. It is a marvel too little noticed that … every displacement of my body … has its place in the same visible universe that I itemize and explore with [my hands], as conversely, every vision takes place somewhere in tactile space. There is double and crossed situation of the visible in the tangible and of the tangible in the visible; the two maps are complete, and yet they do not merge into one. The two parts are total parts and yet are not superposable. (The Visible and the Invisible, quoted in Hass, l. 2031)

Flesh

Flesh is portrayed as one of the most difficult but crucial ideas in Merleau-Ponty’s writings. Hass devotes considerable effort to a close reading on flesh. It caught my eye because I really enjoyed the book Philosophy in the Flesh, and I understand now that nobody can use that word in the title of a philosophy book without alluding to Merleau-Ponty. Here are a couple of especially inspiring passages about flesh:

We can denounce [the living body] as the “prison of the soul” or the principle of sin. We can insist that our analyses exhaust it. We can dismiss it from the penumbra of “legitimate” philosophy as “not intellectual enough” or “touchy-feely.” We can, in our everyday lives, ignore, forget, or overlook it, and as we saw, these tendencies are supported by the outward structure of the body itself. Yet this carnality is what we live, and as Merleau-Ponty’s philosophy shows, we are in fact able to know it and articulate its features so that whole sectors of our experience become intelligible….To be sure, one’s living-conscious body is not a machine… It is, as Merleau-Ponty puts it in his final writings, neither substance nor matter nor machine, but flesh. (l. 1384)

Through our behaviors, we become paired—self and other. More precisely, since you “prolong” my intentions through your projective behavior, since you caress the things with hands not unlike mine, since you look at me with gleaming eyes, I extend my self to you. My corporeal schema and perceptual powers reflexively transfer; your resonant behavior means that you are “Another.” Indeed, Merleau-Ponty shows that our knowledge of other minds or selves is rooted in this experiential coupling of flesh-to-flesh that is lived through our behavior. (l. 3816)

Finally, a quote from Merleau-Ponty that I think both James Gibson and Alva Noë would approve of:

…Body consciousness is “in the first place not a matter of ‘I think that’ but of ‘I can’”. (l. 1290)

I just finished Action In Perception by Alva Noë. It’s a very readable introduction to the enactive view of perceptual consciousness, which argues that perception neither happens in us nor to us; rather, it’s something we do with our bodies, situated in the physical world, over time. Our knowledge of the way in which sensory stimulation varies as we control our bodies is what brings experience about. Without sensorimotor skill, a stimulus cannot constitute a percept. Noë presents empirical evidence for his claim, drawing on the phenomenology of change blindness as well as tactile vision substitution systems. I highly recommend the book.

The emphasis on embodied experience leads to the use of touch as a model for perception, rather than the traditional vision-based approach. Here’s an excerpt:

Touch acquires spatial content—comes to represent spatial qualities—as a result of the ways touch is linked to movement and to our implicit understanding of the relevant tactile-motor dependencies governing our interaction with objects. [Philosopher George Berkeley] is right that touch is, in fact, a kind of movement. When a blind person explores a room by walking about in it and probing with his or her hands, he or she is perceiving by touch. Crucially, it is not only the use of the hands, but also the movement in and through the space in which the tactile activity consists. Very fine movements of the fingers and very gross wanderings across a landscape can each constitute excercises of the sense of touch. Touch, in all such cases, is movement. (At the very least, it is movement of something relative to the perceiver.) These Berkeleyan ideas form a theme, more recently, in the work of [Brian O'Shaughnessy's book "Consciousness and World"]. He writes: “touch is in a certain respect the most important and certainly the most primordial of the senses. The reason is, that it is scarely to be distinguished from the having of a body that can act in physical space”…

But why hold that touch is the only active sense modality? As we have stressed, the visual world is not given all at once, as in a picture. The presence of detail consists not in its representation now in consciousness, but in our implicit knowledge now that we can represent it in consciousness if we want, by moving the eyes or by turning the head. Our perceptual contact with the world consists, in large part, in our access to the world thanks to our possession of sensorimotor knowledge.

Here, no less than in the case of touch, spatial properties are available due to links to movement. In the domain of vision, as in that of touch, spatial properties present themselves to us as “permanent possibilities of movement.” As you move around the rectangular object, its visible profile deforms and transforms itself. These deformations and transformations are reversible. Moreover, the rules governing the transformation are familiar, at least to someone who has learned the relevant laws of visuomotor contingency. How the item looks varies systematically as a function of your movements. Your experience of it as cubical consists in your implicit understanding of the fact that the relevant regularity is being observed.

Virtual and augmented reality interface design practices have already begun to demonstrate these concepts. Head mounted augmented reality displays sense the user’s eye and body movements to construct virtual percepts. Head related transfer functions (HRTFs) synthesize sound as it would be heard by an organism with certain physical characteristics, in a particular environment. It seems to me that an important implication for enactive interface design is that haptic sensory patterns can lead to perceptual experience in all sensory modes (vision, hearing, touch). Thus, all human-computer interaction/user experience can be viewed in a haptic context.