Lecture 2: Multisensory integration

Even though I said that our capacity for attentional processing is limited, we do learn the world from multiple sensory channels. These two things are not contracting to each other. Now you are attending this course, you are watching the video and listening to my lecture at the same time. You are attending to two sources of information – but they are not separate things. You can still handle the course easily because the slide and my voice are integrated.

This is really the most important feat of the human brain: our brain does not just integrate information from multiple resources!

The integration across different sensory modalities is critical to survival. Because we can have a reliable source of information about the environment. For example, when you see something like a carnivore and hear a roar, you know it’s time to run! Moreover, multisensory integration means the processing of one channel may influence the processing of another channel, and such interaction will shape our experience. It’s my honour to present you a very cool study the Ig Nobel Prize. Researchers from the University of Oxford demonstrated that when people are eating crisps, the ‘sound’ of a crisp bit will affect the taste of crisps.

At the moment when you chew a piece of crisp, the more ‘crunching’ it sounds, the better flavour you would feel from the crisp.

And here comes a critical conclusion: our mouth feels things by integrating multiple sensory experiences. Of course, the mouth cannot see or hear things. But our mouth is very sensitive to other sensations, such as taste and touch. That’s why when we taste a piece of steak, we not just taste it but also chew it or hold it by tongue to feel its texture. It may sound strange but even our teeth play a role in sensation. For example, when you try a piece of chocolate, you may hold it in between your front teeth to feel if it is too hard to break. In fact, researchers found that our incisors are very sensitive to a slight change in force when biting things.

In the oral cavity, the teeth, the tongue, the palate and all the soft tissue regions contribute to our perception of ‘what happens in our mouth’. One of the perceptual experiences is oral stereognosis, i.e., the perception of the size and shape of intraoral objects. Researchers can use different test pieces to assess the individual ability of oral stereognosis. For example, the accuracy to distinguish the pieces between different sizes or shapes may differ between individuals. The integration of multiple sensory information is critical to oral stereognosis.

Furthermore, multisensory integration may be associated with some illusions. While visual illusion fools you for what you see, the illusion at the multisensory level fools you for the ‘link’ between different sensory channels. I think this can be best demonstrated by the rubber-hand experiment. Again, you can search for the Rubber Hand illusion from websites, there are plenty of videos about this. In the rubber-hand experiment, the touch sensation from the subject’s hand is real. And the visual experience of the rubber hand is also real. But when touch and vision are linked together, the illusion happens! In the case of the rubber-hand illusion, people mistake the rubber hand for their real hands.

What if we adopt a similar experiment to make an illusion for the whole body, rather than a hand? Researchers have already performed such a ‘body-swap’ experiment and it really works! In this experiment, subjects feel their bodies are touched and, at the same time, see a mannequin being touched. The information is wrongly integrated so that subjects perceive the mannequin as part of the real body. This experiment also demonstrates how body image, the feeling about our own body, is shaped. Similarly, the experience about how my mouth feels like, the oral somaesthesis, is shaped by multisensory integration. Now next time when you see your patients, why not look into their feelings about different sensory aspects?

I have seen patients who are sensitive to the smell of dental materials I put in their mouth. One of my patients even told me he can relate the smell to the hardness of the material! And it is not uncommon to see that patients are sensitive to the temperature of the water during dental scaling or the sound of a handpiece during tooth preparation. There is much evidence showing that patients try to ‘sense’ what happens in their mouth with different sensory channels. Unfortunately, we still know little about multisensory integration in the oral cavity. This would be a critical issue for dental treatment, eating and drinking, and our quality of life.