How The Brain Combines And Enhances Sensory Information: Two New Laws

WRIGHT-PATTERSON AIR FORCE BASE, OH, UNITED STATES

Story by Megan Mudersbach

Naval Medical Research Unit Dayton

By: Dr. Vince Billock, Senior Research Physiologist

Modern multisensory research is about 40 years old. Despite a tremendous research effort, multisensory interactions are not yet properly characterized. This situation contrasts strongly with other sensory phenomena, many of which follow well studied laws and are better understood. Lawful characterization is vital – it guides theory and vets experimentation; it tells you when to be surprised and when to be worried.

For several years I have been studying multisensory enhancement. For example a dim light looks brighter when a sound seems to come from the direction of the light. I’ve studied two kinds of multisensory enhancements: gated amplification and nonlinear combination. I found that gated amplification follows a power law (Billock & Havig, 2018).

Equation 1 says that to predict, for example, visual amplification by auditory stimulation, all we need to know is the unamplified visual response. The audio signal triggers the visual amplification, but the amplification does not depend on audio strength. This law holds for appearance judgements and it is followed by one type of cortical neurons. For both perception and neural spike rates, the agreement between observed values and Equation 1’s predicted values is very high. Similarly, a neural model of this amplification phenomenon, which I developed for rattlesnakes (Billock & Tsou, 2014), has an exponent (0.83) that predicts the most important property of sensory integration – weak signals are amplified more than strong signals.

A different rule governs perceptual sensitivity and bimodal cortical neurons. The law for combining two different sensory channels (A & B) is

CombinedResponse = (ChannelAm + ChannelBm)1/m Eq. 2

In unisensory perceptual combinations (like binocular vision which combines the two eyes), the value of m in Equation 2 is usually between 1 and 8. Billock et al. (2019) compared perceptual sensitivity and neural sensitivity for audio-visual combinations using Equation 2. Cat cortical bimodal cells (sensitive to both audio and visual) had a best fitting m value of 1.75. Remarkably, the best fit to comparable perceptual data had a quite similar m of 1.69.

We now know that both perceptual and neural multisensory enhancement follow just two simple laws (Equations 1 & 2). This means that we can now have some confidence in identifying the neural correlates of our perceptual experience. Efforts are underway to extend these results to visual interactions with tactile and vestibular stimuli. Visual-vestibular interactions are of particular importance to understanding spatial disorientations (see Williams, this issue).

References
Billock VA, Kinney M & Meredith MA (2019) To be presented at the Optical Society Fall Vision Meeting.

Billock VA & Havig PR (2018) Scientific Reports, 8, 7645, 1-7.

Billock VA & Tsou (2014) Journal of Cognitive Neuroscience 26, 1587-1599.