Electrophysiological Signatures of Multisensory Temporal Processing in the Human Brain
:
2018-03-22
Abstract
NEUROSCIENCE
Electrophysiological Signatures of Multisensory Temporal Processing
in the Human Brain
David Michael Simon
Dissertation under the direction of Professor Mark T. Wallace
Events in the natural world frequently generate sensory signals in more than one sensory modality. Multisensory integration, the process of combining these signals into a single coherent perceptual representation, conveys numerous behavioral and perceptual benefits. Integration of audiovisual inputs is particularly relevant to everyday function, as many ecologically important signals such as speech have both auditory and visual elements. To accomplish appropriate integration of audiovisual speech inputs, the nervous system utilizes a number of cues, one of which is the temporal relationship between the auditory and visual signals. We investigated the neurophysiological bases of how the brain uses temporal information to appropriately integrate audiovisual speech inputs using a combination of psychophysics and electroencephalography (EEG). This series of investigations elucidated that, for ecologically valid speech signals, the brain uses temporal concordance to reduce the magnitude of auditory cortical responses and increase the efficiency of cortical processing. Furthermore, we demonstrate that when temporal relationships are task relevant, the neural signals associated with temporal processing are distributed to other brain regions through the formation of functional neural networks. Lastly, perceptual plasticity at the single trial level during temporal processing was found to be associated with changes in the physiological signatures of sensory evidence accumulation in decisional circuits. These experiments offer unique insights into how the brain utilizes temporal concordance to control multiple levels of sensory integration which span low-level cortical responses, transfer of temporal information to higher cognitive systems, and the formation of flexible perceptual decisions. Together they offer the first comprehensive physiological characterization of audiovisual temporal processing in the human brain.
