Narrowing the Window: Multisensory Perceptual Learning and its Neural Correlates

Abstract

The brain’s ability to bind incoming auditory and visual stimuli depends critically on the temporal structure of this information. Specifically, there exists a window of time within which stimuli are highly likely to be bound together and perceived as part of the same environmental event. Several studies have characterized the temporal bounds of this window, but few have investigated its malleability. Here, the plasticity in the size of this temporal binding window was investigated using a perceptual learning paradigm in which participants were given feedback during a two-alternative forced choice (2-AFC) audiovisual simultaneity judgment task. Training resulted in a marked (i.e., approximately 40%) narrowing in the size of the window. To rule out the possibility that this narrowing was the result of changes in cognitive biases, a second experiment using a two-interval forced choice (2-IFC) paradigm was undertaken during which participants were instructed to identify a simultaneously presented audiovisual pair presented within one of two intervals. The 2-IFC paradigm resulted in a narrowing that was similar in both degree and dynamics to that using the 2-AFC approach. In a follow-up neruoimaging study, we sought to reveal the neural substrates of these changes. Eleven subjects completed a two-day 2-IFC audiovisual simultaneity judgment training paradigm, immediately before and after which they performed the same task during an event-related 3T fMRI session. A central zone of multisensory convergence, the posterior superior temporal sulcus (pSTS), as well as areas of auditory and visual cortex, exhibited robust BOLD decreases following training, and resting state and effective connectivity analyses revealed significant increases in functional coupling between these cortices after training. These imaging data represent the first evidence of the neural correlates underlying plastic change in adult multisensory networks that likely represent the substrate for a multisensory temporal binding window. These findings suggest a high degree of flexibility in multisensory temporal processing and have important implications for interventional strategies that may be used to ameliorate clinical conditions (e.g., autism, dyslexia) in which multisensory temporal function may be impaired.