Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Linking multisensory learning, neural processing, and variability in language skills
Poster Session D, Thursday, October 1, 4:30 - 6:30 pm, Wangari Maathai
Carmen Providoli1,2,4, Nina Raduner1,5, Sarah V. Di Pietro1,4, Maya Schneebeli1,4, Joseph Heng1,4, Iliana I. Karipidis1,4, Michael von Rhein2,4, Nora M. Raschle3,4, Christian Ruff4, Silvia Brem1,4; 1University Hospital of Psychiatry, University of Zurich, 2University Children's Hospital Zurich and University of Zurich, 3Jacobs Center for Productive Youth Development, University of Zurich, 4University of Zurich, 5University of Calgary
Effective communication depends on combining information across sensory modalities. During language development, children integrate inputs from different senses and learn cross-modal correspondences to form coherent representations. Multisensory learning may therefore support language development, and individual differences may contribute to variability in language skills. However, the neural mechanisms linking multisensory learning to typical and atypical language development remain poorly understood. A total of 84 children (age = 6.5±0.5 years, [5.7-8.0] completed a feedback-based fMRI learning task designed to assess multisensory associative learning. Children learned associations across three different multisensory conditions: speech-related audio-visual (AVs), non-speech audio-visual (AVns), and tactile-visual (TV) stimuli. All children completed standardized psychometric tests assessing individual differences in language-related abilities of which two composite scores were derived: precursors of literacy skills (letter knowledge, phonological awareness), and receptive language (receptive vocabulary, grammar comprehension). Whole-brain multiple regressions were conducted separately for each multisensory condition to examine associations between neural activity and language abilities during stimulus processing. Condition-specific effects were tested using flexible factorial models with condition as a within-subject factor and language scores as interaction terms. All analyses included age, gender, and handedness as covariates, with significant activation clusters identified using a cluster-level FWE correction (pFWE < .05). Post hoc beta-value extractions characterized effect directions. Regression analyses revealed stronger deactivation in AVns in children with lower literacy scores in the left superior frontal gyrus. Lower receptive language scores were associated with stronger TV activation in the left occipital fusiform gyrus. Flexible factorial analyses revealed significant condition specific effects of literacy scores in the right thalamus: a positive association during AVs but a negative association during AVns, indicating opposite modulations across speech related and non speech audiovisual conditions. These findings suggest that language related abilities influence neuronal activation during multisensory learning in a selective and context dependent manner, rather than through a general increase or decrease. The association between language skills and activation depended on the linguistic relevance and sensory characteristics of the stimuli, indicating that different multisensory contexts place distinct demands on perceptual, attentional, and working memory systems during learning. The association between lower literacy scores and frontal deactivation during non-speech audiovisual processing may reflect reduced engagement of cognitive control mechanisms when processing difficult-to-verbalize sensory associations. Children with higher literacy scores may thus better recruit top-down strategies to support their encoding. The relationship between receptive language and occipital-fusiform activation during TV learning may suggest that children with weaker receptive language abilities rely more on visually driven encoding processes. Finally, the thalamic effects may suggest that language abilities influence how sensory information is prioritized. Opposite patterns for speech- and non-speech audiovisual processing indicate specialization of thalamocortical systems, with stronger literacy abilities supporting more efficient integration of speech-related information and weaker abilities linked to greater sensory-attentional recruitment for arbitrary associations. Given its role in attention, these findings likely reflect differences in allocating neural resources to meaningful versus arbitrary multisensory input during learning in children. Our results may help to explain why children differ in their ability to benefit from multisensory information during learning.
Topic Areas: Multisensory or Sensorimotor Integration, Language Development/Acquisition