Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Integrating optimal neural models for sign language, biological motion, and facial expression processing
Poster Session A, Wednesday, September 30, 11:00 am - 1:00 pm, Wangari Maathai
Jieying He1, Brendan Costello1,2, Pedro M. Paz-Alonso1,2, Manuel Carreiras1,2,3; 1Basque Center on Cognition, Brain and Language, 2Ikerbasque, Basque Foundation for Science, Bilbao, Spain, 3University of the Basque Country (UPV-EHU), Spain
Sign language involves complex actions of the hands, body, and face and thus requires cognitive functions that are relevant to the visual-gestural modality. To date, little is known about the neural networks underlying the integration of functions such as biological motion perception or facial expression recognition into sign language processing. Using dynamic causal modeling (DCM), this study attempts to identify the optimal neural models supporting biological motion perception and facial expression recognition, and then investigates how these models integrate into the sign language network. Three groups of hearing native Spanish-speaking participants with varying experience in Spanish Sign Language took part in the present MRI study: 22 proficient native signers, 20 proficient late signers, and 23 non-signers. Two types of visual stimuli were designed: biological motion stimuli (biological, nonbiological, static dots) and facial expression stimuli (dynamic, neutral, scrambled face). Participants viewed the stimuli while performing a simple in-scanner fMRI detection task (i.e., indicate the direction of an arrow, presented in 10% of trials) to maintain attention. We constructed left-hemispheric DCM models for biological motion perception (V5, fusiform gyrus [FFG], posterior superior temporal sulcus [pSTS], inferior frontal gyrus [IFG]) and facial expression recognition (FFG, pSTS, amygdala [Amyg], lateral orbitofrontal cortex [LOFC]), each including four volumes of interest (VOIs). Bayesian model reduction (BMR) procedure was used to identify the optimal models. Subsequently, using the pSTS as the interface VOI, we integrated the optimal sign language model with the biological motion model and with the facial expression model, respectively, resulting in two models for testing effective connectivity parameters. Our results revealed two main findings. First, the optimal biological motion perception model recruited multiple pathways, including feedforward connections (V5→FFG→IFG; V5→FFG→pSTS→IFG), as well as backward connections (IFG→pSTS→V5; IFG→FFG). The optimal facial expression recognition model recruited three feedforward routes (FFG→pSTS; FFG→Amyg; FFG→LOFC). All of these connections showed strong evidence (posterior probability > .95). These results suggest that biological motion and facial expression processing recruit multiple neural pathways from lower visual to higher-level brain areas. Second, in the models integrating biological motion perception and facial expression recognition with the sign language network, we found evidence that both cognitive functions feed into sign language processing via pSTS. Specifically, biological motion perception projected from V5 to pSTS via FFG (V5→FFG→pSTS), whereas facial expression recognition projected to pSTS via FFG (FFG→pSTS) with an additional forward connection to the amygdala (FFG→Amyg), all of which were statistically significant (FDR corrected p < .05). These integrated models suggest that pSTS serves as a central hub modulating the integration of biological motion and facial expression information into the sign language network through directed effective connectivity pathways. These results suggest that biological motion and facial expression information are channeled into the sign language network via the pSTS. The present work provides the first DCM evidence for a shared neural integration mechanism underlying multimodal perceptual inputs in sign language processing. These findings pave the road for investigating how the sign language network dynamically interacts with modality-relevant cognitive functions such as biological motion and facial expression processing.
Topic Areas: Signed Language and Gesture,