Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Co-speech Gesture as a Conceptual Scaffold in Naturalistic Event Description
Poster Session D, Thursday, October 1, 4:30 - 6:30 pm, Wangari Maathai
Yuyong Liu1, Jiaxin Yan2, Hyeonjeong Jeong2, Sugiura Motoaki3,4; 1Graduate School of Medicine, Tohoku Univ., 2Graduate School of International Cultural Studies, Tohoku Univ., 3Institute of Development, Aging, and Cancer, Tohoku Univ., 4International Research Institute of Disaster Science, Tohoku Univ
Speakers frequently use gestures when describing events, yet the neural mechanisms by which gesture supports speech production remain poorly understood. According to the Gesture-for-Conceptualization Hypothesis (Kita, Alibali, & Chu, 2017), representational gestures do not merely accompany speech; they help speakers activate, manipulate, and package spatio-motoric information for verbal expression. This account predicts that gesture functions as an external conceptual scaffold, reducing cognitive demands associated with organizing dynamic event information for speech. At the neural level, this view predicts that co-speech gesture should reduce recruitment of regions involved in maintaining, selecting, and organizing event representations for verbal production. However, little neuroimaging work has directly examined how gesture supports the production of speech about temporally unfolding events. The present study tested whether co-speech gesture facilitates naturalistic event description by reducing the neural demands associated with conceptualizing events for speech. Thirty-eight participants watched approximately 5-s cartoon clips and subsequently retold each event during fMRI under three production conditions: speech-only (Sp), gesture-only (Ge), and co-speech gesture (Co). Including a gesture-only condition allowed us to distinguish neural effects specific to integrated speech–gesture production from those associated with speech or gesture alone. After excluding participants with excessive head motion, 23 participants were included in the final analysis. If gesture functions as a conceptual scaffold, integrated speech–gesture production should require fewer neural resources than either unimodal production mode. To test this prediction, we first compared speech-only and co-speech production using the contrast Co < Sp, and then conducted a conjunction analysis comparing co-speech production with both unimodal conditions: [(Co < Ge) ∩ (Co < Sp)]. Results were assessed at cluster-level p < .05 FWE correction. Relative to co-speech production, speech-only production elicited greater activation in the right inferior occipital cortex, left lateral temporal regions, including the middle temporal gyrus, and a left inferior frontal cluster. Greater activity in the right inferior occipital cortex may reflect increased reliance on internal visual imagery when gesture was unavailable. Greater activity in the left temporal regions suggests increased demands on contextual semantic integration during unimodal verbal production. Critically, the left inferior frontal cluster was also observed in the conjunction analysis [(Co < Ge) ∩ (Co < Sp)], indicating that both unimodal production modes recruited greater frontal conceptual-control and semantic-selection processes than integrated speech–gesture production. These frontal regions may support the selection, organization, and verbal formulation of event information during narrative production. Together, these findings support the Gesture-for-Conceptualization Hypothesis. Rather than imposing an additional motor burden on speech, gesture appears to serve as a conceptual scaffold that reduces processing demands of maintaining visuo-spatial event information, integrating event meaning, and organizing information for verbal formulation. These findings suggest that gesture contributes directly to conceptualization for speech, rather than functioning merely as an accompanying motor output.
Topic Areas: Signed Language and Gesture, Language Production