Search Abstracts | Symposia | Slide Sessions | Poster Sessions
High gamma power integrates language structure and statistics during speech comprehension
Poster Session F, Friday, October 2, 2:45 - 4:45 pm, Wangari Maathai
Sophie Slaats1, Alexis Hervais-Adelman1,2; 1Department of Basic Neurosciences, University of Geneva, 2Linguistik Zentrum Zürich, University of Zurich
During speech comprehension, the brain dynamically infers a hierarchy of increasingly abstract representations from the sensory input. An important step in the inferential hierarchy is the combination of words to form phrases and sentences. Whether this process is driven primarily by statistical patterns in the linguistic input, or by a mechanism that combines words into hierarchical representations, is a subject of considerable debate that has regained importance with the arrival of large language models. Both statistical regularities and the structure of language (syntax) have been shown to explain variance in the brain’s response to linguistic input, suggesting that both features are jointly relevant during language comprehension rather than one or the other. But how neural activity may represent these types of information, and whether they interact, remains largely unknown. In this study, therefore, we investigated whether local cortical activity (high gamma power; 70-150 Hz) from intracranial recordings is jointly modulated by lexical probability and syntactic structure; and whether lexical probability affects the inference of syntactic structure. To this end, we analyzed an open dataset of intracranial EEG recordings (ECoG) from native speakers of Dutch (N=18, age=18-52) watching a short film (Berezutskaya et al., 2022, Sci. Data). We used multivariate temporal response functions (mTRFs) and a model comparison approach to disentangle contributions from syntactic and probabilistic features. Syntactic features consisted of top-down and bottom-up node counts obtained from a constituency parse; and probabilistic features consisted of surprisal and entropy obtained from GPT2 fine-tuned for Dutch. To study the interaction, we created separate syntactic features for high surprisal (or entropy) words and low surprisal (or entropy) words and compared their contribution to 1,000 models that contained two randomly divided syntactic features. The results indicated that local cortical activity is modulated by both syntactic structure and lexical probability. Structural and statistical features explained variance at sensors distributed in left perisylvian regions, with 70% of sensors being sensitive to one class of features, and 30% being sensitive to both feature classes. The TRF time-courses suggested single-stage responses for surprisal and entropy, which spread from temporal to frontal areas over time. We propose that effects for contextual probability – being sensitive to multiple sources of linguistic information simultaneously – reflect a sum of changes in the neural state as a consequence of the recently processed input. The TRFs of syntactic structure, on the other hand, suggested multi-stage responses in which information is iteratively shared between temporal and frontal areas (for bottom-up) or temporal and parietal areas (for top-down). The results of the interaction analysis indicated that lexical probability can affect cortical responses to syntactic structure. The sensors that displayed the interaction showed stronger modulation by structure for high surprisal words than for low surprisal words. This suggests that lexical probability affects the process of structure building, complementing earlier findings that indicate that probabilistic processing is affected by syntactic information. Taken together, these findings suggest that statistical and structural features are jointly leveraged and integrated during speech comprehension.
Topic Areas: Syntax and Combinatorial Semantics, Speech Perception