Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Exploring the neural dynamics of probabilistic syntactic structure building during naturalistic story listening
Poster F5 in Poster Session F, Friday, October 2, 2:45 - 4:45 pm, Wangari Maathai and Lise Girardin
This poster is part of the Sandbox Series.
Junyuan Zhao1, Jonathan Brennan1; 1University of Michigan
[Introduction] Comprehending language requires inferring hierarchical syntactic structure from sensory input unfolding in time. A body of research has shown neural signatures of incremental syntactic structure building (Brennan & Pylkkänen, 2017; Coopmans et al., 2025; Stanojević et al., 2023). Recent findings show that structure building is modulated by predictability. That is, how predictable a word is changes the timing of structure-building responses (Slaats et al., 2024). Such an interaction is consistent with expectation-based accounts of sentence processing in which the listener builds structure based on probabilistic expectations of upcoming words (Hale, 2001; Levy, 2008). It also reveals a limitation in popular metrics that approximate structure-building, such as node count, which assumes uniform effort (so long as syntactic structure is the same) and does not incorporate word-by-word variability in predictability. Here, we explore the interaction between structure building and prediction, as well as their integration. To this end, we first characterize how structure-building EEG responses interact with next-word and syntactic predictability during naturalistic story listening. Then, we ask whether a parsing measure that incorporates prediction better captures the parsing-by-prediction interaction. [Methods] We estimate the neural dynamics of structure building by fitting temporal response functions (TRFs; Brodbeck et al., 2023) against delta-band EEG using the Alice in Wonderland EEG dataset (Bhattasali et al., 2020). TRFs for structure building are estimated against parsing measures (top-down, bottom-up, left-corner) based on context-free grammar (CFG), alongside control predictors (gammatone spectrogram, gammatone onset, word onset, frequency, prosodic boundary strength), GPT2 surprisal, and part-of-speech surprisal. We fit models in which node count predictors are split along median surprisal (separately for GPT-2 and part-of-speech surprisal) and compare structure-building responses between high- and low-surprisal conditions. To test whether incorporating probabilistic prediction into structure building better reconciles surprisal-induced variability, we explore two further approaches that directly incorporate uncertainty in structure-building operations: (1) weighting node-count by string-probabilities (derived by parsing top-k predicted sentence completions sampled from GPT-2) and (2) node-count derived from a recurrent neural network grammar (RNNG) with beam-search (RNNG, see Brennan et al., 2020; Sugimoto et al., 2024). [Preliminary results] We find that predictors of syntactic parsing reliably explain more variance than acoustic features, single-word properties, and next-word predictability, with top-down measures explaining the most variance among all three predictors (consistent with Coopmans et al., 2025). Preliminary analyses confirm that next-word predictability modulates structure-building: Top-down and left-corner parsing measures elicit stronger EEG response at low GPT2-surprisal words as compared to high-surprisal words, with a similar time course (peaking at around 200 ms). Top-down: t_avg(27) = 2.52, p = 0.013; Left-corner: t_avg(27) = 2.54, p = 0.002. Descriptively, bottom-up parsing elicits later, yet more pronounced, EEG response when surprisal is low (see Coopmans et al., 2025; cf. Slaats et al., 2024). Ongoing work (i) explores the differential effects of next-word and part-of-speech surprisal on structure building and (ii) tests which alternative strategy for incorporating probabilistic predictions into measures of structure building (GPT2-weighted, RNNG) offers the best unified account of syntactic structure building in the brain.
Topic Areas: Computational Approaches, Syntax and Combinatorial Semantics