Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Word meaning, not co-occurrence statistics, is essential for predictive language processing
Poster Session E, Friday, October 2, 11:00 am - 1:00 pm, Wangari Maathai
Andrey Zyryanov1,2, Victoria Pierz1, Yulia Oganian1; 1University of Tübingen, Germany, 2Max Planck Institute for Biological Cybernetics, Tübingen, Germany
Next-word prediction is a key component of human language processing. Human cortical responses that reflect the distance between an anticipated and perceived word are well captured by word surprisal generated by large language models (LLMs). The fact that LLMs rely exclusively on word co-occurrence statistics motivates a hypothesis that human next-word prediction, just as in LLMs, relies on word co-occurrence statistics. In contrast, psycholinguistic models propose that next-word prediction relies on word meaning. Here, we empirically distinguished between these models using sentences where a semantically neutral context is followed by a polysemic word with ambiguous meaning. This creates a unique test case where word co-occurrence statistics are accessible for next-word prediction, but a specific word meaning is not, effectively decorrelating these putative drivers of next-word prediction. In an online self-paced reading experiment (47 – 52 participants per sentence), participants read German sentences similar to ‘My colleague [portrayed / stood on] the head’. We examined processing costs (as reflected in slower reading) incurred by the unpredictable polysemic word (e.g., ‘head’) when its meaning was disambiguated (e.g., ‘stood on the head’) or ambiguous (e.g., ‘portrayed the head’: body part or leader). We found a linear relation between LLM (GPT-2) surprisal and slower reading for disambiguated polysemic words, but this relation was significantly suppressed by ambiguity. Thus, while co-occurrence-based LLM surprisal well captures human processing costs for disambiguated words, it fails to do so for words with ambiguous meaning. This indicates that human next-word prediction, unlike in an LLM, is contingent on word meaning access. To reveal the cortical dynamics of ambiguity processing and its effects on next-word prediction, we analyzed source-localized MEG responses to the same sentences presented auditorily to 27 participants. Both ambiguity and LLM surprisal modulated neural responses to polysemic words in the lateral temporal cortex. Their effects partially converged in timing: both emerged in the N400 window, but only the ambiguity effect continued at least until 750 milliseconds after the polysemic word onset. Consistent with our behavioral findings, ambiguity suppressed the alignment between LLM surprisal and neural responses around 250–500 ms after polysemic word onset, with significant alignment evident exclusively for disambiguated polysemic words. This shows that ambiguity processing and next-word prediction are not merely concurrent, but rather mechanistically linked computations. Our findings reveal a distinctive feature of predictive language processing in humans: it is contingent on word meaning access above and beyond word co-occurrence statistics, which sets it apart from next-word prediction principles of LLMs. The spatiotemporal dynamics of ambiguity processing and next-word prediction reported here can guide the development of computational models that explicate the interface between disambiguation and next-word prediction.
Topic Areas: Speech Perception, Meaning: Lexical Semantics