Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Grounded conceptual relations without linguistic experience in human MEG and large language models
Poster Session C, Thursday, October 1, 10:45 am - 12:45 pm, Wangari Maathai
Fei Gao2, Abigail Licata1, Rico Sennrich2, Sandrien van Ommen1, Timothée Proix3, John Mansfield2; 1University of Geneva, 2University of Zürich, 3ETH Zürich
Humans acquire world knowledge through direct interaction with their environment, a phenomenon called semantic grounding. By contrast, LLMs learn from linguistic inputs. Semantic anomaly studies have already shown that semantic relatedness modulates neural responses (Federmeier & Kutas, 1999) and language model surprisals (Michaelov & Bergen, 2020). However, it remains unclear whether such effects are driven by grounding in world knowledge, or by a lifetime of linguistic input, and whether humans and LLMs represent these relations similarly. In this study, we use typological variation in the lexicon to disentangle grounding and language experience: we use "Colexifiable" concept pairs (e.g., HAIR and FEATHER) that are expressed by the same word in multiple unrelated languages, but not in the language of our participants or in the LLM training corpora. This design allowed us to test whether grounded conceptual relations influence human and model representations without lexical-input confounds. We predicted that Colexifiable alternatives would elicit responses distinct from unrelated Controls in humans, and that newer LLMs would show stronger alignment with human representational structure. We identified concept pairs from the CLICS database that are attested in multiple unrelated languages, but not in major Indo-European languages. These pairs were used to construct sentences ending with three types of critical words: Expected French completions, Colexifiable alternatives, and semantically unrelated Control words. To control expectancy, items were restricted to low-to-medium cloze probability, and behavioral plausibility ratings were collected. MEG recordings were obtained from 25 native French-speaking participants who read sentences, followed by individual structural MRI scans for source reconstruction. We first asked whether MEG responses distinguish the Expected, Colexifiable, and Control conditions using decoding analyses. This showed significant classification accuracy for every pair, and highlighted a late window (603–828 ms) of word processing for the contrast between Colexifiable and Control conditions. Sensor-level cluster permutation tests indicated that Colexifiable alternatives were processed differently from unrelated Controls in the same late window (630–735 ms). Source reconstruction during this window identified a significant cluster for Colexifiable versus Control items from 600–740 ms in the left hemisphere, overlapping a left temporo-parietal network. We then asked whether LLMs could recover the behavioral and representational structure observed in humans. We tested 25 LLMs released between 2018 and 2024. Older models assigned high and weakly differentiated surprisals, whereas newer models showed a pattern closer to behavioral plausibility ratings, with average surprisal ordered as Expected < Colexifiable < Control. RSA revealed a neural geometry shared by half of the participants: Colexifiable-Control closest, Expected-Control most distant, and Expected-Colexifiable intermediate. This dominant geometry was best captured by newer LLMs, whereas older LLMs better fit a smaller subgroup. Together, these results show that cross-linguistic colexification can serve as a principled source of stimuli for testing grounded conceptual relations. Such relations shaped human MEG responses despite not being colexified in French, and newer LLMs aligned better with human neural and behavioral responses. These findings suggest that grounded conceptual relations modulate human neural processing, and can be partially recovered by LLMs despite their lack of direct world experience.
Topic Areas: Meaning: Lexical Semantics, Computational Approaches