Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Do LLMs Encode Human-Aligned Representations of Semantic Concreteness?
Poster Session C, Thursday, October 1, 10:45 am - 12:45 pm, Wangari Maathai
Zhaoqian Yao1, Xufeng Duan1, Bei Xiao1, Zhenguang G. Cai1; 1The Chinese University of Hong Kong
Introduction. Neuroimaging studies have consistently shown that concrete concepts engage specific neural substrates, including angular gyrus (AG) and anterior temporal cortex, as well as modality-related ventral temporal regions. These findings indicate that conceptual concreteness is represented in distributed and specialized cortical networks. Large language models (LLMs) achieve high performance on semantic tasks, but whether their internal representations spontaneously mirror human neural encoding of concreteness remains unclear. This study investigates whether LLM hidden states exhibit human-like concreteness representations and identifies the core neurons whose ablation reduces brain-alignment scores. Methods. We analyzed two fMRI datasets: Pereira et al. (2018) and Tuckute et al. (2024). We first tested whether human concreteness ratings predict fMRI responses by training regression models to predict neural responses in predefined fROIs from human concreteness ratings (Brysbaert et al., 2014). We then examined whether LLM hidden states predict fMRI responses. We extracted hidden states from Gemma2-9B-IT and Qwen3-8B and used them as predictors in the ridge-regression model with cross-validation to calculate the brain score. Finally, we assessed whether concreteness-sensitive LLM units are necessary for behavioral and neural alignment. We identified model neurons in MLP layer whose activations contributed most strongly to concreteness prediction in the training data and zero-ablated those units (activations were set to zero). We evaluated the impact on both the model’s behavioral concreteness prediction and encoding performance. Then, we retrained ridge regression encoding models using the ablated hidden states to quantify changes in brain score, comparing this to a random ablation baseline. Results. Human concreteness ratings predicted neural responses with strong left-hemisphere dominance. In Pereira et al. (2018), mean brain scores across six homologous fROIs were higher in the left hemisphere (r = .303) than right (r = .028; paired t-test p = .042). In Tuckute et al. (2024), concreteness ratings significantly predicted responses in all six left-hemisphere language fROIs (all p < .001), with the largest effect in the left anterior temporal lobe (ATL, r = .310). LLM hidden states showed robust neural alignment. Ablating concreteness-related neurons caused a marked drop in the correlation between model and human ratings (Gemma2-9B-IT: baseline r = .472, ablated r = .189, random r = .423; Qwen3-8B: baseline r = .740, ablated r = .660, random r = .695). In encoding analyses, baseline noise-corrected brain scores were .559 (Gemma2-9B-IT, layer 16) and .564 (Qwen3-8B, layer 20) in Pereira et al. (2018), and .471 and .451, respectively, in Tuckute et al. (2024). Targeted ablation reduced AG encoding scores more than random ablation, with drops of .042 vs. -.004 for Gemma2-9B-IT and .010 vs. -.032 for Qwen3-8B. Conclusion. LLM hidden states encode information about semantic concreteness that predicts variance in human fMRI responses across two datasets. Targeted ablation of concreteness-sensitive units reduced both behavioral concreteness prediction and neural encoding performance relative to random ablation, suggesting that a subset of model units contributes to this alignment. LLM hidden states contain concreteness-related information that partially overlaps with neural variance in human semantic-processing regions, and targeted ablation reduces both behavioral and neural alignment.
Topic Areas: Meaning: Lexical Semantics,