Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Transformer intermediate states reveal representational divergence between conversational production and comprehension
Poster Session F, Friday, October 2, 2:45 - 4:45 pm, Wangari Maathai
Masahiro Yamashita1,2, Shinji Nishimoto1,2; 1The University of Osaka, Japan, 2National Institute of Information and Communications Technology, Japan
Alignment between representations in large language models (LLMs) and brain activity has provided neurocomputational insights into linguistic processing. Although most studies have focused on a single intermediate state within a transformer block, typically the input hidden state, recent studies have shown that other intermediate states can show stronger alignment with brain activity during naturalistic language comprehension (Kumar et al., 2024; Chen and Sivakumar, 2026). These findings suggest that systematic comparisons across transformer intermediate states may clarify the computational mechanisms underlying language processing. However, no systematic investigation has examined how such intermediate states align with neural representations of language production. Here, we tested whether conversational production and comprehension are differentially associated with distinct transformer intermediate states. We analyzed a natural conversation fMRI dataset collected from eight native Japanese speakers (Yamashita et al., 2025). Transcribed utterances were separated into production and comprehension streams, and features were constructed using context windows of 1, 4, and 8 s. From layer 18 of a transformer language model, GPT-NeoX, we extracted 13 internal representations: input hidden state, pre-attention normalized state, per-head query/key/value, per-head query/key with rotary positional embedding (RoPE), per-head context vector, combined attention output, post-attention hidden state, pre-feed-forward network (FFN) normalized state, FFN activated state, and FFN output. Voxel-wise encoding models were fitted separately for production and comprehension features, and variance partitioning was used to isolate production-unique and comprehension-unique variance. Candidate winning features were limited to those showing prediction accuracy of at least r = 0.05 for the corresponding unique component. For each voxel, the “winner” feature was defined as the intermediate state yielding the highest prediction accuracy. Winner ratios were computed by pooling voxel counts across participants and interpreted as descriptive relative preferences among transformer states. Production-unique and comprehension-unique components showed distinct winner profiles across context windows. For production, 1-s windows showed higher winner ratios for per-head key (11.1%) and per-head key with RoPE (11.0%). At 4 s, per-head key with RoPE (13.6%), per-head key (13.5%), and FFN activated states (13.4%) were the most frequent winners. At 8 s, per-head query showed the highest winner ratio (15.2%), followed by key/RoPE-related states. In contrast, comprehension showed a different temporal profile. At 1 s, combined attention output (12.3%) and FFN output (11.4%) were frequent winners. At 4 s, FFN activated states showed the highest winner ratio (15.3%). At 8 s, per-head context vectors became the most frequent winner (15.9%). These results suggest that production-unique and comprehension-unique neural components align with different computations within a transformer block. Production-related activity was most strongly associated with key/query/RoPE states, suggesting that speech production may rely on indexing and selecting relevant information from previously self-produced content to continue the conversation. In contrast, comprehension-related activity shifted from attention-based local integration to FFN-based semantic transformation and broader context-vector representations. These contrasting alignment profiles provide a candidate mechanistic account of how language production and comprehension are differentially organized during real-time conversation.
Topic Areas: Computational Approaches, Language Production