Search Abstracts | Symposia | Slide Sessions | Poster Sessions
From Anatomical Alignment to Shared Responses: SRM Reveals Layer-Wise LLM Encoding in Naturalistic fMRI
Poster Session E, Friday, October 2, 11:00 am - 1:00 pm, Wangari Maathai
Wenbo WANG1, Matthew King-Hang Ma1, Manson Cheuk-Man Fong1; 1The Hong Kong Polytechnic University
Naturalistic language comprehension requires the brain to integrate linguistic information over time. Previous studies showed that large language models (LLMs) provide hierarchical representations that offer a useful computational framework for modeling ECoG data (Goldstein et al., 2025). However, conventional fMRI encoding analyses rely on anatomically aligned voxel responses, potentially ignoring shared response patterns that are distributed differently across individuals. In this study, we aimed to (1) evaluate whether shared response modeling (SRM; Chen et al., 2015) improves LLM encoding of fMRI responses; (2) determine whether SRM-related gains are organized by LLM layer depth and cortical network; and (3) explore whether these gains exhibit hemispheric asymmetry. We analyzed one auditory listening dataset from a Narratives fMRI collection, in which 25 participants listened to the same English naturalistic story. Cortical networks were defined using the Schaefer-400 17-network atlas (400 parcels). For each network, SRM was estimated on the training half of the story and evaluated functional alignment on the held-out half. GPT-2 XL contextual embeddings were then extracted from the story transcript for all layers (0-48), reduced with PCA, temporally delayed to account for the hemodynamic lag. Encoding models were fitted via ridge regression to predict either raw anatomically aligned fMRI responses or SRM-reconstructed responses (SRM+LLM) with the aforementioned embeddings. SRM-reconstructed responses were more predictable from GPT-2 XL, with the strongest SRM+LLM encoding in the temporo-parietal network at layer 40 (r = .18). At participant level, SRM-related encoding gain was modeled across 17 networks, 25 participants, and GPT-2 layers. The layer-wise mixed-effects analysis showed that SRM gain varied with a significant layer-by-network interaction, F(16, 20767) = 79.33, FDR p < .01. Follow-up summaries grouped layers into early, early-middle, middle-late, and late ranges, with many networks showing stronger encoding in the middle-late or late depth. Next, lateralization-index (LI) analysis was used to explore whether SRM-related encoding gain was stronger in different hemispheres. We first tested a series of bilateral SRM+LLM encoding thresholds (r >= .01, .02, .03, .04, .05) and retained only thresholds that preserved at least half of all network-layer estimates. r >= .02 was then selected as the main descriptive filter since it retained 575 of 833 estimates, mean LI .04 and 69.57% of valid estimates were left-hemisphere biased. Follow-up participant-level mixed-effects analyses revealed that SRM-related gain varied strongly with layer depth, F(1, 41625) = 8443.57, FDR p < .01, differed across networks, F(16, 41625) = 2086.18, FDR p < .01, and identified a significant hemisphere effect, F(1, 41625) = 163.53, FDR p < .01. The hemisphere-by-network-by-layer interaction was also significant, F(16, 41625) = 1.93, FDR p = .02, indicating that LH/RH differences depended on both cortical network and GPT-2 layer depth. Follow-up tests showed reliable left-biased SRM-related gains across default-mode and attention/salience networks rather than being only restricted to the temporo-parietal network. Together, our findings suggest that SRM improved LLM encoding of shared fMRI responses. The SRM+LLM gain was strongest in the temporo-parietal network but also extended to default-mode and attention/salience networks in a layer and hemisphere-dependent manner.
Topic Areas: Computational Approaches, Meaning: Lexical Semantics