Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Hierarchical Predictive Alignment in the Human Language Network
Poster Session A, Wednesday, September 30, 11:00 am - 1:00 pm, Wangari Maathai
Xufeng Duan1, Yibo Wang1, Zhenguang Cai1; 1The Chinese University of Hong Kong
Introduction. Language-model representations are increasingly used to explain human neural responses, but it remains unclear whether better next-word prediction uniformly improves brain alignment. We propose a hierarchical predictive alignment framework in which, as models improve during training, different cortical regions and language tasks align at different stages, progressing from broadly predictive representations to higher-level semantic, syntactic, and discourse-sensitive alignment. Methods. We analyzed predictive performance and neural alignment in Pythia-410M across 31 training checkpoints. Predictive performance was measured as negative log-transformed perplexity, or −logPPL, on English Wikipedia, while neural alignment was assessed with cross-validated encoding-model brain scores predicting fMRI responses in cortical language regions. We used publicly available fMRI datasets with varying stimulus structures and integration demands: Blank-2014 natural story listening, Pereira-2018 sentence and passage reading, and Tuckute-2024 LLM-generated sentence stimuli. For each checkpoint and ROI, the best-performing layer was selected within cross-validation. Brain-score and −logPPL trajectories were compared using Pearson correlations and normalized area distances between the two trajectories across checkpoints. Residual regression analyses tested whether semantics and language comprehension benchmark trajectories explained brain-score variation beyond training progress and perplexity. Results. Across datasets, predictive performance improved earlier than neural alignment. −logPPL reached its half-rise at approximately 512M training tokens and saturated near 8B tokens, whereas aggregate brain scores reached half-rise around 4B tokens and saturated closer to 100B tokens. This dissociation suggests that next-word prediction alone does not fully account for neural alignment. Cortical regions showed distinct trajectories: 6 regions were early-aligned, 8 moderately delayed, 23 late-aligned, and 5 weak or outlier, based on half-rise timing, saturation timing, correlation with −logPPL, and trajectory distance. Early alignment was strongest in left anterior temporal cortex and angular gyrus, where brain-score trajectories closely tracked −logPPL. Several posterior temporal regions were also highly PPL-like but moderately delayed. In contrast, frontal regions and several right-hemisphere ROIs showed later or less stable alignment. Task and dataset effects were substantial. ROI-wise correlations with perplexity were highest for Tuckute-2024, followed by Pereira-sentence, Pereira-passage, and Blank-2014 story listening. Story listening showed the weakest correspondence with −logPPL and the largest trajectory distance, consistent with greater higher-level integration demands during naturalistic comprehension. In residual analyses, semantic and comprehension benchmarks explained additional brain-score variance beyond training progress and perplexity, with the largest effects in right posterior temporal cortex and right frontal regions. These results suggest that late-aligned regions, particularly frontal and right-hemisphere areas, reflect higher-level linguistic and discourse processes not captured by next-word prediction alone. Discussion. These findings support a hierarchical predictive account of brain–model alignment. Early next-word prediction gains track alignment in left temporal and angular regions, especially under controlled task conditions, whereas frontal, right-hemisphere, and naturalistic-story responses align later, suggesting reliance on semantic, syntactic, discourse-level, and task-specific integration. This does not imply that LLM training models human language development; rather, checkpoints provide a computational axis for identifying when neural systems become predictable from language-model representations. Overall, next-word prediction appears to provide a foundation for neural alignment, while later-emerging model properties better capture the hierarchical structure of human language processing.
Topic Areas: Computational Approaches,