Search Abstracts | Symposia | Slide Sessions | Poster Sessions
LLM-derived Semantic Encoding of fMRI during Naturalistic Cantonese Listening Reveals Age and Neurodevelopmental Differences in School-aged Children
Poster C26 in Poster Session C, Thursday, October 1, 10:45 am - 12:45 pm, Wangari Maathai and Lise Girardin
Dongyi He1, Tian Jiang1, Wenxiyuan Deng2, Nizhuan Wang1, Wai Ting Siok1; 1The Hong Kong Polytechnic University, 2The University of Hong Kong
Introduction: Naturalistic listening provides an ecologically valid approach to studying how the developing brain tracks speech. Cantonese is a stringent test case because lexical tone, morphosyllabic structure, spoken-written divergence, and limited ASR/NLP resources complicate automatic modeling of continuous speech. It remains unclear whether LLM-derived semantic representations can encode Cantonese story-evoked fMRI responses and reveal developmental variation in school-aged children. Methods: Twenty-two children (10 male; grades 2-6), including children with reading difficulty (n=10) and speech difficulty (n=5), completed one 4:12 naturalistic-listening fMRI run (TR=0.75 s) while passively listening to the Cantonese story Grandma’s Bananas (approximately 1,000 characters). Whole-brain BOLD responses were modeled with leave-one-subject-out voxel-wise encoding. Story audio was manually aligned to token timings. CKIP GPT-2 generated contextual embeddings for each token plus the preceding five-token context; embeddings were resampled to fMRI time points using a three-lobe Lanczos filter and expanded with 2-, 4-, 6-, and 8-s FIR delays (3,072 features/TR). Ridge models trained on all other participants predicted single-TR BOLD responses in each held-out child. Decodable semantic information in the learned BOLD likelihoods was tested with two oracle retrieval analyses: a no-rewrite oracle ranking the true 8-s semantic window against 40 temporally non-adjacent story windows, and a paraphrase oracle ranking each sentence and three DeepSeek-generated paraphrases against candidates from other sentences. Prediction correlations were Fisher-z transformed and summarized whole-brain and within the EvLab/Fedorenko allParcels_language_SN220 atlas. Group effects were tested with permutation Welch t-tests and grade effects with permutation linear-trend models (10,000 permutations). Results: LLM-derived semantic features predicted story-related BOLD activity. Within the language network, participant-specific top 5,000 encoding voxels were concentrated in bilateral posterior and anterior temporal cortices, with ROI-level mean prediction correlations of r=0.1971-0.2053 across these temporal-language regions. Retrieval analyses confirmed decodable semantic information. The no-rewrite oracle achieved 39.1% top-1 and 50.9% top-5 accuracy, exceeding chance (2.44% and 12.20%); the best participant reached 100% top-1 accuracy. The paraphrase oracle achieved 19.8% top-1, 36.3% top-5, and 47.0% top-10 accuracy, also above chance (4.76%, 22.13%, and 40.37%). Neurodevelopmental profiles showed distinct language-network effects: children with speech difficulty had lower encoding performance in left posterior temporal cortex than controls (p=0.0109, Cohen’s d=-1.16), whereas reading difficulty was associated with lower performance in left middle frontal cortex (p=0.0329, d=-0.98). Grade was positively associated with performance in right anterior temporal cortex (beta=0.0075/grade, p=0.0219, r=0.479), left posterior temporal cortex (beta=0.0073, p=0.0226, r=0.489), and left anterior temporal cortex (beta=0.0062, p=0.0265, r=0.472). Gender showed no significant effects. Conclusion: LLM-derived FIR semantic features encoded naturalistic Cantonese story-listening responses, supported semantic retrieval from learned BOLD likelihoods, and revealed grade- and neurodevelopment-related variation within the child language network. These findings support LLM-based semantic encoding as a biologically informative framework for studying naturalistic speech processing across typical and atypical language development.
Topic Areas: Language Development/Acquisition, Computational Approaches