Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Modality-Invariant Cortical Encoding of Generative Semantics Across a Large Intracranial Cohort
Poster Session E, Friday, October 2, 11:00 am - 1:00 pm, Wangari Maathai
Aditya Singh1, Jinlong Li1,2, Tessy Thomas1, Elliot Murphy1, Nitin Tandon1; 1University of Texas Health Science Center at Houston, 2Rice University
The human brain encodes meaning through a hierarchy of symbolic representations, from conceptual semantic categories, through lexical identity, to phonological form, each dissociable in space, time, and the linguistic features that drive them. Despite growing interest in higher-order language BCIs, no large-scale intracranial study has systematically characterized this hierarchy from a decoding perspective, defining what a semantic decoder is and how it differs from a lexical or phonemic one. To address this, we recorded sEEG from 25 patients performing orthographic and auditory naming tasks (>400 trials per modality), in which converging cue words progressively constrained a spoken target, dissociating early lexico-semantic encoding, phrasal integration, and object retrieval across time. We developed a convergent encoding-decoding framework to define this hierarchy at scale. Surface-based linear mixed effects models (sbLME) mapped cohort-level encoding of lexical features (word position, frequency) and contextual features including semantic saliency, operationalized as the cosine similarity of each cue word to the target in Word2Vec embedding space, and narrowing, operationalized as evidence that reduces the cardinality of the set actively entertained for lexical access. Non-negative matrix factorization (NNMF) decomposed neural responses into spatiotemporal motifs harmonized across subjects and tasks, yielding canonical waveforms that served as a shared representational basis for subsequent decoding. Cohort-level temporal generalization matrices dissociated decoding of semantic category (indoor/outdoor spaces, actions, feelings, greetings, tools, objects), lexical identity, and phoneme identity, with classifiers trained and tested cross-modally. sbLME maps revealed a cohort-level dissociation between lexical and contextual feature encoding: word position and frequency were preferentially encoded in IFG and MFG, while semantic saliency and narrowing recruited pMTG and aSTG, tracking progressive reduction of semantic uncertainty consistently across subjects. Canonical waveform analysis identified three archetypal response patterns (pulsatile, transient, and ramping) that were consistent across subjects and tasks, with ramping responses in pMTG and aSTG reflecting sustained integration of semantic constraint regardless of cue modality, establishing these regions as the primary substrate of modality-invariant semantic encoding. Cross-modal decoders trained on these canonical representations confirmed this directly, generalizing consistently across the cohort for semantic category in pMTG, angular gyrus, and IFG, achieving during the cue period prior to speech onset 32.6% when trained on orthographic naming and tested on auditory naming and 31.1% for the reverse (chance: 14.3%, 7 categories, p<0.001). Lexical identity decoding followed in IFG and MFG 500-750 ms pre-articulation, and phonemic identity emerged last in pSTG and premotor cortex. These findings provide the first large-scale intracranial account of how the cortical hierarchy transitions between linguistic scales, from early lexico-semantic encoding, through phrasal integration, to object retrieval, dissociable not just anatomically and temporally, but in the functional motifs and decodable features that define each level.
Topic Areas: Meaning: Lexical Semantics, Language Production