Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Test-retest reliability of TRF-derived measures of higher-level linguistic processing in neurotypical adults
Poster Session A, Wednesday, September 30, 11:00 am - 1:00 pm, Wangari Maathai
Kathleen E. Bradbury-John1, Lokesha S. Pugalenthi2, G. Nike Gnanateja3, Heather R. Dial1; 1University of Houston, 2Rice University, 3University of Wisconsin-Madison
Cortical tracking of speech refers to how the brain tracks acoustic and linguistic features during receptive speech processing. Specifically, activity in the delta (1-4 Hz) and theta (4-8 Hz) bands has been proposed to support processing of words/phrases and syllables, respectively (Gnanateja et al., 2022). Research on temporal response function (TRF) modeling, which quantifies how well the brain tracks specified features, has shown moderate to good reliability (ICC ~0.5-0.8) in tracking acoustic features of speech in healthy controls across the lifespan (Panela et al., 2024; Dial & Gnanateja, 2025). Researchers are increasingly applying TRF modeling to investigate cortical tracking of higher-level linguistic features (e.g., cohort entropy, word frequency, semantic dissimilarity), for which test-retest reliability has not yet been established. This study reports the test-retest reliability of TRF-derived measures of cortical tracking of these higher-level linguistic features in healthy adults. EEG data were recorded in 42 neurotypical adults (15 men; mean age = 43.8 years, SD = 20.5) while they listened to 30 one-minute segments of an audiobook. The data were collected at two timepoints separated by at least one week. Cortical tracking was estimated via linear ridge regression using the mTRF toolbox (Crosse et al., 2016) for delta, theta, and delta-theta bands in a -100 to 1000ms time window using a central electrode cluster that has been used for cortical tracking of linguistic features (Gillis et al., 2021; Verschueren et al., 2022). TRFs were estimated for the following features: multiband speech envelope, phoneme or word onset, and linguistic feature of interest (phoneme-level: cohort entropy; word-level: frequency, semantic dissimilarity). Phoneme onset, word onset, and linguistic feature values were represented as impulse functions at phoneme or word onset and convolved with a Hamming window. Cohort entropy and word frequency were calculated following prior work (Gillis et al., 2021). Semantic dissimilarity was calculated similarly to Broderick et al. (2018) but using vectors from layer 8 of GPT-2 (Caucheteux, Gramfort, & King, 2022). Since prediction accuracies are typically assessed over time and the optimal time window for reliability remains unclear, within-subject ICCs for the two sessions were calculated across the TRF time-course for each feature (envelope, onset, linguistic feature of interest). TRF ICCs from -50 to 950ms were visually assessed (Dial and Gnanateja, 2025). Following Koo & Li (2016), ICC values of <0.5 indicated poor reliability, 0.5-0.75 indicated moderate reliability, 0.75-0.9 indicated good reliability, and 0.9-1.0 indicated excellent reliability. Consistent with Dial and Gnanateja (2025), which includes many overlapping participants with the current dataset, the reliability of the multiband envelope was moderate-good (ICC~0.6-0.8) from ~50-400ms in all bands. However, across the time-course for phoneme onset, word onset, and higher-level linguistic features, reliability was found to be poor (all ICCs<.5). These findings suggest that TRF-derived measures demonstrate acceptable reliability for acoustic features, but show substantially poorer reliability for higher-level linguistic features. As the clinical utility of this approach depends on the method being reliable, future work needs to investigate how different methods (e.g., banded ridge regression, boosted regression) can improve reliability of higher-level feature processing.
Topic Areas: Methods, Speech Perception