Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Different Phoneme Segmentation Strategies Constrain the Neural Readout of Linguistic Representations During Continuous Speech Perception
Poster Session C, Thursday, October 1, 10:45 am - 12:45 pm, Wangari Maathai
Francesco Mantegna1, Oiwi Parker Jones1; 1University of Oxford
A central challenge in speech recognition is transforming continuous acoustic input into discrete linguistic representations. This transformation depends on a segmentation process that identifies boundaries between linguistic units. Segmentation is especially challenging for phonemes, which unfold over short timescales and whose acoustic boundaries are blurred by coarticulation. Listeners compensate for this acoustic variability by integrating information over broader linguistic contexts, such as words and sentences. Recent work has shown that neural representations of phoneme content and order can be recovered independently of their position during naturalistic speech comprehension, suggesting the existence of position-invariant linguistic codes. However, it remains unclear to what extent these invariant representations depend on the temporal precision of phoneme segmentation. One possibility is that variability in phoneme duration is abstracted away during higher-order linguistic processing. Alternatively, precise estimates of phoneme timing may be necessary to recover stable linguistic representations from continuous speech. Different forced-alignment approaches instantiate different computational strategies for phoneme segmentation. CTC-based models primarily rely on acoustic evidence and tend to produce sparse temporal assignments, whereas HMM-GMM approaches integrate acoustic and linguistic constraints to produce more temporally structured estimates of phoneme boundaries. Rather than treating these forced-alignment approaches as alternative approximations of a ground-truth segmentation, we interpret them as computational models implementing different strategies for inferring phoneme boundaries. We hypothesized that differences in segmentation strategy would critically affect the recovery of temporally anchored linguistic representations. We tested this hypothesis in a magnetoencephalography study where thirty-two participants listened to audiobooks. We first characterized the temporal precision of the phoneme boundaries produced by each forced alignment strategy. Compared with CTC-derived boundaries, HMM-GMM boundaries aligned more closely with acoustic peaks, better captured silent intervals between phonemes, more clearly differentiated consonants from vowels, and produced phoneme duration distributions more consistent with the temporal variability characteristic of connected speech. We then asked whether these differences in temporal segmentation affected the recovery of linguistic neural representations. After regressing out low-level acoustic features from the MEG signal, we fitted TRF models using phoneme boundaries derived from each forced alignment approach. HMM-GMM-based segmentation produced stronger model fits than CTC-based segmentation and recruited broader neural responses extending beyond bilateral temporal regions into frontal and parietal regions. Finally, we tested whether segmentation strategy affected the neural decoding of phonological features such as place and manner of articulation and voicing. Using a combination of encoding and decoding models time-locked to phoneme onset, we found that HMM-GMM segmentation yielded more accurate and temporally sharper decoding dynamics, with responses more tightly concentrated along the temporal generalization diagonal. Together, these findings suggest that temporally precise estimates of phoneme boundaries play a critical role in recovering higher-order linguistic representations from continuous speech. Rather than serving as a neutral preprocessing step, forced alignment reflects computational assumptions about how acoustic information is partitioned into linguistic units. Our results further suggest that position-invariant phonological representations remain dependent on the temporal structure of speech, indicating that variability in phoneme boundaries is not merely measurement noise but an important component of linguistic processing itself.
Topic Areas: Speech Perception, Phonology