Search Abstracts | Symposia | Slide Sessions | Poster Sessions
A Hierarchical Recurrent Model Separates Phonological Competition and Semantic Anticipation in Spoken-Word Recognition
Poster Session E, Friday, October 2, 11:00 am - 1:00 pm, Wangari Maathai
Xiao Shao1, Christian Brodbeck1; 1McMaster University
Spoken-word recognition requires listeners to map continuous acoustic input into sublexical, lexical, and semantic representations as speech unfolds. We use computational models to study how hierarchical processing may support human spoken-word recognition. Standard automatic speech recognition (ASR) models are optimized for final transcription rather than for modeling speech perception in the brain, where recognition is incremental, competitive, and shaped by preceding context. Previous research suggests that a long short-term memory (LSTM) model trained to continuously predict the current word develops graded lexical activation and brain-like activity patterns, but this work used artificially constructed stimuli without meaningful language structure above the word level. We ask which target spaces and hierarchical objectives make LSTM speech models better approximate human-like signatures of spoken-word perception in naturalistic connected speech, including incremental lexical competition, separable phonological and semantic organization, and context-sensitive semantic priming. We systematically compared LSTM-based speech models trained on acoustic cochleagram features from LibriSpeech, an audiobook speech corpus, and evaluated over a 20,000-word lexicon, with all models sharing the same acoustic encoder. The models differed in their representational targets and hierarchical objectives, including one-hot word labels, symbolic random vectors (SRVs), phonology-aware SRVs, character-sequence targets trained with connectionist temporal classification (CTC), direct GloVe semantic targets, and explicit phoneme-to-word supervision. As an additional component, we tested whether a higher-level GloVe-based next-word prediction objective could induce context-sensitive semantic anticipation beyond current-word recognition. The most structured variant instantiated these design principles in a three-level hierarchical LSTM: an acoustic-to-phoneme level, a phoneme-to-word level with phonology-aware continuous word targets, and a semantic level mapping lexical-contextual states to GloVe embeddings of upcoming lexical-semantic states. Across analyses, the full hierarchical model most consistently exhibited human-like signatures, while ablations revealed which components contributed to each effect. First, lexical competition analyses measured time-varying similarity between model states and prototypes for target, cohort, rhyme, and unrelated words. The hierarchical model showed incremental competition, with phonologically related competitors remaining closer to the target than unrelated words. Second, word-distance analyses showed that continuous lexical targets preserved graded phonological structure better than LSTM-one-hot and CTC baselines, while phonology-aware targets produced the strongest phonological separation. Third, layer-wise semantic analyses revealed a functional dissociation across levels: phoneme-level states carried strong phonological but little semantic structure, word-level states and outputs remained primarily organized by phonological form, and the semantic recurrent state showed the strongest semantic separation with reduced phonological dominance. In a simulated semantic priming experiment, the hierarchical model showed a related-over-unrelated advantage in the semantic readout, whereas a random-SRV control did not show the same effect. Overall, these results suggest that separating phonological and semantic learning objectives allows recurrent speech models to capture complementary properties of spoken-word processing: graded lexical competition at the word layer and context-sensitive semantic anticipation at a higher recurrent layer. More broadly, the model comparison provides a computational framework for studying which architectural and learning-objective choices make speech models more consistent with human perception, and yields intermediate representations that can be tested against neural responses during speech comprehension.
Topic Areas: Speech Perception, Computational Approaches