Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Next steps in simulating aphasia in the EARSHOT model of human speech processing
Poster Session C, Thursday, October 1, 10:45 am - 12:45 pm, Wangari Maathai
This poster is part of the Sandbox Series.
Ihintza Malharin1,2, M. Belén Saavedra2, Linkai Peng3, Simona Mancini1,4, James S. Magnuson1,3,4; 1BCBL. Basque Center on Cognition, Brain and Language, 2University of the Basque Country UPV/EHU, 3University of Connecticut. Storrs, CT, USA, 4Ikerbasque. Basque Foundation for Science, Bilbao, Spain
EARSHOT is a model of speech processing that learns to map real speech to semantic representations (Magnuson et al., 2020). This study aims at damaging multiple distinct connection layers of this model to simulate semantic and phonological impairments observed in monolingual English aphasia (Landrigan et al., 2021). While the model is purely receptive, we simulated a naming task by presenting spoken words to the model and measuring the difference between the semantic vector outputted by the model to every other word in the lexicon. We operationalized the model's naming/identification response as the word with highest cosine similarity. We evaluated accuracy as well as semantic (cosine) similarity and phonemic similarity (1 - Levenshtein distance normalized by the length of the longest word in a pair) of the response to the target vector leading to four different types of error: semantic, phonological, mixed and unrelated. Ten randomly-initialized instances of the model ('individuals') were trained on 1000 words from the English lexicon. For damage, we randomly removed an increasing proportion of connections from three different locations of the model in 10 different instances of each model, resulting in 100 “simulated patients”. Our hypothesis was that damage at lower levels of processing which are closer to the auditory input would yield more phonological errors than semantic, while damage at higher levels of processing which are closer to the semantic output would yield more semantic errors than phonological. Our preliminary results on 10 “simulated patients” from 1 model, which were presented at SNL 2025, did not completely support our hypothesis. While we did find overall more phonological errors in lower levels of processing than at any other level, semantic errors were not more prominent in the latest level of processing compared to any other level. This study aims at confirming and extending these results by increasing the “simulated patients” sample size and confirm that our previous results were not due to the specific randomly-initialized weights of that unique model. Additionally, this study will control for important psycholinguistic characteristics of the words such as syllables, number of phonological neighbors, frequency and concreteness (Brysbaert et al., 2014). To do so, we will select a subset of items from the lexicon used to train the model with the same distribution of these characteristics as in commonly used naming tests such as the Philadelphia Naming Test (PNT, Roach et al., 1996) and check the proportion of semantic and phonological errors in this testing subset. This will allow our simulated results to be more readily compared to clinical results and account for the important role of these psycholinguistic characteristics in language processing. Using a model previously shown to emulate human aspects of speech processing to simulate linguistic impairments present in aphasia could enhance our understanding of aphasia and provide a useful tool to test intervention strategies.
Topic Areas: Computational Approaches, Disorders: Acquired