Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Neural Evidence for Context-Dependent Sensorimotor Semantics
Poster Session F, Friday, October 2, 2:45 - 4:45 pm, Wangari Maathai
Harshada Vinaya1, Seana Coulson1; 1Department of Cognitive Science, University of California San Diego
Research on grounded semantics often relies on human-sourced ratings of words to quantify the sensorimotor information associated with the relevant concepts. Examples include ratings for concreteness, imageability, and the Lancaster Sensorimotor Norms (LS norms), which are ratings on six perceptual (vision, auditory, haptic, olfaction, gustatory, introspection), and five action effectors (hand/arm, leg/foot, torso, head, mouth) for 40000 English words. However, language comprehension research suggests that word meanings exhibit context-sensitivity, raising the question of whether any context-free ratings can adequately characterize the brain’s response to words in contexts. Here, we examined whether neural activity during comprehension is better explained by context-sensitive or context-free sensorimotor representations, and when this advantage emerges. We employ sentences from Contextualized Sensorimotor Norms (CS norms) database, which provides human ratings on the same 11-dimensions as LS norms, but for words in sentence contexts, yielding a more precise measure of context-sensitive sensorimotor associations. To test whether LS or CS norms better predict brain responses to words, we recorded EEG while 33 healthy adults read sentences from the CS norms database. Participants read 164 sentences one word at a time. Materials featured 82 different sentence-final words (e.g., “market”) that each appeared in two distinct contexts (e.g., “He liked the fish market.” vs. “He liked the stock market..”). Semantic effects in EEG are typically observed after 200 ms, so we modeled successive 100-ms averaged amplitudes elicited by the sentence-final words (e.g., “market”) across six time windows, from 200-300 ms to 700-800 ms. For each window, we compared three linear mixed-effects regression models using single trial EEG data (n=5212). Each model included subject and item-level random effects. First was the WF model that included word frequency (WF) as the only lexical-semantic predictor. WF was included in the remaining two models as a control predictor. Next, the LS model included perceptual strength and action strength derived from the LS norm; these measures reflect the maximum strength value assigned to the most salient perceptual and motor modality, respectively, for a given concept. Lastly, the CS model similarly included corresponding perceptual and action strength measures derived from the CS norms. Model comparison was conducted using Akaike Information Criterion (AIC), with ΔAIC ≥ 10 interpreted as a robust evidence of model improvement. From 200-300 ms, neither the LS or CS improved fit relative to the WF model. However, from 300-400 ms through 700-800 ms, the CS model consistently provided the best fit by robustly outperforming the LS model (ΔAIC > 60). CS norms perceptual strength elicited a frontocentral negativity, similar to concreteness and imageability effects reported in the literature, with the strongest effect observed at 300-400 ms. In comparison, action strength elicited a centroparietal positivity with a distribution resembling the Late Positive Component (LPC), strongest 700-800 ms. The findings suggest that sensorimotor information supporting language comprehension is selectively constrained by ongoing linguistic context, challenging accounts that propose automatic activation of sensorimotor features as we encounter words.
Topic Areas: Meaning: Lexical Semantics,