Search Abstracts | Symposia | Slide Sessions | Poster Sessions
MEG evidence for shared neural representations of phrase grammaticality across serial and parallel presentation
Poster Session A, Wednesday, September 30, 11:00 am - 1:00 pm, Wangari Maathai
Simone Krogh1, Liina Pylkkänen1; 1New York University
Language can occur in highly serial (speech) or fully parallel form (text), yet how our brains adapt to seriality differences is not understood. Here, we held modality constant (visual) and examined whether processing mechanisms underlying grammaticality detection are similar under Rapid Serial Visual Presentation (RSVP) and Rapid Parallel Visual Presentation (RPVP; Snell&Grainger2017Cognition). RSVP delivers words one-by-one (typically somewhat slowly at ~600ms per word), which allows strict control of word processing but simultaneously risks eliciting processes distinct from natural reading. In contrast, RPVP presents multi-word expressions in their entirety for a few hundred milliseconds, thus eliminating prediction from a temporally preceding context. The language system easily accommodates both presentation modes, but to what extent do stimuli with different temporal structures engage shared neural mechanisms? We presented the same three-word phrases in RSVP (each word on for 300ms, off for 500ms) and in RPVP (the entire phrase on for 300ms, off for 500ms) with a simple matching task during magnetoencephalography (MEG) recordings (N = 29). In our primary set of decoding analyses with generalization across time (King&Dehaene2014TiCS), we trained classifiers to discriminate between neural responses to grammatical (e.g., ARABIAN VALLEY TIGER) and ungrammatical phrases (e.g., VALLEY ARABIAN TIGER) presented in one presentation mode and tested the classifier’s generalization to the other. Note that phrases were semantically designed to address questions about animacy that are not reported here. For each significant temporal cluster, we performed spatial decoding to (partially) localize the effect. Group-level significance was determined with cluster-based permutation tests (Maris&Oostenveld2007JNeuroscience). To complement our cross-presentation analyses, we also performed within-presentation analyses. Behaviorally, accuracy was higher and reaction times faster for grammatical over ungrammatical phrases across presentation modes. Neurally, our cross-presentation analyses yielded temporally similar two-stage activity at word 2 in both train-test directions, alongside an early, off-diagonal cluster when training on RPVP and testing on RSVP word 3. No successful cross-presentation decodability was observed at word 1, despite the lexical material differing in the same way at word 1 and word 2. Our spatial decoding analyses primarily revealed recruitment of the left anterior temporal lobe (LATL) for the RPVP-RSVP word 2 effect. In this first direct comparison of RSVP and RPVP, we thus showed that sensitivity to grammaticality elicits certain shared behavioral and neural signatures across presentation modes. That our cross-presentation decoding analyses revealed substantial representational overlap at the word position where (un)grammaticality occurred (word 2), not when it could be predicted (word 1), highlights the role of linguistic knowledge during processing. This conclusion is supported by the recruitment of the LATL, a region consistently implicated in composition regardless of how language is externalized (Bemis&Pylkkänen2011JNeuroscience; Bemis&Pylkkänen2013CerebralCortex; Blanco-Elorrieta&Pylkkänen2017ScientificReports). Conversely, the failure of cross-presentation generalization at other word positions—despite successful within-presentation decoding for both RSVP and RPVP— suggests that RSVP and RPVP neural signals differentially prioritize specific linguistic operations, obscuring others in the process. Together, the findings demonstrate how the temporal structure of the input modulates linguistic processing, underscoring that the choice of presentation mode is a non-trivial decision that can critically influence the outcome.
Topic Areas: Reading, Methods