Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Tree-like neural codes for syntax in the human brain
Poster Session F, Friday, October 2, 2:45 - 4:45 pm, Wangari Maathai
pierre orhan1, Stanislas Dehaene2, Laurent Cohen1,3; 1Paris Brain Institute, ICM, Hôpital de la Pitié-Salpêtrière, Paris, France, 2Cognitive Neuroimaging Unit, Université Paris-Saclay, CEA, INSERM, CNRS ELR9003, NeuroSpin Center, Gif/Yvette, France; Collège de France, Université Paris-Sciences-Lettres (PSL), 11 Place Marcelin Berthelot, Paris, France, 3AP- HP, Hôpital de La Pitié Salpêtrière, Institut de Neurologie, Paris, France
Sentence processing has been theorized to involve a hierarchy of operations parsing words into a tree-like structure, the syntactic tree, but no compact description of the underlying neural code has been proposed. Instead, neural responses to language are currently best predicted by non-interpretable representations emerging in artificial neural networks (ANN), leading to quantitative predictions but no qualitative understanding. Here, we show that both brain signals and ANN states contain low-dimensional subspaces that implement syntactic tree structures, and reveal that they differ in their buffering mechanisms. We quantified the variance of intracranial electroencephalographic (IEEG) recordings during auditory sentence perception that is explained by tree-coding vector spaces. This partitions the cortical processing of words into three representational steps: an early encoding of univariate linguistic features (0.0-0.5s after word offset) surrounded by an anticipation of constituent trees (-1.0-0.0s), and an instantiation of the dependency trees (0.7-1.5s). Thus, constituent and dependency tree-coding vector spaces predicted a novel and unique variance in IEEG recordings with tree-coding electrodes distributed across the inferior-motor cortex, superior temporal gyrus, inferior frontal gyrus, and posterior insular cortex. To explain how the syntactic vector of consecutive words is simultaneously coded, we then compared the predictive power of two different mechanisms: a tape and a buffer. Mirroring the ordinal subspaces decoded from a working memory task in monkeys, we found that individual vector spaces best predicted IEEG activity when duplicated according to a buffer mechanism. Finally, we observe that the same tree-coding and univariate features also explain most of an ANN’s prediction of IEEG, despite the low-dimensionality of these codes. This suggests that if the ANN well predicts the neural activity, it is because of a subspace in the ANNs activations that also instantiates syntactic trees. Together, these findings indicate that, like ANNs, the human brain instantiates syntactic trees, providing a compact description of the codes underlying language processing.
Topic Areas: Syntax and Combinatorial Semantics, Computational Approaches