Search Abstracts | Symposia | Slide Sessions | Poster Sessions
ARENA Language Dataset: An MEG/fMRI dataset for investigating language processing in the brain across contexts and modalities
Poster Session F, Friday, October 2, 2:45 - 4:45 pm, Wangari Maathai
Cosimo Iaia1,2, Emin Çelik3, Mariya Toneva3, Christian J. Fiebach1,2; 1Goethe University Frankfurt, 2Cooperative Brain Imaging Center, 3Max Planck Institute for Software Systems, Saarbrücken, Germany
Understanding how the human brain processes language and represents complex compositional meaning is a challenge that neuroscience and AI as fields are investigating. To this date, there is no single dataset that investigates in the same participants different levels of contextual complexity – naturalistic texts as well as isolated linguistic stimuli. Available datasets typically rely on either auditory or visual presentation and on a single neuroimaging method. Here, we introduce a novel deep-sampling language dataset that combines two neuroimaging modalities (fMRI and MEG) acquired in the same participants while processing linguistic stimuli at different levels of context granularity, spanning from narratives to sentences, to single words. Sentence and word conditions are presented in both the auditory and visual modalities, and for a subset of the word stimuli, brain activation data are also included for the processing of non-linguistic (pictorial) presentations. Eight participants are attending a total of 31 data acquisition sessions each. In the audiobook condition, participants listen to an entire audiobook in German (one chapter per session, ~11 hours). We specifically chose Moonwalking with Einstein as stimulus material, which presents a variety of semantic dimensions as it focuses on memory techniques to recollect concepts and objects, while also including surprising memorable sentences in contrast to other audiobook datasets that have more general language statistics. In the sentence condition, participants are presented with 234 sentences drawn from the book, delivered both auditorily and visually. In the single-word condition, 1,160 words (selected from the audiobook, such as to represent the semantic space of the book) are presented visually and auditorily in separate experimental sessions. A subset of these words also appears in the sentence stimuli. For the object condition, participants are presented with 170 visual objects, whose names are also stimuli in the single-word task. Each condition is repeated in both neuroimaging modalities. Lastly, we acquire functional localizer tasks to individually determine language ROIs and provide task-free (resting-state) data. This dataset will be openly released for scientific use. To increase reusability and comply with FAIR principles, data are stored in BIDS format, minimally pre-processed, and enriched by multiple types of annotations. Here, we will report quality control metrics and results from validation analyses to demonstrate the usefulness of this deep sampling dataset, specifically decoding/encoding analyses of robust linguistic features such as word frequency. In summary, the ARENA Language Dataset is a novel resource for both the neuroscience community and the NLP community, providing one of the largest experimental designs per participant for language processing to date. This deep-sampling dataset is of great value for investigating language processing across different modalities (visual vs. auditory) and across multiple levels of context granularity. Furthermore, it is well-suited for studying questions at the intersection of episodic memory and language processing, and for comparing semantic representations between linguistic and non-linguistic stimuli, and for comprehensive analyses across electrophysiological and haemodynamic functional neuroimaging modalities. Ultimately, it provides both high temporal and high spatial resolution in the same participants.
Topic Areas: Development of Resources, Software, Educational Materials, etc., Computational Approaches