Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Integrating Distributional Semantics and Neuroimaging: Mapping Multidimensional Representations of Emotions
Poster Session A, Wednesday, September 30, 11:00 am - 1:00 pm, Wangari Maathai
This poster is part of the Sandbox Series.
Valentina Alimenti1, Martin Wegrzyn1, Lara von Dombrowski1, Bedia Vidua1, Katalin Havas1, Johanna Kissler1,2; 1Bielefeld University, Germany, 2Center for Cognitive Interaction Technology, Bielefeld, Germany
Growing evidence shows that affective and semantic processes are closely intertwined, supporting models that view emotional features as core components of mental representations of meaning. However, research on semantic memory often overlooks the role of emotions (Hinojosa et al., 2020). The verbal fluency task (VFT) provides insights into semantic memory and lexical access, yet its emotional variants lack methodological consensus, leading to mixed findings (Hegefeld et al., 2023; Sass et al., 2013). Next to behavioral paradigms, neuroimaging methods provide a valuable approach to investigate meaning representation and its neural correlates. Research demonstrates that distinct semantic categories are encoded in different patterns of neural activity. Furthermore, computational models suggest that meaning representation in the brain relates to distributional patterns of words in language and that basic neural activation patterns reflect core semantic features (e.g., semantic features associated with the word “eat” engaging gustatory cortical regions and those associated with “push” engaging motor areas) (Mitchell et al., 2008). However, such models have been primarily tested on concrete nouns, leaving abstract categories, such as emotions, unexplored. Building on these insights, a first completed study (n=416) investigates the semantic structure of affective and non-affective categories through the VFT by using distributional semantics techniques. A second ongoing study (planned n=50, current n=10) incorporates functional neuroimaging to characterize category representations in the brain. Emotion-related categories were distinguished between those prompting emotion-label words, which denote emotional states (e.g., “anger”, “joy”), and emotion-laden words (e.g. “funeral”, “reward”). The latter categories are usually context-dependent and shaped by individual and cultural factors, thus involving more experiential and imagery-based mechanisms (Wu & Zhang, 2025). In the first study, produced items were embedded in the semantic space and semantic dispersion was measured by using distance-based metrics to capture clustering patterns both between and within categories (Lenci & Littell, 2008). Principal component analysis (PCA) was then applied to estimate effective dimensionality of category embeddings, reflecting how semantic information is distributed across distinct semantic features (Musil, 2019). Results show distinct semantic structures between affective and non-affective categories, as well as between emotion-laden and emotion-label categories. Expanding on these findings, the second ongoing study aims to compare the neural and semantic representations of these categories. After completing the VFT across emotion-label, emotion-laden, and non-affective categories, participants undergo fMRI scanning during reprocessing of the self-generated words. They then rate their responses in order to provide autobiographical associations. This paradigm allows us to compare similarity structures of word embeddings with those derived from neural activation patterns, by representing neural responses as multidimensional vectors in a representational space, where local activity patterns correspond to distinct vectorial dimensions (Haxby et al., 2014). We will compare neural and embedding-based similarity matrices using representational similarity analysis (RSA). PCA will be used to examine whether core neural activation patterns correspond to core semantic features, as previous research has suggested. In a later phase, we plan to leverage the autobiographical ratings to assess the contribution of episodic memory to emotion-laden versus emotion-label categories.
Topic Areas: Meaning: Lexical Semantics,