Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Dissecting the emergence of vocal identity representations across conversations
Poster Session C, Thursday, October 1, 10:45 am - 12:45 pm, Wangari Maathai
This poster is part of the Sandbox Series.
Maël Mauchand1, Tianze Xu2, Volker Dellwo2, Alexis Hervais-Adelman1,2; 1University of Geneva, 2University of Zurich
Properly understanding and engaging in a conversation requires listeners to determine who is talking across the conversation. This involves picking up relevant acoustic cues to assign each speaker’s identity, correctly detecting these cues at a turn, and maintaining a representation of the speaker throughout their speech, all while decoding the linguistic and pragmatic meaning of the conversation. This study’s hypothesis is that turn boundaries are critical windows at which listeners typically need to decide whether the next turn is presented by a new speaker (voice discrimination) and who that speaker is (voice identification). This requires a swift processing of identity representations from acoustic identity correlates, but constrained by prior pragmatic inferences on turn-taking. In an MEG experiment, 40 participants will listen to short conversations while performing identity-relevant comprehension tasks. The stimuli will be 280 synthesized, 3-sentence conversations (e.g., “There is almost no wine left. | Is there more in the cellar? | I will go check it out”) involving one to three speakers from a pool of five pre-determined synthesized speakers with controlled acoustic identities. Speakers’ acoustic identities will vary in terms of timbre and pitch. Prior to the task, participants will undergo a familiarization phase to ensure they reliably and consistently assign an identity to each voice. After each conversation, participants will perform a five-alternative forced-choice task via a comprehension question whose answer depends on the correct perception of speaker identity, with the answer being one of the five identities (e.g., “Who is going to the cellar?"). MEG activity in response to acoustic-linguistic properties of the voices will be analyzed throughout the utterances and specifically at the critical turn-switching points using temporal response functions (TRFs) and representational similarity analysis (RSA). The analyses will aim to 1) determine the time course of speaker tracking throughout dialogues, depending on acoustic (pitch, timbre) and linguistic-pragmatic features (word surprisal, turn-taking likelihood) of the stimuli, and 2) assess the neural signature of the representation and discrimination of different speaker identities at turn onsets, offsets, and mid-turn. We predict TRFs will reveal that identity-relevant acoustic information is encoded rapidly in the brain, but parameters may exhibit encoding asymmetries: the salience of pitch may participate more in voice discrimination at turn boundaries, while the stability of timbre may drive voice identification in a secondary stage. Identity representations independent of acoustic changes should emerge at later stages of processing, involve higher-order language and associative cortices (STG, TPJ), and rely on predictive mechanisms dependent on linguistic constraints. In parallel, RSA mapping of identity discrimination from brain signals should depend on “acoustic distance” in early time windows but reflect discrete identity representations at later stages. These effects should be most prominent around turn boundaries rather than in the middle of a turn. The study will not only provide a time course of identity representation in multi-speaker interactions but also explore how neural representations of identity emerge within complex pragmatic contexts involving the appraisal of parallel acoustic and linguistic information.
Topic Areas: Speech Perception, Prosody