Brain representation of acoustic features during goal-oriented dialogues

Abstract

During dialogue, speakers have to rapidly represent different aspects of speech in order to respond and continue engaged. In particular, acoustic aspects indicate different intentions or emotions. Understanding how the brain represents those characteristics, both spatially and temporally, is important for developing better interactive systems. In this work, we analyse electroencephalography (EEG) data and audio recordings from ten participants engaged in a dialogue during a collaborative task. We aim to better understand how the envelope of the previous instants of the speech is encoded in the brain. Our results showed a good performance in predicting EEG signals, in particular in frontal electrodes. Moreover, our findings suggest that the brain encodes the speech envelope information with a latency between 0 and 100ms. These results are widely consistent between participants and also with previous work. The present work paves the way for studying brain representations of speech in natural scenarios with EEG.