Multivariate Phase Synchrony and Auditory Attention Decoding for Speech and Music

¹ Department of Electronic Systems, Aalborg University, Aalborg, Denmark
² Oticon A/S, Copenhagen, Denmark

^* e-mail: asjidt@es.aau.dk

Abstract

This study explores the relationship between auditory attention decoding (AAD), multivariate phase synchrony, and energy (alpha power) in a scenario with competing speech and music, where subjects were asked to attend to a target source in the presence of a distractor. We use an end-to-end deep learning pipeline to extract AAD scores for the target and distracting audio sources and use the circular omega complexity (COC) index, (a recently proposed correlate of listening effort), to extract multivariate phase synchrony from multi-channel electroencephalogram (EEG) signals. The audio sources consist of speech and music combined in four different scenarios for the target and distractor: speech vs. speech, speech vs. music, music vs. speech, and music vs. music. Our results indicate that as the AAD performance decreases, both the COC index and alpha power increase, suggesting increased recruitment of brain regions in conditions characterized by low speech tracking. We use linear mixed models (LMMs) to test the significance of the effect that AAD scores have on COC and alpha power. Our results indicate a significant relationship between target AAD score and COC as well as alpha power, while no significance is seen between distractor AAD score and COC or alpha power.