First Place in the CVPR 2025 Affective Behavior Analysis In-The-Wild Competition
1 May 2025
Emotional mimicry, the human ability to instinctively mirror the expressions and feelings of others, remains a cornerstone of social interaction, empathy, and trust building. In this year's Emotional Mimicry Intensity (EMI) Challenge, held at the 8th Affective Behavior Analysis in the Wild (ABAW) Workshop and Competition, our research team took first place, demonstrating their progress in understanding and modeling this complex social behavior. Over 25 hours of video has been collected for this challenge, in which participants mimic emotions and then rate the intensities they feel.
The team also took second place in the Behavioral Ambivalence/Hesitancy (BAH) Challenge, which focuses on detecting subtle signs of internal conflict or hesitation in communication. For this second task, 3.4 hours of short videos are collected in which participants display ambivalence/hesitancy. Both tasks are presented at CVPR 2025 in Nashville. This research was conducted in collaboration with Tobias Hallmen and Elisabeth André from the University of Augsburg, as well as Robin-Nico Kampa, Fabian Deuser and Norbert Oswald from the University of the Bundeswehr.
While last year's submission focused solely on audio analysis of the provided video, this year's approach expanded to include textual and visual modalities. For the textual modality, we used OpenAI's Whisper model to create transcriptions of the speech present in the videos. The addition of transcribed speech proved to be particularly powerful, providing a significant boost in performance. This finding highlights a key insight: semantic information captured through text often provides a clearer understanding of emotional context than audio alone. It suggests that current audio models still fall short in capturing the deeper meaning and structure of spoken language, underscoring the importance of integrating linguistic content to more accurately interpret emotional communication.
This collaborative work will be presented at CVPR 2025 in Nashville, Tennessee.
Semantic Matters: Multimodal Features for Affective Analysis
Tobias Hallmen, Robin-Nico Kampa, Fabian Deuser, Norbert Oswald, Elisabeth André
