Dynamic multimodal expressions support inference of both presence and relative salience of blended emotions

Abstract

Abstract Few studies explore nonverbal communication of blended emotions, despite common reports of experiencing multiple emotions simultaneously. In two pre-registered experiments, we investigated if participants can accurately judge how prominently different emotions are expressed in dynamic multimodal portrayals of blended emotions. Actors portrayed pairwise combinations of anger, disgust, fear, happiness, and sadness in varying proportions, using facial gestures, body movement and vocal sounds. In Study 1, participants were instructed to choose two scales (out of 5 available scales: anger, disgust, fear, happiness, and sadness) that best described their impression of the emotional content of the portrayals, and rate how clearly both of the two chosen emotions were perceived. In Study 2, participants were instead free to choose any number of scales (out of the 5 available ones) in the emotion rating task. Results were consistent across both studies and showed that all blended emotions were accurately perceived with significantly higher ratings on scales corresponding to intended vs. non-intended emotions. Participants could also accurately judge the relative salience of each emotion, i.e., which was the dominant and the less dominant emotion in the blend. To summarize, results revealed nuanced perception of both presence and relative salience of blended emotions, which may help us navigate in a complex social environment.