How romantic partners interact with each other during a conflict influences how they feel at the end of the interaction and is predictive of whether the partners stay together in the long term. Hence understanding the emotions of each partner is important. Yet current approaches that are used include self-reports which are burdensome and hence limit the frequency of this data collection. Automatic emotion prediction could address this challenge.

Insights from psychology research indicate that partners’ behaviors influence each other’s emotions in conflict interaction and hence, the behavior of both partners could be considered to better predict each partner’s emotion. However, it is yet to be investigated how doing so compares to only using each partner’s own behavior in terms of emotion prediction performance.

In this work, we extracted linguistic features (i.e., what partners said) and paralinguistic features (i.e., how they said it) from a data set of 368 German-speaking Swiss couples (N = 736 individuals) who were videotaped during an 8-minutes conflict interaction in the laboratory. Based on those features, we trained machine learning models to predict if partners feel positive or negative after the conflict interaction.

Our results show that including the behavior of the other partner improves the prediction performance compared to using only one partner’s behavior. Furthermore, for men, considering how their female partners spoke is most important in improving performance and for women, considering what their male partners said is most important in getting better prediction performance. This work is a step towards automatically recognizing each partners’ emotion based on the behavior of both, which would enable a better understanding of couples in research, therapy, and the real world.

For further details on this research article by George Boateng, Peter Hilpert, Guy Bodenmann, Mona Neysari, and Tobias Kowatsch, refer to this article. If this topic is of interest to you, you can learn more about our research into emotion recognition on our dedicated project page.

Reference:

George Boateng, Peter Hilpert, Guy Bodenmann, Mona Neysari, and Tobias Kowatsch. 2021. “You made me feel this way”: Investigating Partners’ Influence in Predicting Emotions in Couples’ Conflict Interactions using Speech Data. In Companion Publication of the 2021 International Conference on Multimodal Interaction (ICMI ’21 Companion), October 18–22, 2021, Montréal, QC, Canada. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3461615.3485424