Generating interaction gestures in dyadic conversations using a diffusion model

Abstract

As expectations for computer graphic (CG) avatars and conversational robots increase, enhancing dialogue skills via multimodal channels is crucial for achieving fluent interactions with humans. Thus, automatic interaction motion generation is essential for autonomous conversation systems. Natural motion generation, such as appropriate nodding, requires considering the behavior and voice of the conversation partner. However, current models generate motion from audio or text, neglecting interaction factors. In this study, we implemented an interaction diffusion model (IDM) that uses a diffusion approach and masking features to generate interaction behaviors for dyadic conversation. IDM accounts for two participants, using masks to generate features from conditional inputs. This allows for accommodating conditions like missing features and forecasting without retraining. The experimental results suggests that the model generates the human-like behaviors during conversation in 30 ms.