Multimodal Ambivalence/Hesitancy Recognition in Videos for Personalized Digital Health Interventions
The study used deep learning to recognize Ambivalence and Hesitancy (A/H) in videos using visual, audio, and text data from a 300-participant dataset. Supervised learning showed context modeling helps. Multimodal fusion was explored, revealing text is strong alone. Domain adaptation using subject-aware alignment performed best for personalization, while zero-shot inference benefited significantly from providing the video transcript.