Real-Time Generation of Game Video Commentary with Multimodal LLMs: Pause-Aware Decoding Approaches
Researchers investigated if multimodal large language models (MLLMs) could generate real-time video commentary by deciding both what to say and when to say it using only prompting. A novel dynamic interval-based decoding approach was introduced, which regulates utterance timing based on previous generation duration. This method showed superior alignment with human utterance timing and pause-awareness compared to fixed-interval methods without needing model fine-tuning.