MMMG: a Comprehensive and Reliable Benchmark for Multitask Multimodal Generation
MMMG is a comprehensive benchmark testing multimodal generative models across four combinations: image, audio, text/image, and text/audio. It features 49 tasks and 937 instructions to rigorously assess model versatility. The benchmark shows that leading models struggle with complex reasoning but highlights strengths in specific modalities like audio, providing a reliable way to measure human-aligned multimodal performance.