Dramarrator: Object-Based Audio Editing for Audio Drama Production from Books
Karim Benharrak, Oriol Nieto, Bryan Wang, Zeyu Jin, Amy Pavel
cs.HC, cs.AI, cs.MM
Submitted: 2026-08-08
Updated: 2026-08-11
Comments: Accepted to UIST 2026
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Audio dramas weave dialogue, sound effects, and music into immersive stories.
Terminology
Abstract
Audio dramas weave dialogue, sound effects, and music into immersive stories. Creators often adapt books into audio dramas, but this process remains labor-intensive, requiring them to interpret source material, author scripts, generate audio assets, and assemble them on a timeline. Because story elements like characters and scenes manifest across many interdependent assets, a single change can ripple into manual updates across the entire project. We present Dramarrator, an audio drama authoring tool built around object-based audio editing, where these story elements are represented as editable objects. Dramarrator extracts these objects from a book, generates linked audio assets (speech, sound effects, and music), and composes a multi-track audio drama. Edits to any object (e.g., a character's voice) automatically propagate to all dependent assets. In a user study with professionals (N=8), Dramarrator significantly lowered task load when creating audio dramas. A listener study (N=300) shows that creator-refined output from Dramarrator approaches the quality of productions made with existing professional tools, and an exploratory study (N=3) suggests object-based editing lowers entry barriers and generalizes beyond audio dramas.
Sources
- MusicLM: Generating Music From Text
- AudioLM: a Language Modeling Approach to Audio Generation
- Stable Audio Open
- AudioStory: Generating Long-Form Narrative Audio with Large Language Models
- AudioGen: Textually Guided Audio Generation
- Keyframer: Empowering Animation Design using Large Language Models
- MM-StoryAgent: Immersive Narrated Storybook Video Generation with a Multi-Agent Paradigm across Text, Image and Audio
Related papers
- Calibrating LLM Explanations to Human Decisions with Reinforcement Learning
- Rethinking Opinion Measurement Interfaces for Human-in-the-Loop Optimization
- EduGage: A Multimodal Dataset and Benchmark for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning
- EvoDesign: Agentic Editable Diagram Creation via Design Expertise Evolution
- HAGI++: Head-Assisted Gaze Imputation and Generation
- Explanation User Interfaces: A Systematic Literature Review