Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatars

The gist The researchers propose Avatar-Forever, a decoupled parallel training framework for high-quality real-time infinite interactive avatars, which treats generation efficiency and long-horizon robustness as two independent capabilities that can be trained in parallel.

How it works

Avatar-Forever decouples the training of efficiency and robustness by treating them as two independent capabilities that can be trained in parallel<ref:2608.12107#pg4>. One branch performs full-parameter distillation to train an efficient generator with high visual quality, while another trains a lightweight long-horizon adapter via Recovery-oriented Rollout Training (RRT), which improves generation robustness under long-horizon inference conditions<ref:2608.12107#pg4>. This decoupled parallel training design simplifies the overall training process and avoids unnecessary objective conflicts between few-step generation and long-horizon adaptation<ref:2608.12107#pg4>.

The framework introduces three core designs<ref:2608.12107#pg4>:

1. Efficiency Branch: This branch aims to obtain a high-quality few-step generator from the pretrained video foundation model by adopting Distribution Matching Distillation (DMD) [44]<ref:2608.12107#pg4>. DMD optimizes the student by matching the generated distribution to the teacher distribution through a reverse-KL objective<ref:2608.12107#pg4> and is responsible only for few-step efficiency<ref:2608.12107#pg4>.

2. Robustness Branch: This branch addresses the train–test mismatch introduced by autoregressive self-conditioning<ref:2608.12107#pg6>. It introduces Recovery-oriented Rollout Training (RRT), which targets the main failure mode of autoregressive generation, i.e., errors accumulate after recursive rollout<ref:2608.12107#pg4>. RRT perturbs an early historical context and applies standard flow-matching supervision only after errors have accumulated<ref:2608.12107#pg6>.

3. ForeverCache: This strategy is introduced to accelerate streaming inference by caching stable historical context features across denoising steps<ref:2608.12107#pg4>. For each generated chunk, ForeverCache executes a full forward pass only at the initial denoising step and reuses the resulting historical features throughout the remaining denoising trajectory<ref:2608.12107#pg7>.

The final Avatar-Forever generator is obtained as a composition of these two branches<ref:2608.12107#pg4>:

theta = theta0 + ΔthetaDMD + ΔthetaRRT<ref:2608.12107#pg4>. This composition keeps the two objectives decoupled during training: full-parameter distillation provides a strong few-step generator, while the LoRA-based RRT branch adds long-horizon robustness with minimal additional training and deployment cost<ref:2608.12107#pg4>.

Key Components and Mechanisms

The paper details specific mechanisms for each component of Avatar-Forever<ref:2608.12107#pg4>:

Distribution Matching Distillation (DMD)

DMD optimizes the student by matching the generated distribution to the teacher distribution through a reverse-KL objective formulated as ∇thetaLDMD = −Et,epsilon,c / ∂Gtheta (epsilon, c) / ∇theta (1)<ref:2608.12107#pg4>. This branch is responsible only for few-step efficiency<ref:2608.12107#pg4>.

Recovery-oriented Rollout Training (RRT)

RRT targets the main failure mode of autoregressive generation by perturbing an early history chunk and propagating this perturbation through multi-step autoregressive rollout without gradients<ref:2608.12107#pg6>. Supervision is applied only to the next ground-truth chunk after errors have accumulated, using a standard flow-matching objective<ref:2608.12107#pg6>. The initial perturbation serves as a trigger for distribution shift<ref:2608.12107#pg6>.

ForeverCache

ForeverCache is an inference-time feature reuse mechanism that eliminates repeated computation on fixed historical context<ref:2608.12107#pg4>. For each autoregressive chunk, it executes a full-window forward pass only at the initial denoising step and reuses the resulting historical features throughout the remaining denoising trajectory<ref:2608.12107#pg4>. This design retains access to long-range context while restricting repeated computation to the evolving current chunk<ref:2608.12107#pg4>.

Synthetic Data and Evaluation

To support scalable adaptation, the framework constructs a fully synthetic data pipeline using video foundation models to synthesize training videos from high-quality conversational prompts<ref:2608.12107#pg5>. This pipeline includes dialogue and prompt synthesis using GPT [55] to filter and refine samples<ref:2608.12107#pg5>. The LTX Video Synthesis then generates avatar videos under controllable visual and motion conditions<ref:2608.12107#pg5>. Quality-aware filtering is performed using multimodal similarity models like ImageBind [56] and Gemini [59] to remove low-quality or semantically inconsistent samples<ref:2608.12107#pg5>.

The experiments evaluate performance on short-horizon quality (5-second split) and long-horizon stability (30-second split)<ref:2608.12107#pg6>. The results show that Avatar-Forever preserves the generation quality of the video foundation model under few-step inference<ref:2608.12107#pg6>. For long-horizon stability, it achieves the best LLM Overall score on all three datasets and improves FID and FVD across all datasets<ref:2608.12107#pg6>.

Main Findings

The main contributions demonstrate that few-step efficiency and long-horizon robustness can be effectively learned in parallel<ref:2608.12107#pg4>. The full Decoupled DMD + RRT model achieves the best result on every metric, spanning perceptual quality, motion, synchronization, and distributional realism<ref:2608.12107#pg4>. The FM only variant shows limited performance because optimizing long-horizon flow matching without the distilled generator substantially degrades visual fidelity and synchronization<ref:2608.12107#pg4>. The effect of RRT horizon shows that larger rollout horizons match more closely the error accumulation process encountered at inference time, with K = 4 producing the most stable long-video results<ref:2608.12107#pg10>.

The ForeverCache variant improves streaming efficiency by reusing fixed historical context features across denoising steps<ref:2608.12107#pg6>. On 30-second generation, it reduces runtime from 38.85s to 26.71s, corresponding to a 31.2% latency reduction and a 45.5% throughput increase<ref:2608.12107#pg6>. The results show that Avatar-Forever preserves identity, facial detail, and motion diversity over extended autoregressive rollout<ref:2608.12107#pg4>.

The work concludes that the quality of few-step generation and the robustness to autoregressive drift are distinct adaptation objectives, which are best learned independently and composed at deployment<ref:2608.12107#pg7>. The framework achieves an end-to-end throughput of high-resolution 768 × 512 videos at 27.2 FPS on a single H100 GPU<ref:2608.12107#pg4>. The results support the principle that learn efficiency and long-horizon robustness separately, then compose them at deployment<ref:2608.12107#pg4>.

Limitations and Future Work

Avatar-Forever achieves 27.2 FPS at 768 × 512 resolution on a single H100 GPU, but it is not yet optimized for consumer-grade hardware<ref:2608.12107#pg4>. Future work will explore domain-specific data construction and training strategies to further unlock Avatar-Forever’s potential for general long-horizon video generation<ref:2608.12107#pg15>.

Ethics Statement

This work aims to advance long-duration audio-driven avatar generation for constructive applications such as virtual communication, education, accessibility, and digital content creation<ref:2608.12107#pg15>. The evaluation data used in this paper are drawn from publicly available and openly accessible research datasets<ref:2608.

1. Improve training stability by adopting a decoupled parallel training framework to treat them [generation efficiency and long-horizon robustness] as two independent capabilities that can be trained in parallel. This simplifies optimization by allowing one branch to perform full-parameter distillation to obtain a high-quality few-step generator while another trains a lightweight adapter for stability.

2. Enhance long-horizon generation robustness using Recovery-oriented Rollout Training (RRT), which targets the main failure mode of autoregressive generation, i.e., errors accumulate after recursive rollout. RRT works by perturbing an early history chunk and applying supervision after errors have accumulated, ensuring training targets recovery under model-induced context drift rather than one-step synthetic corruption.

3. Increase inference throughput during long-horizon streaming by implementing ForeverCache, a mechanism that reuses stable context features across denoising steps. This eliminates redundant computation by performing a full forward pass only at the initial denoising step and reusing those features thereafter, which can improve throughput by 23% during long-horizon generation.

4. Achieve superior long-horizon stability by composing the learned components: The final Avatar-Forever generator is obtained as [theta = theta0 + ΔthetaDMD + ΔthetaRRT], which "keeps the two objectives decoupled during training: full-parameter distillation provides a strong few-step generator, while the LoRA-based RRT branch adds long-horizon robustness with minimal additional training and deployment cost."

5. Produce high-fidelity, coherent, and diverse motion in extended generation by utilizing a fully synthetic data pipeline that uses a 22B video foundation model to synthesize controllable avatar videos from high-quality conversational prompts, ensuring the resulting models maintain persistent identity, coherent motion, accurate lip synchronization over durations exceeding 11 minutes.

Existing streaming video systems often rely on sequential, distillation-centered training pipelines to enable few-step long-video generation. However, this paradigm suffers from two limitations. First, failures or distribution shifts introduced in earlier stages affect later optimization, complicating the training process to converge. Second, the distillation-centric objective favours short-term generation but is prone to quality degradation when autoregressive errors accumulate over long rollouts. We propose Avatar-Forever, a decoupled parallel training framework for high-quality real-time infinite interactive avatars. Instead of coupling generation efficiency and long-horizon robustness under a sequential distillation pipeline, we treat them as two independent capabilities that can be trained in parallel. One branch performs full-parameter distillation to train an efficient generator with high visual quality, while another trains a lightweight long-horizon adapter via Recovery-oriented Rollout Training (RRT), which improves generation robustness under long-horizon inference conditions. Our decoupled parallel training design simplifies the overall training process and avoids unnecessary objective conflicts between few-step generation and long-horizon adaptation. We further introduce ForeverCache, a chunk-wise feature caching mechanism to substantially reduce redundant history computation during streaming inference. Built upon a 22B video foundation model, Avatar-Forever supports unbounded audio-driven avatar generation while maintaining identity consistency, motion coherence, and visual fidelity, enabling an end-to-end throughput of high-resolution 768x512 videos at 27.2 FPS on a single H100 GPU and providing a practical path toward stable digital humans.