Manifold-Constrained Initial Noise Optimization for Efficient Generative Model Alignment
cs.LG, cs.AI, cs.CV
Submitted: 2026-09-30
Updated: 2026-09-30
Comments: 25 pages, 13 figures
Code: https://github.com/christophschuhmann/improved-aesthetic-predictor
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent advances in distillation and flow-map models have enabled deterministic one- or few-step generation for high-quality data, facilitating a new branch of reward alignment approaches that
Terminology
Abstract
Recent advances in distillation and flow-map models have enabled deterministic one- or few-step generation for high-quality data, facilitating a new branch of reward alignment approaches that directly optimize the initial noise from a Gaussian distribution. However, most existing initial-noise optimization methods rely on first-order gradient information, which is either inapplicable or suffers from instability and inefficiency in black-box reward scenarios. Here, we introduce ZeNOVA, a stable and efficient initial noise alignment method in a gradient-free manner. Specifically, we address existing algorithms' major challenge in black-box scenarios through annealed soft-value guidance, manifold-constrained hyperspherical Langevin dynamics, and Metropolis-Hastings jumping. Extensive experiments on image and video generative models show that ZeNOVA outperforms all evaluated zeroth-order baselines by optimizing the initial noise toward higher rewards substantially more stably while exploiting the geometry of the Gaussian prior, demonstrating its practical applicability to various black-box reward alignment.
Sources
- PaddleOCR 3.0 Technical Report
- Mean Flows for One-step Generative Modeling
- Manifold Preserving Guided Diffusion
- Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion
- Gradient-Free Noise Optimization for Reward Alignment in Generative Models
- Manifold-Constrained Noise Optimization for Diverse Diffusion Sampling
- Consistency Models
- Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review
- Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Control
- Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis
- DanceGRPO: Unleashing GRPO on Visual Generation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks