RAMAC: Multimodal Risk-Aware Offline Reinforcement Learning and the Role of Behavior Regularization
RAMAC introduces a simple, model-free framework for offline reinforcement learning that balances high expressiveness with safety. It combines an expressive generative actor (like diffusion) with a distributional critic and optimizes a composite objective mixing behavior cloning and Conditional Value-at-Risk (CVaR). This allows policies to learn from fixed data while explicitly controlling catastrophic lower-tail risks.