COBALT: Contextual Bandit Learning with Offline Trajectories for Multi-Turn Code Generation
COBALT uses contextual bandit learning to train large language models for multi-turn code generation by combining offline data collection with online, single-step optimization. It treats multi-turn coding as a recoverable process, allowing the model to learn efficiently from existing trajectories rather than expensive online reinforcement learning. Results show significant performance gains and improved robustness against misleading feedback.