Self-Play Enhancement via Advantage-Weighted Refinement in Online Federated LLM Fine-Tuning

Name: Self-Play Enhancement via Advantage-Weighted Refinement in Online Federated LLM Fine-Tuning
Uploaded: 2026-10-04T01:05:41.339497+00:00
Duration: 371 s
Description: SPEAR is an online learning algorithm for federated LLM fine-tuning that uses a feedback-guided self-play loop. It constructs contrastive pairs from user feedback to train the model efficiently without needing ground truth or expensive group generations. This method balances training on correct completions with penalizing incorrect ones based on confidence.

SPEAR is an online learning algorithm for federated LLM fine-tuning that uses a feedback-guided self-play loop. It constructs contrastive pairs from user feedback to train the model efficiently without needing ground truth or expensive group generations. This method balances training on correct completions with penalizing incorrect ones based on confidence.

Open the full episode