Words & Weights: Streamlining Multi-Turn Interactions via Co-Adaptation
cs.AI, cs.LG
Submitted: 2026-03-02
Updated: 2026-09-27
License: http://creativecommons.org/licenses/by/4.0/
The gist: Test-time policy adaptation for multi-turn interactions (T2PAM) is essential for aligning Large Language Models (LLMs) with dynamic user needs during inference time.
Terminology
Abstract
Test-time policy adaptation for multi-turn interactions (T2PAM) is essential for aligning Large Language Models (LLMs) with dynamic user needs during inference time. However, existing paradigms commonly treat test-time adaptation as a single-axis problem, either purely refining instructions (Prompt Engineering) or only adjusting weights (Test-Time Training), ignoring that interaction failures stem from a coupled mix of ambiguity and incapacity. We argue that these two optimization paths are not merely additive but synergistic: semantic clarity acts as a pre-conditioner for effective parameter updates. To this end, we propose ROSA2, a framework that reformulates interaction as a joint optimization problem over the heterogeneous space of Words and Weights. By mathematically decomposing the error signal, ROSA2 utilizes textual gradients to rectify intent ambiguity and parameter updates to bridge capability gaps. Theoretically, we prove that this co-adaptation strictly reduces the required parameter shift for convergence. Empirically, ROSA2 outperforms state-of-the-art baselines by 30% on MATH while reducing interaction turns by 40%, demonstrating that refining the context unlocks the true potential of parameter updates.
Sources
- Prompt and Parameter Co-Optimization for Large Language Models
- Evaluating Large Language Models Trained on Code
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- LLMs Get Lost In Multi-Turn Conversation
- Feedback Descent: Open-Ended Text Optimization via Pairwise Comparison
- Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models
- Let's Verify Step by Step
- Training language models to follow instructions with human feedback
- UI-TARS: Pioneering Automated GUI Interaction with Native Agents
- Qwen2.5 Technical Report
- AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning
- Clarifying Ambiguities: on the Role of Ambiguity Types in Prompting Methods for Clarification Generation
- SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines
- OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
- Qwen3 Technical Report
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems