When Both Sides Adapt: The Challenges of Evaluating Interactive LLM Systems
stat.ME, cs.AI
Submitted: 2026-11-12
Updated: 2026-11-12
Terminology
Sources
- What's the Magic Word? A Control Theory of LLM Prompting
- Time series experiments and causal estimands: exact randomization tests and trading
- Design and Analysis of Switchback Experiments
- Operational Dosage: Implications of Capacity Constraints for the Design and Interpretation of Experiments
- Measuring Strategization in Recommendation: Users Adapt Their Behavior to Shape Future Content
- Emergent specialization from participation dynamics and multi-learner retraining
- Drift No More? Context Equilibria in Multi-Turn LLM Interactions
- SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants?
- The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World
- Strategic Classification
- Adopt not equal to Adapt: Longitudinal Analyses of LLM Conversations in the Wild
- Reducing Interference Bias in Online Marketplace Pricing Experiments
- Optimal Targeting in Dynamic Systems
- Switchback Experiments under Geometric Mixing
- Matching of Users and Creators in Two-Sided Markets with Departures
- LLMs Get Lost In Multi-Turn Conversation
- What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction
- Experimenting under Stochastic Congestion
- Novelty and Primacy: A Long-Term Estimator for Online Experiments
- The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance
Related papers
- Learning Robust Treatment Rules for Censored Data
- Neural Estimation of Summary Statistics for Simulation-Based Inference with Non-Asymptotic Guarantees
- Spectral divide-and-conquer MCMC for long stationary time series
- Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE
- Personalized Federated Vector Autoregression with Personalization Diversity
- Doubly Robust and Efficient Calibration of Prediction Sets for Right-Censored Time-to-Event Outcomes