Defense-in-Depth for LLMs: Evaluating Memory Gates Against Activation-Induced and Memory-Induced Sycophancy
cs.AI
Submitted: 2026-10-05
Updated: 2026-10-05
Comments: Accepted to NeurIPS (IAB, RTCA, AIWILD, and CL4FM)
Code: https://github.com/StellarDrift15/AlgoverseRRPR
License: http://creativecommons.org/licenses/by/4.0/
The gist: Long-term memory allows Large Language Models (LLMs) to maintain personalized context across interactions, but retrieved user history can induce memory-induced sycophancy, causing models to favor
Terminology
Abstract
Long-term memory allows Large Language Models (LLMs) to maintain personalized context across interactions, but retrieved user history can induce memory-induced sycophancy, causing models to favor stored user beliefs over objective evidence. Existing defenses primarily operate on retrieved context and are rarely evaluated jointly with internal behavioral bias. We introduce a 2 times 2 defense-in-depth framework separating internal activation steering from external memory handling. We extract sycophancy steering directions from 100 paired prompts and evaluate four open-weight models across 10 steering coefficients and five memory-defense configurations on MemSyco-Bench (answers for all 1,550 items; defense conditions judged on a fixed 250-item subsample), with three LLM judges. Three of the five configurations are new (rewriting every memory, a Router Gate that keeps, rewrites, or drops each memory, and dropping all memory); the other two are MemSyco's baselines. Selective Router Gate filtering preserves substantially more of MemSyco's average accuracy than complete memory removal, and this separation persists when the models are steered toward sycophancy. On Llama 3.1 8B with Router Gate, mild inverse steering (α= -1.5) lowers judge-averaged sycophancy from 35.80% to 31.32% while average accuracy moves from 43.99% to 43.31%; this reduction has the same direction under all three judges but is not statistically significant (paired p = 0.08 to 0.63 on 149 items). External memory filtering is the part of the design that holds up; our data do not show that inverse steering adds to it.
Sources
- PersistBench: When Should Long-Term Memories Be Forgotten by LLMs?
- MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
- Beyond Similarity: Trustworthy Memory Search for Personal AI Agents
- Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention
- Dissociating the Internal Representations of Sycophancy in LLMs
- $C$-$\Delta\Theta$: Circuit-Restricted Weight Arithmetic for Selective Refusal
- When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models
- Steering Llama 2 via Contrastive Activation Addition
- Towards Understanding Sycophancy in Language Models
- Discovering Language Model Behaviors with Model-Written Evaluations
- The Llama 3 Herd of Models
- Qwen2.5 Technical Report
- Mistral 7B
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems