When Unlearning Fails: Reliable Data Deletion under Post-Training in Agent Networks

The gist: MUTE proposes a method to reliably delete data from self-improving federated agent networks by tracing downstream influence, removing model and data residue, and auditing future regeneration under an uplink budget.

How it works

MUTE is designed to address the problem that unlearning fails in self-improving networks because the influence echo of forgotten data can revive behavior during continued operation The core idea is to remove both current model residue and future data-side echo.

1. Influence Provenance involves replays a lightweight server ledger to estimate how the forget data’s influence propagates through aggregation and policy-driven collection without moving raw trajectories off clients. This is done by setting up a lightweight ledger L[t] that records the deployed model version, participating clients, aggregation weights, and deployment windows. The influence spreads through two edges: the aggregation edge and the collection edge.

How it works (continued)

The influence score for a trajectory x is calculated as γ(x) = η γ[tx], where η captures how strongly the deployed policy shapes data collection. This score is useful because the version influence γ[t] decays on its own, and the factor η is an upper bound rather than a fitted constant. The current influence state x[t] summarizes model-side and data-side influence across all clients, allowing the scheduler to track whether influence is decaying over time.

How it works (continued)

Influence-Aware Erasure removes the current residue from the model and blocks future residue from retained trajectories. For model-side erasure, a client holding Df updates its adapter with a forget-retain objective L = L forgetD f; θ, θ ref + λ L retainD r; θ, θ ref. This update uses negative preference optimization (NPO) to suppress behavior on forget trajectories and projects later updates off the dominant forget-gradient subspace to avoid re-fitting the erased direction.

How it works (continued)

For data-side containment, each client grades its trajectories by γ(x) and quarantines or down-weights high-influence retained trajectories based on thresholds τ q and τ d. Quarantined trajectories stay on the client until their score drops below τ q, at which point they can be released once the predicted residue meets the target.

How it works (continued)

Behavioral Audit and Deletion Scheduling sustain deletion through behavioral audit and scheduling. The server audits the behavioral leakage index (BLI) and the influence regeneration rate (IRR) to decide on per-client erasure and containment targets under an uplink budget m. These metrics are defined as BLI(θ) = Area under the ROC curve of a membership inference attack that distinguishes forget data from behavioral traces and IRR measures whether high-influence retained data can revive the forgotten behavior after continued learning.

How it works (continued)

The feasibility of sustained unlearning follows from the condition that the closed-loop transition matrix M∞ under a long-run schedule satisfies ρ(M∞) < 1, which ensures influence decays over time. The scheduler selects the lowest communication schedule that meets deadline and utility constraints while keeping this stability condition, applying actions within the uplink budget m.

How it works (continued)

The method is validated on LIBERO with two VLA backbones and three deletion granularities, showing that MUTE keeps behavioral leakage and influence regeneration low while preserving task utility and using much less communication than full retraining. Experiments on a physical Jetson-based edge testbed show that MUTE keeps behavioral leakage and influence regeneration low while preserving task utility and using much less communication than full retraining.

How it works (continued)

The performance metrics include Success Rate (SR) which measures task utility, Behavioral Leakage Index (BLI) which measures current leakage from the forget data, Influence Regeneration Rate (IRR) which measures how much deleted behavior returns after continued learning, and Communication Cost (Comm). MUTE lowers IRR against retraining in all settings, keeps BLI closer to the chance level, and holds SR within 0.04. The gain that matters for the network is Comm.

How it works (continued)

The sensitivity analysis shows that a looser quarantine threshold τ q keeps more high-influence trajectories, so IRR and BLI rise while SR barely moves. A larger uplink budget m buys more cleaning uploads, so IRR and BLI fall. The calibration η works best at its fitted value; IRR and BLI form a shallow valley near η = 0.6 and grow when η is set too low or too high. A smaller Dirichlet concentration α makes the local data more heterogeneous, which raises IRR and BLI and lowers SR. The two backbones stay close, with π0 a little lower and flatter on average.

How it works (continued)

MUTE addresses the problem that one-time unlearning is insufficient because the forget data can leave an influence echo in later retained trajectories and revive forgotten behavior during continued operation. MUTE addresses this problem by tracing downstream influence from server-side records, erasing model-side residue, containing high-influence retained trajectories, and auditing later regeneration under an uplink budget. The experiments in simulation and on a physical edge testbed show that MUTE suppresses behavioral leakage and influence regeneration while preserving utility with much less communication than retraining.

How it works (continued)

The paper formulates deletion over the network by minimizing the communication cost of deleting Df while preserving utility, meeting the response deadline, and preventing regeneration under constraints (C1) through (C4). This formulation is designed to ensure that deletion remains valid after continued learning. The objective is to find actions a[t] that minimize communication cost while keeping BLI ≤ 0.5 + ϵ, U(θ[t]) ≥ (1 − δ)U0, τ ≤ τmax, and IRR ≤ ϱ.

How it works (continued)

The paper concludes that MUTE addresses the problem that one-time unlearning is insufficient because the forget data can leave an influence echo in later retained trajectories and revive forgotten behavior during continued operation. MUTE addresses this problem by tracing downstream influence from server-side records, erasing model-side residue, containing high-influence retained trajectories, and auditing later regeneration under an uplink budget. The experiments in simulation and on a physical edge testbed show that MUTE suppresses behavioral leakage and influence regeneration while preserving utility with much less communication than retraining.

How it works (continued)

The paper concludes that MUTE addresses the problem that one-time unlearning is insufficient because the forget data can leave an influence echo in later retained trajectories and revive forgotten behavior during continued operation.

1. textbfInfluence Tracing via Server Ledger (Influence Provenance): A Lightweight Ledger for Policy-Driven Influence Tracking. The system can trace how the forget data’s influence propagates through aggregation and policy-driven collection by replays a lightweight server ledger to estimate influence propagation, allowing the system to know where the influence has propagated before deletion even occurs.

2. textbfInfluence-Aware Erasure: Dual-Layer Residue Removal. The improved system can remove both model and data residues by applying a forget-retain objective on the adapter for model erasure and quarantining or down-weighting high-influence retained trajectories for data erasure, as described by the loss function L = L forgetD f; θ, θ ref + λ L retainD r.

3. textbfBehavioral Audit and Dynamic Scheduling: Sustained Deletion via Uplink Budget. The system can sustain deletion over network operation by auditing behavioral leakage and influence regeneration using metrics like the behavioral leakage index (BLI) and influence regeneration rate (IRR), then scheduling actions based on the constraint: "Set per-client erasure and containment targets that keep ρ(M∞) < 1."

4. textbfResource-Constrained Adaptive Communication: Optimized Deletion Overhead. The system can manage the communication cost of deletion by selecting actions under an uplink budget 'm' to minimize traffic, ensuring that a lower Comm means that deletion consumes less network bandwidth, which is crucial for edge agents.

Embedded AI agents on edge devices can learn as a federation and keep improving after deployment. Each agent collects new trajectories with the current policy, and the edge server aggregates them in later rounds. In such self-improving agent networks, deleting data is harder than a one-time model repair. The data to be deleted may have already shaped later retained trajectories, so retraining or model-side unlearning leaves an influence echo that returns as the network keeps learning. We propose Muting Unlearned Trajectories' Echoes (MUTE) for reliable deletion over edge networks. MUTE traces the echo from a lightweight ledger at the edge server, without moving raw trajectories off the devices. It erases the current residue with a forget-retain update, contains high-influence retained trajectories, and audits later behavior to schedule further erasure under the uplink budget and the response deadline. On LIBERO, MUTE keeps the regenerated share of forgotten behavior at or below 0.401 for every request type, against at least 0.466 for five recent methods.