Watermark Robustness and Radioactivity May Be at Odds in Federated Learning
summary
The gist
Federated learning (FL) enables fine-tuning large language models (LLMs) across distributed data sources…
In short
The paper investigates if watermarks embedded in data used to train large language models (LLMs) remain detectable during federated learning (FL). While LLM watermarks are radioactive, meaning they can be detected even after fine-tuning, they are not robust against an active adversary. The adversary uses strong aggregation to filter out the watermark signal, showing a fundamental trade-off between detection and robustness in this setting.
Key concepts
- Federated Learning (FL)
- FL is a machine learning method where a global model is trained across many decentralized devices without sharing their raw data. In this context, clients train local copies of an LLM on their private data and send only the model updates back to a central server for averaging.
- LLM Watermarking
- This technique involves embedding secret statistical signals into text generated by a large language model. These signals are designed so that they can be mathematically detected later, proving that the specific LLM used in training was involved.
- Radioactivity
- Radioactivity refers to the inherent detectability of a signal. In this study, a watermark is radioactive if its statistical signal remains strong enough to be detected after the model has been fine-tuned using distributed updates, even when only a small percentage of the original data was watermarked.
- Active Adversary
- An active adversary is a malicious server in FL that actively tries to hide or destroy provenance tracking signals. This adversary aims to maintain high model utility while ensuring that the watermark signal is filtered out or obscured during the aggregation process.
Terminology used across episodes
This episode discusses
- Watermark Robustness and Radioactivity May Be at Odds in Federated Learning · Paper Radio
- How To Backdoor Federated Learning
- Towards an Accountable and Reproducible Federated Learning: A FactSheets Approach
- Analyzing Federated Learning through an Adversarial Lens
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
- LEAF: A Benchmark for Federated Settings
- Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
- Attacking Byzantine Robust Aggregation in High Dimensions
- Undetectable Watermarks for Language Models
- Recent Advances in Algorithmic High-Dimensional Robust Statistics
- Being Robust (in High Dimensions) Can Be Practical
- Robust Estimators in High Dimensions without the Computational Intractability
- Hierarchical Neural Story Generation
- GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick
- Robust and Heavy-Tailed Mean Estimation Made Simple, via Regret Minimization
- SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
- Adam: A Method for Stochastic Optimization
- A Watermark for Large Language Models
- Outlier-robust moment-estimation via sum-of-squares
- Robust Distortion-free Watermarks for Language Models
- Agnostic Estimation of Mean and Covariance
The paper
Watermark Robustness and Radioactivity May Be at Odds in Federated Learning · Read on arXiv
Georgia Institute of Technology
Federated learning (FL) enables fine-tuning large language models (LLMs) across distributed data sources. As these sources increasingly include LLM-generated text, provenance tracking becomes essential for accountability and transparency. We adapt LLM watermarking for FL data provenance, where a subset of clients computes local updates on watermarked data, and the server averages all updates into the global LLM. We find that several evaluated LLM watermarks remain radioactive in this setup: the watermark signal stays detectable after fine-tuning, with a p- value reaching 10-24 even when only 6.6% of data is watermarked. However, the server can act as an active adversary, aiming to preserve model utility while evading provenance tracking. We observe that updates from watermarked synthetic data appear as outliers relative to clean updates. Our adversary thus applies strong robust aggregation to filter out these outliers along with the watermark signal. All evaluated radioactive watermarks are not robust against such an adversarial server. Our work suggests fundamental trade-offs between radioactivity, robustness, and utility.
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "Watermark Robustness and Radioactivity May Be at Odds in Federated Learning".
Elias: Here's the gist of the paper. Federated learning (FL) enables fine-tuning large language models (LLMs) across distributed data sources, and this paper investigates how watermarking signals behave in this setting under an active adversary who attempts to evade provenance tracking.
Nadia: The key finding is that while LLM watermarks are radioactive in FL, they are not robust against strong Byzantine robust aggregation by an active server.
Elias: Large language models (LLMs) are increasingly used to generate synthetic datasets for fine-tuning, motivated by the high cost of collecting human annotations and expert knowledge.
Nadia: Federated learning (FL) enables fine-tuning across such distributed data sources by training models locally and aggregating updates into a global model.
Paper summary: Nadia: So, we're talking about "Watermark Robustness and Radioactivity May Be at Odds in Federated Learning" again. We've covered the basic idea that watermarks are radioactive, meaning they should be detectable even after fine-tuning, with a p-value hitting ten to the minus twenty-four for just six point six percent of data being marked <ref:2510.17033#pg1>.
Elias: And we also talked about the counterpoint: an active adversary can target these watermarks because they try to maintain model utility while evading provenance tracking, which means they’re trying to keep things looking clean.
Priya: The paper explains that this separation between clean and watermarked updates is very clear, and it uses something called t-SNE visualization of model updates from Maaten and Hinton in two thousand eight to show these distinct patterns <ref:2510.17033#pg2,t-SNE visualization of model updates>.
Nadia: And because the problem of watermark removal in FL simplifies down to robust aggregation, they propose a filtering algorithm that removes any model updates that fall outside the variance of the distribution of clean updates. This is how they test robustness.
Elias: The result is that none of the evaluated radioactive watermarks are robust against this specific filtering technique. They show that even when you increase epsilon to thirty point zero percent, you still don’t see any detectability for KGW+ and KTH+.
Priya: This tells us that the current methods used to embed these secret signals, like hashing in KGW+ or pre-defining random sequences in KTH+, aren't strong enough to resist this type of server manipulation.
Nadia: It suggests that we have a gap between the theoretical detectability and the practical robustness against an active adversary trying to preserve model utility.
Elias: So, when you look at the paper, it’s not just about whether a watermark exists, but whether it can survive the aggregation process in practice under attack.
Priya: It really highlights that we need to consider what the data actually shows before we trust the claimed result about provenance.
Conclusion: Nadia: So, wrapping up this discussion on "Watermark Robustness and Radioactivity May Be at Odds in Federated Learning," the authors show that LLM watermarks are radioactive in this federated learning setting, meaning you can detect synthetic data was used with high confidence.
Elias: But they also introduce a new threat model where an active adversary uses strong robust aggregators to filter out these radioactive watermarks, showing that robustness and radioactivity are at odds.
Priya: In simple terms, what this means is that while the signal is mathematically present, you have to fight against a server trying to keep the global model looking clean by filtering out the outliers.
Nadia: It opens up a new area for research into designing better watermarking techniques specifically tailored for federated learning environments where you expect adversarial servers.
Elias: I think that’s the core implication, focusing on finding schemes that are both detectable and resistant to these types of robust aggregation attacks.
More episodes
- 2608.27496-ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection
- Daily Summary for 2026-11-19
- 2607.07085-Is Randomness Necessary for Adaptive Data Analysis?
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2609.10601-Threshold Choice, Not Sample Size, Bounds Trustless Verification of Nondeterministic Compound AI Workflows
- 2610.38786-Adversarial Process Disturbances at a Deployable Alarm Budget: Detector Evaluation on a Simulator and a Real Plant
- 2511.08660-Binary and Multiclass Cyberattack Classification on GeNIS Dataset
- 2604.27302-Static Attribution of Android Residential Proxy Malware Using Graph Kernels
- Daily Summary for 2026-11-18