Steal the Knowledge, Inherit the Mark: TwinMark for Distillation Watermarking via Second Moments and Class Multiplexing

arXiv:2609.19011 · cs.LG · Submitted 2026-07-18 · Read on arXiv

cs.LG

Submitted: 2026-07-18

Updated: 2026-09-26

Comments: 10 figures, 46 pages

Code: https://github.com/yaoyao-liu/mini-imagenet-tools

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Distillation — 4× in this paper · explained in 2 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Knowledge Distillation — 1× in this paper · explained in 11 episodes

Sources

• BlackMarks: Blackbox Multibit Watermarking for Deep Neural Networks
• Deep Linear Discriminant Analysis
• Distilling the Knowledge in a Neural Network
• Watermark-based Attribution of AI-Generated Content
• Piracy Resistant Watermarks for Deep Neural Networks
• HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Models
• Radioactive Watermarks in Diffusion and Autoregressive Image Generative Models
• FitNets: Hints for Thin Deep Nets
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Recipe for Watermarking Diffusion Models

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks