Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space
Yiwei Chen, Bingqi Shang, Sijia Liu
cs.AI, cs.LG
Submitted: 2026-08-07
Updated: 2026-08-11
Comments: Accepted to COLM 2026
Code: https://github.com/tatsu-lab/stanford_alpaca
License: http://creativecommons.org/licenses/by/4.0/
The gist: Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines, leading to intricate lineage relationships that reflect model origin, ownership, and
Terminology
Abstract
Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines, leading to intricate lineage relationships that reflect model origin, ownership, and evolution. Understanding these relationships is important for model provenance, governance, and supply-chain integrity. In this work, we investigate the notion of LLM "biometrics" (analogous to human biometrics) to ask whether LLMs exhibit intrinsic fingerprints in weight space alone, without access to input data, that reveal their origin and lineage. We formulate this as a lineage discrimination problem, distinguishing among independent-origin, same-series, and shared-base models. To characterize these relationships, we propose a unified geometric fingerprinting framework that analyzes weight matrices from two complementary perspectives: (i) spectral energy, captured by singular value distributions to encode global magnitude patterns, and (ii) subspace alignment, quantified via subspace deviations to capture directional geometry. Our analysis uncovers a clear hierarchy of structural similarity in weight space: spectral energy reliably distinguishes independently trained models and different model families, while subspace alignment enables fine-grained discrimination among closely related models, including variations in dataset scale and post-training procedures. Extensive experiments on over 110 diverse open-weight LLM pairs demonstrate that weight-space geometry provides a robust and interpretable signal for model lineage, enabling coarse-grained regime separation and fine-grained discrimination within shared-base models.
Sources
- Constitutional AI: Harmlessness from AI Feedback
- Detecting Stylistic Fingerprints of Large Language Models
- RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
- Gemma: Open Models Based on Gemini Research and Technology
- The Llama 3 Herd of Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
- Hide and Seek: Fingerprinting Large Language Models with Evolutionary Learning
- The Universal Weight Subspace Hypothesis
- Blackbox Model Provenance via Palimpsestic Membership Inference
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- Model Provenance Testing for Large Language Models
- Proximal Policy Optimization Algorithms
- Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
- SoK: Large Language Model Copyright Auditing via Fingerprinting
- LoRA vs Full Fine-tuning: An Illusion of Equivalence
- Idiosyncrasies in Large Language Models
- Natural Fingerprints of Large Language Models
- LLaMA: Open and Efficient Foundation Language Models
- RAFP: Identifying LLM Lineages via Rare-Region Fingerprints
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems