Auditing Invisible Weight Updates with Reference Traces
cs.LG
Submitted: 2026-07-09
Updated: 2026-08-27
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Pretraining large language models with MXFP4 on Native FP4 Hardware
- Weight-Adjusted Gradients Reveal Parameter Importance and Failure Modes in LLMs
- DeepSeek-V3 Technical Report
- Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention
- dMX: Differentiable Mixed-Precision Assignment for Low-Precision Floating-Point Formats
- FlashOptim: Optimizers for Memory-Efficient Training
- Speeding up and reducing memory usage for scientific machine learning via mixed precision
- Model Compression with Exact Budget Constraints via Riemannian Manifolds
- FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training
- QuRL: Efficient Reinforcement Learning with Quantized Rollout
- M+Adam: Low-Precision Training via Additive-Multiplicative Optimization
- Budget-aware Auto Optimizer Configurator
- STQuant: Spatio-Temporal Adaptive Framework for Optimizer Quantization in Large Multimodal Model Training
- Decoupled Weight Decay Regularization
- Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL
- FP8 Formats for Deep Learning
- ECO: Quantized Training without Full-Precision Master Weights
- Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation
- Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Score
- Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks