A Tropical Geometry View of Forgetting: A Per-Unit Projector for Knowledge-Preserving Fine-Tuning
cs.AI
Submitted: 2026-10-03
Updated: 2026-10-03
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Practical Gauss-Newton Optimisation for Deep Learning
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Gradient Descent Provably Optimizes Over-parameterized Neural Networks
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
- Orthogonal Gradient Descent for Continual Learning
- The Pile: An 800GB Dataset of Diverse Text for Language Modeling
- Gaussian Error Linear Units (GELUs)
- Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons
- Continual Learning with Node-Importance based Adaptive Group Sparse Regularization
- The MiniPile Challenge for Data-Efficient Language Models
- Balancing Stability and Plasticity through Advanced Null Space in Continual Learning
- TRGP: Trust Region Gradient Projection for Continual Learning
- Decoupled Weight Decay Regularization
- Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs
- Continual Learning with Scaled Gradient Projection
- Overcoming catastrophic forgetting with hard attention to the task
- GLU Variants Improve Transformer
- ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
- Put the Space of LoRA Initialization to the Extreme to Preserve Pre-trained Knowledge
- Architecture-Preserving Provable Repair of Deep Neural Networks
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems