AIM-ZO: Activation-Informed Subspace Maintenance for Zeroth-Order LLM Fine-Tuning
cs.LG
Submitted: 2026-09-28
Updated: 2026-09-28
Code: https://github.com/EkkoXy/AIM-ZO
Terminology
Sources
- Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO
- ZO-Act: Efficient Zeroth-Order Fine-Tuning via One-Shot Activation-Informed Low-Rank Subspaces
- Gradient Descent Happens in a Tiny Subspace
- Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
- AGZO: Activation-Guided Zeroth-Order Optimization for LLM Fine-Tuning
- CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks