An Educated Guess: Deriving Statistically Aligned Gradient Estimates for Zeroth-Order Optimisation
cs.LG, cs.AI
Submitted: 2026-11-05
Updated: 2026-11-05
Code: https://github.com/EfficientNeuralSystems/spectral-zeroth-order
Terminology
Sources
- Gradients without Backpropagation
- Online convex optimization in the bandit setting: gradient descent without a gradient
- The Forward-Forward Algorithm: Some Preliminary Investigations
- Adam: A Method for Stochastic Optimization
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
- How to guess a gradient
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis