Learning Sparse Compositional Functions with Norm-Constrained Neural Networks
stat.ML, cs.LG
Submitted: 2026-05-25
Updated: 2026-11-12
Terminology
Sources
- GPT-4 Technical Report
- On Learning Gaussian Multi-index Models with Gradient Flow
- A Theory of Feature Learning in Kernel Models
- The Computational Advantage of Depth: Learning High-Dimensional Hierarchical Functions with Gradient Descent
- Position: A Theory of Deep Learning Must Include Compositional Sparsity
- Optimal scaling laws in learning hierarchical multi-index models
- Norm-based Generalization Bounds for Compositionally Sparse Neural Networks
- Learning Multi-Index Models with Hyper-Kernel Ridge Regression
- On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
- Optimal neural network approximation of smooth compositional functions on sets with low intrinsic dimension
- In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning
- On efficiently computable functions, deep networks and sparse compositionality
- Neural Networks With Dense Computational Graphs Are Not Universal Approximators
- Provable Learning of Random Hierarchy Models and Hierarchical Shallow-to-Deep Chaining
- Deep Network Approximation Characterized by Number of Neurons
- Gemini: A Family of Highly Capable Multimodal Models
- LLaMA: Open and Efficient Foundation Language Models
- Neural Networks Learn Generic Multi-Index Models Near Information-Theoretic Limit
- Understanding deep learning requires rethinking generalization
- Three Mechanisms of Weight Decay Regularization
Related papers
- Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
- Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
- Quantifying Error Propagation and Model Collapse in Diffusion Models
- Behavior of prediction performance metrics with rare events
- Statistical Inference and Learning for Shapley Additive Explanations (SHAP)
- Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference