Two ReLU Layers is All You Need
cs.CC, cs.CG, cs.LG
Submitted: 2026-10-20
Updated: 2026-10-20
License: http://creativecommons.org/licenses/by/4.0/
The gist: We prove that every ReLU network can be exactly simulated by a ReLU network with two hidden layers.
Abstract
We prove that every ReLU network can be exactly simulated by a ReLU network with two hidden layers. We present two proofs: one that is based on fillings in homological algebra and a direct combinatorial proof.
Related papers
- Parameterized Hardness of Zonotope Containment and Neural Network Verification
- Optimal Unambiguous DNFs and Alon-Saks-Seymour
- On the Limits of Quantum Multiparty Simultaneous Communication
- Learning, Compression, and Meta-Complexity
- Hardware-Algorithm Co-Optimization of Early-Exit Neural Networks for Multi-Core Edge Accelerators
- Quantum Fine-Grained Lower Bounds for SetDisjointness via Sub-Linear Reductions from 3SUM