Dimension-Free PAC-Bayes Generalization for Deep Neural Networks via Dirichlet Processes

Classical PAC-Bayesian generalization theory places prior distributions over Euclidean parameter space Θ R D, causing relative entropy to scale with network dimension D. For modern overparameterized neural networks where D n, this parameter-counting dependency yields vacuous generalization bounds (> 1, 000%). In this work, we resolve this barrier by shifting prior specification from internal parameters to the non-parametric space of data distributions M 1(Z) via a Dirichlet Process (DP), DP(α, F 0), inducing parameter distributions through push-forward measures under the learning algorithm. By the Data Processing Inequality, parameter-space relative entropy is strictly bounded by the data-space DP divergence, which we evaluate in exact closed form using Digamma functions. The resulting PAC-Bayesian certificate is strictly invariant to network parameter count D, width, and depth. On intrinsic d* -dimensional submanifolds, optimal partition scaling achieves the minimax non-parametric rate O(n-1/(d*+2) sqrt n). Across vision architectures scaling up to 4.2 million parameters (D = 7, 850 to D = 4, 203, 522) on the full MNIST benchmark, while classical weight-space bounds scale past 940% (becoming vacuous), our data-space DP-PAC-Bayes certificate remains stable, non-vacuous, and certified at 8.8% -- 9.4%.