Hierarchical rank-evolving representation for physics-informed neural networks

The paper proposes a hierarchical rank-evolving (HRE) representation for multivariate functions and builds HRE-PINNs to address two key drawbacks of existing tensor-based PINNs: pre-specified low-rank tensor decompositions with fixed underlying structures and manually tuned ranks. The HRE representation adopts a hierarchical architecture. Its outer representation reduces the computational complexity of function evaluation from exponential to linear with respect to the dimensions, effectively alleviating the curse of dimensionality. The inner representation leverages a customized tensor network decomposition to capture the underlying structure of the target multivariate function. Equipped with learnable rank-evolving vectors and sparse regularization, the method automatically determines the suitable ranks and underlying structure during training, removing tedious manual hyperparameter tuning. Extensive experiments on various PDE benchmarks, including high-dimensional static problems (3D Helmholtz equation and 5D Poisson equation), nonlinear time-dependent problems ((2+1)D Klein-Gordon equation), and complex fluid-dynamics problems ((2+1)D flow mixing equation and (2+1)D Navier-Stokes equation), show HRE-PINN outperforms existing tensor-based PINNs across all error metrics. Ablation studies verify the indispensable performance gains brought by both the hierarchical design and rank-evolving mechanism. In summary, the HRE representation provides a novel function approximation paradigm. The resulting HRE-PINN offers an accurate, easy-to-tune alternative to existing tensor-based PINNs, with broad application prospects for high-dimensional PDE simulation across fluid dynamics, quantum mechanics, and engineering multi-physics modeling.

Improvements to AI systems:

1. Adaptive Rank Learning in Neural PDE Solvers: Implement a dynamic rank-evolving mechanism within neural network architectures that automatically grows or prunes tensor network ranks during training via sparse regularization, eliminating the need for manual hyperparameter tuning of model capacity.

2. Hierarchical Function Approximation for High-Dimensional Data: Replace fixed low-rank tensor decompositions with a hierarchical outer-inner representation that reduces computational complexity from exponential to linear in input dimensionality, enabling scalable modeling of functions in 10+ dimensions without pre-specified structural assumptions.

3. Structure-Agnostic Tensor Network Initialization: Design AI systems that start with a flexible, fully-connected tensor network and use learnable rank-evolving vectors to discover the optimal underlying topology (e.g., tree, loop, or chain) for a given target function, rather than relying on hand-crafted decompositions.

4. Self-Tuning Sparse Regularization for PDE Residual Minimization: Integrate a sparsity-inducing penalty on rank parameters directly into the loss function of physics-informed neural networks, allowing the system to balance approximation accuracy and model complexity autonomously during training.

What the improved AI system can do:

• Solve high-dimensional PDEs (e.g., 5D Poisson, 3D Helmholtz) with significantly lower error than existing tensor-based PINNs, while automatically selecting the appropriate tensor network structure and rank for each problem.

• Handle nonlinear time-dependent and fluid dynamics equations (e.g., Navier-Stokes, Klein-Gordon) without manual rank selection, reducing user expertise requirements and trial-and-error tuning.

• Scale to problems with hundreds of input dimensions by leveraging the hierarchical representation's linear complexity, making it feasible for quantum mechanics and multi-physics engineering simulations.

• Provide interpretable insights into the underlying low-rank structure of the solution manifold, enabling better generalization to unseen boundary/initial conditions and faster retraining for parameter sweeps.