Ghost tasking for parametrized Gaussian Processes solving linear differential equations

Ghost tasking for parametrized Gaussian Processes solving linear differential equations introduces a systematic procedure using auxiliary tasks to render any non-parametrizable system effectively parametrizable, enabling algorithmic construction of parametrized Gaussian Processes while keeping the number of required tasks and latent functions low The gist Ghost tasking for parametrized Gaussian Processes solving linear differential equations introduces a systematic procedure using auxiliary tasks to render any non-parametrizable system effectively parametrizable, enabling algorithmic construction of parametrized Gaussian Processes while keeping the number of required tasks and latent functions low.

How it works

The main idea of ghost tasking is to (1) embed an non-parametrizable system represented by its operator matrix in a higher-dimensional space by adding ghost tasks, (2) compute the now fully parametrizable null space of the embedded operator matrix, (3) apply the null space as a linear operator to parametrize a base kernel of a GP and (4) re-constrain the embedded parametrized GP to the original solution space by conditioning the ghost tasks to be point wise zero. This process is formalized through Theorem 3.2, which states that there exists a matrix Λ(x) with at most m = 2 columns such that the embedded system Pϕ(x) = Rϕ(x) −Λ(x) is stably free, and hence parametrizable over any sufficiently smooth function space. The construction depends on the residue class of Λ i e.g., Λ = Λ + RX yields the same result for any X in Dp x r.

Key Contributions

The work establishes that ghost tasking is the optimum on the spectrum between PIGP method [9] and NSB method [10], because it provides a method for rendering a physical system encoded in a Gaussian Process minimally parametrizable. The contributions include proving theoretical validity for all systems with full row-rank, and polynomial, rational, or generally meromorphic coefficients. Furthermore, the authors show improved accuracy over Raissi et al. [9] in low data settings for inverse problems in three experiments through systematic comparisons. The method is fully compatible with the already freely available python package PCGP [25] and includes tutorials for using ghost tasking with computer algebra programs OreModules [26] and Macaulay2 [27].

Theoretical Foundations

The algebraic analysis relies on studying modules over rings D, where D can be Weyl algebras An(R) or Bn(R) depending on the coefficients. The parametrization is studied by defining the left quotient row-module M = D 1×p / (D 1×qR), and its properties are characterized using concepts like stably free, projective, and torsion-free modules. Theorem 2.4 states that if M is torsion-free, there exists a non-trivial parametrization matrix Q in Dp x n such that imF(Q) = kerF(R). The distinction between the two formulations lies entirely in how the inhomogeneities f(x) are incorporated, with ghost tasking being able to achieve parametrizability while keeping tasks and latent functions at a minimum.

Experimental Validation

The method was tested in three experiments: an ODE system (tripendulum), a PDE with two variables (coupled transport), and anisotropic magnetostatics. In Experiment 1, the GT+ method showed strong improvements especially for the ratio of the parameters g/l in the measured case compared to PIGP. In Experiment 3 (magnetostatics), GT outperformed PIGP in every setting. The performance comparison shows that ghost tasking achieves performance comparable to PIGP on forward problems while requiring fewer training data and outperforms PIGP when using the same amount of training data on inverse problems.

Practical Considerations

Practically, finding a suitable generator Λ is algorithmic for holonomic systems but often relies on a heuristic ansatz that is verified computationally. Ghost tasking allows collocation points to be added freely without requiring measurements or real data, although this trades off accuracy against computation time, which scales cubically with the total amount of training data. The authors recommend using the weaker, torsion-free variant in practice due to computational overheads associated with the projective variant.

Conclusion

Ghost tasking is introduced as a method to extend parametrizability to physical systems that on their own are not parametrizable, while retaining a minimal latent function space by adding one, but at most two, additional (ghost) tasks. It is understood as the optimally data-efficient parametrization on the spectrum between PIGP and NSB. Ghost tasking achieves performance comparable to PIGP on forward problems while requiring fewer training data and outperforms PIGP when using the same amount of training data on inverse problems. It demonstrates that a substantially broader class of differential equation systems can be encoded more efficiently as priors in GPs without sacrificing predictive performance.

How it works

The paper introduces the structure of ghost tasking by showing how the PIGP construction of Raissi et al. [9] can be interpreted as a parametrization obtained by applying the matrix QPIGP(x) = Iq Lϕ(x)! to the base kernel kuu(x, x′). This demonstrates that RϕPIGP(x) has a right inverse, choosing SϕPIGP(x) = S 1 Lϕ(x)S 1 − Iq! for any S 1 in Dq x q. By point (3) of Lemma 2.3, the system presented by RϕPIGP(x) is projective and hence parametrizable over any sufficiently smooth function space.

Key Findings on Latent Functions

The comparison shows that Q ν0 GT+B effectively has column rank n = 1 in Experiment 3, while Q ν0 PIGP has column rank 3, indicating that a single latent function is sufficient to fit all training data across five tasks in the inverse problem. This contrasts with the PIGP method where three latent functions must be fit from training data across six tasks.

Performance Metrics

The performance is measured by RMSE on test points for the forward problem and MAP error relative to true values for the inverse problem. The results show that GT+ improves especially the g/l results up to 5 orders of magnitude in Experiment 1 when using known f(t). Overall, ghost tasking frequently attains several orders of magnitude higher accuracy than PIGP in inverse problems.

Comparison with Existing Methods

Ghost tasking is understood as the sweet spot in the trade-off between PIGP and NSB methods, preserving strong coupling while relaxing parametrizability constraints for general systems. It is theoretically applicable to a broad class of systems, including those with meromorphic coefficients, although practical implementation is mediated by the need to compute parametrizations via computer algebra programs. The method inherits limitations of current computer algebra implementations, which are largely restricted to polynomial or rational coefficients.

Future Directions

Future work may involve investigating parametrizations based on the Jacobson Normal Form (JNF) analogous to the Smith Normal Form employed in [11], and integrating ghost tasking with approaches that incorporate boundary conditions directly into the kernel. The authors suggest that extending parametrization-based methods to the wider class of weak formulations of differential operators remains an important direction for future research.

Acknowledgements

J.M. would like to thank A. Quadrat for helpful discussions and A. Leykin for providing the foundation of the Macaulay2 code snippets. J.M. and C.A. were financially supported by the Austrian Research Promotion Agency (FFG) project VENTUS within the AI for Green funding program, Grant no. 910263. S.R. was financially supported by the Austrian Science Fund (FWF), Grant DOI: 10.55776/J4774. The work has been carried out within the framework of the EUROfusion Consortium, funded by the European Union via the Euratom Research and Training Programme (Grant Agreement No 101052200 — EUROfusion).

Declaration

During the preparation of this work the authors used Claude (Anthropic) and GPT (OpenAI) in order to help with code development, brainstorming, and text polishing. The authors take full responsibility for the content of the published article. The work is subject to AI-assisted technologies in manuscript preparation processes.

References

[1] L. P. Swiler, M. Gulian, A. L. Frankel, C. Safta, J. D. Jakeman, A survey of constrained gaussian process regression: Approaches and implementation challenges, Journal of Machine Learning for Modeling and Computing 1 (2) (2020).

[9] M. Raissi, P. Perdikaris, G.

1. Bold header: Ghost tasking for non-parametrizable systems

This method renders any non-parametrizable system effectively parametrizable by embedding it in a higher dimension and then re-constraining the embedded parametrized GP to the original solution space by conditioning the ghost tasks to be point wise zero. This enables algorithmic construction of GPs for systems with non-constant coefficients, which is a limitation for existing methods.

2. Bold header: Improved data efficiency in inverse problems

Ghost tasking improves accuracy over Raissi (2017) in low-data inverse problems and can achieve several orders of magnitude higher accuracy compared to PIGP when learning physical parameters, particularly when the system dynamics are oscillatory. This is achieved because it results in a latent function space that is minimally parametrizable, leading to a sharply peaked, informative data-fit term.

3. Bold header: Enhanced performance in high-dimensional PDEs

For coupled transport systems (Experiment 2), ghost tasking allows for the construction of parametrization matrices that are either projective or torsion-free, offering a choice between guaranteed realizability and computational efficiency. The torsion-free version vastly outperform[s] PIGP in terms of convergence times when the full system is not parametrizable.