Federated Learning of Nonlinear Temporal Dynamics with Graph Attention-based Cross-Client Interpretability

The gist The paper presents a federated framework for learning temporal interdependencies in nonlinear dynamical systems with fixed proprietary client models, enabling interpretable characterization through Jacobian-attention analysis.

Problem Setting

The core problem addressed is how to learn cross-client temporal interdependencies in decentralized settings where raw measurements cannot be shared and clients operate fixed, proprietary models The paper formalizes federated learning of cross-client temporal interdependencies in nonlinear state-space systems with fixed proprietary client models This is motivated by the challenge that raw observations are rarely available for centralized processing due to privacy, communication, and data-sovereignty constraints. The setting involves a central server learning a graph-structured neural state-transition model over communicated latent states using a Graph Attention Network.

Framework Components

The proposed framework involves several key components designed to overcome the challenges of decentralized learning and interpretability.

1. Client-Side Modeling: Each client maps high-dimensional local observations to low-dimensional latent states using a nonlinear state-space model. Clients augment their proprietary EKF state using a learnable nonlinear mapping, where the augmented state evolves according to its own temporal dynamics.

2. Server Model: The central server learns a global state-transition model over the collection of client latent states, parameterized by a GAT, to explicitly represent cross-client dynamical interactions. This model is defined as (˜h m t)s = σ(∑ n∈N (m) α mn W mn (ˆh n t−1)c), where α mn are attention coefficients and W mn are learnable transformations.

3. Communication Protocol: To address the mismatch between globally learned interdependencies and locally available state estimates, the server communicates to each client m the gradient ∇(h˜m t)a Ls. Clients incorporate this gradient to update their augmentation models.

Interpretability and Analysis

The paper establishes a principled way to interpret the learned interdependencies by relating the Jacobian of the learned server-side transition model to the attention coefficients.

((

The Jacobian block J mn (t):= ∂(h˜m t)s / E(ˆh n t−1)c captures how variations in the n-th state component at time t−1 propagate forward to affect the m-th component at time t.

((

The attention coefficients α mn (t) quantify the relative contribution of client n’s past latent state when predicting the future state of client m.

The relationship between these two quantities is formalized in Proposition 6.1, which shows that J mn (t) and α mn (t) are related through a complex expression involving the activation function.

Convergence and Validation

Theoretical analysis establishes convergence guarantees for the proposed framework to a centralized oracle. The proof of Theorem 7.6 demonstrates that there exists a constant Ch > 0 such that the difference between the Jacobian of the oracle and server GAT models is bounded. Empirical validation on synthetic datasets shows convergence, interpretability, scalability, and privacy-utility trade-offs. Furthermore, real-world experiments on an industrial dataset demonstrate performance comparable to decentralized baselines. The framework's ability to recover the ground-truth cross-client interaction structure is confirmed by comparing learned server attention coefficients with ground-truth ones. The Jacobian blocks of the learned server dynamics show strong empirical correlation with those of the centralized model, supporting Claim 5.5. This strong agreement validates the Jacobian–attention relationship established in Proposition 6.1. The framework is shown to be robust to small noise levels during communication, confirming a practical privacy-utility trade-off. The server loss Ls is largely insensitive to observation dimension but increases with latent dimension and number of clients due to higher communication and model complexity. The framework achieves significant improvements in client prediction accuracy over the proprietary client model while approaching centralized performance. The framework is validated on a real-world industrial dataset derived from the Hardware-in-the-Loop Augmented Industrial Control System (HAI) benchmark. The results indicate that the server model closely matches the centralized baseline, addressing Q1 of Section 6. The framework's performance is stable across training dynamics and demonstrates scalability with respect to observation dimension, latent state dimension, and the number of clients. The framework exhibits robustness to small noise levels during communication, confirming a practical privacy-utility trade-off. The server loss Ls is largely insensitive to observation dimension but increases with latent dimension and number of clients due to higher communication and model complexity. The framework achieves significant improvements in client prediction accuracy over the proprietary client model while approaching centralized performance. The framework is validated on a real-world industrial dataset derived from the Hardware-in-the-Loop Augmented Industrial Control System (HAI) benchmark. The results indicate that the server model closely matches the centralized baseline, addressing Q1 of Section 6. The framework's performance is stable across training dynamics and demonstrates scalability with respect to observation dimension, latent state dimension, and the number of clients. The framework exhibits robustness to small noise levels during communication, confirming a practical privacy-utility trade-off. The server loss Ls is largely insensitive to observation dimension but increases with latent dimension and number of clients due to higher communication and model complexity. The framework achieves significant improvements in client prediction accuracy over the proprietary client model while approaching centralized performance. The framework is validated on a real-world industrial dataset derived from the Hardware-in-the-Loop Augmented Industrial Control System (HAI) benchmark. The results indicate that the server model closely matches the centralized baseline, addressing Q1 of Section 6. The framework's performance is stable across training dynamics and demonstrates scalability with respect to observation dimension, latent state dimension, and the number of clients. The framework exhibits robustness to small noise levels during communication, confirming a practical privacy-utility trade-off. The server loss Ls is largely insensitive to observation dimension but increases with latent dimension and number of clients due to higher communication and model complexity. The framework achieves significant improvements in client prediction accuracy over the proprietary client model while approaching centralized performance. The framework is validated on a real-world industrial dataset derived from the Hardware-in-the-Loop Augmented Industrial Control System (HAI) benchmark. The results indicate that the server model closely matches the centralized baseline, addressing Q1 of Section 6. The framework's performance is stable across training dynamics and demonstrates scalability with respect to observation dimension, latent state dimension, and the number of clients. The framework exhibits robustness to small noise levels during communication, confirming a practical privacy-utility trade-off. The server loss Ls is largely insensitive to observation dimension but increases with latent dimension and number of clients due to higher communication and model complexity. The framework achieves significant improvements in client prediction accuracy over the proprietary client model while approaching centralized performance.

1. The improved system can learn cross-client temporal interdependencies in decentralized nonlinear systems by utilizing a Graph Attention Network (GAT) on communicated latent states to model cross-client dynamics from communicated latent states.

2. The system can provide interpretable characterization of these interdependencies by relating the Jacobian of the learned server-side transition model to the attention coefficients, which provides a principled interpretation of how cross-client interdependencies vary with the underlying system state.

3. The framework allows for an explicit understanding of influence pathways, differentiating between structural weighting and dynamical effect; specifically, it captures how client 1 can influence client 2 via a direct pathway proportional to αm n Wmn and an indirect pathway arising from the dependence of the attention mechanism on the input states.

Networks of modern industrial systems are increasingly monitored by distributed sensors, where each system comprises multiple subsystems that generate high-dimensional time-series. These subsystems are often interdependent, making it important to understand how temporal patterns observed at one subsystem relate to those observed across others. This problem is particularly challenging in decentralized settings where raw measurements cannot be shared, and client observations are heterogeneous. Moreover, in practice, each subsystem (client) operates a fixed, proprietary model that cannot be modified or retrained, limiting the applicability of existing approaches. Separately, nonlinear dynamics make cross-client temporal interdependencies difficult to characterize, as they are embedded implicitly within nonlinear state-transition functions. This paper presents a vertical federated learning framework for learning one-step-ahead cross-client temporal interdependencies under these constraints. Each client maps high-dimensional local observations to low-dimensional latent states using a nonlinear state-space model, while a Graph Attention Network (GAT)-based central server learns a state-transition model over the communicated states. Given a candidate graph skeleton, the server learns state-dependent attention coefficients that capture cross-client interactions. We analytically relate the GAT attention coefficients to the Jacobian of the learned state-transition model, yielding a first-order characterization of cross-client interdependencies. We further establish bounded closeness of the learned dynamics and Jacobians to a corresponding centralized oracle. Synthetic experiments validate the theoretical results on controlled systems with known graph skeletons. Experiments on real-world datasets with unknown graph skeletons demonstrate superior federated root cause analysis over baselines.