Auditing Finite-Lag Statistics of Recurrent Representations

The gist The framework develops finite-lag operator geometry for recurrent representations by defining a directed empirical transport law on observed source–successor pairs to measure deterministic motion at a chosen lag.

How it works

The primitive is the conditional transport law Q∆(dy x), which is estimated by a dense Gaussian source-smoothing operator, Qb∆(dy xq) (Page 13). This directed finite-lag law allows for the derivation of key geometric objects: the source-centered transport tensor G∆(x) (Page 13), which decomposes exactly into conditional spread and coherent displacement, and an antisymmetric coordinate circulation Wρ∆ (Page 13).

The decomposition is formalized by Proposition 1, stating that whenever the conditional second moment exists, 2G∆(x) = C∆(x) + τ m∆(x)m∆(x) (Page 18). Here, C∆ measures conditional spread and τm∆m⊤ measures coherent displacement that would be removed by conditional-mean centering (Page 18). The total transport scale is given by tr(G¯ρ∆), which separates the two components as 2 tr(G¯ρ∆) = Ztr(C∆(x)) ρ(dx) conditional spread trace + τZm∆(x)2 ρ(dx) coherent displacement trace (Page 14).

Structural Properties

The framework establishes four structural properties for the finite-lag construction (Page 4). First, the tensorial observables are affine-covariant, meaning they transform predictably under affine maps ϕ(x) = Ax + b, with formulas like G′∆ = A G∆A⊤ (Page 21). Second, the dense Gaussian estimator is Lipschitz on bounded trajectory clouds at fixed bandwidth (Page 23). Third, finite-lag source-centered transport separates from infinitesimal carré du champ geometry by retaining deterministic displacement at the chosen lag (Page 5). Fourth, a linear-Gaussian model provides a closed form calibration for G¯∆ and W∆ (Page 27).

Key Analytical Results

Theorem 4 provides the linear-Gaussian closed form, where m∆(x) = τ−1(A∆ − I)x, C∆(x) = τ−1Σξ, G¯∆ = 1/2τΣξ + (A∆ − I)Σ(A∆ − I)⊤, and W∆ = τ−1ΣA⊤Δ − A∆Σ (Page 27). This form separates the mechanisms: innovation covariance Σξ contributes conditional spread, the update offset A∆ - I contributes coherent displacement, and the antisymmetric mismatch ΣA⊤Δ − A∆Σ contributes directed coordinate circulation (Page 27).

Experimental Validation

Controlled experiments validate the decomposition, circulation, affine covariance, and stability predictions (Page 5). In performance-matched repeat-copy networks, the framework reveals architecture-dependent differences in total transport scale and coherent displacement trace (Page 7). The most robust finite-lag differences in this case study are total transport scale and coherent displacement trace (Page 9). Furthermore, the phase-profile experiment localizes where finite-lag transport differences arise within a solved recurrent computation, showing that Elman has substantially larger transport and coherent displacement during write and recall phases (Page 38).

Finite Lag vs. Infinitesimal Geometry

The distinction between finite-lag geometry and infinitesimal carré du champ geometry is structural (Page 10). Theorem 3 shows that deterministic recurrent motion can be visible to G∆ even when the infinitesimal carré du champ of a first-order deterministic generator is zero (Page 6). The framework captures finite-step motion at the operating lag ∆, which is distinct from the limiting object in regime one where an exact diffusion description exists (Page 32).

Conclusion

Finite-lag operator geometry provides a trajectory-directed counterpart to static diffusion operator geometry by using a directed empirical transport law on observed source–successor pairs (Page 6). This approach is well-defined even when the hidden state alone is not autonomously Markov, making it suitable for general trained recurrent networks operating on integer time steps (Page 31).

--- Page 1 ---

Kanishka Reddy Department of Applied Mathematics University of Washington Seattle, WA 98105 kani@uw.edu Abstract Recurrent representations are trajectories, but representation geometry is often measured from static snapshots. We develop finite-lag operator geometry for recurrent hidden states from observed source–successor pairs (Xt, Xt+∆). The primitive is the conditional transport law Q∆(dy x), estimated by a dense Gaussian sourcesmoothing operator. From this directed finite-lag law we derive a source-centered transport tensor G∆, which decomposes exactly into conditional spread and coherent displacement, and an antisymmetric coordinate circulation Wρ∆, which summarizes directed lagged flow. We prove affine covariance with explicit metric dependence of scalar summaries, dense estimator stability on bounded trajectory clouds, and a finite-lag separation result showing that source-centered transport detects deterministic recurrent motion not recorded by infinitesimal carré-du-champ geometry. A linear-Gaussian closed form calibrates the quantities in terms of the update A∆, source covariance, and innovation covariance. Controlled experiments validate the decomposition, circulation, and stability predictions. In performance matched repeat-copy networks, the framework reveals architecture dependent differences in total transport scale and coherent displacement trace, while coherent displacement fraction is metric and resolution dependent.

--- Page 2 ---

Source-centering is the key choice. If m∆(x) = τ−1E[Y − X X = x], C∆(x) = τ−1 Cov(Y − X X = x), then 2G∆(x) = C∆(x) + τm∆(x)m∆(x)⊤. Thus finite-lag transport decomposes exactly into conditional spread and coherent displacement. Centering at the conditional mean would remove the second term, while source-centering keeps deterministic recurrent motion as part of the geometry. We also define an antisymmetric coordinate circulation Wρ∆, a lagged source–successor cross-moment that summarizes directed flow in representation coordinates. These objects are related to transfer-operator, Koopman, TICA, and VAMP methods [11–16], but the use is different. Those methods estimate predictive operators or extract slow modes. Here the finite-lag law is used geometrically, with its second moments describing transport scale, conditional spread, coherent displacement, and directed circulation in hidden-state space.

--- Page 3 ---

We define Q∆(dy x) = L(Xt+∆ ∈ dy Xt = x), where the law is induced jointly by the recurrent update and the sequence/input distribution. The empirical estimator smooths over nearby source states and transports to their attached successors. This gives a directed finite-lag operator even when the hidden state alone is not an autonomous Markov state.

--- Page 4 ---

2G∆(x) = 1/2τ Z(y − x)(y − x)⊤ Q∆(dy x), τ = ∆ dt. Preprint arXiv:2607.01746v1 [cs.LG] 2 Jul 2026

--- Page 5 ---

Source-centering is the key choice. If m∆(x) = τ−1E[Y − X X = x], C∆(x) = τ−1 Cov(Y − X X = x), then 2G∆(x) = C∆(x) + τm∆(x)m∆(x)⊤. Thus finite-lag transport decomposes exactly into conditional spread and coherent displacement. Centering at the conditional mean would remove the second term, while source-centering keeps deterministic recurrent motion as part of the geometry. We also define an antisymmetric coordinate circulation Wρ∆, a lagged source–successor cross-moment that summarizes directed flow in representation coordinates.

--- Page 6 ---

Contributions. We make four contributions. 1. We define finite-lag operator geometry for recurrent representations through the conditional transport law Q∆, the source-centered transport tensor G∆, its exact spread/displacement decomposition, and the coordinate circulation Wρ∆ (Page 6).

--- Page 7 ---

Table 1: Finite-lag observables derived from Q∆. Trace quantities depend on the chosen metric. Experiments report center-RMS Euclidean summaries unless stated otherwise. Quantity Definition Interpretation tr(G¯ρ∆) source-centered second moment total finite-lag transport scale R tr(C∆) dρ conditional covariance trace spread over successors / smoothing variation τR m∆2 dρ drift-energy trace coherent displacement at lag ∆ F ρ∆ coherent trace fraction attribution of transport to coherent motion∥Wρ∆F antisymmetric lagged cross-moment directed coordinate circulation G∆(x) is a positive semidefinite finite-lag quadratic form on coordinate functions. It is centered at the source state rather than at the conditional mean successor, so it retains coherent motion at the chosen lag.

--- Page 8 ---

Proposition 1 (Source-centered decomposition). Whenever the conditional second moment exists, 2G∆(x) = C∆(x) + τ m∆(x)m∆(x)⊤. The identity is the second-moment decomposition for D = Y − X. Its role is conceptual, C∆ measuring conditional spread, while τmDeltam⊤ measures coherent displacement that would be removed by conditional-mean centering. Averaging and taking traces gives 2 tr(G¯ρ∆) = Ztr(C∆(x)) ρ(dx) conditional spread trace + τZm∆(x)2 ρ(dx) coherent displacement trace (Page 8).

1. Bold header: Source-centered transport tensor G∆ for Recurrent Trajectories

The system can measure coherent displacement which would be removed by conditional-mean centering, allowing it to detect deterministic recurrent motion not recorded by infinitesimal carré-du-champ geometry.

2. Bold header: Coordinate Circulation Wρ∆ for Directed Flow Analysis

The AI system can summarize directed lagged flow via the antisymmetric coordinate circulation statistic, which distinguishes between isotropic contraction and oriented rotation.

3. Bold header: Metric-dependent Observables and Affine Covariance

The framework provides observables that are affine-covariant, meaning they transform predictably under affine maps, while scalar summaries require a metric choice, enabling analysis of how transport scale and coherent displacement change with coordinate systems.

4. Bold header: Dense Estimator Stability on Bounded Trajectory Clouds

The dense Gaussian source-smoothing estimator exhibits Lipschitz stability on bounded trajectory clouds at fixed bandwidth, ensuring that derived observables like Gb∆ are robust to small perturbations in the data cloud.

5. Bold header: Finite-Lag Detection of Deterministic Motion

The system can use the finite-lag tensor G¯ρ∆ to detect deterministic motion where it is otherwise invisible, as G∆(x) ̸= 0 whenever T∆(x) ≠ x even when the infinitesimal carré-du-champ geometry is zero.

6. Bold header: Closed-Form Calibration for Linear-Gaussian Models

For systems where the update is linear and innovation is additive Gaussian noise, the framework provides a linear-Gaussian closed form calibrates the quantities in terms of the update A∆, source covariance, and innovation covariance, allowing for analytic calibration of spread, coherent displacement, and circulation.

7. Bold header: Phase-Resolved Geometry for Temporal Dynamics

By splitting source–successor pairs into write, cue, delay, and recall phases, the system can localize where finite-lag transport differences arise within a solved recurrent computation.

8. Bold header: Robustness to Resolution Changes

The framework allows users to assess how geometric summaries behave across resolutions; for instance, Table 5 shows that the coherent displacement fraction is generally larger at narrower bandwidths, showing that it is a resolution-dependent scalar summary.

Hidden-state trajectories are computational traces that record how recurrent representations evolve as networks process sequences. When exact state repeats are unavailable, one way to estimate conditional motion is to borrow successors from nearby states, yielding a finite-bandwidth successor smoother. However, this procedure can change the displacement being measured. At a median-distance bandwidth, write-phase displacement energy is about 1.88 times the exact value for GRU and LSTM. Elman instead shows near agreement in total energy despite large, opposing finite-bandwidth contributions. We therefore compute the conditional-mean energy fraction exactly on repeat-copy. Training reverses the architecture ordering of this fraction, with GRU and LSTM rising to about 94% while Elman falls to about 55%. The centered remainder is small in energy but can still expose input information. For LSTM, linear four-bit decoding rises from 0.804 on total displacement to 1.000 after exact conditional centering.