From Recoverability to Functional Use: Certifying Temporal Reports in Time-Series Forecasting

The paper separates three questions for time-series models with known delay structure: can the true delay be recovered from the observed data, does the model report it, and does the forecast actually use the same history? The authors derive input-conditioned recoverability measures that separate intrinsic ambiguity from model error, prove that a delay report can become arbitrarily reliable while forecast risk approaches the oracle even though the predictor still uses the wrong lag, and show this failure appears in finite samples on the point-delay task: among forecasts with a correct delay report and normalized excess risk within 10% of the oracle, the reported history is functionally unused under their matched masking test in 55.4% of N-HiTS cases and 92.7% of TCN cases. The main conclusion is that a good forecast, even with a correct delay report, does not show that the model used the right history.

The paper studies three controlled mechanisms: P1 (point delay, Yt = αUt−d + ϵt), P2 (output memory, Yt = aYt−1 + bUt−d + ϵt), and P3 (finite kernel library, Yt = aYt−1 + Σ hkUt−k + ϵt). For P1, the KL divergence between two candidate delays is DKL(d∥d′u) = (α2/2σ2)∥Sdu − Sd′u∥22, and the exact binary Bayes error is Φ(−√(DKL/2)). For P2, a wrong delay can refit other parameters, so the authors define a profile separation score κARX = min Qd′/(nP2(σ⋆)2) that measures how much mismatch remains after nuisance refitting. For P3, kernel recovery depends on whether the realized input separates candidate mean responses, with worst-case separation γ(U,H) = min ∥U(h−h′)∥2/∥h−h′∥2.

The central theoretical result is Proposition 4.1: two structures can become arbitrarily reliably distinguishable at the trajectory level while using the wrong structure costs asymptotically nothing under average forecast MSE. The proof shows that structural evidence adds over the whole trajectory, whereas forecast MSE averages the mismatch over coordinates. A concrete choice Dn = nβ for any 0 < β < 1 makes structural decision error vanish while wrong-structure excess MSE is only 2σ2nβ−1.

The experiments mirror the three theoretical links. Table 1 shows that matched references are close to the favorable-region limit while learned structural readouts remain much worse. For example, in P2, MLP-NARX reaches forecast MSE.520 (close to N-HiTS.540) but its structural error is.784 and recoverable error is.783. Table 2 shows forecast and structural rankings disagree: TCN has lower structural error than N-HiTS while N-HiTS has lower forecast MSE in both P2 and P3, with all paired bootstrap intervals excluding zero.

The stronger test conditions on reporting and forecasting simultaneously. Table 3 shows that even when the report is correct and the forecast is near the oracle (NER ≤.10), the reported history is often unused: the conditional unused rate is.554 for N-HiTS and.927 for TCN. Table 4 shows this is insensitive to the NER cutoff, with N-HiTS staying near.53–.55 unused and TCN above.91 across four thresholds.

Routing is then tested as a mechanism fix. The paper proves Proposition 4.2: if prediction is forced through Fstr(u,v;r) = G(v, r, g(r)⊙u) with no other function of the historical block, then changing coordinates with gate entry gk(r) = 0 cannot change the forecast. Corollary 4.3 gives exact point-delay alignment under nondegenerate one-hot routing: if g(r) = er and the selected coordinate has nonzero intervention response, every off-report response is zero and the unique response peak is r. Table 5 shows hard one-hot routing removes the measured fixed report–use gap from 9.846/13.791 lags to zero for N-HiTS/TCN. Table 6 verifies the theorem numerically: every fixed-report off-support response is zero to 10−7, every selected response is nonzero, and every response peak equals the report. Table 7 shows cross-task empirical coupling reduces the report–use gap by about 9.8–13.1 lags across P1–P3 with all intervals excluding zero. Table 8 shows the routing effect transfers to real-input semi-synthetic tests with Beijing air-quality and hydraulic input histories, with all 12 backbone–task–source contrasts favoring coupling.

The paper concludes that whether this mismatch matters depends on the role of the report. If the only goal is in-distribution forecast MSE, exploiting a correlated proxy lag may be harmless. The issue arises when a reported delay is interpreted as evidence about the process or used for sensor selection, diagnosis, or control. The scope remains narrow: P2 uses profile separation rather than a composite Bayes oracle, the exact P3 calibration uses the a = 0 special case, the semi-synthetic tests retain controlled output mechanisms, and the interventions measure predictor sensitivity rather than real-world causality.

Improvements to AI systems:

1. Add a functional use verification layer to any time-series model that outputs a delay or lag report. After training, run a matched masking test: zero out the reported lag’s input coordinates and measure forecast change. If the change is below a threshold (e.g., 1% of oracle risk), flag the report as not functionally used. This prevents models from reporting a delay that is statistically correct but causally inert.

2. Implement hard one-hot routing for delay-sensitive architectures (e.g., N-HiTS, TCN, MLP-NARX). Modify the model so that the forecast is forced through a gate: F str(u,v;r) = G(v, r, g(r) u), where g(r) = e r (one-hot). This guarantees that off-report lags have zero influence, eliminating the report–use gap. The improved system can then be trusted for sensor selection, control, and diagnosis because the reported delay is provably the only history used.

3. Add a profile separation score (κ ARX) to autoregressive models with exogenous inputs. Before deployment, compute the minimal residual variance after refitting nuisance parameters for each candidate delay. If κ ARX is low, the model cannot distinguish delays even if forecast MSE is good—so the system should abstain from reporting a delay or explicitly warn that the delay is unidentifiable.

4. Use the input-conditioned recoverability measure (D KL or γ(U,H)) as a training regularizer. During training, penalize models that achieve low forecast MSE but high structural error (i.e., mismatch between reported and used lag). This pushes the model toward using the true delay when the input signal is sufficiently informative, rather than exploiting correlated proxies.

5. Build a report–use consistency audit into model selection pipelines. When comparing architectures (e.g., N-HiTS vs. TCN), do not rely on forecast MSE alone. Compute both forecast risk and structural error (e.g., via the matched masking test). Select models only if they pass a joint criterion: NER ≤ 10% AND functional use of the reported lag ≥ 90%. This avoids deploying models like the 92.7% unused-rate TCN.

6. For control or diagnosis tasks, replace standard delay reports with routed forecasts. Use the Proposition 4.2 architecture so that the reported delay directly determines the input mask. The improved system can then output a delay that is both accurate (matches true process) and actionable (changing the report changes the forecast in the intended way), verified by the zero off-support response property.

What the improved AI system can do:

• Reliably report causal delays in time-series forecasting, with a guarantee that the reported lag is the one actually driving predictions—not a correlated proxy.

• Detect and flag unidentifiable delays (e.g., in P2 scenarios) where nuisance refitting masks the true lag, preventing false confidence in downstream decisions.

• Achieve zero report–use gap in point-delay tasks, enabling safe use of delay reports for sensor placement, anomaly diagnosis, and control loops where wrong-lag usage could cause instability.

• Provide interpretable forecasts where users can verify that masking the reported lag changes the output, and masking any other lag does not—making the model auditable by non-experts.

• Improve model selection by rejecting architectures that look good on MSE but fail structural tests, reducing deployment of brittle models in high-stakes settings.