Discretizing Continuous Time Series for Imputation with Masked Diffusion Training

arXiv:2608.19119 · cs.LG, cs.AI · Submitted 2026-08-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training".

Jane: Here's the gist of the paper. Time series imputation is addressed by proposing the Masked Diffusion Time-series Imputation Model (MDTIM), which bridges discrete masked diffusion and continuous time series modeling The MDTIM framework leverages the training paradigm of masked diffusion model for imputation tasks Addressing Representation Limitations Existing approaches suffer from two main limitations: missing and observed values are embedded within the same representation space without explicit structural separation Furthermore, continuous diffusion-based methods are trained to predict added noise rather than the original signal The

MASK: token is structurally orthogonal to valid observations, which provides a clearer separation between missing and observed states in the representation This structural orthogonality allows the model to directly predict the original values rather than the noise, more closely matching the imputation goal.

Tom: Bridging Discrete and Continuous Spaces The fundamental challenge is bridging discrete masked diffusion and continuous time series modeling because time series data are inherently continuous and ordinal, whereas language tokens are discrete and categorical To solve this, the authors introduce Stochastic Discretization to map continuous values to ordinal-aware tokens while preserving continuous dynamics They also introduce Ordinal-Aware Soft Labeling to capture the ordinal relationships between tokens.

Jane: Training Signal Construction The training objective is formulated as a dual-domain objective combining an Ordinal-Aware Soft Cross-Entropy and a Spectral Consistency Loss The Ordinal-Aware Masked Diffusion loss decomposes into (Equation 6) L = (1 − ϵ)LNELBO + ϵ DKL(u∥pθ) + ϵ H(u) const, which encourages the predicted distribution to align with the ordinal neighborhood of y.

Tom: Additionally, a Spectral Consistency Loss is incorporated to enforce global temporal coherence by minimizing the L1 distance between Fourier representations of the reconstructed and ground-truth signals.

Title and authors: Tom: So we're looking at this paper called "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training," and the authors are Dongbin Kim, Seungyun Lee, Geonwoo Shin, and Jaewook Lee from Seoul National University.

Jane: It sounds like they are trying to solve a really tough problem in time series analysis where you have continuous data but you're using tools that are built for discrete things.

Lu: Basically, the main idea is to take the training method from masked diffusion models and adapt it for continuous time series imputation, which is pretty novel because those two worlds don't usually mix well.

Tom: So what's the core problem they are tackling here, Jane? What makes this approach different from just using a standard diffusion model on time series data?

Jane: Well, the paper points out that existing methods have two big flaws: first, missing and observed values are in the same representation without any clear separation.

Tom: And second, continuous diffusion models usually try to predict some added noise instead of actually figuring out what the original signal was.

Lu: Exactly. They propose a model called the Masked Diffusion Time-series Imputation Model or MDTIM, which they say leverages that masked diffusion training paradigm specifically for imputation tasks.

Jane: And they make a key structural argument about how their approach fixes the representation issue by using a special token called the mask token which is structurally orthogonal to any valid observation.

Meng: Orthogonal to observations—that sounds cleaner than just mixing everything together—but what does that actually mean for the prediction part?

Tom: Because if it's truly orthogonal, can the model directly predict the original values instead of predicting noise?

Lu: Yes, that's the goal. They say this structural orthogonality allows the model to directly predict those original values rather than just noise, which is what we want for imputation. It’s more closely aligned with what imputation actually requires.

Jane: But how do you handle the fact that time series data are continuous and ordinal, while language tokens, which diffusion models usually use, are discrete?

Tom: That's where they introduce Stochastic Discretization to map those continuous values into ordinal-aware tokens while still keeping the continuous dynamics intact.

Jane: And they also add something called Ordinal-Aware Soft Labeling to capture those important relationships between the tokens themselves.

Lu: The training objective is quite specific here, it's a dual-domain objective combining an Ordinal-Aware Soft Cross Entropy and a Spectral Consistency Loss.

Tom: That spectral consistency loss is interesting because they use it to enforce global temporal coherence by minimizing the L1 distance between Fourier representations of the reconstructed signal and the ground truth signal.

Title and authors: Jane: It sounds like they are trying to ensure that when you reconstruct a sequence, not only does it look right locally, but it also maintains a consistent pattern across its entire length in time.

Tom: And they do that by using this dual objective, which encourages the predicted distribution to align with the ordinal neighborhood of the target values.

Meng: From my side as someone who has to worry about deployment, does this complexity translate into a manageable training process?

Lu: They state that the Ordinal-Aware Masked Diffusion loss decomposes into one part involving L NELBO plus some terms with epsilon, which encourages the predicted distribution to align with the ordinal neighborhood of the target values.

Jane: And they say this decomposition shows that for epsilon greater than zero, that additional term penalizes any probability mass placed outside of the ordinal neighborhood of the true value.

Tom: That’s a concrete way they are enforcing those ordinal relationships we talked about earlier.

Lu: They also incorporate a Spectral Consistency Loss to enforce global temporal coherence by minimizing the L1 distance between Fourier representations of the reconstructed and ground truth signals, which is what they call L FFT.

Jane: So, in short, they are taking a masked diffusion model and adding tools to handle continuous ordinal data so it can accurately reconstruct those time series.

Tom: And I see a lot of promise there because if you can get that structural separation right between missing and observed states, you bypass some of the typical noise prediction issues inherent in these models.

Jane: But let's look at how they actually train this thing, because the training setup seems quite involved.

Lu: The training objective is formulated as a dual-domain objective combining an Ordinal-Aware Soft Cross Entropy and a Spectral Consistency Loss, which they call L total equals L diff plus lambda L FFT.

Tom: And they set the spectral loss weight lambda to one point zero for their main experiments on the datasets like Energy, ETTh, and Weather.

Meng: If we're talking about running this at scale, how does this training setup affect computational cost?

Lu: They show that it's parameter efficient; the zero point six five million small variant is smaller than the eight eight point two four million large SAITS variant <ref:2608.19119#pg2>.

Jane: That’s a significant reduction in size, which is good for deployment, especially when we compare it to some of the larger continuous diffusion baselines.

Title and authors: Tom: And on inference speed, they show that the Small MDTIM variant completes inference in about four point zero five seconds, which is much faster than the continuous diffusion baseline which takes up to eight hundred ninety four seconds at large scale.

Jane: That difference in time is huge for real-time applications, so that's a big win for practical use of this model.

Lu: They also show that the performance advantage of MDTIM gets bigger as the sequence length increases, which suggests it manages to capture global context without losing information through attention dilution.

Tom: That’s what we want when dealing with long-term dependencies in time series.

Jane: The paper shows that they used Expectation-based Unmasking to recover the continuous values by computing the probability weighted expectation of the predicted token distribution across multiple independent noise samples.

Tom: So, how does that help with uncertainty?

Lu: They use a Continuous Ranked Probability Score or CRPS for probabilistic settings to assess both calibration and sharpness, which rewards both aspects.

Meng: I need to know about the robustness here. The results show it performs well under geometric masking where most other baselines suffer significant degradation.

Jane: That's important because real-world data often has those kinds of missing patterns, and this model seems pretty resilient to them compared to other methods.

Lu: And they have an ablation study showing that Expectation-based Unmasking consistently reduces error across all datasets, which helps mitigate the quantization error that comes from using the Argmax selection method.

Jane: But what about the Ordinal-Aware Soft Labeling part? Does it always help?

Meng: The contribution of Soft Labeling scales with how regular the underlying data is; it's most effective when there's clear ordinal or periodic structure in the signal.

Lu: They found that on the Sine dataset, where there is known periodicity, Soft Labeling yields substantial additional gain.

Jane: But on datasets like Energy, it didn't offer any further improvement beyond just using Expectation-based Unmasking.

Tom: And they found an optimal vocabulary size of four zero for the best balance between how detailed the discretization is and how complex the classification task becomes.

Jane: So, to summarize what we have on this paper, it successfully adapts masked diffusion training to continuous time series using stochastic discretization and ordinal-aware soft labeling.

Lu: It consistently outperforms both deterministic and generative baselines in reconstruction accuracy across different missing scenarios.

Tom: And when you look at downstream tasks, like forecasting on the Energy dataset, MDTIM reduces the mean absolute error by nineteen point zero percent to twenty six point zero percent compared to SAITS, and by sixteen point zero percent to nineteen point zero percent compared to CSDI.

Title and authors: Jane: That propagation of accuracy is telling; it means that how well this model reconstructs the data directly translates into how much better you can forecast things down the line on things like energy usage.

Meng: I'm still thinking about that PhysioNet two thousand twelve dataset result, where MDTIM achieved the lowest mean absolute error across all settings, outperforming SAITS by a clear margin.

Tom: That’s a strong claim because it shows this approach generalizes well to real-world sparse and irregular observations when you compare it to traditional frequency domain priors which they say become unreliable for those signals.

Lu: But the paper does have some limitations, and one is that the spectral consistency regularization assumes the presence of stable frequency components, so they disabled that part on benchmarks dominated by irregular sampling like PhysioNet A data.

Jane: That's a fair limitation to note; it means the method relies on certain signal characteristics to work best.

Tom: And they point out another future direction: designing a data adaptive discretization scheme that automatically selects the resolution of the vocabulary size K based on the range and granularity of the underlying signal, since they found K=forty was optimal for their tests <ref:2608.19119#pg2>.

Jane: And extending this to extremely long sequences or non-stationary domains like financial tick data is definitely where future work needs to go.

Lu: To wrap up, this paper, "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training," successfully adapts the masked diffusion training paradigm to continuous time series imputation by introducing stochastic discretization and ordinal-aware soft labeling.

Tom: It consistently outperforms both deterministic and generative baselines in reconstruction accuracy and robustness across diverse missing scenarios.

Jane: It’s a solid piece of work because it shows that the way we structure the model, by making the structural separation between valid observations and missing placeholders explicit, provides a real benefit beyond just using masked diffusion training.

Meng: And when you look at inference time, achieving four point zero five seconds on small models is much faster than some continuous diffusion baselines which take up to eight hundred ninety four seconds.

Tom: And the results show that this imputation accuracy translates directly into downstream forecasting performance, reducing the mean absolute error by nineteen point zero percent to twenty six point zero percent on energy forecasts compared to SAITS and CSDI.

Jane: So, we're looking at a model that handles continuous ordinal data in a way that is both computationally feasible and actually improves performance on real-world time series.

The paper's summary: Tom: Essentially, they introduced this Masked Diffusion Time-series Imputation Model or MDTIM. Jane It’s an attempt to bridge the gap between discrete diffusion and continuous time series modeling by using the masked diffusion training paradigm for imputation tasks.

Lu: The core idea is fixing two big problems in existing methods—first, missing and observed values are mixed up in the same representation space without any clear separation. Tom And second, continuous diffusion models usually try to predict some added noise instead of figuring out what the original signal actually was.

Jane: So they solve that by adding a special mask token that is structurally different from any valid observation, which lets the model directly predict the original values instead of just predicting noise. Lu That’s a nice structural trick for cleaning up the representation.

Meng: But how do you handle continuous time series data? They have to deal with them being continuous and ordinal, while diffusion models usually work with discrete tokens. Tom That’s where they introduce Stochastic Discretization to map those continuous values into tokens that respect the ordinal relationships, while still keeping the underlying dynamics smooth.

Jane: And they also layer on this Ordinal-Aware Soft Labeling to make sure those ordinal relationships between the tokens themselves are captured correctly during training. Lu They say this setup leads to a dual-domain objective that combines an Ordinal-Aware Soft Cross Entropy with a Spectral Consistency Loss.

Tom: That Spectral Consistency Loss is important because it forces the model to maintain global temporal coherence by minimizing the distance between the Fourier representations of the reconstructed signal and the true signal. Jane It’s like making sure that when you reconstruct a whole sequence, not only does it look right locally, but it also keeps a consistent pattern across its entire length in time.

Lu: Exactly. The training objective is set up so that if you use those epsilon terms in the loss decomposition, any probability mass the model puts outside the correct ordinal neighborhood of the true value gets penalized. Jane It’s a concrete way to enforce that structural separation we talked about earlier.

Meng: So, what does this mean for actually deploying it? Does this complexity translate into a manageable training process? Tom They show that it’s parameter efficient; the smaller model variant is only point six five million parameters compared to the larger SAITS variant. Jane That size reduction is good for production because it means less memory and faster training times.

Lu: And for inference speed, they show that the small MDTIM variant can complete a run in about four point zero five seconds, which beats some continuous diffusion baselines that need up to eight hundred ninety four seconds at large scale. Tom That difference in time is huge for real-time applications.

Jane: The paper also shows that this approach performs better even when the sequence length gets longer, meaning it captures global context without losing information through attention dilution as the sequence grows. Lu And they use a Continuous Ranked Probability Score or CRPS to measure uncertainty, which gives a better picture of both how well the model predicts and how sure it is about its answer.

Tom: So, in short, they’ve successfully adapted masked diffusion training to continuous time series imputation by using stochastic discretization and ordinal-aware soft labeling. Jane This method consistently beats deterministic and generative baselines in accuracy across different missing scenarios.

Lu: And when you look at downstream tasks, like forecasting on the energy dataset, MDTIM reduces the mean absolute error by nineteen point zero percent to twenty six point zero percent compared to SAITS and by sixteen point zero percent to nineteen point zero percent compared to CSDI. Jane That propagation of accuracy means that how well this model reconstructs data directly translates into how much better you can forecast things down the line.

Meng: I’m still thinking about the PhysioNet two thousand twelve dataset result where MDTIM achieved the lowest mean absolute error across all settings, outperforming SAITS by a clear margin. Tom That’s a strong claim because it shows this approach generalizes well to real-world sparse and irregular observations when you compare it to traditional frequency domain priors which they say become unreliable for those signals.

Jane: But there is one limitation they flag: the spectral consistency regularization assumes the presence of stable frequency components, so they disabled that part on datasets dominated by irregular sampling like PhysioNet A data. Lu That’s a fair caveat; it means the method relies on certain signal characteristics to work best.

Tom: And looking ahead, they suggest a future direction: designing a data adaptive discretization scheme that automatically picks the resolution of the vocabulary size K based on how granular the underlying signal is. Jane That would make it more flexible for different types of data.

Lu: And they also point out that extending MDTIM to extremely long sequences or non-stationary domains like financial tick data is definitely where future work needs to go.

The paper's improvements: Tom: So, we've seen how they adapted masked diffusion for continuous time series imputation using those discretization methods we talked about earlier. Jane Now let's talk about what the authors actually suggest to do next with this work and what that means for future research in this area.

Lu: They point out a few key things they think are important improvements. First, they say we need to design a data adaptive discretization scheme that automatically picks the resolution of the vocabulary size K based on how granular the underlying signal is. Jane That would be smart because K being fixed at forty might not work for every single time series dataset.

Jane: Yeah, if you have a very smooth signal and another one with lots of sharp spikes, having a fixed vocabulary size doesn't make sense for both cases. Tom So they want the model to figure out the best token resolution automatically instead of us guessing it ahead of time.

Tom: Exactly. And beyond just choosing K, Lu says they’re looking at developing an adaptive frequency objective to modulate that spectral consistency loss based on how regular or irregular the local signal is. Jane That means you tune the global coherence part of the model to match the specific local patterns in your data.

Lu: It’s about making it more flexible. They also mention extending this model to handle extremely long sequences and non-stationary domains, like financial tick data, because their current advantage grows with sequence length, which is good but not infinite. Tom That means we need to see if this structure holds up when the time series is really long or constantly changing.

Jane: So they’re looking at making it more robust to those extreme conditions. And they are also evaluating performance on those real-world data sets, like PhysioNet two thousand twelve to see how well it generalizes compared to traditional frequency domain priors for irregular signals. Lu That's a crucial test because those priors often break down when the signal isn't smooth.

Tom: The implication is that if we can automate the discretization choice and the frequency objective, we might create an imputation method that is much more robust across a wider range of real-world applications, not just controlled benchmarks. Jane That moves it from a specific technique to a more universal framework.

Conclusion: Tom: So we've covered how they took masked diffusion and made it work for continuous time series imputation by introducing stochastic discretization and ordinal labeling in this paper called "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training." Jane It really bridges two worlds that usually don't talk to each other.

Jane: Exactly. The main implication is that we can use the powerful training methods from diffusion models to actually reconstruct continuous data, not just discrete tokens. Lu That means we’re getting better at handling those ordinal relationships in time series data.

Lu: I think the big change here is showing that this structural separation between missing and observed states provides a real benefit over standard masked diffusion training alone. Tom It confirms that the way we structure the model makes a difference in how it learns to impute.

Meng: From an engineering standpoint, the inference speed is what really caught my attention because they showed that the small variant runs in about four point zero five seconds, which is much faster than some continuous diffusion baselines that take up to eight hundred ninety four seconds. Jane That’s a massive win for real-time deployment if we can get it stable.

Jane: It really shows the practical benefit of this structural change because speed matters when you have high-frequency data streams, and it does translate into better forecasting accuracy down the line on things like energy usage. Tom Yes, that propagation of accuracy is what makes this work for many real-world systems.

Lu: And I'm confident in the evidence because they showed it generalizes effectively to real-world sparse and irregular observations when tested on PhysioNet two thousand twelve data, which is a pretty tough test for any imputation model. Jane It shows the ordinal-aware discrete formulation holds up even when the signal isn't perfectly smooth.

Tom: And I just want to mention that they have some limitations, like they had to turn off the spectral consistency regularization on some irregular datasets because it assumes stable frequency components. Meng That’s a necessary caveat; we need to know where this method stops working effectively.

Meng: And the future work Lu mentioned is key, so designing a data adaptive discretization scheme that automatically picks the right resolution for vocabulary size K based on signal granularity seems like the most important thing to watch for. Jane It sounds like they are moving toward a system that can adapt to whatever kind of time series you feed it.

Lu: Right, and extending this to extremely long sequences or non-stationary domains, like financial tick data, is where we need to look next because that's the next frontier for these types of models. Tom So, "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training" gives us a solid foundation here. Jane It’s a lot to digest, but it’s definitely pushing the boundaries of how we use diffusion models for continuous data.

Seoul National University

cs.LG, cs.AI

Submitted: 2026-08-19

Updated: 2026-11-06

Comments: NeurIPS 2026

Code: https://github.com/zhouhaoyi/ETDataset3https:

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Time series imputation is addressed by proposing the Masked Diffusion Time-series Imputation Model (MDTIM)…

Key concepts

Stochastic Discretization
This technique maps the continuous numerical values in a time series onto discrete tokens suitable for language models. It is designed to preserve the continuous dynamics of the signal while allowing the model to operate within a discrete token space, effectively bridging continuous and discrete modeling paradigms.
Ordinal-Aware Soft Labeling
This method captures the inherent order or relationship between different tokens in a sequence. By introducing this labeling, the model learns to respect how values relate to each other in terms of magnitude and order, which is crucial for accurately reconstructing continuous time series data.
Spectral Consistency Loss
This regularization term enforces global temporal coherence by comparing the Fourier representations of the reconstructed signal against the ground truth. It ensures that the imputed sequence maintains consistent periodic structures across different frequencies, enhancing the model's ability to capture long-range dependencies.

Terminology

Summary

Time series imputation is addressed by proposing the Masked Diffusion Time-series Imputation Model (MDTIM), which bridges discrete masked diffusion and continuous time series modeling The MDTIM framework leverages the training paradigm of masked diffusion model for imputation tasks

Addressing Representation Limitations

Existing approaches suffer from two main limitations: missing and observed values are embedded within the same representation space without explicit structural separation Furthermore, continuous diffusion-based methods are trained to predict added noise rather than the original signal The [MASK] token is structurally orthogonal to valid observations, which provides a clearer separation between missing and observed states in the representation This structural orthogonality allows the model to directly predict the original values rather than the noise, more closely matching the imputation goal.

Bridging Discrete and Continuous Spaces

The fundamental challenge is bridging discrete masked diffusion and continuous time series modeling because time series data are inherently continuous and ordinal, whereas language tokens are discrete and categorical To solve this, the authors introduce Stochastic Discretization to map continuous values to ordinal-aware tokens while preserving continuous dynamics They also introduce Ordinal-Aware Soft Labeling to capture the ordinal relationships between tokens.

Training Signal Construction

The training objective is formulated as a dual-domain objective combining an Ordinal-Aware Soft Cross-Entropy and a Spectral Consistency Loss The Ordinal-Aware Masked Diffusion loss decomposes into (Equation 6) L = (1 − ϵ)LNELBO + ϵ DKL(u∥pθ) + ϵ H(u) const, which encourages the predicted distribution to align with the ordinal neighborhood of y. Additionally, a Spectral Consistency Loss is incorporated to enforce global temporal coherence by minimizing the L1 distance between Fourier representations of the reconstructed and ground-truth signals. p. 4

Where the paper says it

During training, we compute a continuous estimate of the full sequence, xˆt, by applying expectation-based decoding (Sec. We then minimize the L1 distance between the Fourier representations of the reconstructed and ground-truth signals: LFFT = Et [∥F(xˆt) − F(x0)∥1] , (7) where F(·) denotes the Fast Fourier Transform (FFT).

Performance and Robustness Gains

MDTIM consistently outperforms state-of-the-art deterministic and generative baselines across diverse benchmarks The model achieves the lowest MAE on Energy, ETTh, and Weather across both Uniform and Geometric missing patterns. Furthermore, MDTIM consistently outperforms the diffusion-based CSDI baseline across all datasets The model also exhibits notable robustness under geometric masking, where most baselines suffer significant degradation. p. 6

Where the paper says it

Compared to the diffusion-based CSDI [33], MDTIM consistently yields lower MAE across all datasets, owing to our Expectation-based Reconstruction that recovers continuous values deterministically from the predicted token distribution and mitigates the sampling variance inherent in continuous diffusion. MDTIM also exhibits notable robustness under geometric masking, where most baselines suffer significant degradation; for instance, on Energy at 30%, MDTIM increases only from 0.

Computational Efficiency and Scalability

MDTIM addresses the latency bottleneck of continuous diffusion by completing inference in 4.05s for the Small MDTIM variant, which is significantly faster than CSDI at the Large scale. p. 7

Where the paper says it

63 MDTIM addresses the latency bottleneck of continuous diffusion: while CSDI requires up to 894s at the Large scale, the Small MDTIM completes inference in 4. 05s while halving the MAE (0.

The model is also notably parameter-efficient, with the 0. p. 7
Where the paper says it

05s while halving the MAE (0. It is also notably parameter-efficient—the 0.

65M Small variant surpassing the 88. p. 7
Where the paper says it

It is also notably parameter-efficient—the 0. 65M Small variant surpasses the 88.

24M Large SAITS. p. 7
Where the paper says it

65M Small variant surpasses the 88. 24M Large SAITS, suggesting that the masked diffusion objective captures temporal dynamics more densely than scaling deterministic Transformers.

The performance advantage of MDTIM grows substantially at longer sequence lengths, indicating it captures global context without suffering from attention dilution. p. 6
Where the paper says it

145), indicating that MDTIM captures global context without suffering from the attention dilution often observed in baselines. Table 2: Robustness analysis on extended sequence lengths (L ∈ {96, 192}).

Ablation Study Insights

The ablation studies confirm that Expectation-based Unmasking consistently reduces error across all datasets, mitigating quantization error inherent in Argmax selection. The contribution of Soft Labeling scales with the structural regularity of the data, yielding substantial additional gain on the highly periodic Sine dataset. p. 8

Where the paper says it

The contribution of Soft Labeling, in contrast, scales with the structural regularity of the data. On the highly periodic Sine dataset, Soft Labeling yields a substantial additional gain on top of Expectation-based Unmasking, as ordinal-aware guidance aligns naturally with the smooth, predictable transitions of periodic signals.

Conversely, on Energy, Soft Labeling offers no further improvement beyond Expectation-based Unmasking. p. 8
Where the paper says it

A similar but milder gain is observed on ETTh, whose complex temporal dynamics still benefit from ordinal regularization under higher uncertainty. On Energy, where the dynamics are less structured, Soft Labeling offers no further improvement beyond Expectation-based Unmasking.

The optimal vocabulary size K=40 achieves the best balance between discretization resolution and classification complexity. p. 8
Where the paper says it

Conversely, K=60 slightly degrades performance, as a larger vocabulary increases the difficulty of the discrete classification task. K=40 achieves the best balance, providing sufficient resolution to approximate the continuous signal while remaining tractable to optimize.

Conclusion and Future Directions

MDTIM successfully adapts the masked diffusion training paradigm to continuous time series imputation by introducing Stochastic Discretization and Ordinal-Aware Soft Labeling The framework consistently outperforms both deterministic and generative baselines in reconstruction accuracy and robustness across diverse missing scenarios. Future work should focus on designing a data-adaptive discretization scheme for the optimal vocabulary size K and developing an adaptive frequency objective to modulate spectral consistency loss based on local signal regularity. p. 9

Where the paper says it

Designing an adaptive frequency objective that modulates its influence based on local signal regularity is a promising direction. Finally, our evaluation focuses on standard imputation benchmarks of moderate length; extending MDTIM to extremely long sequences and to multi-scale or non-stationary domains such as financial tick data remains for future work.

The evaluation extends to extremely long sequences and non-stationary domains like financial tick data for future investigation. p. 9
Where the paper says it

Designing an adaptive frequency objective that modulates its influence based on local signal regularity is a promising direction. Finally, our evaluation focuses on standard imputation benchmarks of moderate length; extending MDTIM to extremely long sequences and to multi-scale or non-stationary domains such as financial tick data remains for future work.

Evaluation Metrics

The authors evaluate performance using Mean Absolute Error (MAE) and Mean Squared Error (MSE) averaged over the masked entries M For probabilistic settings, they use the Continuous Ranked Probability Score (CRPS) to assess calibration and sharpness. The CRPS approximation involves averaging stochastic samples generated by the model over multiple independent noise realizations. p. 5

Where the paper says it

To obtain a robust estimate, we run inference M times (e. , M = 10) with independent noise samples and average the predicted probability distributions, p¯t,c = 1 M PM m=1 p (m) t,c .

p. 5
Where the paper says it

To obtain a robust estimate, we run inference M times (e. , M = 10) with independent noise samples and average the predicted probability distributions, p¯t,c = 1 M PM m=1 p (m) t,c .

Visualization of Results

Comprehensive visualizations illustrate that MDTIM effectively captures the complex temporal dynamics and periodicity of multivariate time series The green lines denote the imputed values reconstructed by MDTIM, showing how it effectively captures the complex temporal dynamics and periodicity of the multivariate time series.

Downstream Task Performance

The imputation accuracy of MDTIM translates to downstream tasks, as it yields substantially lower forecasting error than both baselines across all horizons on Energy This indicates that the imputation accuracy of MDTIM propagates to downstream forecasting. The results show that MDTIM reduces MAE by 19–26% over SAITS and 16–19% over CSDI on Energy forecasts. p. 6

Where the paper says it

2 Imputation Performance Main Results. As shown in Table 1, MDTIM achieves the lowest MAE on Energy, ETTh, and Weather across both Uniform and Geometric missing patterns.

p. 6
Where the paper says it

2 Imputation Performance Main Results. As shown in Table 1, MDTIM achieves the lowest MAE on Energy, ETTh, and Weather across both Uniform and Geometric missing patterns.

Comparison with Continuous Modeling

Comparing MDTIM (Disc) against a continuous-modeling variant MDTIM (Cont) shows that MDTIM (Disc) consistently achieves lower MAE across all missing rates and missing types This confirms that the explicit structural separation between valid observations and missing placeholders provides a benefit beyond the masked diffusion training scheme alone. MDTIM (Cont) already performs competitively with strong baselines, indicating that the masked diffusion paradigm of learning to reconstruct from varying corruption levels with timestep conditioning is itself effective even in continuous spaces.

Imputation Under Real-World Data

MDTIM achieves the lowest MAE across all settings when evaluated on PhysioNet 2012, outperforming SAITS by a clear margin This demonstrates that our ordinal-aware discrete formulation generalizes effectively to real-world sparse and irregular observations. The frequency-domain priors become unreliable for highly irregular signals where dominant spectral components are weak or unstable, whereas our ordinal-aware discrete formulation generalizes effectively to real-world sparse and irregular observations. p. 7

Where the paper says it

Notably, FGTI exhibits the largest degradation here, even underperforming BRITS. This indicates that frequency-domain priors become unreliable for highly irregular signals where dominant spectral components are weak or unstable, whereas our ordinal-aware discrete formulation generalizes effectively to real-world sparse and irregular observations.

Inference Time Comparison

MDTIM addresses the latency bottleneck of continuous diffusion: while CSDI requires up to 894s at the Large scale, the Small MDTIM completes inference in 4.05s while halving the MAE. p. 7

Where the paper says it

63 MDTIM addresses the latency bottleneck of continuous diffusion: while CSDI requires up to 894s at the Large scale, the Small MDTIM completes inference in 4. 05s while halving the MAE (0.

This shows that the masked diffusion objective encourages inference of global temporal structure rather than reliance on local interpolation. p. 6
Where the paper says it

MDTIM also exhibits notable robustness under geometric masking, where most baselines suffer significant degradation; for instance, on Energy at 30%, MDTIM increases only from 0. 053, indicating that the masked diffusion objective encourages inference of global temporal structure rather than reliance on local interpolation.

A Preliminaries on Diffusion Models

Diffusion models are a class of latent-variable generative models that learn to invert a fixed noise-injection process The forward process gradually corrupts a data sample into pure Gaussian noise over T steps The reverse process incrementally denoises noise back into the data manifold.

Masked Diffusion Language Models

Masked Diffusion Language Models (MDLM) adapt the diffusion framework to discrete sequences by replacing Gaussian corruption with an absorbing-state Markov chain The forward process replaces each token by [MASK] with probability 1 − αt. The reverse process iteratively unmasks positions back to data tokens. p. 2

Where the paper says it

To stabilize local statistics against distribution shifts, we normalize each input window x ∈ R T ×C into x˜. Specifically, the normalization statistics are computed solely using the observed values (excluding masked positions), ensuring that there is no data-leakage in the training and imputation process.

Derivation of Soft-Label Loss Decomposition

The per-token loss decomposes as L = (1 − ϵ)LNELBO + ϵ DKL(u∥pθ) + ϵ H(u) const, which is derived by splitting the cross-entropy and applying the KL form of the second term. This decomposition shows that for epsilon > 0, the additional non-negative term penalizes probability mass placed outside the ordinal neighborhood of y. p. 3

Where the paper says it

Crucially, we assign zero probability to the [MASK] token (s0 = 0) and distant bins. This concentrates probability mass on valid ordinal neighbors while preserving the structural separation of the mask state.

Continuous Reconstruction via Expectation

Continuous values are recovered by computing the probability-weighted expectation of the predicted token distribution, enabling precise dense reconstruction from discrete outputs This is achieved by averaging the predicted probability distributions across M independent noise samples. The result is then denormalized using the instance-wise statistics to recover the original scale. p. 5

Where the paper says it

The continuous value is then recovered as the expectation over the averaged distribution: xˆt,c = X K k=1 p¯ (k) t,c · vk, (8) where vk is the center value of the k-th bin within [−1. The result is then denormalized using the instance-wise statistics to recover the original scale.

Experimental Settings Summary

The experiments are conducted on Energy, ETTh, Weather, and Sine datasets They simulate data corruption using Uniform masking and Geometric masking across missing rates of 30%, 50%, and 70%. The comparison includes a comprehensive suite of 19 baselines spanning various architectural paradigms. p. 5

Where the paper says it

For both scenarios, we assess performance across varying missing rates of 30%, 50%, and 70%. We compare MDTIM against a comprehensive suite of 19 baselines spanning various architectural paradigms.

All experiments are conducted with dual Intel Xeon Gold 6444Y CPUs and a single NVIDIA H100 PCIe GPU (80GB). The evaluation reports the averaged Mean Absolute Error (MAE) across three independent inference trials using distinct random seeds for mask generation. p. 6
Where the paper says it

evaluate each trained model across three independent inference trials using distinct random seeds for mask generation, and report the averaged Mean Absolute Error (MAE). 2 Imputation Performance Main Results.

Ablation Study Summary

The ablation study on unmasking and labeling strategies confirms that Expectation-based Unmasking consistently reduces error across all datasets Soft Labeling's contribution scales with the structural regularity of the data, being most effective when the underlying signal exhibits clear ordinal or periodic structure. The sensitivity to vocabulary size K shows that K=40 achieves the best balance between discretization resolution and classification complexity. p. 8

Where the paper says it

208 Sensitivity to Vocabulary Size (K). We further investigate the impact of the vocabulary size K on imputation accuracy (Table 8), which exhibits a tradeoff between discretization resolution and classification complexity.

Limitations and Future Work

A limitation is that the spectral consistency regularization assumes the presence of stable frequency components and is therefore disabled on benchmarks dominated by irregular sampling such as PhysioNet A data-adaptive discretization scheme that automatically selects the resolution would further improve generalization. Extending MDTIM to extremely long sequences and multi-scale or non-stationary domains like financial tick data remains for future work. p. 9

Where the paper says it

Designing an adaptive frequency objective that modulates its influence based on local signal regularity is a promising direction. Finally, our evaluation focuses on standard imputation benchmarks of moderate length; extending MDTIM to extremely long sequences and to multi-scale or non-stationary domains such as financial tick data remains for future work.

Data Specifications Summary

The datasets include Energy with 28 channels, ETTh with 7 channels, Weather with 21 channels, and Sine with 5 channels These datasets cover a broad range of dynamics observed in long-horizon multivariate time-series. The Energy dataset contains electricity-related measurements with strong seasonality and occasional irregular variations. p. 1

Where the paper says it

2 Related Work 2. 1 Time Series Imputation Time series data are often partially observed due to sensor faults, irregular sampling, or datacollection constraints [35].

The ETTh2 dataset is a standard benchmark derived from Electricity Transformer Dataset [43] and is commonly used to evaluate long-range dependency modeling. p. 5
Where the paper says it

Our experiments focus on imputation accuracy, robustness to complex missing patterns, and computational efficiency. 1 Experimental Setup We evaluate our method on widely used benchmarks: Energy, ETTh, Weather, and the synthetic Sine dataset.

The Weather3 dataset includes multivariate meteorological observations characterized by nonlinear interactions, seasonal trends, and stochastic fluctuations. p. 5
Where the paper says it

1 Experimental Setup We evaluate our method on widely used benchmarks: Energy, ETTh, Weather, and the synthetic Sine dataset. To rigorously test the models, we simulate data corruption using two distinct mechanisms: Uniform masking, where observations are dropped randomly to mimic independent failures, and Geometric masking, which removes consecutive time steps to simulate prolonged sensor malfunctions or transmission errors.

The synthetic Sine dataset offers a controlled environment with known periodic structure to validate fundamental forecasting behavior under minimal uncontrolled factors. p. 8
Where the paper says it

On Energy, where the dynamics are less structured, Soft Labeling offers no further improvement beyond Expectation-based Unmasking. These results indicate that Soft Labeling is most effective when the underlying signal exhibits clear ordinal or periodic structure that ordinal-aware guidance can exploit.

Evaluation Metrics Summary

MAE and MSE are reported averaged over the masked entries M For probabilistic settings, the Continuous Ranked Probability Score (CRPS) is used to assess calibration and sharpness. The CRPS approximation involves averaging stochastic samples generated by the model over multiple independent noise realizations.

Visualization Summary

Figures 5, 6, 7, and 8 provide comprehensive visualizations of the imputation results across all benchmark datasets The green lines denote the imputed values reconstructed by MDTIM, effectively capturing the complex temporal dynamics and periodicity of the multivariate time series.

Forecasting Performance

MDTIM yields substantially lower forecasting error than both baselines across all horizons on Energy This indicates that the imputation accuracy of MDTIM propagates to downstream forecasting. The results show that MDTIM reduces MAE by 19–26% over SAITS and 16–19% over CSDI on Energy forecasts.

Training Configuration Details

The architecture of MDTIM is based on the Factorized Temporal-Variate Backbone described in Section 4 The utilized hyperparameters include a Hidden Size (D) of 256, Attention Heads of 16, DiT Blocks (L) of 5, and a Vocabulary size (K) of 40. The training employs a Log-Linear Noise Schedule and an importance sampling via a time-dependent loss weight w(t). p. 4

Where the paper says it

For a set of masked indices Mt at timestep t, the diffusion loss is formulated as: Ldiff = Et " w(t) X l∈Mt − s (l) , log pθ(z (l) t ) # , (5) where log pθ(z (l) t ) ∈ R K is the predicted log-probability vector, and s (l) ∈ R K denotes the soft target distribution. The detailed formulation of time-dependent weight w(t) and the corresponding noise schedule are detailed in Appendix C.

The spectral loss weight lambda is set to 1. p. 4
Where the paper says it

For a set of masked indices Mt at timestep t, the diffusion loss is formulated as: Ldiff = Et " w(t) X l∈Mt − s (l) , log pθ(z (l) t ) # , (5) where log pθ(z (l) t ) ∈ R K is the predicted log-probability vector, and s (l) ∈ R K denotes the soft target distribution. The detailed formulation of time-dependent weight w(t) and the corresponding noise schedule are detailed in Appendix C.

0.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved AI system can perform time series imputation by leveraging masked diffusion model training paradigms to directly predict original values rather than added noise, aligning both representation and learning objectives with the imputation task.

  2. The system can handle continuous and ordinal nature of time series data by introducing Stochastic Discretization, which maps continuous values to ordinal-aware tokens while preserving continuous dynamics, and Ordinal-Aware Soft Labeling to capture ordinal relationships between tokens.

  3. The system can achieve precise continuous reconstruction by employing an Expectation-based Unmasking strategy and Spectral Consistency Regularization during training, ensuring the model reconstruct[s] continuous time series precisely.

  4. The system can be made robust to various missing scenarios by utilizing a dual-domain objective combining Ordinal-Aware Masked Diffusion with Spectral Consistency Regularization, which minimizes the total training objective Ltotal = Ldiff + λLFFT.

  5. The system can be made computationally efficient for inference by using an Expectation-based Unmasking strategy to compute the expectation over the averaged distribution for continuous reconstruction, allowing it to complete inference in as little as 4.05 seconds on small models compared to continuous diffusion baselines which require up to 894 seconds.

  6. The system can be adapted for long-term dependencies by extending evaluations to sequence lengths L = 192, where the performance advantage of MDTIM grows with the sequence length, indicating it captures global context without attention dilution often observed in baselines.

  7. The system can provide reliable uncertainty estimates for imputation by using a Continuous Ranked Probability Score (CRPS) based on sample-based estimation, which rewards both calibration and sharpness, leading to lower CRPS values compared to diffusion-based baselines like CSDI.

  8. The system can generalize effectively to real-world sparse and irregular observations by demonstrating that MDTIM's ordinal-aware discrete formulation generalizes effectively to real-world sparse and irregular observations when evaluated on the PhysioNet 2012 dataset, outperforming frequency-domain priors which become unreliable for such signals.

  9. The system can be made data adaptive by incorporating a data adaptive discretization scheme that automatically selects the resolution of the vocabulary size K based on the range and granularity of the underlying signal, addressing the optimal vocabulary size K may vary across datasets.

Abstract

Time series imputation is a crucial area for reliable time series analysis, yet it remains challenging due to the complex temporal dynamics and noise of real-world data. Existing approaches, however, exhibit two limitations: missing and observed values are embedded within the same representation space without explicit structural separation, and continuous diffusion-based methods recover the missing values only through many iterative denoising steps rather than by predicting them directly. To address these, we propose the Masked Diffusion Time-series Imputation Model (MDTIM), which leverages the training paradigm of masked diffusion model for imputation tasks. The [MASK] token is structurally orthogonal to valid observations, and because the corrupted state is the partially observed series itself, the model is trained to predict the missing values directly from the observed ones, naturally aligning both the representation and the learning objective with the imputation task. To bridge the gap between discrete masked diffusion and the continuous, ordinal nature of time series, we further introduce Stochastic Discretization, which maps continuous values to ordinal-aware tokens while preserving continuous dynamics. Our experiments on diverse benchmarks confirm that MDTIM achieves superior robustness and scalability, consistently outperforming state-of-the-art deterministic and generative baselines across various missing scenarios. Code and project page are available at https://dongbeank.github.io/MDTIM/.

Sources

Related papers