Discretizing Continuous Time Series for Imputation with Masked Diffusion Training
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training".
Jane: Here's the gist of the paper. Time series imputation is addressed by proposing the Masked Diffusion Time-series Imputation Model (MDTIM), which bridges discrete masked diffusion and continuous time series modeling The MDTIM framework leverages the training paradigm of masked diffusion model for imputation tasks Addressing Representation Limitations Existing approaches suffer from two main limitations: missing and observed values are embedded within the same representation space without explicit structural separation Furthermore, continuous diffusion-based methods are trained to predict added noise rather than the original signal The
MASK: token is structurally orthogonal to valid observations, which provides a clearer separation between missing and observed states in the representation This structural orthogonality allows the model to directly predict the original values rather than the noise, more closely matching the imputation goal.
Tom: Bridging Discrete and Continuous Spaces The fundamental challenge is bridging discrete masked diffusion and continuous time series modeling because time series data are inherently continuous and ordinal, whereas language tokens are discrete and categorical To solve this, the authors introduce Stochastic Discretization to map continuous values to ordinal-aware tokens while preserving continuous dynamics They also introduce Ordinal-Aware Soft Labeling to capture the ordinal relationships between tokens.
Jane: Training Signal Construction The training objective is formulated as a dual-domain objective combining an Ordinal-Aware Soft Cross-Entropy and a Spectral Consistency Loss The Ordinal-Aware Masked Diffusion loss decomposes into (Equation 6) L = (1 − ϵ)LNELBO + ϵ DKL(u∥pθ) + ϵ H(u) const, which encourages the predicted distribution to align with the ordinal neighborhood of y.
Tom: Additionally, a Spectral Consistency Loss is incorporated to enforce global temporal coherence by minimizing the L1 distance between Fourier representations of the reconstructed and ground-truth signals.
Title and authors: Tom: So we're looking at this paper called "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training," and the authors are Dongbin Kim, Seungyun Lee, Geonwoo Shin, and Jaewook Lee from Seoul National University.
Jane: It sounds like they are trying to solve a really tough problem in time series analysis where you have continuous data but you're using tools that are built for discrete things.
Lu: Basically, the main idea is to take the training method from masked diffusion models and adapt it for continuous time series imputation, which is pretty novel because those two worlds don't usually mix well.
Tom: So what's the core problem they are tackling here, Jane? What makes this approach different from just using a standard diffusion model on time series data?
Jane: Well, the paper points out that existing methods have two big flaws: first, missing and observed values are in the same representation without any clear separation.
Tom: And second, continuous diffusion models usually try to predict some added noise instead of actually figuring out what the original signal was.
Lu: Exactly. They propose a model called the Masked Diffusion Time-series Imputation Model or MDTIM, which they say leverages that masked diffusion training paradigm specifically for imputation tasks.
Jane: And they make a key structural argument about how their approach fixes the representation issue by using a special token called the mask token which is structurally orthogonal to any valid observation.
Meng: Orthogonal to observations—that sounds cleaner than just mixing everything together—but what does that actually mean for the prediction part?
Tom: Because if it's truly orthogonal, can the model directly predict the original values instead of predicting noise?
Lu: Yes, that's the goal. They say this structural orthogonality allows the model to directly predict those original values rather than just noise, which is what we want for imputation. It’s more closely aligned with what imputation actually requires.
Jane: But how do you handle the fact that time series data are continuous and ordinal, while language tokens, which diffusion models usually use, are discrete?
Tom: That's where they introduce Stochastic Discretization to map those continuous values into ordinal-aware tokens while still keeping the continuous dynamics intact.
Jane: And they also add something called Ordinal-Aware Soft Labeling to capture those important relationships between the tokens themselves.
Lu: The training objective is quite specific here, it's a dual-domain objective combining an Ordinal-Aware Soft Cross Entropy and a Spectral Consistency Loss.
Tom: That spectral consistency loss is interesting because they use it to enforce global temporal coherence by minimizing the L1 distance between Fourier representations of the reconstructed signal and the ground truth signal.
Title and authors: Jane: It sounds like they are trying to ensure that when you reconstruct a sequence, not only does it look right locally, but it also maintains a consistent pattern across its entire length in time.
Tom: And they do that by using this dual objective, which encourages the predicted distribution to align with the ordinal neighborhood of the target values.
Meng: From my side as someone who has to worry about deployment, does this complexity translate into a manageable training process?
Lu: They state that the Ordinal-Aware Masked Diffusion loss decomposes into one part involving L NELBO plus some terms with epsilon, which encourages the predicted distribution to align with the ordinal neighborhood of the target values.
Jane: And they say this decomposition shows that for epsilon greater than zero, that additional term penalizes any probability mass placed outside of the ordinal neighborhood of the true value.
Tom: That’s a concrete way they are enforcing those ordinal relationships we talked about earlier.
Lu: They also incorporate a Spectral Consistency Loss to enforce global temporal coherence by minimizing the L1 distance between Fourier representations of the reconstructed and ground truth signals, which is what they call L FFT.
Jane: So, in short, they are taking a masked diffusion model and adding tools to handle continuous ordinal data so it can accurately reconstruct those time series.
Tom: And I see a lot of promise there because if you can get that structural separation right between missing and observed states, you bypass some of the typical noise prediction issues inherent in these models.
Jane: But let's look at how they actually train this thing, because the training setup seems quite involved.
Lu: The training objective is formulated as a dual-domain objective combining an Ordinal-Aware Soft Cross Entropy and a Spectral Consistency Loss, which they call L total equals L diff plus lambda L FFT.
Tom: And they set the spectral loss weight lambda to one point zero for their main experiments on the datasets like Energy, ETTh, and Weather.
Meng: If we're talking about running this at scale, how does this training setup affect computational cost?
Lu: They show that it's parameter efficient; the zero point six five million small variant is smaller than the eight eight point two four million large SAITS variant <ref:2608.19119#pg2>.
Jane: That’s a significant reduction in size, which is good for deployment, especially when we compare it to some of the larger continuous diffusion baselines.
Title and authors: Tom: And on inference speed, they show that the Small MDTIM variant completes inference in about four point zero five seconds, which is much faster than the continuous diffusion baseline which takes up to eight hundred ninety four seconds at large scale.
Jane: That difference in time is huge for real-time applications, so that's a big win for practical use of this model.
Lu: They also show that the performance advantage of MDTIM gets bigger as the sequence length increases, which suggests it manages to capture global context without losing information through attention dilution.
Tom: That’s what we want when dealing with long-term dependencies in time series.
Jane: The paper shows that they used Expectation-based Unmasking to recover the continuous values by computing the probability weighted expectation of the predicted token distribution across multiple independent noise samples.
Tom: So, how does that help with uncertainty?
Lu: They use a Continuous Ranked Probability Score or CRPS for probabilistic settings to assess both calibration and sharpness, which rewards both aspects.
Meng: I need to know about the robustness here. The results show it performs well under geometric masking where most other baselines suffer significant degradation.
Jane: That's important because real-world data often has those kinds of missing patterns, and this model seems pretty resilient to them compared to other methods.
Lu: And they have an ablation study showing that Expectation-based Unmasking consistently reduces error across all datasets, which helps mitigate the quantization error that comes from using the Argmax selection method.
Jane: But what about the Ordinal-Aware Soft Labeling part? Does it always help?
Meng: The contribution of Soft Labeling scales with how regular the underlying data is; it's most effective when there's clear ordinal or periodic structure in the signal.
Lu: They found that on the Sine dataset, where there is known periodicity, Soft Labeling yields substantial additional gain.
Jane: But on datasets like Energy, it didn't offer any further improvement beyond just using Expectation-based Unmasking.
Tom: And they found an optimal vocabulary size of four zero for the best balance between how detailed the discretization is and how complex the classification task becomes.
Jane: So, to summarize what we have on this paper, it successfully adapts masked diffusion training to continuous time series using stochastic discretization and ordinal-aware soft labeling.
Lu: It consistently outperforms both deterministic and generative baselines in reconstruction accuracy across different missing scenarios.
Tom: And when you look at downstream tasks, like forecasting on the Energy dataset, MDTIM reduces the mean absolute error by nineteen point zero percent to twenty six point zero percent compared to SAITS, and by sixteen point zero percent to nineteen point zero percent compared to CSDI.
Title and authors: Jane: That propagation of accuracy is telling; it means that how well this model reconstructs the data directly translates into how much better you can forecast things down the line on things like energy usage.
Meng: I'm still thinking about that PhysioNet two thousand twelve dataset result, where MDTIM achieved the lowest mean absolute error across all settings, outperforming SAITS by a clear margin.
Tom: That’s a strong claim because it shows this approach generalizes well to real-world sparse and irregular observations when you compare it to traditional frequency domain priors which they say become unreliable for those signals.
Lu: But the paper does have some limitations, and one is that the spectral consistency regularization assumes the presence of stable frequency components, so they disabled that part on benchmarks dominated by irregular sampling like PhysioNet A data.
Jane: That's a fair limitation to note; it means the method relies on certain signal characteristics to work best.
Tom: And they point out another future direction: designing a data adaptive discretization scheme that automatically selects the resolution of the vocabulary size K based on the range and granularity of the underlying signal, since they found K=forty was optimal for their tests <ref:2608.19119#pg2>.
Jane: And extending this to extremely long sequences or non-stationary domains like financial tick data is definitely where future work needs to go.
Lu: To wrap up, this paper, "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training," successfully adapts the masked diffusion training paradigm to continuous time series imputation by introducing stochastic discretization and ordinal-aware soft labeling.
Tom: It consistently outperforms both deterministic and generative baselines in reconstruction accuracy and robustness across diverse missing scenarios.
Jane: It’s a solid piece of work because it shows that the way we structure the model, by making the structural separation between valid observations and missing placeholders explicit, provides a real benefit beyond just using masked diffusion training.
Meng: And when you look at inference time, achieving four point zero five seconds on small models is much faster than some continuous diffusion baselines which take up to eight hundred ninety four seconds.
Tom: And the results show that this imputation accuracy translates directly into downstream forecasting performance, reducing the mean absolute error by nineteen point zero percent to twenty six point zero percent on energy forecasts compared to SAITS and CSDI.
Jane: So, we're looking at a model that handles continuous ordinal data in a way that is both computationally feasible and actually improves performance on real-world time series.
The paper's summary: Tom: Essentially, they introduced this Masked Diffusion Time-series Imputation Model or MDTIM. Jane It’s an attempt to bridge the gap between discrete diffusion and continuous time series modeling by using the masked diffusion training paradigm for imputation tasks.
Lu: The core idea is fixing two big problems in existing methods—first, missing and observed values are mixed up in the same representation space without any clear separation. Tom And second, continuous diffusion models usually try to predict some added noise instead of figuring out what the original signal actually was.
Jane: So they solve that by adding a special mask token that is structurally different from any valid observation, which lets the model directly predict the original values instead of just predicting noise. Lu That’s a nice structural trick for cleaning up the representation.
Meng: But how do you handle continuous time series data? They have to deal with them being continuous and ordinal, while diffusion models usually work with discrete tokens. Tom That’s where they introduce Stochastic Discretization to map those continuous values into tokens that respect the ordinal relationships, while still keeping the underlying dynamics smooth.
Jane: And they also layer on this Ordinal-Aware Soft Labeling to make sure those ordinal relationships between the tokens themselves are captured correctly during training. Lu They say this setup leads to a dual-domain objective that combines an Ordinal-Aware Soft Cross Entropy with a Spectral Consistency Loss.
Tom: That Spectral Consistency Loss is important because it forces the model to maintain global temporal coherence by minimizing the distance between the Fourier representations of the reconstructed signal and the true signal. Jane It’s like making sure that when you reconstruct a whole sequence, not only does it look right locally, but it also keeps a consistent pattern across its entire length in time.
Lu: Exactly. The training objective is set up so that if you use those epsilon terms in the loss decomposition, any probability mass the model puts outside the correct ordinal neighborhood of the true value gets penalized. Jane It’s a concrete way to enforce that structural separation we talked about earlier.
Meng: So, what does this mean for actually deploying it? Does this complexity translate into a manageable training process? Tom They show that it’s parameter efficient; the smaller model variant is only point six five million parameters compared to the larger SAITS variant. Jane That size reduction is good for production because it means less memory and faster training times.
Lu: And for inference speed, they show that the small MDTIM variant can complete a run in about four point zero five seconds, which beats some continuous diffusion baselines that need up to eight hundred ninety four seconds at large scale. Tom That difference in time is huge for real-time applications.
Jane: The paper also shows that this approach performs better even when the sequence length gets longer, meaning it captures global context without losing information through attention dilution as the sequence grows. Lu And they use a Continuous Ranked Probability Score or CRPS to measure uncertainty, which gives a better picture of both how well the model predicts and how sure it is about its answer.
Tom: So, in short, they’ve successfully adapted masked diffusion training to continuous time series imputation by using stochastic discretization and ordinal-aware soft labeling. Jane This method consistently beats deterministic and generative baselines in accuracy across different missing scenarios.
Lu: And when you look at downstream tasks, like forecasting on the energy dataset, MDTIM reduces the mean absolute error by nineteen point zero percent to twenty six point zero percent compared to SAITS and by sixteen point zero percent to nineteen point zero percent compared to CSDI. Jane That propagation of accuracy means that how well this model reconstructs data directly translates into how much better you can forecast things down the line.
Meng: I’m still thinking about the PhysioNet two thousand twelve dataset result where MDTIM achieved the lowest mean absolute error across all settings, outperforming SAITS by a clear margin. Tom That’s a strong claim because it shows this approach generalizes well to real-world sparse and irregular observations when you compare it to traditional frequency domain priors which they say become unreliable for those signals.
Jane: But there is one limitation they flag: the spectral consistency regularization assumes the presence of stable frequency components, so they disabled that part on datasets dominated by irregular sampling like PhysioNet A data. Lu That’s a fair caveat; it means the method relies on certain signal characteristics to work best.
Tom: And looking ahead, they suggest a future direction: designing a data adaptive discretization scheme that automatically picks the resolution of the vocabulary size K based on how granular the underlying signal is. Jane That would make it more flexible for different types of data.
Lu: And they also point out that extending MDTIM to extremely long sequences or non-stationary domains like financial tick data is definitely where future work needs to go.
The paper's improvements: Tom: So, we've seen how they adapted masked diffusion for continuous time series imputation using those discretization methods we talked about earlier. Jane Now let's talk about what the authors actually suggest to do next with this work and what that means for future research in this area.
Lu: They point out a few key things they think are important improvements. First, they say we need to design a data adaptive discretization scheme that automatically picks the resolution of the vocabulary size K based on how granular the underlying signal is. Jane That would be smart because K being fixed at forty might not work for every single time series dataset.
Jane: Yeah, if you have a very smooth signal and another one with lots of sharp spikes, having a fixed vocabulary size doesn't make sense for both cases. Tom So they want the model to figure out the best token resolution automatically instead of us guessing it ahead of time.
Tom: Exactly. And beyond just choosing K, Lu says they’re looking at developing an adaptive frequency objective to modulate that spectral consistency loss based on how regular or irregular the local signal is. Jane That means you tune the global coherence part of the model to match the specific local patterns in your data.
Lu: It’s about making it more flexible. They also mention extending this model to handle extremely long sequences and non-stationary domains, like financial tick data, because their current advantage grows with sequence length, which is good but not infinite. Tom That means we need to see if this structure holds up when the time series is really long or constantly changing.
Jane: So they’re looking at making it more robust to those extreme conditions. And they are also evaluating performance on those real-world data sets, like PhysioNet two thousand twelve to see how well it generalizes compared to traditional frequency domain priors for irregular signals. Lu That's a crucial test because those priors often break down when the signal isn't smooth.
Tom: The implication is that if we can automate the discretization choice and the frequency objective, we might create an imputation method that is much more robust across a wider range of real-world applications, not just controlled benchmarks. Jane That moves it from a specific technique to a more universal framework.
Conclusion: Tom: So we've covered how they took masked diffusion and made it work for continuous time series imputation by introducing stochastic discretization and ordinal labeling in this paper called "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training." Jane It really bridges two worlds that usually don't talk to each other.
Jane: Exactly. The main implication is that we can use the powerful training methods from diffusion models to actually reconstruct continuous data, not just discrete tokens. Lu That means we’re getting better at handling those ordinal relationships in time series data.
Lu: I think the big change here is showing that this structural separation between missing and observed states provides a real benefit over standard masked diffusion training alone. Tom It confirms that the way we structure the model makes a difference in how it learns to impute.
Meng: From an engineering standpoint, the inference speed is what really caught my attention because they showed that the small variant runs in about four point zero five seconds, which is much faster than some continuous diffusion baselines that take up to eight hundred ninety four seconds. Jane That’s a massive win for real-time deployment if we can get it stable.
Jane: It really shows the practical benefit of this structural change because speed matters when you have high-frequency data streams, and it does translate into better forecasting accuracy down the line on things like energy usage. Tom Yes, that propagation of accuracy is what makes this work for many real-world systems.
Lu: And I'm confident in the evidence because they showed it generalizes effectively to real-world sparse and irregular observations when tested on PhysioNet two thousand twelve data, which is a pretty tough test for any imputation model. Jane It shows the ordinal-aware discrete formulation holds up even when the signal isn't perfectly smooth.
Tom: And I just want to mention that they have some limitations, like they had to turn off the spectral consistency regularization on some irregular datasets because it assumes stable frequency components. Meng That’s a necessary caveat; we need to know where this method stops working effectively.
Meng: And the future work Lu mentioned is key, so designing a data adaptive discretization scheme that automatically picks the right resolution for vocabulary size K based on signal granularity seems like the most important thing to watch for. Jane It sounds like they are moving toward a system that can adapt to whatever kind of time series you feed it.
Lu: Right, and extending this to extremely long sequences or non-stationary domains, like financial tick data, is where we need to look next because that's the next frontier for these types of models. Tom So, "Discretizing Continuous Time Series for Imputation with Masked Diffusion Training" gives us a solid foundation here. Jane It’s a lot to digest, but it’s definitely pushing the boundaries of how we use diffusion models for continuous data.
Seoul National University
cs.LG, cs.AI
Submitted: 2026-08-19
Updated: 2026-11-06
Comments: NeurIPS 2026
Code: https://github.com/zhouhaoyi/ETDataset3https:
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 91/100
The gist: Time series imputation is addressed by proposing the Masked Diffusion Time-series Imputation Model (MDTIM)…
Key concepts
- Stochastic Discretization
- This technique maps the continuous numerical values in a time series onto discrete tokens suitable for language models. It is designed to preserve the continuous dynamics of the signal while allowing the model to operate within a discrete token space, effectively bridging continuous and discrete modeling paradigms.
- Ordinal-Aware Soft Labeling
- This method captures the inherent order or relationship between different tokens in a sequence. By introducing this labeling, the model learns to respect how values relate to each other in terms of magnitude and order, which is crucial for accurately reconstructing continuous time series data.
- Spectral Consistency Loss
- This regularization term enforces global temporal coherence by comparing the Fourier representations of the reconstructed signal against the ground truth. It ensures that the imputed sequence maintains consistent periodic structures across different frequencies, enhancing the model's ability to capture long-range dependencies.
Terminology
Summary
Time series imputation is addressed by proposing the Masked Diffusion Time-series Imputation Model (MDTIM), which bridges discrete masked diffusion and continuous time series modeling The MDTIM framework leverages the training paradigm of masked diffusion model for imputation tasks
Addressing Representation Limitations
Existing approaches suffer from two main limitations: missing and observed values are embedded within the same representation space without explicit structural separation Furthermore, continuous diffusion-based methods are trained to predict added noise rather than the original signal The [MASK] token is structurally orthogonal to valid observations, which provides a clearer separation between missing and observed states in the representation This structural orthogonality allows the model to directly predict the original values rather than the noise, more closely matching the imputation goal.
Bridging Discrete and Continuous Spaces
The fundamental challenge is bridging discrete masked diffusion and continuous time series modeling because time series data are inherently continuous and ordinal, whereas language tokens are discrete and categorical To solve this, the authors introduce Stochastic Discretization to map continuous values to ordinal-aware tokens while preserving continuous dynamics They also introduce Ordinal-Aware Soft Labeling to capture the ordinal relationships between tokens.
Training Signal Construction
The training objective is formulated as a dual-domain objective combining an Ordinal-Aware Soft Cross-Entropy and a Spectral Consistency Loss The Ordinal-Aware Masked Diffusion loss decomposes into (Equation 6) L = (1 − ϵ)LNELBO + ϵ DKL(u∥pθ) + ϵ H(u) const, which encourages the predicted distribution to align with the ordinal neighborhood of y. Additionally, a Spectral Consistency Loss is incorporated to enforce global temporal coherence by minimizing the L1 distance between Fourier representations of the reconstructed and ground-truth signals. p. 4 During training, we compute a continuous estimate of the full sequence,
xˆt, by applying expectation-based decoding (Sec. We then minimize the L1 distance between
the Fourier representations of the reconstructed and ground-truth signals:
LFFT = Et [∥F(xˆt) − F(x0)∥1] , (7)
where F(·) denotes the Fast Fourier Transform (FFT).Where the paper says it
Performance and Robustness Gains
MDTIM consistently outperforms state-of-the-art deterministic and generative baselines across diverse benchmarks The model achieves the lowest MAE on Energy, ETTh, and Weather across both Uniform and Geometric missing patterns. Furthermore, MDTIM consistently outperforms the diffusion-based CSDI baseline across all datasets The model also exhibits notable robustness under geometric masking, where most baselines suffer significant degradation. p. 6 Compared to the diffusion-based CSDI [33], MDTIM
consistently yields lower MAE across all datasets, owing to our Expectation-based Reconstruction
that recovers continuous values deterministically from the predicted token distribution and mitigates
the sampling variance inherent in continuous diffusion. MDTIM also exhibits notable robustness
under geometric masking, where most baselines suffer significant degradation; for instance, on Energy
at 30%, MDTIM increases only from 0.Where the paper says it
Computational Efficiency and Scalability
MDTIM addresses the latency bottleneck of continuous diffusion by completing inference in 4.05s for the Small MDTIM variant, which is significantly faster than CSDI at the Large scale. p. 7 63
MDTIM addresses the latency bottleneck of continuous diffusion: while CSDI requires up to
894s at the Large scale, the Small MDTIM completes inference in 4. 05s while halving the MAE
(0. 05s while halving the MAE
(0. It is also notably parameter-efficient—the 0. It is also notably parameter-efficient—the 0. 65M Small variant surpasses the 88. 65M Small variant surpasses the 88. 24M
Large SAITS, suggesting that the masked diffusion objective captures temporal dynamics more
densely than scaling deterministic Transformers. 145), indicating that MDTIM captures global context without suffering from
the attention dilution often observed in baselines. Table 2: Robustness analysis on extended sequence lengths (L ∈ {96, 192}).Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Ablation Study Insights
The ablation studies confirm that Expectation-based Unmasking consistently reduces error across all datasets, mitigating quantization error inherent in Argmax selection. The contribution of Soft Labeling scales with the structural regularity of the data, yielding substantial additional gain on the highly periodic Sine dataset. p. 8 The contribution of
Soft Labeling, in contrast, scales with the structural regularity of the data. On the highly periodic Sine
dataset, Soft Labeling yields a substantial additional gain on top of Expectation-based Unmasking, as
ordinal-aware guidance aligns naturally with the smooth, predictable transitions of periodic signals. A similar but milder gain is observed on ETTh, whose complex temporal dynamics still benefit from
ordinal regularization under higher uncertainty. On Energy, where the dynamics are less structured,
Soft Labeling offers no further improvement beyond Expectation-based Unmasking. Conversely, K=60 slightly
degrades performance, as a larger vocabulary increases the difficulty of the discrete classification
task. K=40 achieves the best balance, providing sufficient resolution to approximate the continuous
signal while remaining tractable to optimize.Where the paper says it
Where the paper says it
Where the paper says it
Conclusion and Future Directions
MDTIM successfully adapts the masked diffusion training paradigm to continuous time series imputation by introducing Stochastic Discretization and Ordinal-Aware Soft Labeling The framework consistently outperforms both deterministic and generative baselines in reconstruction accuracy and robustness across diverse missing scenarios. Future work should focus on designing a data-adaptive discretization scheme for the optimal vocabulary size K and developing an adaptive frequency objective to modulate spectral consistency loss based on local signal regularity. p. 9 Designing an adaptive frequency objective that modulates its influence based on
local signal regularity is a promising direction. Finally, our evaluation focuses on standard imputation
benchmarks of moderate length; extending MDTIM to extremely long sequences and to multi-scale
or non-stationary domains such as financial tick data remains for future work. Designing an adaptive frequency objective that modulates its influence based on
local signal regularity is a promising direction. Finally, our evaluation focuses on standard imputation
benchmarks of moderate length; extending MDTIM to extremely long sequences and to multi-scale
or non-stationary domains such as financial tick data remains for future work.Where the paper says it
Where the paper says it
Evaluation Metrics
The authors evaluate performance using Mean Absolute Error (MAE) and Mean Squared Error (MSE) averaged over the masked entries M For probabilistic settings, they use the Continuous Ranked Probability Score (CRPS) to assess calibration and sharpness. The CRPS approximation involves averaging stochastic samples generated by the model over multiple independent noise realizations. p. 5 To obtain a robust estimate, we run inference M times (e. , M = 10) with
independent noise samples and average the predicted probability distributions, p¯t,c =
1
M
PM
m=1 p
(m)
t,c . To obtain a robust estimate, we run inference M times (e. , M = 10) with
independent noise samples and average the predicted probability distributions, p¯t,c =
1
M
PM
m=1 p
(m)
t,c .Where the paper says it
Where the paper says it
Visualization of Results
Comprehensive visualizations illustrate that MDTIM effectively captures the complex temporal dynamics and periodicity of multivariate time series The green lines denote the imputed values reconstructed by MDTIM, showing how it effectively captures the complex temporal dynamics and periodicity of the multivariate time series.
Downstream Task Performance
The imputation accuracy of MDTIM translates to downstream tasks, as it yields substantially lower forecasting error than both baselines across all horizons on Energy This indicates that the imputation accuracy of MDTIM propagates to downstream forecasting. The results show that MDTIM reduces MAE by 19–26% over SAITS and 16–19% over CSDI on Energy forecasts. p. 6 2 Imputation Performance
Main Results. As shown in Table 1, MDTIM achieves the lowest MAE on Energy, ETTh, and
Weather across both Uniform and Geometric missing patterns. 2 Imputation Performance
Main Results. As shown in Table 1, MDTIM achieves the lowest MAE on Energy, ETTh, and
Weather across both Uniform and Geometric missing patterns.Where the paper says it
Where the paper says it
Comparison with Continuous Modeling
Comparing MDTIM (Disc) against a continuous-modeling variant MDTIM (Cont) shows that MDTIM (Disc) consistently achieves lower MAE across all missing rates and missing types This confirms that the explicit structural separation between valid observations and missing placeholders provides a benefit beyond the masked diffusion training scheme alone. MDTIM (Cont) already performs competitively with strong baselines, indicating that the masked diffusion paradigm of learning to reconstruct from varying corruption levels with timestep conditioning is itself effective even in continuous spaces.
Imputation Under Real-World Data
MDTIM achieves the lowest MAE across all settings when evaluated on PhysioNet 2012, outperforming SAITS by a clear margin This demonstrates that our ordinal-aware discrete formulation generalizes effectively to real-world sparse and irregular observations. The frequency-domain priors become unreliable for highly irregular signals where dominant spectral components are weak or unstable, whereas our ordinal-aware discrete formulation generalizes effectively to real-world sparse and irregular observations. p. 7 Notably, FGTI
exhibits the largest degradation here, even
underperforming BRITS. This indicates that
frequency-domain priors become unreliable for
highly irregular signals where dominant spectral
components are weak or unstable, whereas our
ordinal-aware discrete formulation generalizes
effectively to real-world sparse and irregular observations.Where the paper says it
Inference Time Comparison
MDTIM addresses the latency bottleneck of continuous diffusion: while CSDI requires up to 894s at the Large scale, the Small MDTIM completes inference in 4.05s while halving the MAE. p. 7 63
MDTIM addresses the latency bottleneck of continuous diffusion: while CSDI requires up to
894s at the Large scale, the Small MDTIM completes inference in 4. 05s while halving the MAE
(0. MDTIM also exhibits notable robustness
under geometric masking, where most baselines suffer significant degradation; for instance, on Energy
at 30%, MDTIM increases only from 0. 053, indicating that the masked diffusion objective
encourages inference of global temporal structure rather than reliance on local interpolation.Where the paper says it
Where the paper says it
A Preliminaries on Diffusion Models
Diffusion models are a class of latent-variable generative models that learn to invert a fixed noise-injection process The forward process gradually corrupts a data sample into pure Gaussian noise over T steps The reverse process incrementally denoises noise back into the data manifold.
Masked Diffusion Language Models
Masked Diffusion Language Models (MDLM) adapt the diffusion framework to discrete sequences by replacing Gaussian corruption with an absorbing-state Markov chain The forward process replaces each token by [MASK] with probability 1 − αt. The reverse process iteratively unmasks positions back to data tokens. p. 2 To stabilize local statistics against distribution shifts, we
normalize each input window x ∈ R
T ×C into x˜. Specifically, the normalization statistics are
computed solely using the observed values (excluding masked positions), ensuring that there is no
data-leakage in the training and imputation process.Where the paper says it
Derivation of Soft-Label Loss Decomposition
The per-token loss decomposes as L = (1 − ϵ)LNELBO + ϵ DKL(u∥pθ) + ϵ H(u) const, which is derived by splitting the cross-entropy and applying the KL form of the second term. This decomposition shows that for epsilon > 0, the additional non-negative term penalizes probability mass placed outside the ordinal neighborhood of y. p. 3 Crucially, we assign zero probability to the [MASK] token (s0 = 0) and distant
bins. This concentrates probability mass on valid ordinal neighbors while preserving the structural
separation of the mask state.Where the paper says it
Continuous Reconstruction via Expectation
Continuous values are recovered by computing the probability-weighted expectation of the predicted token distribution, enabling precise dense reconstruction from discrete outputs This is achieved by averaging the predicted probability distributions across M independent noise samples. The result is then denormalized using the instance-wise statistics to recover the original scale. p. 5 The continuous value is then recovered as the expectation over the averaged distribution:
xˆt,c =
X
K
k=1
p¯
(k)
t,c · vk, (8)
where vk is the center value of the k-th bin within [−1. The result is then denormalized using
the instance-wise statistics to recover the original scale.Where the paper says it
Experimental Settings Summary
The experiments are conducted on Energy, ETTh, Weather, and Sine datasets They simulate data corruption using Uniform masking and Geometric masking across missing rates of 30%, 50%, and 70%. The comparison includes a comprehensive suite of 19 baselines spanning various architectural paradigms. p. 5 For both scenarios, we assess performance across varying missing rates of
30%, 50%, and 70%. We compare MDTIM against a comprehensive suite of 19 baselines spanning various architectural
paradigms. evaluate each trained model across three independent inference trials using distinct random seeds for
mask generation, and report the averaged Mean Absolute Error (MAE). 2 Imputation Performance
Main Results.Where the paper says it
Where the paper says it
Ablation Study Summary
The ablation study on unmasking and labeling strategies confirms that Expectation-based Unmasking consistently reduces error across all datasets Soft Labeling's contribution scales with the structural regularity of the data, being most effective when the underlying signal exhibits clear ordinal or periodic structure. The sensitivity to vocabulary size K shows that K=40 achieves the best balance between discretization resolution and classification complexity. p. 8 208
Sensitivity to Vocabulary Size (K). We further
investigate the impact of the vocabulary size K on
imputation accuracy (Table 8), which exhibits a tradeoff between discretization resolution and classification complexity.Where the paper says it
Limitations and Future Work
A limitation is that the spectral consistency regularization assumes the presence of stable frequency components and is therefore disabled on benchmarks dominated by irregular sampling such as PhysioNet A data-adaptive discretization scheme that automatically selects the resolution would further improve generalization. Extending MDTIM to extremely long sequences and multi-scale or non-stationary domains like financial tick data remains for future work. p. 9 Designing an adaptive frequency objective that modulates its influence based on
local signal regularity is a promising direction. Finally, our evaluation focuses on standard imputation
benchmarks of moderate length; extending MDTIM to extremely long sequences and to multi-scale
or non-stationary domains such as financial tick data remains for future work.Where the paper says it
Data Specifications Summary
The datasets include Energy with 28 channels, ETTh with 7 channels, Weather with 21 channels, and Sine with 5 channels These datasets cover a broad range of dynamics observed in long-horizon multivariate time-series. The Energy dataset contains electricity-related measurements with strong seasonality and occasional irregular variations. p. 1 2 Related Work
2. 1 Time Series Imputation
Time series data are often partially observed due to sensor faults, irregular sampling, or datacollection constraints [35]. Our experiments focus on imputation accuracy, robustness to complex missing patterns,
and computational efficiency. 1 Experimental Setup
We evaluate our method on widely used benchmarks: Energy, ETTh, Weather, and the synthetic Sine
dataset. 1 Experimental Setup
We evaluate our method on widely used benchmarks: Energy, ETTh, Weather, and the synthetic Sine
dataset. To rigorously test the models, we simulate data corruption using two distinct mechanisms:
Uniform masking, where observations are dropped randomly to mimic independent failures, and
Geometric masking, which removes consecutive time steps to simulate prolonged sensor malfunctions
or transmission errors. On Energy, where the dynamics are less structured,
Soft Labeling offers no further improvement beyond Expectation-based Unmasking. These results
indicate that Soft Labeling is most effective when the underlying signal exhibits clear ordinal or
periodic structure that ordinal-aware guidance can exploit.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Evaluation Metrics Summary
MAE and MSE are reported averaged over the masked entries M For probabilistic settings, the Continuous Ranked Probability Score (CRPS) is used to assess calibration and sharpness. The CRPS approximation involves averaging stochastic samples generated by the model over multiple independent noise realizations.
Visualization Summary
Figures 5, 6, 7, and 8 provide comprehensive visualizations of the imputation results across all benchmark datasets The green lines denote the imputed values reconstructed by MDTIM, effectively capturing the complex temporal dynamics and periodicity of the multivariate time series.
Forecasting Performance
MDTIM yields substantially lower forecasting error than both baselines across all horizons on Energy This indicates that the imputation accuracy of MDTIM propagates to downstream forecasting. The results show that MDTIM reduces MAE by 19–26% over SAITS and 16–19% over CSDI on Energy forecasts.
Training Configuration Details
The architecture of MDTIM is based on the Factorized Temporal-Variate Backbone described in Section 4 The utilized hyperparameters include a Hidden Size (D) of 256, Attention Heads of 16, DiT Blocks (L) of 5, and a Vocabulary size (K) of 40. The training employs a Log-Linear Noise Schedule and an importance sampling via a time-dependent loss weight w(t). p. 4 For a set of masked indices Mt at timestep t, the diffusion loss is formulated as:
Ldiff = Et
"
w(t)
X
l∈Mt
−
s
(l)
, log pθ(z
(l)
t
)
#
, (5)
where log pθ(z
(l)
t
) ∈ R
K is the predicted log-probability vector, and s
(l) ∈ R
K denotes the soft
target distribution. The detailed formulation of time-dependent weight w(t) and the corresponding
noise schedule are detailed in Appendix C. For a set of masked indices Mt at timestep t, the diffusion loss is formulated as:
Ldiff = Et
"
w(t)
X
l∈Mt
−
s
(l)
, log pθ(z
(l)
t
)
#
, (5)
where log pθ(z
(l)
t
) ∈ R
K is the predicted log-probability vector, and s
(l) ∈ R
K denotes the soft
target distribution. The detailed formulation of time-dependent weight w(t) and the corresponding
noise schedule are detailed in Appendix C.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved AI system can perform time series imputation by leveraging masked diffusion model training paradigms to directly predict original values rather than added noise, aligning both representation and learning objectives with the imputation task.
-
The system can handle continuous and ordinal nature of time series data by introducing Stochastic Discretization, which maps continuous values to ordinal-aware tokens while preserving continuous dynamics, and Ordinal-Aware Soft Labeling to capture ordinal relationships between tokens.
-
The system can achieve precise continuous reconstruction by employing an Expectation-based Unmasking strategy and Spectral Consistency Regularization during training, ensuring the model
reconstruct[s] continuous time series precisely.
-
The system can be made robust to various missing scenarios by utilizing a dual-domain objective combining Ordinal-Aware Masked Diffusion with Spectral Consistency Regularization, which minimizes the total training objective Ltotal = Ldiff + λLFFT.
-
The system can be made computationally efficient for inference by using an Expectation-based Unmasking strategy to compute the
expectation over the averaged distribution
for continuous reconstruction, allowing it to complete inference in as little as 4.05 seconds on small models compared to continuous diffusion baselines which require up to 894 seconds. -
The system can be adapted for long-term dependencies by extending evaluations to sequence lengths L = 192, where the performance advantage of MDTIM
grows with the sequence length,
indicating it captures global context without attention dilution often observed in baselines. -
The system can provide reliable uncertainty estimates for imputation by using a Continuous Ranked Probability Score (CRPS) based on sample-based estimation, which rewards both calibration and sharpness, leading to lower CRPS values compared to diffusion-based baselines like CSDI.
-
The system can generalize effectively to real-world sparse and irregular observations by demonstrating that MDTIM's
ordinal-aware discrete formulation generalizes effectively to real-world sparse and irregular observations
when evaluated on the PhysioNet 2012 dataset, outperforming frequency-domain priors which become unreliable for such signals. -
The system can be made data adaptive by incorporating a data adaptive discretization scheme that automatically selects the resolution of the vocabulary size K based on the range and granularity of the underlying signal, addressing
the optimal vocabulary size K may vary across datasets.
Abstract
Time series imputation is a crucial area for reliable time series analysis, yet it remains challenging due to the complex temporal dynamics and noise of real-world data. Existing approaches, however, exhibit two limitations: missing and observed values are embedded within the same representation space without explicit structural separation, and continuous diffusion-based methods recover the missing values only through many iterative denoising steps rather than by predicting them directly. To address these, we propose the Masked Diffusion Time-series Imputation Model (MDTIM), which leverages the training paradigm of masked diffusion model for imputation tasks. The [MASK] token is structurally orthogonal to valid observations, and because the corrupted state is the partially observed series itself, the model is trained to predict the missing values directly from the observed ones, naturally aligning both the representation and the learning objective with the imputation task. To bridge the gap between discrete masked diffusion and the continuous, ordinal nature of time series, we further introduce Stochastic Discretization, which maps continuous values to ordinal-aware tokens while preserving continuous dynamics. Our experiments on diverse benchmarks confirm that MDTIM achieves superior robustness and scalability, consistently outperforming state-of-the-art deterministic and generative baselines across various missing scenarios. Code and project page are available at https://dongbeank.github.io/MDTIM/.
Sources
- Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models
- TSI-Bench: Benchmarking Time Series Imputation
- Classifier-Free Diffusion Guidance
- DiffWave: A Versatile Diffusion Model for Audio Synthesis
- Score-Based Generative Modeling through Stochastic Differential Equations
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis