Neural Realized Measures: Forecast-Supervised Intraday Aggregation for Volatility Forecasting

arXiv:2611.07287 · q-fin.ST, stat.ML · Submitted 2026-11-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Quantitative Finance. The day in quantitative finance: models of markets, risk and prices.

Ruth: We're looking at the paper: "Neural Realized Measures".

Warren: Here's the gist of the paper. High-frequency volatility forecasting typically separates measurement from prediction: intraday observations are first compressed into a daily realized measure using a fixed aggregation rule, and the resulting statistic is then passed to a forecasting model The Core Proposal The paper proposes Neural Realized Measures (NRM), which is a primitive-preserving, forecast-supervised aggregation framework that retains the original measure-specific intraday contributions but replaces uniform aggregation with positive, mean-normalized context-dependent weights NRM is a primitive-preserving, forecast-supervised aggregation framework that retains the original measure-specific intraday contributions but replaces uniform aggregation with positive, mean-normalized context-dependent weights <ref:2611.07287#pg4>.

Ruth: Uniform weights exactly recover the corresponding raw realized measure (RRM), so the classical construction is nested as a special case <ref:2611.07287#pg5>.

Warren: Because NRM remains an explicit weighted sum of established primitive contributions, it also retains direct bar-level attribution <ref:2611.07287#pg4>.

Ruth: Primitive Preservation and Structure NRM is built upon the concept of retaining the statistically defined primitive and making only its aggregation task-adaptive <ref:2611.07287#pg4>.

Paper summary: Ruth: We just talked about how this paper, Neural Realized Measures: Forecast-Supervised Intraday Aggregation for Volatility Forecasting, suggests that instead of treating measurement as a fixed step before forecasting, we can learn the aggregation weights directly from the forecast objective >

Warren: The core proposal is NRM. It's a primitive preserving framework that keeps the original measure specific contributions but uses context dependent positive weights to replace uniform aggregation >

Iris: They show that uniform weights perfectly recover the raw realized measure, which they call RRM, so the classical construction is just a special case of this framework >

Ruth: So the authors are saying they can learn these weights by training a neural network where the loss function penalizes the average daily divergence between their forecast and what actually occurred >

Warren: And to keep things honest, they include a KL to uniform regularization term, which specifically punishes deviations from that classical uniform aggregation benchmark >

Iris: The structure they are building is important because it keeps the primitive fixed—that core measurement concept doesn't change—but only the aggregation weights learn based on the downstream task >

Ruth: This puts NRM in a structured middle ground between traditional econometric measurement and pure end-to-end neural representation learning >

Warren: It’s not just throwing away the old measure for some black box representation, it's adapting how that measure is summarized for forecasting >

Iris: And because the primitive contribution remains explicit, this framework allows for a direct interpretation of which bar level gets which weight in the final measure >

Ruth: That means we can actually examine what's driving the learned allocation rather than just looking at some abstract latent feature vector >

Conclusion: Warren: So to wrap up, this paper, Neural Realized Measures: Forecast-Supervised Intraday Aggregation for Volatility Forecasting, offers a way to learn the aggregation structure directly from forecasting objectives while maintaining the statistical integrity of realized measures >

Iris: The authors show that by using positive mean normalized context dependent weights instead of uniform ones, they can achieve better forecast performance in various volatility regimes across different assets >

Ruth: The implication is that for high frequency volatility forecasting, this isn't about completely abandoning the measure construction pipeline, but making it task specific >

Warren: It provides a structured alternative to both just using fixed realized measure aggregation and totally unrestricted end-to-end neural forecasting methods >

Iris: The paper notes that while they found improvements, the method is forecast specificationspecific, so it's better viewed as a representation tailored for that task rather than something universal >

Ruth: And they also flag some boundaries. For instance, the cryptocurrency setting provides consistent twenty-four hour UTC days but doesn't generalize well to markets with sessions or overnight gaps > <ref:2611.07287#pg1>

Warren: Plus, the simulation regimes are controlled stresses instead of fully calibrated microstructure models, and it excludes richer volume and order book information >

Brian Sing Fan Chan, Jennifer So Kuen Chan

University of Sydney

q-fin.ST, stat.ML

Submitted: 2026-11-05

Updated: 2026-11-05

Comments: 9 pages, 3 figures, 5 tables. Accepted at the ACM International Conference on AI in Finance (ICAIF 2026), Milan, Italy

DOI: 10.1145/3858063.3858785

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: High-frequency volatility forecasting typically separates measurement from prediction: intraday observations are first compressed into a daily realized measure using a fixed aggregation rule…

Key concepts

Neural Realized Measures (NRM)
NRM is a method that aggregates high-frequency intraday price movements by learning optimal, positive weights for each contribution. It keeps the original data structure intact while replacing simple uniform averaging with weights that adapt based on the specific forecasting task.
Primitive Preservation
This concept means NRM keeps the fundamental, statistically defined building blocks (primitives) of volatility measures unchanged. The learning process only focuses on how these fixed primitives should be combined together, rather than redefining what the underlying measure itself is.
Forecast-Supervised Aggregation
The aggregation weights and model parameters are learned by training the system to minimize prediction errors (the loss). This means the aggregation method is guided directly by the goal of making accurate volatility forecasts, linking data processing directly to predictive performance.

Terminology

Summary

High-frequency volatility forecasting typically separates measurement from prediction: intraday observations are first compressed into a daily realized measure using a fixed aggregation rule, and the resulting statistic is then passed to a forecasting model

The Core Proposal

The paper proposes Neural Realized Measures (NRM), which is a primitive-preserving, forecast-supervised aggregation framework that retains the original measure-specific intraday contributions but replaces uniform aggregation with positive, mean-normalized context-dependent weights NRM is a primitive-preserving, forecast-supervised aggregation framework that retains the original measure-specific intraday contributions but replaces uniform aggregation with positive, mean-normalized context-dependent weights <ref:2611.07287#pg4>. Uniform weights exactly recover the corresponding raw realized measure (RRM), so the classical construction is nested as a special case <ref:2611. p. 0

Where the paper says it

We propose Neural Realized Measures (NRM), a primitive-preserving, forecast-supervised aggregation framework that retains the original measure-specific intraday contributions but replaces uniform aggregation with positive, mean-normalized context-dependent weights. Uniform weights exactly recover the corresponding raw realized measure (RRM), so the classical construction is nested as a special case.

07287#pg5>. Because NRM remains an explicit weighted sum of established primitive contributions, it also retains direct bar-level attribution <ref:2611. p. 0
Where the paper says it

Uniform weights exactly recover the corresponding raw realized measure (RRM), so the classical construction is nested as a special case. Because NRM remains an explicit weighted sum of established primitive contributions, it also retains direct bar-level attribution.

07287#pg4>.

Primitive Preservation and Structure

NRM is built upon the concept of retaining the statistically defined primitive and making only its aggregation task-adaptive <ref:2611.07287#pg4>. The primitive remains fixed; learning operates only through the aggregation weights w m t,k <ref:2611. p. 1

Where the paper says it

ICAIF ’26, November 14–17, 2026, Milan, Italy Brian Sing Fan Chan and Jennifer So Kuen Chan For measure family 𝑚, 𝑋 𝑚 𝑡 = ∑︁𝐾 𝑘=1 𝑤 𝑚 𝑡,𝑘𝑥 𝑚 𝑡,𝑘, 1 𝐾 ∑︁𝐾 𝑘=1 𝑤 𝑚 𝑡,𝑘 = 1. (2) The primitive 𝑥 𝑚 𝑡,𝑘 remains fixed; learning operates only through the aggregation weights 𝑤 𝑚 𝑡,𝑘 .

07287#pg5>. This framework occupies a structured middle ground between fixed econometric measurement and end-to-end neural representation learning <ref:2611. p. 1
Where the paper says it

Rather than replacing the realized measure with an unrestricted latent representation, it retains the statistically defined primitive and makes only its aggregation task-adaptive. In this sense, NRM occupies a structured middle ground between fixed econometric measurement and end-to-end neural representation learning.

07287#pg4>. The primitive x m t,k remains fixed; learning operates only through the aggregation weights w m t,k <ref:2611. p. 1
Where the paper says it

ICAIF ’26, November 14–17, 2026, Milan, Italy Brian Sing Fan Chan and Jennifer So Kuen Chan For measure family 𝑚, 𝑋 𝑚 𝑡 = ∑︁𝐾 𝑘=1 𝑤 𝑚 𝑡,𝑘𝑥 𝑚 𝑡,𝑘, 1 𝐾 ∑︁𝐾 𝑘=1 𝑤 𝑚 𝑡,𝑘 = 1. (2) The primitive 𝑥 𝑚 𝑡,𝑘 remains fixed; learning operates only through the aggregation weights 𝑤 𝑚 𝑡,𝑘 .

07287#pg5>.

How the Training Signal is Built

The neural network learns the aggregation weight w m t,k in Eq. (9) for NRM in Eq. p. 0

Where the paper says it

We ask whether the aggregation stage itself can instead be learned from the downstream forecasting objective while preserving the statistical structure of established realized measures. We propose Neural Realized Measures (NRM), a primitive-preserving, forecast-supervised aggregation framework that retains the original measure-specific intraday contributions but replaces uniform aggregation with positive, mean-normalized context-dependent weights.

(10) and the forecast model parameters <ref:2611. p. 3
Where the paper says it

(9) for NRM in Eq. (10) and the forecast model parameters.

07287#pg4>. The network loss penalizes the average daily divergence D m t <ref:2611. p. 3
Where the paper says it

To quantify departure from equal-weight aggregation, define the implied simplex weights 𝛼 𝑚 𝑡,𝑘 = 𝑤 𝑚 𝑡,𝑘 /𝐾, the uniform allocation 𝑢𝑘 = 1/𝐾, and the daily divergence 𝐷 𝑚 𝑡 := 𝐷KL (𝛼 𝑚 𝑡 ∥𝑢) = 1 𝐾 ∑︁𝐾 𝑘=1 𝑤 𝑚 𝑡,𝑘 log𝑤 𝑚 𝑡,𝑘 . (12) For a mini-batch B of daily observations, the network loss penalizes the average daily divergence: Loss = QLIKE + 𝜆L 𝑚 KL, L 𝑚 KL = 1 |B| ∑︁ 𝑡 ∈ B 𝐷 𝑚 𝑡 .

07287#pg4>. The KL-to-uniform regularization penalizes deviations from classical uniform aggregation <ref:2611. p. 3
Where the paper says it

where 𝜏 > 0 controls weight concentration. We fix 𝜏 = 1 so that concentration is governed by the learned score scale, while validationselected KL-to-uniform regularization penalizes deviations from classical uniform aggregation.

07287#pg4>. QLIKE is given by QLIKE(V t,V b t) = V t V b t - log V t / V b - 1 <ref:2611. p. 4
Where the paper says it

The tuning parameter 𝜆 is selected together with the other network hyperparameters. QLIKE is given by QLIKE(𝑉𝑡 ,𝑉b𝑡 ) = 𝑉𝑡 𝑉b𝑡 − log 𝑉𝑡 𝑉b𝑡 − 1, (15) The network outputs the log-variance forecast 𝑦b𝑡 = log𝑉b𝑡 , while 𝑦𝑡 = log𝑉𝑡 denotes the corresponding target.

07287#pg4>.

Where the Gains Come From

Controlled simulations show lower reported losses for contemporaneous and next-day latent integrated variance across the evaluated continuous, jump, and conditional event-stress regimes <ref:2611.07287#pg6> Across cryptocurrency data, paired RRM–NRM comparisons show broad improvements under SF, HAR, and MIDAS forecasting specifications <ref:2611. p. 8

Where the paper says it

Controlled simulations show lower reported losses for contemporaneous and next-day latent integrated variance across the evaluated continuous, jump, and conditional event-stress regimes. In cryptocurrency data, paired RRM–NRM comparisons show broad gains under SF, HAR, and MIDAS forecasting specifications, with particularly consistent improvements under QLIKE.

07287#pg6>. Validation-selected NRM specifications attain the lowest mean QLIKE across the six assets among the evaluated approaches <ref:2611. p. 6
Where the paper says it

Performance remains heterogeneous across assets and metrics, and the strongest competing benchmark differs across markets. Across the six assets, however, the validation-selected NRM specifications attain the lowest mean QLIKE among the compared approaches.

07287#pg6>. Ablation evidence shows that Time and Local inputs produce the largest average deterioration in QLIKE for HAR–NRM specifications <ref:2611. p. 6
Where the paper says it

3 Feature-group ablation Table 5 reports controlled leave-one-group-out ablations of the HAR–NRM specifications. Removing Time and Local produces the largest average deterioration, increasing QLIKE by 4.

07287#pg6>.

Mechanism and Interpretation

Because NRM retains explicit weights on established primitive contributions, its learned allocation can be examined directly rather than only through a latent representation <ref:2611.07287#pg4>. The mechanism analysis examines the validation-selected SF–NRM score networks for BTC–RV, ETH–RV, and SOL–RV as representative liquid-asset cases <ref:2611. p. 6

Where the paper says it

6 Mechanism and Interpretation Because NRM retains explicit weights on established primitive contributions, its learned allocation can be examined directly rather than only through a latent representation. The mechanism analysis therefore examines the validation-selected SF–NRM score networks for BTC–RV, ETH–RV, and SOL–RV as representative liquid-asset cases.

07287#pg4>. The targeted SHAP display focuses on the Time and Local groups, which have the largest and most directly interpretable bar-level effects <ref:2611. p. 7
Where the paper says it

The late-day pattern may partly reflect the UTC boundary or forecast origin and should not be read as a universal session effect. 2 Local-context and time effects The targeted SHAP display [20] focuses on the Time and Local groups, which have the largest and most directly interpretable barlevel effects.

07287#pg4>. The common direction suggests that the reallocation mechanism is not specific to one cryptocurrency, although differences in magnitude indicate asset-level heterogeneity <ref:2611. p. 7
Where the paper says it

Locally small realized-variance contributions receive positive score adjustments, whereas locally elevated contributions generally receive negative adjustments. The common direction suggests that the reallocation mechanism is not specific to one cryptocurrency, although differences in magnitude indicate asset-level heterogeneity.

07287#pg4>.

Conclusion and Limitations

NRM provides a structured alternative to both fixed realized-measure aggregation and unrestricted end-to-end neural forecasting <ref:2611.07287#pg4>. The evidence has several boundaries, as NRM is forecasting-specificationspecific and should be interpreted as a task-specific representation rather than a reusable universal measure <ref:2611. p. 8

Where the paper says it

The evidence has several boundaries. NRM is forecasting-specificationspecific and should be interpreted as a task-specific representation rather than a reusable universal measure.

07287#pg4>. The cryptocurrency setting provides consistent 24/7 UTC days but does not establish generalization to markets with sessions and overnight gaps <ref:2611. p. 8
Where the paper says it

NRM is forecasting-specificationspecific and should be interpreted as a task-specific representation rather than a reusable universal measure. The cryptocurrency setting provides consistent 24/7 UTC days but does not establish generalization to markets with sessions and overnight gaps.

07287#pg4>. The empirical target is a noisy realized proxy, the simulation regimes are controlled stresses rather than fully calibrated microstructure models, and STLG excludes richer volume and order-book information <ref:2611. p. 8
Where the paper says it

The cryptocurrency setting provides consistent 24/7 UTC days but does not establish generalization to markets with sessions and overnight gaps. The empirical target is a noisy realized proxy, the simulation regimes are controlled stresses rather than fully calibrated microstructure models, and STLG excludes richer volume and order-book information.

07287#pg4>. Future work can examine transferable aggregation, alternative day boundaries, traditional asset classes, and economic-value applications <ref:2611. p. 8
Where the paper says it

The empirical target is a noisy realized proxy, the simulation regimes are controlled stresses rather than fully calibrated microstructure models, and STLG excludes richer volume and order-book information. Future work can examine transferable aggregation, alternative day boundaries, traditional asset classes, and economic-value applications.

07287#pg4>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Learned aggregation for volatility prediction allows AI systems to treat measurement construction as part of the forecasting problem by estimating aggregation weights from a downstream objective. This enables systems to learn how much each intraday observation contributes based on its predictive power, rather than relying on a fixed aggregation rule like the raw realized measure (RRM).

  2. Primitive preservation ensures that AI systems retain the statistical primitive that defines the realized measure, meaning they do not discard established volatility metrics. The framework retains the original measure-specific intraday contributions but replaces uniform aggregation with positive, mean-normalized context-dependent weights.

  3. Interpretable allocation allows researchers to inspect the learned weights directly, revealing systematic effects. Ablation and SHAP analyses show systematic time-of-day and nonlinear local-context effects in the learned allocation, allowing for direct attribution of volatility information based on specific market conditions.

  4. Model flexibility is applied narrowly to the aggregation stage, aligning with research emphasizing task-specific inductive bias over architectural scale. This means AI systems can adapt their structure to identify which part of that structure benefits from task-adaptive learning rather than using larger models universally for all tasks.

  5. The system can perform controlled simulation and empirical forecasting across various market stresses, including continuous diffusion, price jumps, and gradual jump regimes. It can evaluate how accurately the constructed daily measure aligns with targets like contemporaneous IV oracle and next-day log realized variance.

  6. The AI system can be deployed to predict latent integrated variance in Heston simulations and cryptocurrency data using Neural Realized Measures (NRM). Empirical results show NRM improves QLIKE in all paired out-of-sample comparisons, with mean gains of up to 9.22% under the MIDAS specification.

  7. The system can achieve lower mean QLIKE across six different cryptocurrencies compared to established approaches like HARNet, DeepVol, and XGBoost when using validationselected NRM specifications as benchmarks. This demonstrates that the learned aggregation provides a robust improvement in predictive content over fixed econometric measurements.

Abstract

High-frequency volatility forecasting typically separates measurement from prediction: intraday observations are first compressed into a daily realized measure using a fixed aggregation rule, and the resulting statistic is then passed to a forecasting model. This conventional pipeline therefore treats measure construction as forecast-agnostic preprocessing. We ask whether the aggregation stage itself can instead be learned from the downstream forecasting objective while preserving the statistical structure of established realized measures. We propose Neural Realized Measures (NRM), a primitive-preserving, forecast-supervised aggregation framework that retains the original measure-specific intraday contributions but replaces uniform aggregation with positive, mean-normalized context-dependent weights. Uniform weights exactly recover the corresponding raw realized measure (RRM), so the classical construction is nested as a special case. Because NRM remains an explicit weighted sum of established primitive contributions, it also retains direct bar-level attribution. Across SF, HAR, and MIDAS forecasting specifications, NRM is evaluated against corresponding RRM specifications under controlled comparisons. In Heston simulations, NRM reduces RMSE for both contemporaneous and next-day latent integrated variance across all evaluated primitive--regime combinations. On six cryptocurrencies using five-minute data, NRM improves QLIKE in all 54 paired out-of-sample comparisons, with mean gains of 7.23%, 8.15%, and 9.22% under SF, HAR, and MIDAS, respectively. Validation-selected NRM specifications also achieve the lowest mean QLIKE across the six assets among the evaluated approaches. Ablation and SHAP analyses reveal systematic time-of-day and nonlinear local-context effects in the learned allocation.

Sources

Related papers