BEAM3R: Beam's-eye-view architecture with Mamba-3 for implicit dose reconstruction
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.
Kai: I'm Kai.
Mira: And I'm Mira, here with Lev.
Kai: We're looking at the paper: "BEAM3R: Beam's-eye-view architecture with Mamba-3 for implicit dose reconstruction".
Paper summary: Kai: So, we're diving into this paper, "BEAM3R: Beam’s-eye-view architecture with Mamba-three for implicit dose reconstruction <ref:2609.04747#pg1,BEAM3R: Beam’s-eye-view architecture with Mamba-3 for implicit dose>." Essentially, the authors are tackling the need for super fast and accurate dose calculation when you're doing photon control point or proton beamlet dose estimation in the DoseRAD2026 challenge.
Mira: It claims they present BEAM3R, a framework that operates in beam’s-eye-view or BEV. The core idea is combining a Mamba state-space depthsequence core with physics based transport conditioning to handle longrange depth transport without using expensive three dimensional convolutions.
Lev: So, you're saying they are trying to model how the dose travels deep into the patient without needing those heavy three dimensional convolutions, which usually take a lot of computational power.
Kai: Exactly. They share a two dimensional convolutional encoder decoder architecture for both photon and proton dose tasks, processing these per plane BEV slices. And for proton beamlets specifically, they condition them on water equivalent thickness and the remaining range, which are the parameters that determine where the Bragg peak is going to be.
Mira: For the photon models, they use a bidirectional Mamba-three core to capture those dose contributions coming from materials downstream of the calculation point <ref:2609.04747#pg1,use a bidirectional Mamba-3 core to capture>. That's their specific approach for photon modeling.
Lev: That bidirectional aspect sounds interesting because it lets you see what’s happening on both sides of the calculation point simultaneously. What is this about?
Kai: The input handling is also quite specific to the task; for MRI tasks, they generate synthetic computed tomography or sCT using a patch based conditional generative adversarial network with a SwinUNETR backbone.
Mira: And for proton inputs, they feed it the normalized computed tomography, the Gaussian spot fluence, the cumulative water equivalent thickness and that remaining range we just talked about. They snap the BEV lattice to CT slice positions along the axial axis, which eliminates interpolation errors in that direction.
Lev: That sounds like a lot of specific input conditioning to handle different types of patient data and beam physics simultaneously. How do they train this whole thing?
Kai: They used a staged learning approach for the dose prediction models to speed up how fast they converge. For photon training, it started with a unidirectional model trained on the thorax cohort before fine tuning on combined cohorts.
Mira: The losses are defined in two parts: one for photons that's mean squared error plus a mean absolute error term weighted by something called P i Mi, and then for protons it's similar but with an extra term involving a valid loss to make sure the predictions are physically sensible.
Lev: Those losses look like they are trying to balance just getting the right number with ensuring the result is also plausible in terms of dose distribution. What did you find out about performance?
Paper summary: Kai: The CT based photon and proton models achieved similar beam level accuracy with a masked mean absolute error of zero point eighty-four for both tasks. But they noted that performance decreased when planning CT was replaced by sCT, with a substantially larger reduction for the proton prediction than for the photon prediction.
Mira: They also experimented with increasing model representational capacity through BEV extent and bidirectional depth modelling, and grid snapping along the S I direction and training in CT space utilizing super sampling enabled significant accuracy improvement because it reduced interpolation error.
Lev: That’s a lot of tuning to get that accuracy boost, especially with the grid snapping. What about those reverse depth models they mentioned?
Kai: The reverse depth Mamba branch improved the IDD distance more than masked mean absolute error, which aligns with its intended role in modeling those depth dependent effects.
Mira: Their limitations are that the performance of the sCT model depends on scanner characteristics, and they also used simplified approximations for the residual stopping power and range, which might not generalize beyond just the energies and anatomy in that challenge dataset.
Lev: So what would someone who only listens to this show actually take away from this work?
Kai: They should understand that by using this BEAM3R architecture, you can get accurate and rapid photon control point and proton beamlet dose calculation. It’s about making those calculations much faster while keeping the accuracy high enough for clinical use in environments like the DoseRAD2026 challenge.
Mira: The authors are showing how a Mamba architecture, combined with specific physics conditioning, can model deep transport without needing those heavy three dimensional convolutions. It’s moving away from older methods that rely on explicit three dee geometry <ref:2609.04747#pg1>.
Lev: It seems like the main argument is that this approach provides a way to improve beam level modelling for flexible plan optimisation and dose accumulation workflows, which is what people in clinical planning want.
Kai: To sum up, the paper shows how this BEAM3R architecture can deliver good beam level accuracy for both photon and proton tasks using a Mamba core on per plane slices within a beam's-eye-view framework.
Mira: The final point is that while it’s accurate on the test set with metrics like stratified mean absolute error of zero point eighty-four the performance still depends on things like the quality of sCT and those simplified range approximations they mentioned.
Lev: And for real hardware implementation, we need to keep in mind that even with this fast framework, you're still relying on these physics based transport conditionings for protons.
Kai: Exactly. The paper is a solid piece of work because it shows a viable path toward faster dose estimation using modern sequence models and state space architectures instead of just brute force methods.
Conclusion: Kai: So, we're wrapping up on BEAM3R, which is this framework that uses beam’s-eye view modeling with Mamba to estimate photon and proton doses for radiotherapy planning and delivery.
Mira: It seems like the authors are really focusing on making deep transport modeling faster by using state space models instead of those heavy three dimensional convolutions.
Lev: From an error correction standpoint, if you could run that on real hardware, it would be a huge win because we're still wrestling with approximating those long range dose contributions.
Kai: They did show that they got similar beam level accuracy for both photon and proton tasks using this method, though they pointed out some trade-offs when swapping in synthetic computed tomography for the actual CT scans.
Mira: The core idea is that the bidirectional Mamba core helps capture those things happening downstream without needing to process every single voxel in a massive three dimensional grid.
Lev: So it’s not just about speed; it’s about how they condition the input data, like using water equivalent thickness and remaining range to tell the proton model exactly what to expect at the Bragg peak position.
Kai: Yeah, and they found that increasing the model's capacity by extending that beam’s-eye view depth actually helped improve accuracy because it cut down on interpolation errors in a specific direction.
Mira: But they also had those limitations, you know, like how much the sCT model performs really depends on the scanner itself and those simplified range approximations they used.
Lev: Right, so even if the math looks clean and fast, you still have to worry about how well that physics conditioning holds up in a real clinical setting.
Kai: It’s a solid piece of work because it shows a viable path toward faster dose estimation using sequence models instead of just brute force methods.
Mira: And it opens up new ways to handle the complexities of photon and proton calculations in planning workflows that are currently much slower than what’s needed for real time guidance.
Lev: Next up, we need to look at how this framework compares when you're talking about actually running these things on real hardware versus just theoretical accuracy.
Chen Cheng, Michael Ferraro, James Grover, David E J Waddington, Emily Hewson
Image X Institute, The University of Sydney
physics.med-ph, cs.CV
Submitted: 2026-09-04
Updated: 2026-11-16
Comments: 14 pages, 5 figures
Code: https://github.com/doserad-IX/ImageXDoseRAD2026
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 89/100
The gist: To enable accurate and rapid photon control point and proton beamlet dose calculation in the DoseRAD2026 challenge…
Key concepts
- Beam's-eye-view (BEV)
- This is a visualization method where the patient is viewed from directly above, like looking down onto a map. In this framework, it allows the model to see the entire cross-section of tissues along an axis (like axial slices) simultaneously, which helps in accurately modeling how radiation travels through different depths.
- Mamba-3 State-Space Depthsequence Core
- This is a specific type of neural network architecture used as the core engine for modeling dose transport. It uses state-space principles to efficiently capture long-range dependencies in the data, allowing it to model how radiation interacts with materials deep within the body without needing complex 3D convolutions.
- Proton Beamlet Conditioning
- For proton calculations, this involves feeding the model specific information about the beamlet's path. This includes factors like water equivalent thickness and remaining range, which are crucial physical parameters that determine where the Bragg peak (the maximum dose) will occur for that specific treatment.
- Bidirectional Mamba-3 Core
- This is a specialized version of the Mamba-3 core used in photon dose modeling. It allows the model to look both forward and backward along the depth axis. This capability helps capture complex, depth-dependent dose contributions from materials located downstream of the calculation point.
Terminology
Summary
To enable accurate and rapid photon control point and proton beamlet dose calculation in the DoseRAD2026 challenge, BEAM3R presents a dose estimation framework operating in beam’s-eye-view (BEV) This paper introduces BEAM3R, a dose estimation framework operating in beam’s-eye-view (BEV)
Core Innovation and Architecture
The core innovation of BEAM3R combines a Mamba-3 state-space depthsequence core with physics-based transport conditioning to model longrange depth transport without expensive 3D convolutions The framework shares a 2D CNN encoder-decoder architecture for photon and proton dose tasks, processing per-plane BEV slices Proton beamlets are conditioned on water equivalent thickness and remaining range, encoding the parameters determining Bragg peak position Photon models use a bidirectional Mamba-3 core to capture dose contributions from materials downstream of the calculation point
Input Conditioning and Data Handling
The model incorporates specific conditioning for different tasks, such as proton transport conditioning For MRI-based tasks, synthetic CTs (sCT) are generated by a patch-based conditional GAN with a SwinUNETR backbone Proton inputs consisted of the normalised CT, Gaussian spot fluence, cumulative water-equivalent thickness (WET), and remaining range The BEV lattice was snapped to CT slice positions along the axial axis, eliminating interpolation in that direction
Model Training Strategy
The training employed a staged learning approach for dose prediction models to accelerate model convergence Photon training comprised three stages, starting with a unidirectional model trained on the thorax cohort before fine-tuning on combined cohorts The proton model was trained only on the thorax training cohort without the Bragg residual head initially Losses were defined as Lphoton = MSE(ˆy, y) + λMAE P i Mi yˆi − yi / P i Mi, (5) and Lproton = MSE(ˆy, y) + λMAE P i Mi yˆi − yi / P i Mi + λvalid P i Vi max(0, −yˆi) squared / P i Vi, (6)
Performance and Design Trade-offs
CT-based photon and proton models achieved similar beam-level accuracy with masked MAE of 0.0084 for both Performance decreased when planning CT was replaced by sCT, with a substantially larger reduction for proton than photon prediction The development experiments favored increased greater model representational capacity through BEV extent and bidirectional depth modelling Grid snapping along the S-I direction and CT-space training utilising super-sampling enabled significant accuracy improvement via reduced interpolation error The reverse-depth Mamba branch improved IDD distance more than masked MAE, consistent with its intended role in modelling depth-dependent effects Limitations include the dependence of sCT model performance on scanner characteristics and the simplified RSP and range approximations
Evaluation Metrics
Beam-level performance was evaluated using several official plan-level metrics, including stratified MAE, 3D local gamma pass rate at 1%/1 mm, and a DVH-based clinical score Packaged models were additionally evaluated on the hidden test set through the DoseRAD2026 Grand Challenge platform The runtime was estimated using a non-negative linear model of fixed, per-image, and per-dose-map costs The final results showed CT to photon/proton tasks with runtimes around 23.4 s and 18.4 s respectively The MRI-based pipelines exhibited increased runtimes, reaching 39. p. 77 s and 42.8 s for the corresponding tasks
Author Contributions
D.E.J.W — Writing – review & editing, Supervision & Resources E.H. — Writing – review & editing, Supervision C.C. – Conceptualisation, Methodology, Software, Validation, Formal Analysis, Investigation, Data Curation, Writing – Original Draft M. p. 7F. – Conceptualisation, Methodology, Software, Validation, Formal Analysis, Investigation J. p. 7G. – Conceptualisation, Methodology, Software, Validation & Investigation
References
-
Bortfeld, T. p. 3: An analytical approximation of the Bragg curve for therapeutic proton beams
-
Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H.R., Xu, D.: Swin UNETR: Swin transformers for semantic segmentation of brain tumors in MRI images
-
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks
-
Keall, P.J., El Naqa, I., Fast, M.F., Hewson, E.A., Hindley, N., Poulsen, P., Sengupta, C., Tyagi, N., Waddington, D.E.: Real-time dose-guided radiation therapy
-
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization
-
Kontaxis, C., Bol, G.H., Lagendijk, J.J.W., Raaymakers, B.W.: Deepdose: Towards a fast dose calculation engine for radiation therapy using deep learning
-
Lahoti, A.S., Li, K., Chen, B., Wang, C., Bick, A., Kolter, Z., Dao, T., Gu, A.: Mamba-3: Improved sequence modeling using state space principles
-
Li, M., Choulilitsa, E., Fankhauser, L., Albertini, F., Lomax, A., Zhang, Y.: Neural network-driven direct CBCT-based dose calculation for head-and-neck proton treatment planning
-
Li, M., Winterhalter, C., Li, X., Safai, S., Lomax, A., Zhang, Y.: A proofof-concept study of direct magnetic resonance imaging-based proton dose calculation for brain tumors via neural networks with Monte Carlo-comparable accuracy
-
Lim-Reinders, S., Keller, B.M., Al-Ward, S., Sahgal, A., Kim, A.: Online adaptive radiation therapy
-
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization
-
Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Smolley, S.P.: Least squares generative adversarial networks
-
Neishabouri, A., Wahl, N., Mairani, A., Köthe, U., Bangert, M.: Long short-term memory networks for proton dose calculation in highly heterogeneous tissues
-
Pastor-Serrano, O., Dong, P., Huang, C., Xing, L., Perkó, Z.: Sub-second photon dose prediction via transformer neural networks
-
Pastor-Serrano, O., Perkó, Z.: Millisecond speed deep learning based proton dose calculation with Monte Carlo accuracy
-
Rogers, D.W.O.: Fifty years of monte carlo simulations for medical physics
-
Rogowski, V., Terpstra, M.L., Wahl, N., Kamp, F., van der Bijl, E., Galapon Jr, A., Kurz, C., Xin, B., Sun, Z., Min, H.: Generating synthetic computed tomography for radiotherapy: SynthRAD2025 challenge report
-
Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network
-
Thummerer, A., van der Bijl, E., Galapon, A.J., Kamp, F., Savenije, M., Muijs, C., Aluwini, S., Steenbakkers, R.J., Beuel, S., Intven, M.P.: SynthRAD2025 Grand Challenge dataset: Generating synthetic CTs for radiotherapy from head to abdomen
-
Witte, M., Sonke, J.J.: A deep learning based dynamic arc radiotherapy photon dose engine trained on Monte Carlo dose distributions
-
Xiao, F., Delopoulos, N., Wahl, N., Volz, L., Bucher, L., Maspero, M., Palacios, M., Li, M., Schulz, S., Rogowski, V., Zhang, Y.: Doserad2026 challenge dataset: Ai accelerated photon and proton dose calculation for radiotherapy (2026)
-
Xiao, F., Wahl, N., Belka, C., Kurz, C., Dedes, G., Landry, G.: Multi-model study of fast VMAT segment dose calculation with deep learning
Summary of Findings
CT-based photon and proton models achieved similar beam-level accuracy with masked MAE of 0.0084 for both Performance decreased when planning CT was replaced by sCT, with a substantially larger reduction for proton than photon prediction The greater proton sensitivity is consistent with the dependence of proton range on integrated stopping power Architectural design choices favored increased greater model representational capacity through BEV extent and bidirectional depth modelling Grid snapping along the S-I direction and CT-space training utilising super-sampling enabled significant accuracy improvement via reduced interpolation error The reverse-depth Mamba branch improved IDD distance more than masked MAE, consistent with its intended role in modelling depth-dependent effects The validation cohorts contained only two or three patients per task, preventing reliable confidence intervals or formal patient-level significance testing The temporal convolution, energy prefix, slice scaler, and Bragg residual head were not independently ablated Due to compute and time constraints, proton fine-tuning was not completed on the full training cohort potentially limiting anatomical generalisability The sCT model performance may also depend on scanner and acquisition characteristics The simplified RSP and range approximations may not generalise beyond the energies and anatomy represented in the challenge dataset
Ablation Insights
The development experiments favored increased greater model representational capacity through BEV extent, increased model width, and bidirectional depth modelling Grid snapping along the S-I direction and CT-space training utilising super-sampling enabled significant accuracy improvement via reduced interpolation error Increasing BEV depth from 256 to 384 planes reduced masked MAE by 8.5% and IDD distance by 30. p. 70% on the thorax cohort Doubling model capacity reduced masked MAE by 15. p. 95% Adding the reverse-depth Mamba branch reduced masked MAE by 5. p. 93% and the IDD distance by 20. p. 7
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Beam’s-eye-view architecture with Mamba-3 for implicit dose reconstruction enables accurate and rapid photon control point and proton beamlet dose calculation by combining a Mamba-3 state-space depthsequence core with physics-based transport conditioning to model longrange depth transport without expensive 3D convolutions.
-
The system can predict the dose from one control point (photon model) or one beamlet (proton model) using task-specific sequence cores and inputs, such as proton transport conditioning, allowing for flexible plan optimization and dose accumulation workflows.
-
Axial grid alignment of BEV lattices with CT slices and an implicit super-resolution representation via sub-pixel phase packing, evaluated by a differentiable Triton-accelerated resampler, reduces interpolation artifacts and supports high spatial resolution in the output field.
-
For MRI tasks, the system can generate synthetic CTs (sCT) using a patch-based conditional GAN with a SwinUNETR backbone to create realistic input data for CT-to-dose models.
-
The proton model is conditioned on derived transport quantities like cumulative WET and remaining range along each ray, directly encoding the physical determinant of Bragg-peak position.
Sources
- Adam: A Method for Stochastic Optimization
- Decoupled Weight Decay Regularization
- Generating synthetic computed tomography for radiotherapy: SynthRAD2025 challenge report
- DoseRAD2026 Challenge dataset: AI accelerated photon and proton dose calculation for radiotherapy
Related papers
- BART Online Open-Source Sequence Toolbox for Computational MRI
- Constrained estimation of rotational invariants of the cumulant expansion (RICE) for rapid tensor-valued diffusion MRI
- Improved Myelin Water Fraction Estimation and Uncertainty Mapping in Multicomponent T2 Relaxometry via NNLS Bootstrap Aggregation
- Radiation damage to normal mammalian tissue in vivo with laser-driven protons at ultra-high instantaneous dose rate
- MORSE-PI -- Flexible and artefact-free image reconstruction for structural and functional QSM and other phase-critical imaging applications
- RadMaps: A Geospatial Framework for Simultaneously Modelling Capacity and Geographic Constraints on Radiotherapy Access