Action-Guided Reachability for Deterministic World Models of Vision-Based Systems

arXiv:2512.08991 · cs.CV, cs.LG · Submitted 2025-12-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Action-Guided Reachability for Deterministic World Models of Vision-Based Systems".

Jane: Here's the gist of the paper. End-to-end image controllers that map raw camera frames directly to control actions are increasingly deployed in safetycritical systems However, none of these tools can be directly applied when the controller input is a high-dimensional camera image because two fundamental challenges arise The authors propose a Deterministic World Model (DWM), a latent-free neural decoder that maps physical states directly to synthetic camera images, enabling closed-loop reachability analysis without the overapproximation caused by stochastic latent variables.

Tom: Pruning redundant reasoning tokens The DWM is designed to resolve two main challenges: by providing a closed-form, deterministic stateto-image mapping, it closes the verification loop and by keeping the input dimension equal to the physical state dimension, it reduces the high-dimensional image variability that causes verification to explode.

Jane: The DWM is trained with a novel dual loss combining saliency-map reconstruction and a controlconsistent term that preserves behavioral consistency with the real controller.

Tom: How the training signal is built The world model is trained using a weighted sum of two complementary losses The first loss, Saliency-Weighted Reconstruction Loss, assigns greater weights to controller-relevant pixels by estimating a saliency map from the controller itself by occlusion.

Paper summary: Jane: So, to conclude this paper on Action-Guided Reachability for Deterministic World Models of Vision-Based Systems, the authors present a framework that uses a deterministic world model as a latent-free decoder <ref:2512.08991#pg1>.

Tom: This approach uses a dual loss combining saliency map reconstruction and control consistency to ensure the model stays faithful where it matters for control <ref:2512.08991#pg3>.

Lu: The process involves constructing this verifiable world model, analyzing its reachability to get a reachable tube, and then transferring that guarantee to the real system using conformal prediction <ref:2512.08991#pg2>.

Jane: The key finding is that the DWM produces tighter reachable tubes than cGAN baselines and meets the target coverage after inflation across all four tested plant dynamics <ref:2512.08991#pg7>.

Tom: The main contribution is this deterministic world modeling approach that creates verifiable generative surrogates of camera sensors <ref:2512.08991#pg2>.

Lu: It also provides the first extension of star-based reachability analysis for closed-loop verification of vision-based systems <ref:2512.08991#pg3>.

Jane: This work shows that with probability at least one minus alpha, the inflated reachable sets contain the real trajectories <ref:2512.08991#pg3>.

Conclusion: Tom: So, we've been talking about this new work on action-guided reachability for deterministic world models of vision-based systems by these authors in this paper "Action-Guided Reachability for Deterministic World Models of Vision-Based Systems".

Jane: Right, so what's the actual big picture here? Basically, they’re taking those complicated end to end vision controllers and trying to figure out if they can safely control the real thing.

Lu: They replace the camera with this deterministic world model that takes a physical state directly and makes a synthetic image. It’s not using those fuzzy latent variables from other models.

Meng: So, they’re building a fake camera that’s super predictable based on where you are in the system. That sounds like it could be useful for checking safety before you deploy anything real.

Tom: Exactly, and the way they trained this model is clever too. They use two losses at once: one to make sure it reconstructs the image correctly, and another to keep the actions consistent with what the real controller does.

Jane: That consistency loss is interesting because it forces the fake camera to behave like a real one, which is crucial for safety checks.

Lu: And then they use symbolic propagation—this Star set thing—to find a reachable tube over each cell in the state space, which gives them a sound reachable area.

Tom: Before that, they used conformal prediction to take that tube and inflate it into a guarantee that holds for the real system with at least one minus alpha probability. That’s how they transfer the safety guarantee to reality.

Meng: So, what does this actually change for someone shipping models? It means we can get a tighter bound on what can happen, which is better than just relying on some baseline like a cGAN or a trajectory predictor.

Jane: It really shows that by making the world model deterministic and training it to match real control behavior, you get these much tighter tubes for safety verification.

Tom: The authors show they did this across four different plant dynamics, including things like braking systems and those Gym environments we mentioned before.

Lu: The results are pretty consistent; the deterministic approach yields tighter reachable tubes than the cGAN baselines on Pendulum and other test cases.

Meng: That’s what I wanted to hear. If we can get a model that gives us a more precise safety boundary, it makes deploying vision-based controllers way less risky at scale.

Jane: So, this framework essentially offers a reliable way to verify the safety of complex vision-based controllers by using a deterministic surrogate camera.

Tom: And the authors are pretty confident in this because they maintained a precision of one point across all those benchmarks, even after inflation.

Yuang Geng, Zhongzheng Zhang, Chengzhen Jiang, Yanru Li, Xinyang Wang, Zhuoyang Zhou, Hoang-Dung Tran, Ivan Ruchkin

University of Florida

cs.CV, cs.LG

Submitted: 2025-12-08

Updated: 2026-11-10

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: End-to-end image controllers that map raw camera frames directly to control actions are increasingly deployed in safetycritical systems However…

Key concepts

Deterministic World Model (DWM)
A latent-free neural decoder that maps physical states directly to synthetic camera images. Unlike other models, it has a fixed output for any given state, eliminating uncertainty from latent variables and enabling precise reachability analysis.
Saliency-Weighted Reconstruction Loss
A training loss that prioritizes pixels in the image reconstruction based on how important they are for control. It uses the controller's saliency map to weight which parts of the image are most critical for accurate state prediction.
Control-Consistency Loss
A training loss designed to ensure that the actions derived from a reconstructed image match those from the original real image. This loss enforces behavioral fidelity, ensuring the model learns to generate images that result in correct control outputs.

Terminology

Summary

End-to-end image controllers that map raw camera frames directly to control actions are increasingly deployed in safetycritical systems However, none of these tools can be directly applied when the controller input is a high-dimensional camera image because two fundamental challenges arise The authors propose a Deterministic World Model (DWM), a latent-free neural decoder that maps physical states directly to synthetic camera images, enabling closed-loop reachability analysis without the overapproximation caused by stochastic latent variables. p. 0

Where the paper says it

Deterministic World Models for Closed-loop Reachability Analysis of End-to-End Vision-based Control Yuang Geng, Zhongzheng Zhang, Chengzhen Jiang, Yanru Li, Xinyang Wang, Zhuoyang Zhou, Hoang-Dung Tran, Ivan Ruchkin University of Florida, Gainesville, FL, USA Abstract End-to-end image controllers that map raw camera frames directly to control actions are increasingly deployed in safetycritical systems. However, formally verifying their closed-loop behavior remains an open challenge because cameras produce high-dimensional images whose generation cannot easily be described in a closed mathematical form.

Pruning redundant reasoning tokens

The DWM is designed to resolve two main challenges: by providing a closed-form, deterministic stateto-image mapping, it closes the verification loop and by keeping the input dimension equal to the physical state dimension, it reduces the high-dimensional image variability that causes verification to explode. The DWM is trained with a novel dual loss combining saliency-map reconstruction and a controlconsistent term that preserves behavioral consistency with the real controller. p. 0

Where the paper says it

, position and velocity) directly to synthetic camera images, enabling closed-loop reachability analysis without the overapproximation caused by stochastic latent variables. The DWM is trained with a novel dual loss combining saliency-map reconstruction and a controlconsistent term that preserves behavioral consistency with the real controller.

How the training signal is built

The world model is trained using a weighted sum of two complementary losses The first loss, Saliency-Weighted Reconstruction Loss, assigns greater weights to controller-relevant pixels by estimating a saliency map from the controller itself by occlusion. Specifically, the weighting function is defined as H˜ij = 1/Qij Sum q∈Qij d(q), where d(q) = C(I(q)) − C(I). p. 2

Where the paper says it

We slide a k × k patch with stride σ over the image and replace the pixels Bq under the patch at position q with a reference image B: I (q) ij = Bij , (i, j) ∈ Bq, Iij , otherwise. (5) Each patch is scored by how far it moves the action, d (q) = |C(I (q) ) − C(I)|, and each pixel takes the mean score of the patches covering it: H˜ ij = 1 |Qij | X q∈Qij d (q) , Qij = {q : (i, j) ∈ Bq}.

The second loss, Control-Consistency Loss, penalizes the discrepancy between the control actions derived from the reconstructed image and the original image. p. 3
Where the paper says it

2025), we penalize the discrepancy between the control actions derived from the reconstructed image and the original image: Lctrl(I, ˆI) = C( ˆI) − C(I) 2 2 . (10) The final world model training loss combines the two above into a weighted sum: L(θ) = Lrec(I, ˆI) + λLctrl(I, ˆI), (11) The λ balances the trade-off between visual accuracy and behavioral consistency.

The final objective function is L(θ) = Lrec(I, ˆI) + λLctrl(I, ˆI), which balances visual accuracy and behavioral consistency. p. 3
Where the paper says it

2025), we penalize the discrepancy between the control actions derived from the reconstructed image and the original image: Lctrl(I, ˆI) = C( ˆI) − C(I) 2 2 . (10) The final world model training loss combines the two above into a weighted sum: L(θ) = Lrec(I, ˆI) + λLctrl(I, ˆI), (11) The λ balances the trade-off between visual accuracy and behavioral consistency.

Where the gains come from

The framework proceeds in three steps: construct a verifiable world model to replace the camera from the abundant data, analyze its reachability to obtain the reachable tube, and transfer the guarantee to the real system using scarce real trajectories. The reachability analysis uses symbolic Star sets which encode admissible states as an affine transformation of a predicate space. p. 3

Where the paper says it

Symbolic Reachability via Star Sets To enable sound reachability, we represent state uncertainty with Star sets (Tran et al. 2019, 2025), which encode admissible states as an affine transformation of a predicate space: S0 = s = c0 + V0α C0α ≤ d0, ℓ0 ≤ α ≤ u0 , (12) where c0 ∈ R d is the center, V0 ∈ R d×p the basis vectors of uncertainty, and α ∈ R p the predicate variables.

This symbolic propagation yields a sound reachable tube over each cell, satisfying the soundness requirement of Subproblem 2. p. 4
Where the paper says it

, T − 1, (15) yields the world model reachable tube {Rsym t } T t=0. Because every operator above is either exact or a sound overapproximation, the resulting tube satisfies the soundness requirement of Subproblem 2: Proposition 1 (Sound Surrogate Reachability).

Downstream Safety Verification Performance

The final step involves transferring the guarantee to the real system using Conformal Prediction (CP). This process, called Reachable Tube Based Inflation, uses a Tube-Robustness Score to bound the total trajectory-level discrepancy directly from observed trajectory pairs with conformal prediction. p. 4

Where the paper says it

From World-Model Reachability to Real-World Guarantees The DWM is a learned surrogate model of the real camera, but no surrogate is perfect. To solve Subproblem 3, we apply a data-driven approach called Reachable Tube Based Inflation and bound the total trajectory-level discrepancy directly from observed trajectory pairs with conformal prediction (Balasubramanian, Ho, and Vovk 2014).

The result is Theorem 1, which states that with probability at least 1 − α, the inflated reachable sets contain the real trajectories. p. 4
Where the paper says it

, T} : st ∈ Rˆ t(s0) i ≥ 1 − α. Inflating our world model with the conformal inflation yields a realistic safety guarantee: with probability at least 1−α, the inflated reachable sets contain the real trajectories.

Experiments on four dynamics show that the DWM produces tighter reachable tubes than baselines and supports sound downstream safety by maintaining a precision of 1. p. 6
Where the paper says it

Symbolic Star-set reachability then yields a sound reachable tube over each cell, and conformal prediction inflates it into a tube that contains the real trajectory with probability at least 1 − α. Across four benchmarks, the DWM produces tighter reachable tubes than baseline and supports sound downstream safety.

0 across all benchmarks. p. 6
Where the paper says it

Table 2: Comparison of reachable tube construction across four benchmarks. is the empirical coverage of held-out real trajectories (α = 0.

p. 6
Where the paper says it

Table 2: Comparison of reachable tube construction across four benchmarks. is the empirical coverage of held-out real trajectories (α = 0.

p. 6
Where the paper says it

Table 2: Comparison of reachable tube construction across four benchmarks. is the empirical coverage of held-out real trajectories (α = 0.

The results demonstrate that the DWM produces substantially tighter reachable tubes than a cGAN and trajectory predictor baselines while meeting the target coverage after conformal inflation. p. 0
Where the paper says it

We integrate the DWM into closed-loop reachability analysis and apply conformal prediction to inflate the reachable sets by a distribution-free trajectory-tube deviation bound, transferring the surrogate guarantee to the real system with high probability. Experiments on a CARLA braking system and three Gym benchmarks (CartPole, MountainCar, Pendulum) show that the DWM produces substantially tighter reachable tubes than a cGAN and trajectory predictor baselines while meeting the target coverage after conformal inflation.

Conclusion

The paper presents a framework for closed-loop reachability analysis of end-to-end vision-based controllers. A Deterministic World Model replaces the camera with a latent-free, stateto-image decoder, trained with a saliency-weighted reconstruction loss and a control-consistency term so that it stays faithful where it matters for control. p. 6

Where the paper says it

Conclusion We presented a framework for closed-loop reachability analysis of end-to-end vision-based controllers. A Deterministic World Model replaces the camera with a latent-free, stateto-image decoder, trained with a saliency-weighted reconstruction loss and a control-consistency term so that it stays faithful where it matters for control.

Symbolic Star-set reachability then yields a sound reachable tube over each cell, and conformal prediction inflates it into a tube that contains the real trajectory with probability at least 1 − α. p. 6
Where the paper says it

A Deterministic World Model replaces the camera with a latent-free, stateto-image decoder, trained with a saliency-weighted reconstruction loss and a control-consistency term so that it stays faithful where it matters for control. Symbolic Star-set reachability then yields a sound reachable tube over each cell, and conformal prediction inflates it into a tube that contains the real trajectory with probability at least 1 − α.

Across four benchmarks, the DWM produces tighter reachable tubes than baseline and supports sound downstream safety. p. 6
Where the paper says it

Symbolic Star-set reachability then yields a sound reachable tube over each cell, and conformal prediction inflates it into a tube that contains the real trajectory with probability at least 1 − α. Across four benchmarks, the DWM produces tighter reachable tubes than baseline and supports sound downstream safety.

The paper's contributions include a deterministic world modeling approach that creates verifiable generative surrogates of camera sensors and conformal transfer of world-model reachability results to the real system. The DWM produces tighter reachable tubes than the cGAN (e. p. 0
Where the paper says it

We integrate the DWM into closed-loop reachability analysis and apply conformal prediction to inflate the reachable sets by a distribution-free trajectory-tube deviation bound, transferring the surrogate guarantee to the real system with high probability. Experiments on a CARLA braking system and three Gym benchmarks (CartPole, MountainCar, Pendulum) show that the DWM produces substantially tighter reachable tubes than a cGAN and trajectory predictor baselines while meeting the target coverage after conformal inflation.

p. 0
Where the paper says it

We integrate the DWM into closed-loop reachability analysis and apply conformal prediction to inflate the reachable sets by a distribution-free trajectory-tube deviation bound, transferring the surrogate guarantee to the real system with high probability. Experiments on a CARLA braking system and three Gym benchmarks (CartPole, MountainCar, Pendulum) show that the DWM produces substantially tighter reachable tubes than a cGAN and trajectory predictor baselines while meeting the target coverage after conformal inflation.

g., for Pendulum A¯ = 0. p. 0
Where the paper says it

We integrate the DWM into closed-loop reachability analysis and apply conformal prediction to inflate the reachable sets by a distribution-free trajectory-tube deviation bound, transferring the surrogate guarantee to the real system with high probability. Experiments on a CARLA braking system and three Gym benchmarks (CartPole, MountainCar, Pendulum) show that the DWM produces substantially tighter reachable tubes than a cGAN and trajectory predictor baselines while meeting the target coverage after conformal inflation.

p. 0
Where the paper says it

We integrate the DWM into closed-loop reachability analysis and apply conformal prediction to inflate the reachable sets by a distribution-free trajectory-tube deviation bound, transferring the surrogate guarantee to the real system with high probability. Experiments on a CARLA braking system and three Gym benchmarks (CartPole, MountainCar, Pendulum) show that the DWM produces substantially tighter reachable tubes than a cGAN and trajectory predictor baselines while meeting the target coverage after conformal inflation.

p. 0
Where the paper says it

We integrate the DWM into closed-loop reachability analysis and apply conformal prediction to inflate the reachable sets by a distribution-free trajectory-tube deviation bound, transferring the surrogate guarantee to the real system with high probability. Experiments on a CARLA braking system and three Gym benchmarks (CartPole, MountainCar, Pendulum) show that the DWM produces substantially tighter reachable tubes than a cGAN and trajectory predictor baselines while meeting the target coverage after conformal inflation.

703 vs. 1.824). The results show that the approach maintains a precision of 1. p. 6
Where the paper says it

6639 baseline in Table 4. Our approach maintains a precision of 1.

0 across all benchmarks. The DWM demonstrates better coverage before inflation and after inflation, meeting the coverage target in all cases. p. 5
Where the paper says it

Without CP inflation, empirical coverage falls far below the 95% target, since a raw surrogate tube does not certify the real system; Braking is the exception (100%), reflecting its smaller modeling gap. DWM demonstrates better coverage before inflation. p. 5

Where the paper says it

Without CP inflation, empirical coverage falls far below the 95% target, since a raw surrogate tube does not certify the real system; Braking is the exception (100%), reflecting its smaller modeling gap. DWM demonstrates better coverage before inflation.

The authors conclude by showing that the DWM produces tighter reachable tubes than baseline and supports sound downstream safety. p. 6
Where the paper says it

Symbolic Star-set reachability then yields a sound reachable tube over each cell, and conformal prediction inflates it into a tube that contains the real trajectory with probability at least 1 − α. Across four benchmarks, the DWM produces tighter reachable tubes than baseline and supports sound downstream safety.

The final result is that with probability at least 1 − α, the inflated reachable sets contain the real trajectories. p. 4
Where the paper says it

, T} : st ∈ Rˆ t(s0) i ≥ 1 − α. Inflating our world model with the conformal inflation yields a realistic safety guarantee: with probability at least 1−α, the inflated reachable sets contain the real trajectories.

--- Page 4 ---

The authors evaluate their world model on four plant dynamics: a braking system in Carla and three OpenAI Gym environments: CartPole, MountainCar, and Pendulum. They show that the reachability analysis result of our world model produces tighter bound on the real system than baselines. p. 1

Where the paper says it

We evaluated our world model on four plant dynamics: a braking system in Carla and three OpenAI Gym environments: CartPole, MountainCar, and Pendulum. Our experiments show that the reachability analysis result of our world model produces tighter bound on the real system than baselines.

--- Page 7 ---

The DWM demonstrates better coverage before inflation and after inflation, meeting the coverage target in all cases. After inflation, both constructions meet the coverage target. p. 5

Where the paper says it

DWM demonstrates better coverage before inflation. After inflation, both constructions meet the coverage target.

p. 5
Where the paper says it

DWM demonstrates better coverage before inflation. After inflation, both constructions meet the coverage target.

The DWM produces consistently tighter tubes than the cGAN (e. p. 5
Where the paper says it

Note that the coverage of a particular held-out set of real trajectories is random; while it fluctuates, its average is guaranteed to be at least 95%. Under an identical reachability method, the DWM produces consistently tighter tubes than the cGAN (e.

g., for Pendulum A¯ = 0.703 vs. 1.824). Between the two reachability methods, symbolic and sampling tubes methods produce varied sizes and can therefore be used as complements, preferring whichever tubes are tighter. p. 5
Where the paper says it

, for Pendulum A¯ = 0. Between the two reachability methods, symbolic and sampling tubes methods produce varied sizes and can therefore be used as complements, preferring whichever tubes are tighter.

Symbolic verification has a bigger area than the sampling method in Cartpole: it has the most strongly nonlinear dynamics, so the sound overapproximation in PyBDR accumulates conservatism and inflates the tube, whereas sampling sidesteps this cost. p. 5
Where the paper says it

Between the two reachability methods, symbolic and sampling tubes methods produce varied sizes and can therefore be used as complements, preferring whichever tubes are tighter. Symbolic verification has a bigger area than the sampling method in Cartpole: it has the most strongly nonlinear dynamics, so the sound overapproximation in PyBDR accumulates conservatism and inflates the tube, whereas sampling sidesteps this cost.

--- Page 10 ---

The cGAN incorporates a latent variable z sampled from a truncated uniform distribution: IcGAN = Gan(s, z) s ∈ S0, z ∼ N (0, I). In contrast, the image set generated by our DWM depends solely on the state input, defined formally as: IWM = gθ(s) s ∈ S0. p. 5

Where the paper says it

The cGAN incorporates a latent variable z sampled from a truncated uniform distribution: IcGAN = { Gan(s, z) | s ∈ S0, z ∼ N (0, I)}. In contrast, the image set generated by our DWM depends solely on the state input, defined formally as: IWM = { gθ(s) | s ∈ S0 }.

Visual comparisons in Figure 2 show that the baseline model produces significantly larger pixel-level intervals than our approach. p. 5
Where the paper says it

In contrast, the image set generated by our DWM depends solely on the state input, defined formally as: IWM = { gθ(s) | s ∈ S0 }. Visual comparisons in Figure 2 show that the baseline model produces significantly larger pixel-level intervals than our approach.

The baseline’s weaknesses are high background noise, uncontrolled latent variables z inject noise into the background regions that are irrelevant to control, further inflating the pixel interval. p. 5
Where the paper says it

The baseline’s weaknesses are: • High background noise. Uncontrolled latent variables z inject noise into the background regions that are irrelevant to control, further inflating the pixel interval.

Downstream impact on reachability is that when these widened pixel intervals propagate through the controller and plant dynamics over multiple steps, overapproximation errors accumulate rapidly. p. 5
Where the paper says it

• Downstream impact on reachability. When these widened pixel intervals propagate through the controller and plant dynamics over multiple steps, overapproximation errors accumulate rapidly.

--- Page 6 ---

Table 7 evaluates how the DWM tube is constructed (symbolic Star propagation vs. per-cell sampling) and whether it is transferred with conformal prediction. p. 5

Where the paper says it

Reachability Construction and Transfer Table 2 evaluates how the DWM tube is constructed (symbolic Star propagation vs. per-cell sampling) and whether it is transferred with conformal prediction.

Without CP inflation, empirical coverage falls far below the 95% target, since a raw surrogate tube does not certify the real system; Braking is the exception (100%). p. 5
Where the paper says it

per-cell sampling) and whether it is transferred with conformal prediction. Without CP inflation, empirical coverage falls far below the 95% target, since a raw surrogate tube does not certify the real system; Braking is the exception (100%), reflecting its smaller modeling gap.

DWM demonstrates better coverage before inflation. After inflation, both constructions meet the coverage target.

--- Page 7 ---

Table 3 ablates the two design choices behind our world model: removing the latent variable and adding the controlconsistency loss Lctrl. Removing the latent variable alone already shrinks the bound by one to two orders of magnitude over the cGAN baseline, most sharply on CartPole (1. p. 6

Where the paper says it

We measure their effect on the trajectory-based maximum discrepancy, which measures how far real trajectories deviate from the DWM (smaller is tighter). Removing the latent variable alone already shrinks the bound by one to two orders of magnitude over the cGAN baseline, most sharply on CartPole (1.

472 → 0.0116) and Pendulum (0. p. 6
Where the paper says it

Removing the latent variable alone already shrinks the bound by one to two orders of magnitude over the cGAN baseline, most sharply on CartPole (1. 0116) and Pendulum (0.

358 → 0.0717). Adding Lctrl yields a further consistent reduction across all benchmarks. p. 6
Where the paper says it

0116) and Pendulum (0. Adding Lctrl yields a further consistent reduction across all benchmarks (e.

The trajectory-based maximum discrepancy between world models and real system (lower means better model) is shown in Table 3. p. 6
Where the paper says it

0027 on MountainCar). Table 3: Trajectory-based maximum discrepancy between world models and real system (lower means better model).

--- Page 4 ---

Figure 3 shows failure and success cases of closed-loop reachability analysis of the world model. The DWM produces tighter reachable tubes than the cGAN (e.g., for Pendulum A¯ = 0.703 vs. 1.824).

--- Page 5 ---

Figure 4 shows the DWM Closed-loop verification with camera roll-out ground truth across four benchmarks.

--- Page 6 ---

Figure 5 visualizes the pixel-level reachable sets produced by the proposed DWM-based verification pipeline under identical initial state intervals. The interval width image is defined as W(p) = I(p) − I(p), which characterizes the uncertainty induced by the initial state set through the decoder. p. 4

Where the paper says it

Following prior work (Cai, Fan, and Bak 2025), we replace our DWM decoder with a conditional GAN whose generator takes the concatenation of the state st and a latent vector z ∼ N (0, I) and outputs an image ˜It = Gan(st, z). During reachability analysis, both s and z must be bounded; we set z interval to [−0.

--- Page 7 ---

Table 7 shows a comparison of reachable tube construction across four benchmarks. Cov. is the empirical coverage of held-out real trajectories (α = 0. p. 6

Where the paper says it

Table 2: Comparison of reachable tube construction across four benchmarks. is the empirical coverage of held-out real trajectories (α = 0.

p. 6
Where the paper says it

Table 2: Comparison of reachable tube construction across four benchmarks. is the empirical coverage of held-out real trajectories (α = 0.

05, target ≥ 95%); A¯ is the average tube area (reported in units of 10−3). p. 6
Where the paper says it

is the empirical coverage of held-out real trajectories (α = 0. 05, target ≥ 95%); A¯ is the average tube area (reported in units of 10−3 ).

p. 6
Where the paper says it

is the empirical coverage of held-out real trajectories (α = 0. 05, target ≥ 95%); A¯ is the average tube area (reported in units of 10−3 ).

--- Page 8 ---

Table 4 shows downstream quantitative safety results across CartPole, MountainCar, Pendulum, and Braking System. Our approach maintains a precision of 1. p. 6

Where the paper says it

6639 baseline in Table 4. Our approach maintains a precision of 1.

0 across all benchmarks.

--- Page 4 ---

Figure 4 shows the DWM Closed-loop verification with camera roll-out ground truth across four benchmarks.

--- Page 6 ---

Figure 5 visualizes the pixel-level reachable sets produced by the proposed DWM-based verification pipeline under identical initial state intervals. Each row corresponds to a different case study, namely the Advanced Emergency Braking System (top), Pendulum (middle), and MountainCar (bottom). p. 4

Where the paper says it

This partitioning also enables efficient parallel computation, as the reachability analysis for different cells can be performed independently. For reachability analysis, we compute reachable tubes {Rˆ t} T t=0 with a time horizon of T = 20 for CartPole, MountainCar, and Pendulum, and T = 10 for the Braking System.

--- Page 4 ---

Table 7 shows a comparison of reachable tube construction across four benchmarks. Cov. is the empirical coverage of held-out real trajectories (α = 0.05, target ≥ 95%); A¯ is the average tube area (reported in units of 10−3).

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Deterministic World Model Training: Train a state-to-image decoder using a loss function that combines a saliency-weighted reconstruction loss and a control consistency loss to ensure fidelity in control-critical regions while sacrificing global fidelity for background pixels.

  2. Closed-Loop Verification: Integrate the trained Deterministic World Model into a Star-based closed-loop reachability analysis pipeline, where state uncertainty is represented using Star sets to ensure sound overapproximations of image propagation through the controller and plant dynamics.

  3. Real-System Transfer: Apply Conformal Prediction to inflate the reachable tubes derived from the world model with a distribution-free trajectory-tube deviation bound, ensuring that with high probability, these inflated sets contain the real system trajectories.

  4. Improved AI System Capability: The resulting AI system can perform safety certification for end-to-end vision-based controllers in safety-critical applications by providing a sound guarantee that the closed-loop trajectory will remain within a specified safe region with high confidence.

Abstract

End-to-end vision-based controllers map raw camera images directly to control actions, yet verifying their closed-loop behavior remains challenging: no analytical model maps a physical state to a camera image, and propagating a set of states through high-dimensional images yields loose reachable sets. We address both challenges with two techniques. First, we propose a Deterministic World Model (DWM), a latent-free decoder that generates camera images only from the physical state, trained with a saliency-weighted reconstruction loss and a control-consistency loss. The DWM replaces the camera in the closed loop, so a sound and tight reachable tube of the surrogate closed loop can be computed without bounding any latent variables. Second, we propose an action-guided partition-and-verify scheme that certifies as many initial sets as possible within a fixed computation budget. A Gaussian process learned from cheap simulations orders the cells and splits those that lie across the property boundary, while every other unknown cell is split along the state dimension that most changes the controller's actions along the simulated closed loop, i.e., the one that most reduces the pixel uncertainty reaching the controller. On a CARLA braking system and three Gym benchmarks, the DWM is more accurate to the real camera than a cGAN surrogate and yields 1.9x to 69x narrower robustness intervals. The vision part causes most of the remaining overapproximation, and our splitting reduces the unknown fraction of the four-dimensional CartPole initial set from 21.88% to 8.18% within the budget.

Sources

Related papers