Can Webcam Gaze Constrain Mesa-Objectives in Driving Models? An Instrument Precision Analysis

arXiv:2608.08947 · cs.CV, cs.HC, cs.LG · Submitted 2026-08-09 · Read on arXiv

Lennox Anderson, Ahmed Boutar, Jonah Mulcrone, Tal Erez

cs.CV, cs.HC, cs.LG

Submitted: 2026-08-09

Updated: 2026-08-11

Comments: 6 pages, 3 figures, 4 tables

Code: https://github.com/Onyx-AILLC/Human-Alignment-Hazardous-Driving-Detection

License: http://creativecommons.org/licenses/by/4.0/

The gist: Current hazard detection systems in autonomous driving may develop mesa objectives, learned internal goals that achieve high training performance through spurious correlations rather than genuine

Terminology

Calibration — 30× in this paper · explained in 9 episodes
Transformer — 9× in this paper · explained in 2 episodes
AUC — 7× in this paper · explained in 2 episodes
Accuracy — 6× in this paper · explained in 1 episode
Benchmark — 5× in this paper · explained in 2 episodes
Embeddings — 4× in this paper · explained in 2 episodes
Distillation — 3× in this paper · explained in 2 episodes
Conference on Computer Vision and Pattern Recognition — 2× in this paper
Eye Tracking Research and Applications — 2× in this paper
Hyperparameters — 2× in this paper · explained in 2 episodes
Calibration Failure — 1× in this paper · explained in 2 episodes
Data Leakage — 1× in this paper · explained in 2 episodes
Deep Reinforcement Learning — 1× in this paper · explained in 3 episodes
Evaluation Framework — 1× in this paper · explained in 3 episodes
Feature Engineering — 1× in this paper · explained in 3 episodes
Feature Extraction — 1× in this paper · explained in 3 episodes
Knowledge Distillation — 1× in this paper · explained in 15 episodes
Reinforcement Learning — 1× in this paper · explained in 4 episodes
Training Objective — 1× in this paper · explained in 3 episodes
Transformer Architecture — 1× in this paper · explained in 4 episodes
Transformers — 1× in this paper · explained in 2 episodes

Abstract

Current hazard detection systems in autonomous driving may develop mesa objectives, learned internal goals that achieve high training performance through spurious correlations rather than genuine hazard recognition. We investigate whether human gaze patterns, captured via webcam-based eye tracking (WebGazer.js), can serve as privileged information to constrain mesa-objective formation. We collected 137,663 frame-level gaze samples synchronized with hazard annotations across 388 real dashcam clips, then test this hypothesis across two calibration protocols (9-point/45-click and 11-point/440-click), two model architectures (Random Forest and causal Transformer), and five random seeds per experiment with paired t-tests. No experiment yields a statistically significant improvement from gaze (p = 0.919, 0.578, and 0.667 respectively). A geometric analysis reveals the root cause: WebGazer's reported error (130-257 px depending on configuration) exceeds 93% of detected hazard object sizes (median 36 px), rendering object-level gaze attribution physically impossible at this instrument precision.

Can Webcam Gaze Constrain Mesa-Objectives in Driving Models? An Instrument Precision Analysis

Terminology

Abstract

Sources

Related papers