PathoGaze1.0: An AI-Ready Navigation, Eye-Tracking, and Decision Dataset in Pathology

arXiv:2510.24653 · cs.CV, cs.HC · Submitted 2025-10-28 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "PathoGaze1.0: An AI-Ready Navigation, Eye-Tracking, and Decision Dataset in Pathology".

Paper summary: Tom: So wrapping up the PathoGaze1 point 0 paper, the authors are giving us this comprehensive behavioral dataset of eye-tracking and decision-making data from nineteen pathologists working on whole slide images > <ref:2510.24653#pg1>

Jane: The title itself, "PathoGaze1 point 0: An AI-Ready Navigation, Eye-Tracking, and Decision Dataset in Pathology," suggests they're aiming to make this behavior data usable for building actual AI systems that can navigate and diagnose cancer slides > <ref:2510.24653#pg1>

Lu: It’s important because it directly addresses the lack of behavioral data in this field that was holding back diagnostic accuracy estimates around seventy percent >

Meng: For production, the implication is that if an AI system can learn from these viewing strategies, it might be able to correct those inconsistencies where different human pathologists make different choices >

Lalam: The paper lays out how they structured the data—all the files and metadata are available according to FAIR principles, making it accessible for long-term analysis by other researchers >

Tom: It’s a big step because they didn't just look at the final diagnosis; they captured the entire journey of visual search, from loading the image to making that final decision >

Jane: So what this means simply is that we can finally start studying *how* humans look at these massive images, which is something we completely lacked before >

Conclusion: Tom: So we’ve seen how PathoGaze1 point zero is tracking what pathologists do when they look at whole slide images and making decisions, and now we’re at the end of this segment to wrap up what this paper actually means for us <ref:2510.24653#pg3>.

Jane: This whole dataset is built around capturing the actual viewing behavior—the eye movements, the mouse clicks, and those diagnostic decisions—from nineteen different pathologists working on cancer diagnoses. It gives us a real look into that process.

Lu: The authors are basically saying they’ve created a comprehensive behavioral dataset to finally explain why pathologists sometimes get inconsistent results when looking at these massive whole slide images.

Meng: So what does that inconsistency actually mean for deploying an AI? Can we use this data to make sure the AI follows the right path, or is it just another way to train a model on human mistakes?

Lalam: It means we have something concrete to train on—not just the final diagnosis—so if an AI can learn these viewing strategies, it might be able to navigate those tricky areas better.

Tom: That’s the core idea, right? They didn't just give us pictures of slides and answers; they gave us eighteen point six nine hours of actual eye tracking and interaction data.

Jane: Right, it’s about making the process transparent so we can finally measure how much variability there is in how people diagnose these things.

Lu: The key finding they highlight is that tumor size itself is a huge factor influencing accuracy, with participants being ninety-one point zero percent accurate on slides with macrometastasis.

Meng: That's a big number, but I wonder if that accuracy holds up when we move from lab conditions to actual clinical environments where things are much more cluttered and the lighting is bad.

Lalam: The data is structured really well, following FAIR principles so other researchers can actually use it long-term. You have the data and the code available online at go osu edu pathogaze.

Tom: Exactly. It’s about accessibility for building better AI, not just for publishing a new paper in a journal that nobody reads.

Jane: So what we're looking at here is how to bridge the gap between knowing *what* they decided and understanding *how* they got there, which is super important for building reliable diagnostic tools.

Veronica Thai, Rui Li, Meng Ling, Shuning Jiang, Jeremy Wolfe, Raghu Machiraju, Yan Hu, Zaibo Li, Anil Parwani, ¢Jian Chen

The Ohio State University Computer Science and Engineering Department of Computer Science and Engineering The Ohio State University Medical Center Department of Pathology The Harvard Medical School Departments of Ophthalmology and Radiology Brigham and Women’s Hospital

cs.CV, cs.HC

Submitted: 2025-10-28

Updated: 2026-11-16

Comments: 20 pages, 11 figures, 3 tables, submitted to Nature Scientific Data

Project page: https://openseadragon.github.io

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Interpretation of giga-pixel whole-slide images (WSIs) is an important but difficult task for pathologists Their diagnostic accuracy is estimated to average around 70% The field lacks adequate…

Key concepts

Whole-Slide Images (WSIs)
These are high-resolution digital scans of entire tissue samples used by pathologists for diagnosis. The paper focuses on how experts navigate these complex images to make a diagnosis, highlighting the difficulty of this task.
Eye-Tracking and Mouse Interaction Data
This data captures the dynamic visual search patterns, including where a pathologist looks (gaze) and how they use their mouse to interact with the software. This behavioral data helps researchers understand the decision-making process during diagnosis.
Ecological Validity (PTAH)
Data collection occurred in an application-grounded testbed called PTAH, which simulates real diagnostic workflows. This method ensures the collected behavior reflects actual clinical practice rather than artificial laboratory settings.
Diagnostic Accuracy Metrics
The study measured how accurate pathologists were at diagnosing cancer based on their viewing behaviors. A key finding was that tumor size was a major factor, with accuracy reaching 91.0% for macrometastasis slides.

Terminology

Summary

Interpretation of giga-pixel whole-slide images (WSIs) is an important but difficult task for pathologists Their diagnostic accuracy is estimated to average around 70% The field lacks adequate behavioral data to explain diagnostic errors and inconsistencies. p. 0

Where the paper says it

ABSTRACT Interpretation of giga-pixel whole-slide images (WSIs) is an important but difficult task for pathologists. Their diagnostic accuracy is estimated to average around 70%.

Dataset Composition

PathoGaze1.0 presents a comprehensive behavioral dataset capturing the dynamic visual search and decision-making processes of the full diagnostic workflow during cancer diagnosis. p. 0

Where the paper says it

To fill in this gap, we present PathoGaze1. 0, a comprehensive behavioral dataset capturing the dynamic visual search and decision-making processes of the full diagnostic workflow during cancer diagnosis.

This dataset comprises 18. p. 0
Where the paper says it

0, a comprehensive behavioral dataset capturing the dynamic visual search and decision-making processes of the full diagnostic workflow during cancer diagnosis. The dataset comprises 18.

69 hours of eye-tracking, mouse interaction, stimulus tracking, viewport navigation, and diagnostic decision data (EMSVD) collected from 19 pathologists interpreting 397 WSIs. p. 0
Where the paper says it

The dataset comprises 18. 69 hours of eye-tracking, mouse interaction, stimulus tracking, viewport navigation, and diagnostic decision data (EMSVD) collected from 19 pathologists interpreting 397 WSIs.

In total, the researchers recorded 171,909 fixations, 263,320 saccades, and 1,867,362 mouse interaction events. p. 0
Where the paper says it

The data collection process emphasizes ecological validity through an application-grounded testbed, called PTAH. In total, we recorded 171,909 fixations, 263,320 saccades, and 1,867,362 mouse interaction events.

Data Collection Methodology

The data collection process emphasizes ecological validity through an application-grounded testbed, called PTAH. The researchers recruited 19 pathologists across two experiments (Experiment I with 10 and Experiment II with 9) to study complementary aspects of diagnostic reasoning. p. 0

Where the paper says it

Our data collection uses application-grounded evaluation, where data collection is conducted within a replication of application tools: our testbed, called PTAH, has assembled features from the state-of-the-art clinical diagnosis platforms. We recruited 19 pathologists across two experiments (10 in Experiment I and 9 in Experiment II), designed to study complementary aspects of diagnostic reasoning.

Experiment I focused on behavior consistency between pathologists, where participants labeled cancerous tissue regions that led to their diagnosis. p. 2
Where the paper says it

Experiment I (P10S60T600) Data Selection, Tasks, and Participants. Experiment I focused on behavior consistency between pathologists.

In contrast, Experiment II asked participants to look at as many WSIs as possible for broad behavior monitoring. p. 0
Where the paper says it

As a result, we can measure behavior consistency between pathologists. In contrast, Experiment II, P9D397T540, asked participating pathologists to look at as many WSIs as possible for broad behavior monitoring.

Data Processing and Synchronization

Coordinate system transformation is necessary because the mouse, eye-tracking device attached to the screen, graphical interface, and WSI each operated in distinct coordinate systems. The researchers computed the coordinate conversion from screen space to image space using a specific formula involving viewport offsets and monitor dimensions. p. 4

Where the paper says it

The coordinate conversion from screen position to image position was computed as follows: ximage yimage = "wviewport wwindow 0 0 hviewport hwindow # xscreen yscreen + xviewport yviewport (1) where (ximage, yimage) denote the image-space coordinates, (xscreen, yscreen)the raw screen-space coordinates, and (xviewport, yviewport) the viewport offset. wviewport and hviewport represent the viewport width and height reported by Seadragon, while wwindow and hwindow correspond to the monitor width (2,560 px) and height (1,440 px), respectively.

Synchronization between mouse and eye-gaze was achieved by aligning timestamps recorded by the eye-tracker with those logged by the testbed. p. 4
Where the paper says it

We first aggregated gaze samples into single fixation or saccade points (fixation and saccade identification done by the Tobii I-VT filter). The testbed recorded the timestamps from the eye-tracker, allowing synchronization between these eye-tracker fixations and testbed events, so that fixations could be associated with testbed data such as viewport position, image coordinate, and zoom level.

Each trial was defined as the period from when the WSI image finished loading to when a participant clicked the “Next” button to proceed to the next trial. p. 4
Where the paper says it

The average gaze position between the left and right eye was used. Each trial was defined as the period from the moment the WSI image finished loading to the time the participant clicked the “Next” button to proceed to the next trial.

Key Findings and Metrics

The analysis revealed distinct viewing strategies, such as Participant P5 making a diagnostic decision without scanning all tumor areas, whereas Participant P10 examined most regions before responding. The study observed that mouse movement generally aligns with gaze points, but exceptions exist where the mouse does not follow the gaze. p. 1

Where the paper says it

In general, we observe mouse movement (blue lines and arrows) aligns with gaze points, such as the examples from Participant P10. However, there are a few cases, such as Participant P5, where the mouse does not follow the gaze, in this case, spanning a wider area than the gaze.

Technical validation showed that tumor size emerges as the primary influencing factor on diagnostic accuracy, with participants being substantially more accurate on macrometastasis slides (91. p. 0
Where the paper says it

Their diagnostic accuracy is estimated to average around 70%. Adding a second pathologist does not substantially improve decision consistency.

0%). Furthermore, the data allows for investigation into how experts allocate attention under cognitive constraints. p. 0
Where the paper says it

Background & Summary Visual search expertise in digital pathology depends on how pathologists allocate attention across complex tissue landscapes1–3 . Collecting gaze data can provide insight into the causes of diagnostic inconsistency4 , support decision-making5 , model visual search errors6 , and enable the development of machine learning methods to assist pathologists in daily diagnostic tasks7 , or to create a digital twin of a pathologist’s diagnostic process8 .

Data Availability and Code

The dataset is structured to maximize transparency, accessibility, and long-term usability in accordance with FAIR principles. All data files are accompanied by metadata, standardized naming conventions, and consistent formatting to ensure interoperability across analysis platforms. p. 4

Where the paper says it

0 is structured and documented to maximize transparency, accessibility, and long-term usability. All data files are accompanied by metadata, standardized naming conventions, and consistent formatting to ensure interoperability across analysis platforms.

The complete dataset along with analysis code is available at https://go. p. 0
Where the paper says it

All experiments were preregistered at https://osf. io/hj9a7, and the complete dataset along with analysis code is available at https://go.

osu.edu/pathogaze. p. 0
Where the paper says it

io/hj9a7, and the complete dataset along with analysis code is available at https://go. edu/pathogaze.

p. 0
Where the paper says it

io/hj9a7, and the complete dataset along with analysis code is available at https://go. edu/pathogaze.

The authors released all data processing code online at https://go. p. 5
Where the paper says it

Both subdirectories contain the same type of data with the same organization: experiment specific code, raw testbed and eye-tracker data, screen recordings, and processed data. Publicly available via https://go.

osu.edu/pathogaze.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improve AI systems by incorporating gaze data to explain diagnostic errors and inconsistencies, as The field lacks adequate behavioral data to explain diagnostic errors and inconsistencies. This allows AI models to understand why a human pathologist might miss or misclassify a tumor region based on where attention was allocated.

  2. Develop machine learning methods that assist pathologists in daily diagnostic tasks by using the dataset to model visual search errors and create a digital twin of a pathologist’s diagnostic process. This enables AI tools to mimic expert visual search strategies, such as those related to Looked But Failed to See errors.

  3. Enhance AI systems by validating whether mouse-action and eye-gaze are correlated as in other imaging modalities, or by identifying the interaction between image features, actions, individual differences, and tasks. This provides a richer understanding of how multimodal behavior influences pathology diagnosis compared to single-modality datasets like CAMELYON16.

  4. Train AI systems using PathoGaze1.0 to classify specific search errors, such as determining if an error is caused by the target was fixated or related to the zoom level at which it was viewed, offering a finer taxonomy for error classification than current methods allow.

Abstract

Performing cancer diagnosis on giga-pixel whole-slide images (WSIs) is an important but difficult visual search task. The large giga-pixel image size forces pathologists to navigate by zooming and panning to search tissue for malignancy. Diagnostic accuracy averages around 70%, and adding a second pathologist does not substantially improve consistencies between diagnoses. There is a lack of publicly available behavioral data to explain these diagnostic errors and inconsistencies. To fill this gap, we curated PathoGaze1.0, a comprehensive behavioral dataset capturing high-fidelity clinical diagnostic workflows across 11 different pathologist participants over two experiments 12-months apart (8 overlapping pathologists across both experiments), as they interpreted 397 WSIs from CAMELYON16, a set of publicly available breast cancer lymph node WSIs. The dataset comprises 18.69 hours of eye-tracking (171,909 fixations), and navigational scanning data along with 1,140 diagnostic decisions, all collected using a newly-developed testbed PTAH. The testbed supports interaction techniques found in clinical tools and collects viewport, mouse actions, and diagnostic decisions, enabling cross-links between WSI image stimulus, gaze, navigation behaviors, and diagnostic decisions. We share preregistered experimental materials, the datasets, and processing methods on the Open Science Framework (OSF).

Related papers