CG-Probes: Recovering Guardrail Directions from Patient Query Embeddings
cs.CL, cs.IR
Submitted: 2026-09-25
Updated: 2026-09-25
Comments: Accepted as a short paper at CIKM '26 (35th ACM International Conference on Information and Knowledge Management), Rome, Italy. 7 pages, 1 figure, 2 tables. Code and benchmark: https://github.com/mrehacek/cg-probes
Code: https://github.com/mrehacek/cg-probes
License: http://creativecommons.org/licenses/by/4.0/
The gist: Patient-facing AI assistants promise valuable support to patients, but incoming queries can pose medical risks.
Terminology
Abstract
Patient-facing AI assistants promise valuable support to patients, but incoming queries can pose medical risks. To create guardrails, we work with oncologists to define three ordinal risk axes: Medical Urgency, Psychological Urgency, and Topic Sensitivity. We propose Clinical Guardrail Probes (CG-Probes) to measure the risks from query embeddings. We probe for each axis in the normalized embedding space of frozen embedders via the difference-in-means method, treating each axis as a potential linear direction. To train the probes, we cluster 79,658 Czech oncology search queries with BERTopic and use these clusters to generate pairs of queries with contrastive risk levels via few-shot prompting. We evaluate the approach on 200 queries (90 real, 110 synthetic), each graded by two oncologists, against two open-weight LLMs and a frontier LLM. We find that urgency-based axes are recoverable as linear directions, and the probes are competitive with open-weight LLMs (no significant differences in quadratic-weighted kappa) at a fraction of the latency. Each axis yields a scalar score that clinicians can inspect and use to set escalation thresholds. The pipeline requires only search logs, axis definitions, and black-box access to the embedding model, suggesting transferability across healthcare domains. Robust validation on new queries and axes remains future work.
Sources
- Understanding intermediate layers using linear classifier probes
- Refusal in Language Models Is Mediated by a Single Direction
- LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
- Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
- Language Models are Few-Shot Learners
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- BERTopic: Neural topic modeling with a class-based TF-IDF procedure
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- gpt-oss-120b & gpt-oss-20b Model Card
- Refusal Direction is Universal Across Safety-Aligned Languages
- Representation Engineering: A Top-Down Approach to AI Transparency
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering