How Robust Is Multimodal Claim Verification to LLM Rewriting?
cs.CL
Submitted: 2026-10-02
Updated: 2026-10-02
Comments: Accepted to AACL 2026 (Main Conference). 18 pages
Code: https://github.com/yunangwu/llm-rewrite-claim-verification
License: http://creativecommons.org/licenses/by/4.0/
The gist: LLMs are known to introduce stylistic changes into generated text, yet how these stylistic shifts affect model decisions on scientific tasks remains underexplored.
Terminology
Abstract
LLMs are known to introduce stylistic changes into generated text, yet how these stylistic shifts affect model decisions on scientific tasks remains underexplored. In this paper, we focus on multimodal claim verification, where the goal is to determine whether a textual claim is grounded in a given piece of evidence. We apply two rewriting strategies: natural rewriting, which simulates how researchers routinely use LLMs to polish academic text, and controlled injection, which inserts a single LLM-associated word to isolate the effect of vocabulary choice. We evaluate 11 open-weight models spanning five VLM families and ranging from 2B to 38B parameters. We find that models are robust to these modifications: most show no significant drop in accuracy, and compared to prior work on review-score manipulation, verification appears far more stable. However, consistent probability shifts do occur. Hedging-oriented conditions produce significant shifts across nearly all models, while boosting conditions show a weaker effect and general polishing conditions (e.g., grammar correction, fluency improvement) have little effect.
Sources
- Qwen3-VL Technical Report
- Shifting norms in scholarly publications: trends in readability, objectivity, authorship, and AI use
- GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
- Paraphrasing Adversarial Attack on LLM-as-a-Reviewer
- gpt-oss-120b & gpt-oss-20b Model Card
- ABCI 3.0: Evolution of the leading AI infrastructure in Japan
- Kimi-VL Technical Report
- Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering