Jako Tako or Fluent? Presenting PoVisLE: A Polish Vision-Language Evaluation

arXiv:2608.07763 · cs.CL, cs.CV · Submitted 2026-08-07 · Read on arXiv

Anna Kołos, Grzegorz Statkiewicz, Karolina Seweryn, Katarzyna Kowol, Karolina Piosek, Wojciech Kusa

cs.CL, cs.CV

Submitted: 2026-08-07

Updated: 2026-08-11

Comments: 28 pages. Preprint under review

Code: https://github.com/NASK-NLP/PoVisLE

License: http://creativecommons.org/licenses/by-sa/4.0/

The gist: Vision-language models (VLMs) have achieved strong performance on tasks such as image captioning, visual question answering, and image-to-text generation.

Terminology

Abstract

Vision-language models (VLMs) have achieved strong performance on tasks such as image captioning, visual question answering, and image-to-text generation. However, they are predominantly trained on English-centric data, which limits their ability to handle culturally grounded visual understanding and leads to failures in interpreting region-specific meanings, symbolic content, and context-dependent visual cues. Existing benchmarks for cultural competence are often template-driven and focused on surface-level recognition, making them insufficient for evaluating deeper linguistic and pragmatic understanding in culturally situated settings. We introduce PoVisLE, a monocultural vision-language benchmark for Polish designed to evaluate culturally grounded multimodal understanding under a grounded evaluation paradigm, where language is interpreted in interaction with visual context. The dataset contains 1,117 images and 2,366 manually annotated VQA pairs. Overall, our dataset provides a controlled and challenging resource for assessing culturally grounded vision-language understanding beyond surface-level recognition.

Sources

Related papers