Do Vision Encoders Exhibit Human-like Color Thresholds?
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Do Vision Encoders Exhibit Human-like Color Thresholds?".
Jane: Here's the gist of the paper. Understanding and characterizing human color perception is a longstanding research goal <ref:2607.16540#pg2> One of the most traditional approaches is looking for the human color discrimination thresholds, the minimum chromatic differences perceptible to human observers <ref:2607.16540#pg2>.
Tom: In this work, we present a large-scale exploratory study probing the chromatic sensitivity of more than 50 pretrained vision encoders, including convolutional networks and vision transformers, against human discrimination thresholds <ref:2607.16540#pg2>.
Jane: Our analysis reveals generally weak alignment between model representations and human perceptual thresholds across all model families, with the best mIoU.
Tom: Pruning redundant reasoning tokens The study systematically evaluates the chromatic representation geometry of deep vision encoders against human discrimination behavior <ref:2607.16540#pg2>.
Paper summary: Tom: So, wrapping up this discussion on "Do Vision Encoders Exhibit Human-like Color Thresholds?", the paper confirms that while deep vision encoders develop structured and anisotropic chromatic similarity geometries in their latent spaces, these representations only partially align with human perceptual models.
Jane: The alignment varies systematically across chromaticity regions and saturation levels, and it depends strongly on the training paradigm; self-supervised and supervised models outperformed language-supervised and generative approaches in terms of this alignment.
Lu: The overall message is that even the best performing models remain far from human-level chromatic discrimination when tested against these specific human perceptual thresholds.
Meng: It means we can't just assume high performance on a benchmark automatically translates to accurate color perception, especially since the alignment depends so heavily on what kind of training signal you feed the AI.
Tom: Exactly. The implication is that for any application where precise chromatic discrimination is critical, we need to consider both the model architecture and the specific training objective alongside just looking at raw performance metrics.
Jane: It's a caution to keep in mind as we move forward with these large foundation models, reminding us that internal feature representations don't always map perfectly onto established human psychophysical data.
Conclusion: Tom: So we’re done with looking at the mechanics of how these vision encoders are trained, and now we’re just looking at what this whole study actually means for color science.
Jane: The title is pretty direct, right? "Do Vision Encoders Exhibit Human-like Color Thresholds?" It’s asking if the AI's internal understanding of color matches what a person can actually see when they look at two colors side by side.
Lu: Basically, they took fifty different types of vision models, from older convolutional networks to the newest vision transformers, and compared how well their internal representations line up with actual human color discrimination thresholds.
Meng: And the main finding is that the alignment isn't great across the board; it’s weak generally. They found that while these models develop some structured geometry in their latent space, they don't perfectly mirror how humans perceive those differences.
Tom: So, if we think about this for a practical application—say, an autonomous vehicle needing to detect subtle color changes—this study suggests we can't just trust the raw output of these massive models when it comes to precise color discrimination.
Jane: Exactly. It shows that even the best performing models still have a gap between their internal math and real human perception, and this gap changes depending on the specific shade of blue you’re looking at or how saturated it is.
Lu: The study points out that the alignment varies wildly based on the training signal; self-supervised methods actually kept their color relationships better than language-supervised ones did.
Meng: That’s important because if we are deploying these models, choosing the right training method matters more than just picking the biggest model size.
Tom: It really highlights that achieving high scores in a test doesn't automatically mean you’ve captured human sensory reality; there's still a distance to cover.
Jane: And it leaves us wondering what that distance looks like when we move beyond simple flat color patches and look at complex, real-world textures. That’s where the next part of this line gets interesting.
Computer Vision Center, Spain · Universitat Autonoma de Barcelona, Spain · Image Processing Lab, Universitat de Valencia, Spain · Arab Academy for Science, Technology, and Maritime Transport
cs.CV
Submitted: 2026-07-17
Updated: 2026-11-02
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 77/100
The gist: Understanding and characterizing human color perception is a longstanding research goal <ref:2607.16540#pg2> One of the most traditional approaches is looking for the human color discrimination…
Key concepts
- Human Color Discrimination Thresholds
- These are the minimum differences in color that a human observer can perceive as distinct. The study uses these thresholds as a benchmark to see how well computer vision models understand what humans can see in terms of color separation.
- Model Representations Geometry
- This refers to the shape and structure of the colors learned by deep vision encoders inside their internal feature spaces. The researchers analyzed whether these learned color patterns resemble the natural, anisotropic shapes (like ellipses) that describe how humans perceive color differences.
- mIoU (Mean Intersection over Union)
- This is a metric used to measure the overlap between the chromatic discrimination thresholds derived from a specific vision model and the actual human discrimination ellipses. A higher mIoU indicates better alignment between what the AI model thinks about color and what humans actually perceive.
Terminology
Summary
Understanding and characterizing human color perception is a longstanding research goal <ref:2607. p. 0 Do Vision Encoders Exhibit Human-like Color Thresholds? Engy Ehab 1,2,4
, Pablo Hernandez-C ´ amara ´
3
, Nahla Belal 4
, Jes ´us Malo 3
, Javier Vazquez-Corral, 1,2
, Alexandra Gomez-Villa 1,2
1 Computer Vision Center, Spain
2 Universitat Autonoma de Barcelona, Spain `
3
Image Processing Lab, Universitat de Valencia, Spain `
4 Arab Academy for Science, Technology, and Maritime Transport, Egypt
Abstract
Understanding and characterizing human color perception
is a longstanding research goal. Engy Ehab 1,2,4
, Pablo Hernandez-C ´ amara ´
3
, Nahla Belal 4
, Jes ´us Malo 3
, Javier Vazquez-Corral, 1,2
, Alexandra Gomez-Villa 1,2
1 Computer Vision Center, Spain
2 Universitat Autonoma de Barcelona, Spain `
3
Image Processing Lab, Universitat de Valencia, Spain `
4 Arab Academy for Science, Technology, and Maritime Transport, Egypt
Abstract
Understanding and characterizing human color perception
is a longstanding research goal. One of the most traditional approaches is looking for the human color discrimination thresholds, the minimum chromatic differences perceptible to human
observers. Despite the widespread use of deep vision encoders, few
studies have investigated whether their internal representations
exhibit human-like discrimination thresholds. In this work, we
present a large-scale exploratory study probing the chromatic
sensitivity of more than 50 pretrained vision encoders, including convolutional networks and vision transformers, against human discrimination thresholds.Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The study systematically evaluates the chromatic representation geometry of deep vision encoders against human discrimination behavior <ref:2607.16540#pg2>. This framework allows us to quantify the extent to which modern vision encoders align with human perceptual chromatic thresholds <ref:2607. p. 0 Using controlled chromatic stimuli
and similarity fields derived from latent feature embeddings, we
compare model responses against human discrimination thresholds defined in CIEDE2000 space. This framework allows us to
quantify the extent to which modern vision encoders align with
human perceptual chromatic thresholds. By comparing the structure of these similarity fields with human
discrimination thresholds, we can evaluate the extent to which
modern vision encoders exhibit human-aligned chromatic thresholds. Evaluation Metrics
To quantify alignment between model-derived similarity
fields and human discrimination thresholds, we compute a
region-overlap metric based on intersection over union (IoU,
Eq.Where the paper says it
Where the paper says it
How the training signal is built
The researchers evaluate a large and heterogeneous collection of 50 pretrained vision encoders spanning convolutional neural networks, vision transformers, and hybrid architectures <ref:2607.16540#pg2>. The benchmark includes multiple variants within each model family, with variability arising from differences in scale, training objectives, and optimization strategies <ref:2607. p. 1 Methodology
Vision Encoders
We evaluate a large and heterogeneous collection of 50 pretrained vision encoders spanning a broad spectrum of contemporary visual representation learning paradigms. Rather than relying on a single representative per architecture, the benchmark
includes multiple variants within each model family, with variability arising from differences in scale, training objectives, and
optimization strategies.Where the paper says it
Where the gains come from
The results indicate that self-supervised encoders generally achieve the strongest alignment scores <ref:2607.16540#pg2>. Self-supervised objectives preserve low-level chromatic relationships more effectively, suggesting that objectives preserving local image structure and visual consistency may better retain more perceptually human-like chromatic thresholds <ref:2607. p. 3 Nevertheless, the results reveal trends across training paradigms. Self-supervised encoders generally achieve the strongest
alignment scores, suggesting that objectives preserving local image structure and visual consistency may better retain
more perceptually human-like chromatic thresholds. Our analysis reveals generally weak alignment between model representations
and human perceptual thresholds across all model families, with
the best mIoU < 0. Moreover, we find that self-supervised encoders consistently outperform supervised ones, while languagesupervised models show the most polarized behavior, occupying
both the top and bottom of the ranking.Where the paper says it
Where the paper says it
Dependence on Hue Region and Chroma Level
The alignment is highly dependent on both chromatic location and saturation regime <ref:2607.16540#pg2>. Certain hue regions, particularly blues, consistently produce lower IoU scores across nearly all encoders <ref:2607. p. 3 The results reveal that alignment is highly dependent on
both chromatic location and saturation regime. Certain hue regions (particularly blues, which are also known to exhibit strong
perceptual non-uniformities in classical psychophysical datasets)
consistently produce lower IoU scores across nearly all encoders,
indicating that these areas of color space are systematically more
difficult for learned representations to organize in a psychophysically consistent manner. Certain hue regions (particularly blues, which are also known to exhibit strong
perceptual non-uniformities in classical psychophysical datasets)
consistently produce lower IoU scores across nearly all encoders,
indicating that these areas of color space are systematically more
difficult for learned representations to organize in a psychophysically consistent manner. This behavior is especially pronounced
at high chroma levels, where the human discrimination ellipses
are larger and their anisotropic structure more complex.Where the paper says it
Where the paper says it
Conclusions
The results show that while deep vision encoders consistently develop structured and anisotropic chromatic similarity geometries, these representations only partially align with human perceptual models <ref:2607.16540#pg2>. Alignment varies systematically across chromaticity regions and saturation levels, and depends strongly on training paradigm, with self-supervised and supervised models outperforming language-supervised and generative approaches <ref:2607. p. 4 Our results show that while deep vision encoders consistently develop structured and anisotropic chromatic similarity geometries, these representations only partially align with human
perceptual models. Alignment varies systematically across chromaticity regions and saturation levels, and depends strongly on
training paradigm, with self-supervised and supervised models
outperforming language-supervised and generative approaches. Alignment varies systematically across chromaticity regions and saturation levels, and depends strongly on
training paradigm, with self-supervised and supervised models
outperforming language-supervised and generative approaches. However, even the best-performing models remain far from
human-level chromatic discrimination, suggesting that humanlike chromatic discrimination does not naturally emerge from
current large-scale visual training objectives.Where the paper says it
Where the paper says it
Limitations and future work
This study has some limitations, first focusing on static, synthetic stimuli designed to isolate chromatic effects under controlled conditions <ref:2607.16540#pg2>. Extending the evaluation to include structured stimuli, such as textured surfaces and object-based renderings, would help determine whether chromatic alignment improves under more ecologically valid conditions <ref:2607. p. 5 this enables precise comparison with MacAdam human discrimination thresholds, it necessarily abstracts away from richer visual contexts present in natural scenes. Extending the evaluation to include structured stimuli, such as textured surfaces and
object-based renderings, would help determine whether chromatic alignment improves under more ecologically valid conditions. Extending the evaluation to include structured stimuli, such as textured surfaces and
object-based renderings, would help determine whether chromatic alignment improves under more ecologically valid conditions. In particular, a promising direction is to quantify whether
models preserve the directional structure of MacAdam-like ellipses across different spatial contexts, including full objects
rather than flat color patches. Such an analysis would help disentangle whether current models encode chromatic sensitivity primarily at the pixel level or whether these properties persist in
higher-level representations. Additionally, our evaluation is based on cosine distance
in frozen feature spaces, which may not fully capture nonlinear decoding processes in downstream tasks.Where the paper says it
Where the paper says it
Where the paper says it
Acknowledgments
The work was partially funded by Grants PID2023-152133NB-I00, PID2024-162555OB-I00; by MICIU/AEI/10.13039/501100011033, ERDF/EU and the FEDER; by Spanish MIU under FPU21/02256; by the grant Catedra ENIA UAB-Cru¨ılla (TSI-100929-2023- 2) from the Ministry of Economic Affairs and Digital Transition of Spain; by the grant BBVA Foundations of Science program: Maths, Stats, Comp. p. 5 Acknowledgments
The work was partially funded by Grants PID2023-
152133NB-I00, PID2024-162555OB-I00; by MICIU/AEI/10. 13039/501100011033, ERDF/EU and the FEDER;
by Spanish MIU under FPU21/02256; by the grant Catedra `
ENIA UAB-Cru¨ılla (TSI-100929-2023- 2) from the Ministry
of Economic Affairs and Digital Transition of Spain; by the
grant BBVA Foundations of Science program: Maths, Stats,
Comp. and AI (VIS4NN). JVC also acknowledges the 2025
Leonardo Grant for Scientific Research and Cultural Creation
from the BBVA Foundation. JVC also acknowledges the 2025
Leonardo Grant for Scientific Research and Cultural Creation
from the BBVA Foundation. The BBVA Foundation accepts
no responsibility for the opinions, statements and contents
included in the project and/or the results thereof, which are
entirely the responsibility of the authors.Where the paper says it
Where the paper says it
Where the paper says it
References
[1] David L MacAdam. Visual sensitivities to color differences in daylight <ref:2607. p. 5 References
[1] David L MacAdam. Visual sensitivities to color differences in daylight.Where the paper says it
[2] Ming Ronnier Luo and Bob Rigg. Chromaticity-discrimination ellipses for surface colours <ref:2607. p. 5 [2] Ming Ronnier Luo and Bob Rigg. Chromaticity-discrimination
ellipses for surface colours.Where the paper says it
[3] Roy S. Berns, David H. p. 5 Color Research & Application,
11(1):25–42, 1986. Berns, David H. Berns, David H. Alman, Lisa Reniff, G. Alman, Lisa Reniff, G. Snyder, and
M. Snyder, and
M. Balonon-Rosen.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[4] Commission Internationale de l’Eclairage Colorimetry <ref:2607.16540#pg2>
[5] Roy S. Berns Billmeyer and Saltzman’s Principles of Color Technology <ref:2607. p. 5 15, Central Bureau of the CIE, Vienna,
Austria, 1976. Billmeyer and Saltzman’s Principles of Color Technology.Where the paper says it
[6] F. J. J. Clarke, R. McDonald, and B. p. 5 John Wiley & Sons, Hoboken, NJ, 4th edition, 2019. McDonald, and B. McDonald, and B. Modification to the
JPC79 colour-difference formula.Where the paper says it
Where the paper says it
[7] Commission Internationale de l’Eclairage Industrial colour-difference evaluation <ref:2607.16540#pg2>
[8] M Ronnier Luo, Guihua Cui, and Bryan Rigg The development of the cie 2000 colour-difference formula Ciede2000 <ref:2607.16540#pg2>
[9] Alexandra Gomez-Villa Kai Wang C. Alejandro Parraga Bartlomiej Twardowski Jesus Malo Javier Vazquez-Corral and Joost van den Weijer The art of deception Color visual illusions and diffusion models <ref:2607. p. 5 Alejandro Parraga, Bartlomiej Twardowski, Jesus Malo, Javier Vazquez-Corral, and Joost
van den Weijer. The art of deception: Color visual illusions and
diffusion models.Where the paper says it
[10] Alex Gomez-Villa Adrian Mart´ın Javier Vazquez-Corral Marcelo Bertalm´ıo and Jesus Malo Color illusions also deceive cnns for low-level vision tasks Analysis and implications <ref:2607.16540#pg2>
[11] Jaykishan Patel Alban Flachot Javier Vazquez-Corral David H. Brainard Thomas S. p. 5 [11] Jaykishan Patel, Alban Flachot, Javier Vazquez-Corral, David H. Brainard, Thomas S. Brainard, Thomas S. Wallis, Marcus A. Wallis, Marcus A. Brubaker, and Richard F. Brubaker, and Richard F. A deep convolutional neural network trained for lightness
constancy is susceptible to lightness illusions.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[12] Guillem Arias Ariadna Sola Mart ´ı Armengod, and Maria Vanrell Color encoding in latent space of stable diffusion models <ref:2607.16540#pg2>
[13] Yancheng Cai Fei Yin Dounia Hammou, and Rafal Mantiuk Do computer vision foundation models learn the low-level characteristics of the human visual system In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition CVPR 2025 <ref:2607.16540#pg2>
[14] Nipun Ravindu Wickramanayaka and Masafumi Oizumi Systematic comparison of color representations between humans and deep neural networks towards predicting human color perception in a vast color space bioRxiv pages 2025–12, 2025 <ref:2607.16540#pg2>
[15] Alec Radford Jong Wook Kim Chris Hallacy Aditya Ramesh, et al. Learning transferable visual models from natural language supervision arXiv preprint arXiv:2103. p. 5 Learning transferable visual models from natural language
supervision. arXiv preprint arXiv:2103. arXiv preprint arXiv:2103. 00020, 2021.Where the paper says it
Where the paper says it
[16] Gabriel Ilharco et al. Openclip 2021 GitHub repository <ref:2607. p. 5 Openclip, 2021. GitHub repository.Where the paper says it
[17] Maxime Oquab Timothee Darcet Theo Moutakanni, et al. Dinov2 Learning robust visual features without supervision arXiv preprint arXiv:2304. p. 5 Dinov2: ´
Learning robust visual features without supervision. arXiv preprint
arXiv:2304. arXiv preprint
arXiv:2304. 07193, 2023.Where the paper says it
Where the paper says it
[18] Stability AI Stable diffusion 3 https://stability.ai, 2024 <ref:2607. p. 0 They found that some models, particularly DINOv2 and OpenCLIP, exhibit partial alignment with
human vision, although substantial differences remain Closer
to our work, Wickramanayaka and Oizumi [14] systematically
compared the internal color representations of 16 DNNs against
human color similarity judgments of 93 colors, finding that only
CLIP models achieved fine-grained structural congruence with
human perceptual data. Crucially, however, their analysis targets
the global representational geometry of color space — whether
arXiv:2607.Where the paper says it
[19] X. Zhang et al. p. 5 https://stability. Zhang et al. Pe-clip: Perception-enhanced clip for image-text
representation learning. arXiv preprint, 2023.Where the paper says it
Where the paper says it
[20] Karen Simonyan and Andrew Zisserman Very deep convolutional networks for large-scale image recognition arXiv preprint arXiv:1409.1556, 2014 <ref:2607. p. 5 arXiv preprint
arXiv:1409. 1556, 2014.Where the paper says it
[21] Gaurav Sharma Wencheng Wu and Edul N Dalal The ciede2000 color-difference formula Implementation notes supplementary test data and mathematical observations Color Research & Application, 30(1):21–30, 2005 <ref:2607.16540#pg2>
[22] IEC A standard default color space for the internet - srgb, 1996 IEC 61966-2-1 <ref:2607.16540#pg2>
[23] Alexey Dosovitskiy et al. An image is worth 16x16 words Transformers for image recognition at scale arXiv preprint arXiv:2010. p. 5 An image is worth 16x16 words:
Transformers for image recognition at scale. arXiv preprint
arXiv:2010. arXiv preprint
arXiv:2010. 11929, 2020.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improve color fidelity in digital art preservation and fine-grained product inspection by using self-supervised models, such as DINOv2 or Perception Encoder, for latent space representations because "Self-supervised encoders generally achieve the strongest alignment scores, suggesting that objectives preserving local image structure and visual consistency may better retain more perceptually human-like chromatic thresholds."
-
Develop color analysis tools that are robust across different saturation regimes by prioritizing models trained with self-supervised objectives over purely supervised ones, as the results show
the superiority of self-supervised models over purely supervised shows a clear dependence and the importance of the optimization goal.
-
Enhance model generalization in color tasks by utilizing language-supervised models like AIMv2 when seeking top performance, as these models
show the most polarized behavior, occupying both the top and bottom of the ranking,
suggesting their training objectives capture broader organizational principles. -
Implement a systematic method for diagnosing latent space failures by using cosine distance similarity fields to identify where
the precise geometric organization of the learned fields remains substantially misaligned with human references,
allowing developers to pinpoint specific color-space regions where models fail to reproduce perceptual non-uniformities. -
Design color processing pipelines that account for chromatic location dependence by dynamically adjusting encoding based on hue and chroma levels, as the study found that
alignment is highly dependent on both chromatic location and saturation regime
and noted thatcertain hue regions (particularly blues, which are also known to exhibit strong perceptual non-uniformities in classical psychophysical datasets) consistently produce lower IoU scores.
Abstract
Understanding and characterizing human color perception is a longstanding research goal. One of the most traditional approaches is looking for the human color discrimination thresholds, the minimum chromatic differences perceptible to human observers. In recent years, deep neural networks have become the standard networks for computer vision tasks. In particular, deep vision encoders, foundation models trained on large-scale visual data, map images into latent feature representations. Despite the widespread use of deep vision encoders, few studies have investigated whether their internal representations exhibit human-like discrimination thresholds. In this work, we present a large-scale exploratory study probing the chromatic sensitivity of more than 50 pretrained vision encoders, including convolutional networks and vision transformers, against human discrimination thresholds. Using controlled chromatic stimuli at multiple chroma levels, we compare model-derived chromatic discrimination thresholds with human discrimination ellipses through a region-overlap metric (mIoU). Our analysis reveals generally weak alignment between model representations and human perceptual thresholds across all model families, with the best mIoU < 0.25. Moreover, we find that self-supervised encoders consistently outperform supervised ones, while language-supervised models show the most polarized behavior, occupying both the top and bottom of the ranking. These findings suggest that human-like chromatic sensitivity does not emerge naturally from current large-scale visual training objectives for any of the analyzed architectures.
Sources
- Color encoding in Latent Space of Stable Diffusion Models
- Learning Transferable Visual Models From Natural Language Supervision
- DINOv2: Learning Robust Visual Features without Supervision
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs