Co-Evolutionary Prompt Optimization with Cross-Category Transfer for Zero-Shot Anomaly Detection
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Co-Evolutionary Prompt Optimization with Cross-Category Transfer for Zero-Shot Anomaly Detection".
Jane: Here's the gist of the paper. Zero-shot anomaly detection (ZSAD) has gained significant attention for its practical value in industrial inspection The proposed framework achieves state-of-the-art performance across multiple anomaly detection datasets Framework Introduction CoEvoAD is a co-evolutionary framework for discrete prompt selection that performs prompt search in the discrete natural-language space using an evolutionary algorithm It preserves the interpretability and composability of natural language by iteratively generating, evaluating, and selecting candidate prompts throughout population evolution Furthermore, it introduces a Cross-Category Transfer Objective (CCTO) which treats held-out source categories as proxies for unseen categories to score prompt rules based on their estimated cross-category transferability Discrete Prompt Optimization CoEvoAD formulates prompt optimization as a discrete search over natural-language prompt rules rather than relying on opaque blackbox vector representations It maintains separate populations, Pn and Pa, for normal and abnormal roles, updating them through role-preserving mutation while keeping the role type fixed to prevent mixing descriptions during co-evolution The fitness function combines the evaluation score on source data with the semantic contrast between normal and abnormal descriptions to enable a transparent optimization process without gradients Cross-Category Transfer Objective (CCTO) The key idea behind CCTO is to construct systematic held-out splits within the source domain, where a subset of source categories is treated as proxy unseen categories This design enables the evaluation of candidate prompt rules under a simulated zero-shot transfer scenario, allowing assessment of their consistency and robustness when applied across categories The roleadjusted selection scores are defined as ψn(p, c) = (1 − λn)ηn + λnχn, ψa(p, c) = ηa + λa (ηa − χa), where the normal branch interpolates between in-category performance and cross-category consistency Experimental Validation Extensive experiments validate the effectiveness of CoEvoAD, showing it achieves state-of-the-art performance across multiple anomaly detection datasets The framework shows that interpretable prompt search is feasible for ZSAD via role-separated co-evolution guided by held-out cross-category signals Table 14 demonstrates that CoEvoAD consistently improves both transfer directions, suggesting the gain is not explained by candidate budget alone The framework's inference latency is competitive, running only under 0.7 ms difference compared to the class-name control CoEvoAD attains the highest mean image AUROC/AP (95.0/96.2) and pixel AUROC/PRO (96.9/92.5) among compared CLIP-based ZSAD baselines The benefits of CoEvoAD concentrate on pixellevel localization rather than image-level AUROC, and the gains are not uniform across target categories The selected rules remain auditable as natural-language strings, complementing continuous prompt embeddings References Institute of Information Science, Beijing Jiaotong University, Beijing, China Extensive experiments are conducted to validate the effectiveness of CoEvoAD, and the results show that it achieves state-of-the-art performance across multiple anomaly detection datasets.
Tom: The code is available at https://github.com/rstao-bjtu/CoEvoAD.
Jane: To this end, we propose CoEvoAD, a co-evolutionary framework for discrete prompt selection CoEvoAD performs prompt search in the discrete natural-language space using an evolutionary algorithm.
Tom: Candidate prompts are iteratively generated, evaluated, and selected throughout population evolution, thus preserving the interpretability and composability of natural language.
Paper summary: Jane: So, wrapping up the discussion on this paper, what’s the real takeaway for someone just listening who isn't deep in the math? What does "Co-Evolutionary Prompt Optimization with Cross-Category Transfer for Zero-Shot Anomaly Detection" actually mean in simple terms?
Tom: Well, it means they found a way to make prompt selection interpretable and scalable by searching through natural language rules instead of using complex vector embeddings. The key is using that cross-category transfer objective to ensure the prompts they select aren't just good for the categories they trained on, but actually work when faced with new ones.
Lu: It’s about building a system where you can audit exactly which natural language rules are being used, and that system is robust because it has been tested on simulated zero-shot transfers using those held-out source categories.
Meng: From my side, the fact that they keep the prompt search discrete means it’s more manageable for deployment than if we were trying to optimize a massive continuous embedding space. It gives us something auditable at scale.
Jane: And the authors claim they achieved state-of-the-art performance across multiple anomaly detection datasets, specifically mentioning a mean image AUROC of nine point five and pixel AUROC of nine point six point two. So, they’re showing solid results for this zero-shot capability.
Tom: But I do have to mention the caveat: the benefits are concentrated on pixellevel localization rather than just overall image-level AUROC, and those gains aren't uniform across target categories; for example, the MVTecAD to VisA direction showed a worst category pixel AP regression on pcb4.
Lu: That’s a fair point about uneven gains. It suggests that while the framework is powerful for finding good rules, the quality of those rules still depends on which specific target category you are looking at.
Jane: So, in short, they offer a way to get interpretable prompts that generalize better across different categories than existing methods do, even if the performance bump isn't perfectly even everywhere. That’s what this paper is about.
Conclusion: Tom: So, we're wrapping up this look at CoEvoAD, which is Co-Evolutionary Prompt Optimization with Cross-Category Transfer for Zero-Shot Anomaly Detection.
Jane: To start off, what’s the actual big idea of that title? It sounds pretty dense.
Lu: It’s about using an evolutionary algorithm to pick the best natural language prompts for anomaly detection, and they add this cross-category transfer objective to make sure those prompts work even when seeing something totally new.
Meng: So, we’re talking about searching through language rules instead of just tweaking the numbers in a vector space, right?
Tom: Exactly. They’re keeping the search discrete so it stays interpretable and you can actually read what the AI is doing, which is huge for us when we need to debug something.
Jane: But what does that cross-category transfer thing actually *do* for the detector?
Lu: It treats held-out categories as if they were unseen ones, so it scores the prompt rules based on how well they might generalize across different kinds of anomalies.
Tom: The results show state-of-the-art performance across several datasets, hitting image AUROC numbers around nine point five and pixel AUROC around nine point six.
Jane: Those are high numbers for zero-shot detection, but what’s the catch?
Lu: The authors admit that while the gains are good for pixellevel localization—finding where exactly something is—the improvements aren't uniform across every single target category they tested.
Meng: That means if you're focusing on one specific type of anomaly, you might see a bigger jump than someone else.
Tom: Right. And the rules themselves are still just natural language strings, which means we can actually look at them and understand why the AI picked that particular phrase.
Jane: So it gives us a way to build models that are both powerful for finding anomalies and transparent about how they arrived at those findings.
Lu: It’s a solid framework for showing how you can optimize discrete choices while ensuring those choices aren't just memorizing the training data.
Institute of Information Science, Beijing Jiaotong University · Institute of Automation, Chinese Academy of Sciences
cs.CV
Submitted: 2026-08-29
Updated: 2026-11-11
Comments: 25 pages, 25 figures. Accepted to EMNLP 2026 Main Conference. v2: removes an inaccurate description in Sec. 3.3 and corrects the VisA image AP in Table 1 (from 89.7 to 89.8); all other results are unchanged
Code: https://github.com/rstao-bjtu/CoEvoAD
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 88/100
The gist: Zero-shot anomaly detection (ZSAD) has gained significant attention for its practical value in industrial inspection The proposed framework achieves state-of-the-art performance across multiple…
Key concepts
- CoEvoAD
- A co-evolutionary framework that uses an evolutionary algorithm to search for the best discrete natural-language prompts. It iteratively generates, evaluates, and selects candidate prompts while keeping the roles (normal/abnormal) separate to maintain interpretability.
- Discrete Prompt Optimization
- This method treats prompt optimization as a discrete search over natural-language rules instead of using complex vector representations. It maintains distinct populations for normal and abnormal roles, updating them through role-preserving mutation without mixing descriptions.
- Cross-Category Transfer Objective (CCTO)
- A criterion designed to improve cross-category generalization by treating held-out source categories as proxies for unseen categories. It scores prompt rules based on their estimated transferability between these related categories, simulating zero-shot transfer.
- Role Adjustment Scores
- Mathematical functions used to define the selection score for prompts. They blend in-category performance (normal branch) with cross-category consistency (abnormal branch), allowing the framework to intelligently select rules that are both accurate and transferable.
Terminology
Summary
Zero-shot anomaly detection (ZSAD) has gained significant attention for its practical value in industrial inspection The proposed framework achieves state-of-the-art performance across multiple anomaly detection datasets
Framework Introduction
CoEvoAD is a co-evolutionary framework for discrete prompt selection that performs prompt search in the discrete natural-language space using an evolutionary algorithm It preserves the interpretability and composability of natural language by iteratively generating, evaluating, and selecting candidate prompts throughout population evolution Furthermore, it introduces a Cross-Category Transfer Objective (CCTO) which treats held-out source categories as proxies for unseen categories to score prompt rules based on their estimated cross-category transferability
Discrete Prompt Optimization
CoEvoAD formulates prompt optimization as a discrete search over natural-language prompt rules rather than relying on opaque blackbox vector representations It maintains separate populations, Pn and Pa, for normal and abnormal roles, updating them through role-preserving mutation while keeping the role type fixed to prevent mixing descriptions during co-evolution The fitness function combines the evaluation score on source data with the semantic contrast between normal and abnormal descriptions to enable a transparent optimization process without gradients
Cross-Category Transfer Objective (CCTO)
The key idea behind CCTO is to construct systematic held-out splits within the source domain, where a subset of source categories is treated as proxy unseen categories This design enables the evaluation of candidate prompt rules under a simulated zero-shot transfer scenario, allowing assessment of their consistency and robustness when applied across categories The roleadjusted selection scores are defined as ψn(p, c) = (1 − λn)ηn + λnχn, ψa(p, c) = ηa + λa (ηa − χa), where the normal branch interpolates between in-category performance and cross-category consistency
Experimental Validation
Extensive experiments validate the effectiveness of CoEvoAD, showing it achieves state-of-the-art performance across multiple anomaly detection datasets The framework shows that interpretable prompt search is feasible for ZSAD via role-separated co-evolution guided by held-out cross-category signals Table 14 demonstrates that CoEvoAD consistently improves both transfer directions, suggesting the gain is not explained by candidate budget alone The framework's inference latency is competitive, running only under 0.7 ms difference compared to the class-name control
Conclusion and Limitations
CoEvoAD attains the highest mean image AUROC/AP (95.0/96.2) and pixel AUROC/PRO (96. p. 5 9 – – 95. 2
Table 2: Pixel-level benchmark comparison (%; AUROC and Per-Region Overlap (PRO)).Where the paper says it
References
Institute of Information Science, Beijing Jiaotong University, Beijing, China
Extensive experiments are conducted to validate the effectiveness of CoEvoAD, and the results show that it achieves state-of-the-art performance across multiple anomaly detection datasets. The code is available at https://github. p. 0 Extensive experiments are conducted to validate
the effectiveness of CoEvoAD, and the results show that it achieves state-of-the-art performance across multiple anomaly detection
datasets. The code is available at https://
github. The code is available at https://
github. com/rstao-bjtu/CoEvoAD.Where the paper says it
Where the paper says it
To this end, we propose CoEvoAD, a co-evolutionary framework for discrete prompt selection CoEvoAD performs prompt search in the discrete natural-language space using an evolutionary algorithm. p. 0 To this
end, we propose CoEvoAD, a co-evolutionary
framework for discrete prompt selection. CoEvoAD performs prompt search in the discrete natural-language space using an evolutionary algorithm. CoEvoAD performs prompt search in the discrete natural-language space using an evolutionary algorithm. Candidate prompts are iteratively generated, evaluated, and selected
throughout population evolution, thus preserving the interpretability and composability of natural language. Candidate prompts are iteratively generated, evaluated, and selected
throughout population evolution, thus preserving the interpretability and composability of natural language. Furthermore, we introduce a Cross-Category Transfer Objective
(CCTO), which treats held-out source categories as proxies for unseen categories and
scores prompt rules based on their estimated
cross-category transferability, effectively improving cross-category generalization.Where the paper says it
Where the paper says it
Where the paper says it
CoEvoAD treats cross-category ZSAD as discrete prompt-rule selection, as shown in Figure 2. p. 2 3 Method
3. 1 Overview
CoEvoAD treats cross-category ZSAD as discrete
prompt-rule selection, as shown in Figure 2. 1 Overview
CoEvoAD treats cross-category ZSAD as discrete
prompt-rule selection, as shown in Figure 2. The
prompt-conditioned scorer is kept fixed during
rule search, so the optimization only changes
the natural-language rule strings used to render
prompts.Where the paper says it
Where the paper says it
We maintain two separate populations, Pn = (p n1,. p. 3 Candidates are generated through role-preserving
mutation and selected by the paired fitness function and CCTO, while keeping the role type fixed
prevents normal and abnormal descriptions from
being mixed during co-evolution. We maintain two separate populations,
P
n = {p
n
1
, . , pn
N },
P
a = {p
a
1
, . , pa
N },
(2)
where N is the population size and each p
r
i
is a roletyped rule. , pa
N },
(2)
where N is the population size and each p
r
i
is a roletyped rule. Initial populations are constructed from
role-specific templates and descriptor pools; since
each candidate is an explicit natural-language rule,
the same rule can be instantiated for another category by substituting the class-name placeholder.Where the paper says it
Where the paper says it
Where the paper says it
To address the cross-category generalization issue, we introduce the Cross-Category Transfer Objective (CCTO), a prompt-rule selection criterion specifically designed for cross-category transfer. p. 1 Overall, the
framework transforms prompt-rule search from a
single black-box vector into a structured, multirole, multi-candidate co-evolutionary system, effectively improving interpretability and scalability. To address the cross-category generalization issue, we introduce the Cross-Category Transfer Objective (CCTO), a prompt-rule selection criterion
specifically designed for cross-category transfer. To address the cross-category generalization issue, we introduce the Cross-Category Transfer Objective (CCTO), a prompt-rule selection criterion
specifically designed for cross-category transfer. The key idea behind CCTO is to construct systematic held-out splits within the source domain,
where a subset of source categories is treated as
proxy unseen categories. The key idea behind CCTO is to construct systematic held-out splits within the source domain,
where a subset of source categories is treated as
proxy unseen categories. This design enables the
evaluation of candidate prompt rules under a simulated zero-shot transfer scenario, allowing us to
assess their consistency and robustness when applied across categories.Where the paper says it
Where the paper says it
Where the paper says it
CoEvoAD runs the same frozen scorer call as the class-name control, differing only in the prompt string: the two differ by under 0. p. 8 Table 8 compares per-image
inference latency under one implementation-level
protocol: single RTX 4090, batch size 1, 30 warmup followed by 300 timed images per direction,
with the timer wrapping the entire per-image call. CoEvoAD runs the same frozen scorer call as the
class-name control, differing only in the prompt
string: the two differ by under 0. CoEvoAD runs the same frozen scorer call as the
class-name control, differing only in the prompt
string: the two differ by under 0. 7 ms with opposite signs across the two directions, so no systematic overhead is attributable to the evolved
rules; the routing lookup, measured separately,
adds 0. Evaluation metrics. Following common ZSAD
evaluation, we report image-level AUROC and AP
and pixel-level AUROC and Per-Region Overlap
(PRO) for benchmark comparison. 31 cls_N ×
Semantic / Template route
𝑒𝑡
Global &
Patch Features
Evaluate
rule pairs
Role-preserving
mutation
Selected
rule pair
for a source
category
𝒑𝒏,⋆, 𝒑 Select Elite 𝒂,⋆
Trained
Scorer
❄
Figure 2: Overview of CoEvoAD. A frozen source-trained scorer evaluates role-separated normal and abnormal rule
populations; CCTO uses held-out source categories to select rules for unseen-category inference. CoEvoAD’s inference is 1. 0× faster than AnomalyCLIP, MRAD, AdaCLIP, and Bayes-PFL retimed under the same protocol.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The benefits of CoEvoAD concentrate on pixellevel localization rather than image-level AUROC: matched-control gains in image AUROC are modest and not uniformly positive, and CCTO gains vary across target datasets. p. 8 The saved rules
remain auditable as natural-language strings, complementing continuous prompt embeddings. Limitations
The benefits of CoEvoAD concentrate on pixellevel localization rather than image-level AUROC:
matched-control gains in image AUROC are modest and not uniformly positive, and CCTO gains
vary across target datasets. Limitations
The benefits of CoEvoAD concentrate on pixellevel localization rather than image-level AUROC:
matched-control gains in image AUROC are modest and not uniformly positive, and CCTO gains
vary across target datasets. The gains are not
uniform across target categories: the MVTecAD→VisA direction shows a worst-category pixelAP regression on pcb4 (Table 25).Where the paper says it
Where the paper says it
Table 10 lists the full implementation and hyperparameters. p. 8 Inference latency. Table 8 compares per-image
inference latency under one implementation-level
protocol: single RTX 4090, batch size 1, 30 warmup followed by 300 timed images per direction,
with the timer wrapping the entire per-image call.Where the paper says it
Table 29: Evolved prompt rules per VisA source category, for the VisA→MVTec-AD direction.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
CoEvoAD improves prompt search by performing
prompt search in the discrete natural-language space using an evolutionary algorithm
instead of relying onopaque blackbox vector representations,
which enhances interpretability and composability of natural language rules. -
The Cross-Category Transfer Objective (CCTO) allows the system to score prompt rules based on their
estimated cross-category transferability,
effectively improving cross-category generalization by treating held-out source categories asproxies for unseen categories.
-
The system can generate interpretable, human-readable rule strings, such as
a photo of a damaged pcb4
orfaulty flawed deteriorated pcb4,
which are stored for auditability and can be instantiated with any category name via the class-name slot. -
The framework enables role separation by maintaining
independent populations for the two roles
(normal and abnormal), ensuring that normal descriptions remain focused onacceptable appearance
while abnormal descriptions focus ondefect evidence.
-
The system can perform inference for an unseen target category by using a
prespecified rule-transfer policy,
selecting a stored pair based only on category names, and rendering the rules to produce anomaly scores.
Abstract
Zero-shot anomaly detection (ZSAD) has gained significant attention for its practical value in industrial inspection. Recently, CLIP-based approaches have been widely adopted in ZSAD due to their strong vision-language generalization capabilities. However, existing methods commonly employ continuous prompt embeddings for prompt optimization and encode semantics in latent vectors, which lack interpretability and scalability. To this end, we propose CoEvoAD, a co-evolutionary framework for discrete prompt selection. CoEvoAD performs prompt search in the discrete natural-language space using an evolutionary algorithm. Candidate prompts are iteratively generated, evaluated, and selected throughout population evolution, thus preserving the interpretability and composability of natural language. Furthermore, we introduce a Cross-Category Transfer Objective (CCTO), which treats held-out source categories as proxies for unseen categories and scores prompt rules based on their estimated cross-category transferability, effectively improving cross-category generalization. Extensive experiments are conducted to validate the effectiveness of CoEvoAD, and the results show that it achieves state-of-the-art performance across multiple anomaly detection datasets. The code is available at https://github.com/rstao-bjtu/CoEvoAD.
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs