AOI-SSL: Self-Supervised Framework for Efficient Segmentation of Wire-bonded Semiconductors In Optical Inspection

arXiv:2605.12430 · cs.CV · Submitted 2026-05-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "AOI-SSL: Self-Supervised Framework for Efficient Segmentation of Wire-bonded Semiconductors In Optical Inspection".

Title and authors: Tom: Now we’re looking at the paper titled AOI-SSL: Self-Supervised Framework for Efficient Segmentation of Wire-bonded Semiconductors In Optical Inspection and who came up with it.

Jane: So the main idea is that these models are usually device specific, meaning you have to retrain them every time you get a new device or when things shift around in production.

Lu: The authors introduce AOI-SSL as a training-efficient framework for semantic segmentation of wire-bonded semiconductors by combining small domain self supervised pretraining of vision transformers with in context inference that minimizes the need for labeled examples.

Meng: So what does that mean in practice? They are trying to reduce the amount of manual labeling we have to do when we introduce new equipment.

Tom: Exactly, they pre-train state of the art self supervised algorithms on a small industrial inspection dataset and found that Masked Autoencoders are the most effective in this small data setting.

Jane: And they showed that this helps improve downstream segmentation while reducing the amount of labeled fine tuning effort required later.

Lu: They also introduce retrieval based segmentation, which is an in context test time adaptation strategy that minimizes needing labeled examples for every new device layout.

Tom: So it’s about making the adaptation process much faster and less reliant on having a huge set of perfectly labeled examples for every single variant.

The paper's summary: Tom: Let's talk about what AOI-SSL actually does, putting together the main points of the paper.

Jane: Basically, they built a system that doesn't need massive amounts of labeled data just to understand the specific geometry of wire bonds in inspection.

Lu: The core implication is that we can move away from painstakingly labeling every new device setup toward a much more automated way to adapt our visual models as production shifts.

Meng: For me, it means reduced downtime when we introduce new equipment because the adaptation happens on the memory bank instead of requiring a full retraining cycle.

Lalam: I’m confident in this direction because my analysis shows that domain pre trained vision transformers capture those precise geometric features way better than standard convolutional models or generic ImageNet weights.

Tom: And they proved it with numbers showing a fifty point seven percent improvement in segmentation quality under just an eight hour training budget when using Masked Autoencoders for efficient segmentation of wire-bonded semiconductors In Optical Inspection.

Jane: It really shows that this is a practical tool for industry because it’s data efficient and gives us high quality results quickly without needing huge teams of annotators.

Lu: The biggest thing is the hybrid FasterViT zero architecture, which gives us better performance on those fine wire details compared to a standard vision transformer tiny model.

Meng: It’s good to hear about efficiency but I still have my eye on those scaling questions they mentioned earlier regarding higher resolution images and production requirements.

Tom: Exactly. The paper flags that future work needs to assess scaling in higher resolution scenarios and meet the segmentation quality requirements for actual production environments.

The paper's improvements: Tom: Let’s go into what they actually improved, beyond just the initial concept of AOI-SSL.

Jane: They introduced a three component framework: domain specific self supervised pretraining, time budgeted fine tuning with a lightweight decoder, and retrieval based in context segmentation.

Lu: In the pre training stage, they looked at whether domain specific SSL can yield representations that rival or surpass those of established encoders pre trained on massive generic datasets like ImageNet.

Meng: So they tested three different self supervised paradigms: Masked Autoencoders called MAE, DINO, and iBOT.

Tom: And they found that Masked Autoencoders are the most effective in this small data setting, leading to faster convergence and higher segmentation performance compared to DINO and iBOT.

Jane: MAE uses a reconstruction objective driven by the mean squared error loss between the unmasked input image and the reconstructed image.

Lu: DINO focuses on aligning global class tokens, while iBOT extends this framework by incorporating a Masked Image Modeling applied to patch level tokens.

Tom: They also adapted MAE using a modified pre-training strategy motivated by DINO and iBOT for FasterViT architecture, which they call FasterViT.

Meng: What I’m interested in is that the retrieval based segmentation outperforms finetuning when targeting single device images, allowing for near instant adaptation to difficult samples.

Conclusion: Tom: So we've gone through how AOI-SSL uses Masked Autoencoders for pre training to get fast models and retrieval for near instant adaptation on new devices, and now we're wrapping up what all this means in terms of concrete improvements.

Jane: So while this is a strong framework now the next step is proving it can handle the real world's noise and scale without breaking down.

Lu: This AOI-SSL work lays a solid foundation by combining that Masked Autoencoder pre training with the retrieval mechanism to tackle data scarcity in specialized visual domains.

Meng: Overall it’s a really smart way to handle device specific segmentation by shifting the burden from constant manual labeling to efficient self supervised learning.

Lalam: I'm sure this method is going to be a solid tool for anyone dealing with complex visual inspection tasks where data is naturally scarce.

Tom: It sounds like the whole point of AOI-SSL is moving away from needing constant manual labeling toward a system that learns and adapts on its own within the constraints of the domain.

Jane: That's really what this paper suggests shifting the burden from massive human annotation to efficient self supervised learning for these specific inspection tasks.

Lu: We're seeing how effectively vision transformers can be adapted to these industrial inspection tasks when given the right self supervised training signal.

Meng: I’m just thinking about how we would benchmark this against a production environment where the variability across devices is huge and unpredictable.

Tom: That sets up a clear path forward for making these specialized inspection models more practical for real shop floor use.

Joaquán Figueira, Rob van Gastel, Giacomo D’Amicantonio, Zhuoran Liu, Ioan Gabriel Bucur, Faysal Boughorbel, Egor Bondarev

AIMS Lab, Eindhoven University of Technology 2CoC, ASMPT 3 · iCIS, Radboud University

cs.CV

Submitted: 2026-05-12

Updated: 2026-05-12

Comments: Accepted to the AI4RWC Workshop at CVPR 2026

Journal ref: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2026, AI4RWC Workshop

Code: https://github.com/jacomof/aoi-ssl

License: http://creativecommons.org/licenses/by-sa/4.0/

Importance score: 82/100

The gist: Segmentation models in automated optical inspection of wire-bonded semiconductors are typically device-specific and must be re-trained when new devices or distribution shifts appear AOI-SSL…

Key concepts

Domain-specific Self-Supervised Pretraining
This involves training a Vision Transformer (ViT) using a reconstruction task like Masked Autoencoders (MAE) on a small dataset of wire-bonded semiconductors. The goal is for the model to learn meaningful visual representations specific to this domain, which helps it understand the unique patterns of these devices much better than models trained on general images.
Masked Autoencoders (MAE)
MAE is a self-supervised learning technique where parts of an input image are masked or hidden, and the model must reconstruct them. In this context, MAE is used to pretrain the ViT by forcing it to learn rich features from the visual data of semiconductor inspection images without needing manual labels.
Retrieval-based Segmentation
This technique allows a model to perform segmentation almost instantly on new devices by using a memory bank of previously labeled examples. When presented with a new image, the system retrieves relevant labels from this bank to predict the segmentation mask, enabling near-instant adaptation without retraining.

Terminology

Summary

Segmentation models in automated optical inspection of wire-bonded semiconductors are typically device-specific and must be re-trained when new devices or distribution shifts appear AOI-SSL, a training-efficient framework for semantic segmentation of wire-bonded semiconductors by combining small-domain self-supervised pretraining of vision transformers with in-context inference that minimizes the need of labeled examples We pre-train SOTA self-supervised algorithms in a small industrial inspection dataset and find that Masked Autoencoders are the most effective in this small-data setting, improving downstream segmentation while reducing the labeled fine-tuning effort Retrieval based segmentation outperforms finetuning when targeting single device images, allowing for near-instant adaptation to difficult samples. p. 0

Where the paper says it

We introduce AOI-SSL, a training-efficient framework for semantic segmentation of wire-bonded semiconductors by combining small-domain self-supervised pretraining of vision transformers with in-context inference that minimizes the need of labeled examples. We pre-train SOTA self-supervised algorithms in a small industrial inspection dataset and find that Masked Autoencoders are the most effective in this small-data setting, improving downstream segmentation while reducing the labeled fine-tuning effort.

Pruning redundant reasoning tokens

The paper introduces AOI-SSL, a comprehensive framework tailored for domain-specific automated optical inspection by integrating selfsupervised pre-training and fine-tuning strategies designed for low-data regimes The approach utilizes efficient Vision Transformer (ViT) architectures and in-context test-time adaptation. The framework has three main components: domain-specific self-supervised pretraining, time-budgeted fine-tuning with a lightweight decoder, and retrieval-based, in-context segmentation. p. 3

Where the paper says it

Our approach has three components: (i) domain-specific self-supervised pretraining, (ii) time-budgeted fine-tuning with a lightweight decoder, and (iii) retrieval-based, in-context segmentation that avoids re-training by transferring labels from a labeled memory as detailed in Fig. Moreover, since capturing the fine-grained geometry of the components is of the upmost importance for AOI, we use the mean Intersection over Union (mIoU) metric as our main evaluation tool.

How the training signal is built

The authors investigate whether domain-specific SSL can yield representations that rival or surpass those of established encoders pre-trained on massive, generic datasets such as ImageNet. Specifically, MAE pre-training is particularly effective in this setting, leading to faster convergence and higher segmentation performance compared to alternative pre-training strategies. p. 1

Where the paper says it

Our focus is on improving the downstream performance of the encoders for semantic segmentation and minimizing the amount of costly human-labeled data. We show that, in this setting, MAE pre-training is particularly effective, leading to faster convergence and higher segmentation performance compared to alternative pre-training strategies.

The authors adapt three SSL paradigms—MAE, DINO, and iBOT—to the pre-training dataset. p. 3
Where the paper says it

ViT-Tiny additionally includes image level embeddings with a special class token: [CLS]. Domain-Specific Self-Supervised Pre-training We adapt three SSL paradigms to our pre-training dataset: MAE, DINO, and iBOT.

** MAE uses a reconstruction objective driven by the mean squared error (MSE) loss between the unmasked input image and the reconstructed image. p. 3

Where the paper says it

It is done by masking part of the input via the elimination of random patch tokens, which are then reconstructed in pixel space using a light-weight transformer-based decoder head. Learning is driven by the mean squared error (MSE) loss between the unmasked input image and the reconstructed image.

1 **

** DINO focuses on aligning global [CLS] tokens, while iBOT extends this framework by incorporating a Masked Image Modeling (MIM) objective applied to patch-level tokens. The authors also adapt MAE using a modified pre-training strategy motivated by iBOT and DINO for FasterViT. p. 4

Where the paper says it

For MAE, a modified pre-training strategy motivated by iBOT and DINO is used, where learnable masked tokens are included in both the encoder and decoder inputs. This is a necessary adaptation to allow the use of the technique in FasterViT, which is not capable of processing an arbitrary number of tokens per-image as required in the original MAE masking strategy [20].

1 **

Where the gains come from

The paper demonstrates significant improvements in performance and efficiency through several key findings.

  1. Small-data AOI pre-training protocol: MAE outperforms DINO and iBOT in the small-data regime, leading to faster convergence and higher segmentation performance. p. 1

    Where the paper says it

    Small-data AOI pre-training protocol: We develop an SSL pre-training protocol tailored to small AOI datasets and tiny models under adverse conditions (two-channel input, low variance, black borders, fine structures, and edge compute), and show that MAE outperforms DINO and iBOT in this regime. Efficiency gain: We demonstrate that MAE pre-training substantially accelerates fine-tuning and yields superior segmentation quality (up to ∼50.

  2. Efficiency gain: MAE pre-training substantially accelerates fine-tuning and yields superior segmentation quality (up to ∼50. p. 1

    Where the paper says it

    Small-data AOI pre-training protocol: We develop an SSL pre-training protocol tailored to small AOI datasets and tiny models under adverse conditions (two-channel input, low variance, black borders, fine structures, and edge compute), and show that MAE outperforms DINO and iBOT in this regime. Efficiency gain: We demonstrate that MAE pre-training substantially accelerates fine-tuning and yields superior segmentation quality (up to ∼50.

    7% relative improvement) under a fixed ∼8 hour training budget. p. 1
    Where the paper says it

    Efficiency gain: We demonstrate that MAE pre-training substantially accelerates fine-tuning and yields superior segmentation quality (up to ∼50. 7% relative improvement) under a fixed ∼8 hour training budget.

  3. Practical retrieval-based segmentation: Retrieval strategies achieve training-free, near instant test-time adaptation by transferring labels from a labeled memory bank. p. 1

    Where the paper says it

    7% relative improvement) under a fixed ∼8 hour training budget. Practical retrieval-based segmentation: We introduce image/patch retrieval strategies for AOI achieving training-free, near instant test-time adaptation, and show that patch-level cosine similarity aggregation is often sufficient, while attention-based aggregation is not universally superior for multi-label dense retrieval.

Furthermore, the hybrid FasterViT-0 architecture yields superior performance compared to standard ViT-Tiny due to its integration of hierarchical downsampling and overlapping tokenization. p. 6

Where the paper says it

A critical finding in our method comparison is the superior performance of the hybrid FasterViT-0 architecture. While the standard ViT-Tiny achieves a respectable 53.

Limitations

The authors acknowledge several avenues for future optimization, including incorporating a knowledge distillation module from a larger encoder to refine feature representations. They also considered alternative loss formulations like Focal Loss and class re-weighting to address extreme class imbalance. p. 7

Where the paper says it

Our approach focuses on direct SSL pre-training; however, incorporating a knowledge distillation module from a larger encoder could further refine the feature representations. Additionally, while the extreme class imbalance of AOI segmentation was addressed through region-based losses, alternative formulations such as the Focal Loss [25] and class re-weighting were considered.

Finally, future work is required to assess scaling in higher resolution scenarios and meet the segmentation quality requirements of inspection models in production. p. 7
Where the paper says it

Given the substantial overhead of hyperparameter tuning for this setting, we leave such strategies for future work. Lastly, although our evaluation demonstrates efficacy in center-cropped images, future work is required to assess scaling in higher resolution scenarios and meet the segmentation quality requirements of inspection models in production.

Conclusion

This work introduces AOI-SSL, a data-efficient framework for semiconductor AOI that overcomes data scarcity by combining domain-specific MAE pre-training with a patch-level retrieval mechanism Our results demonstrate that domain pre-trained ViTs significantly outperform convolutional models and generic ImageNet weights in capturing precise geometric features. Furthermore, the integration of retrieval mechanisms enables near-instant adaptation to new device layouts Ultimately, this hybrid approach provides a robust solution to the challenges of segmentation in AOI. p. 7

Where the paper says it

Furthermore, the integration of retrieval mechanisms enables near-instant adaptation to new device layouts. Ultimately, this hybrid approach provides a robust solution to the challenges of segmentation in AOI.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Domain-specific self-supervised pre-training protocol: The system can learn meaningful representations for wire-bonded semiconductors by adapting Masked Autoencoders (MAE) to small industrial inspection datasets, which improves downstream segmentation while reducing labeled fine-tuning effort.

  2. Retrieval-based segmentation for near instant adaptation: The system can achieve training free, near instant test time adaptation to new devices by using patch level retrieval methods that predict masks directly from dense encoder embeddings with negligible additional training.

  3. Improved feature representation capture: By utilizing domain pre-trained Vision Transformers (ViT) significantly outperform convolutional models and generic ImageNet weights in capturing precise geometric features, especially for fine-grained structures like wires.

  4. Enhanced architectural inductive bias: Incorporating convolutional elements, such as FasterViT, yields significant gains for fine-grained structures over traditional vision transformers by providing hierarchical attention with downsampling stages.

  5. Superior segmentation quality under budget constraints: The system can achieve superior segmentation quality (up to 50.7% relative improvement) under a fixed training budget of approximately 8 hours when using MAE pre-training compared to training from scratch or ImageNet backbones.

  6. Robustness against data shifts: The retrieval mechanism enables adaptation to new devices by updating a labeled memory bank rather than re-training a full decoder, allowing for near instant adaptation to covariate shifts from capture and process variation.

Abstract

Segmentation models in automated optical inspection of wire-bonded semiconductors are typically device-specific and must be re-trained when new devices or distribution shifts appear. We introduce AOI-SSL, a training-efficient framework for semantic segmentation of wire-bonded semiconductors by combining small-domain self-supervised pre-training of vision transformers with in-context inference that minimizes the need of labeled examples. We pre-train SOTA self-supervised algorithms in a small industrial inspection dataset and find that Masked Autoencoders are the most effective in this small-data setting, improving downstream segmentation while reducing the labeled fine-tuning effort. We further introduce in-context, patch-level retrieval methods that predict masks directly from dense encoder embeddings with negligible additional training. We show that, in this setting, simple similarity-based retrieval performs on par with more complex attention-based aggregation used currently in the literature. Furthermore, our experiments demonstrate that self-supervised pre-training significantly improves segmentation quality compared to training from scratch and to ImageNet pre-trained backbones under a fixed fine-tuning computational budget. Finally, the results reveal that retrieval based segmentation outperforms fine-tuning when targeting single device images, allowing for near-instant adaptation to difficult samples.

Sources

Related papers