AOI-SSL: Self-Supervised Framework for Efficient Segmentation of Wire-bonded Semiconductors In Optical Inspection
summary
The gist
Segmentation models in automated optical inspection of wire-bonded semiconductors are typically device-specific and must be re-trained when new devices or distribution shifts appear AOI-SSL…
In short
AOI-SSL is a training-efficient framework for segmenting wire-bonded semiconductors by combining self-supervised pretraining with retrieval methods. It uses Masked Autoencoders (MAE) on small industrial datasets to learn device features, which significantly improves segmentation quality and reduces the need for extensive labeled fine-tuning. This allows the model to adapt quickly to new devices.
Key concepts
- Domain-specific Self-Supervised Pretraining
- This involves training a Vision Transformer (ViT) using a reconstruction task like Masked Autoencoders (MAE) on a small dataset of wire-bonded semiconductors. The goal is for the model to learn meaningful visual representations specific to this domain, which helps it understand the unique patterns of these devices much better than models trained on general images.
- Masked Autoencoders (MAE)
- MAE is a self-supervised learning technique where parts of an input image are masked or hidden, and the model must reconstruct them. In this context, MAE is used to pretrain the ViT by forcing it to learn rich features from the visual data of semiconductor inspection images without needing manual labels.
- Retrieval-based Segmentation
- This technique allows a model to perform segmentation almost instantly on new devices by using a memory bank of previously labeled examples. When presented with a new image, the system retrieves relevant labels from this bank to predict the segmentation mask, enabling near-instant adaptation without retraining.
Terminology used across episodes
This episode discusses
- AOI-SSL: Self-Supervised Framework for Efficient Segmentation of Wire-bonded Semiconductors In Optical Inspection · Paper Radio
- BEiT: BERT Pre-Training of Image Transformers
- Rethinking Atrous Convolution for Semantic Image Segmentation
- DINOv3
The paper
AOI-SSL: Self-Supervised Framework for Efficient Segmentation of Wire-bonded Semiconductors In Optical Inspection · Read on arXiv
Joaquán Figueira, Rob van Gastel, Giacomo D’Amicantonio, Zhuoran Liu, Ioan Gabriel Bucur, Faysal Boughorbel, Egor Bondarev
AIMS Lab, Eindhoven University of Technology 2CoC, ASMPT 3 · iCIS, Radboud University
Segmentation models in automated optical inspection of wire-bonded semiconductors are typically device-specific and must be re-trained when new devices or distribution shifts appear. We introduce AOI-SSL, a training-efficient framework for semantic segmentation of wire-bonded semiconductors by combining small-domain self-supervised pre-training of vision transformers with in-context inference that minimizes the need of labeled examples. We pre-train SOTA self-supervised algorithms in a small industrial inspection dataset and find that Masked Autoencoders are the most effective in this small-data setting, improving downstream segmentation while reducing the labeled fine-tuning effort. We further introduce in-context, patch-level retrieval methods that predict masks directly from dense encoder embeddings with negligible additional training. We show that, in this setting, simple similarity-based retrieval performs on par with more complex attention-based aggregation used currently in the literature. Furthermore, our experiments demonstrate that self-supervised pre-training significantly improves segmentation quality compared to training from scratch and to ImageNet pre-trained backbones under a fixed fine-tuning computational budget. Finally, the results reveal that retrieval based segmentation outperforms fine-tuning when targeting single device images, allowing for near-instant adaptation to difficult samples.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "AOI-SSL: Self-Supervised Framework for Efficient Segmentation of Wire-bonded Semiconductors In Optical Inspection".
Title and authors: Tom: Now we’re looking at the paper titled AOI-SSL: Self-Supervised Framework for Efficient Segmentation of Wire-bonded Semiconductors In Optical Inspection and who came up with it.
Jane: So the main idea is that these models are usually device specific, meaning you have to retrain them every time you get a new device or when things shift around in production.
Lu: The authors introduce AOI-SSL as a training-efficient framework for semantic segmentation of wire-bonded semiconductors by combining small domain self supervised pretraining of vision transformers with in context inference that minimizes the need for labeled examples.
Meng: So what does that mean in practice? They are trying to reduce the amount of manual labeling we have to do when we introduce new equipment.
Tom: Exactly, they pre-train state of the art self supervised algorithms on a small industrial inspection dataset and found that Masked Autoencoders are the most effective in this small data setting.
Jane: And they showed that this helps improve downstream segmentation while reducing the amount of labeled fine tuning effort required later.
Lu: They also introduce retrieval based segmentation, which is an in context test time adaptation strategy that minimizes needing labeled examples for every new device layout.
Tom: So it’s about making the adaptation process much faster and less reliant on having a huge set of perfectly labeled examples for every single variant.
The paper's summary: Tom: Let's talk about what AOI-SSL actually does, putting together the main points of the paper.
Jane: Basically, they built a system that doesn't need massive amounts of labeled data just to understand the specific geometry of wire bonds in inspection.
Lu: The core implication is that we can move away from painstakingly labeling every new device setup toward a much more automated way to adapt our visual models as production shifts.
Meng: For me, it means reduced downtime when we introduce new equipment because the adaptation happens on the memory bank instead of requiring a full retraining cycle.
Lalam: I’m confident in this direction because my analysis shows that domain pre trained vision transformers capture those precise geometric features way better than standard convolutional models or generic ImageNet weights.
Tom: And they proved it with numbers showing a fifty point seven percent improvement in segmentation quality under just an eight hour training budget when using Masked Autoencoders for efficient segmentation of wire-bonded semiconductors In Optical Inspection.
Jane: It really shows that this is a practical tool for industry because it’s data efficient and gives us high quality results quickly without needing huge teams of annotators.
Lu: The biggest thing is the hybrid FasterViT zero architecture, which gives us better performance on those fine wire details compared to a standard vision transformer tiny model.
Meng: It’s good to hear about efficiency but I still have my eye on those scaling questions they mentioned earlier regarding higher resolution images and production requirements.
Tom: Exactly. The paper flags that future work needs to assess scaling in higher resolution scenarios and meet the segmentation quality requirements for actual production environments.
The paper's improvements: Tom: Let’s go into what they actually improved, beyond just the initial concept of AOI-SSL.
Jane: They introduced a three component framework: domain specific self supervised pretraining, time budgeted fine tuning with a lightweight decoder, and retrieval based in context segmentation.
Lu: In the pre training stage, they looked at whether domain specific SSL can yield representations that rival or surpass those of established encoders pre trained on massive generic datasets like ImageNet.
Meng: So they tested three different self supervised paradigms: Masked Autoencoders called MAE, DINO, and iBOT.
Tom: And they found that Masked Autoencoders are the most effective in this small data setting, leading to faster convergence and higher segmentation performance compared to DINO and iBOT.
Jane: MAE uses a reconstruction objective driven by the mean squared error loss between the unmasked input image and the reconstructed image.
Lu: DINO focuses on aligning global class tokens, while iBOT extends this framework by incorporating a Masked Image Modeling applied to patch level tokens.
Tom: They also adapted MAE using a modified pre-training strategy motivated by DINO and iBOT for FasterViT architecture, which they call FasterViT.
Meng: What I’m interested in is that the retrieval based segmentation outperforms finetuning when targeting single device images, allowing for near instant adaptation to difficult samples.
Conclusion: Tom: So we've gone through how AOI-SSL uses Masked Autoencoders for pre training to get fast models and retrieval for near instant adaptation on new devices, and now we're wrapping up what all this means in terms of concrete improvements.
Jane: So while this is a strong framework now the next step is proving it can handle the real world's noise and scale without breaking down.
Lu: This AOI-SSL work lays a solid foundation by combining that Masked Autoencoder pre training with the retrieval mechanism to tackle data scarcity in specialized visual domains.
Meng: Overall it’s a really smart way to handle device specific segmentation by shifting the burden from constant manual labeling to efficient self supervised learning.
Lalam: I'm sure this method is going to be a solid tool for anyone dealing with complex visual inspection tasks where data is naturally scarce.
Tom: It sounds like the whole point of AOI-SSL is moving away from needing constant manual labeling toward a system that learns and adapts on its own within the constraints of the domain.
Jane: That's really what this paper suggests shifting the burden from massive human annotation to efficient self supervised learning for these specific inspection tasks.
Lu: We're seeing how effectively vision transformers can be adapted to these industrial inspection tasks when given the right self supervised training signal.
Meng: I’m just thinking about how we would benchmark this against a production environment where the variability across devices is huge and unpredictable.
Tom: That sets up a clear path forward for making these specialized inspection models more practical for real shop floor use.
More episodes
- 2603.28113-The multiplicative Lipschitz bound as an inductive bias
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound