Neural Object Detection for 4D STEM: High-Throughput Sub-Pixel Electron Diffraction Pattern Recognition

arXiv:2506.04477 · cond-mat.mtrl-sci · Submitted 2025-06-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Condensed Matter. The day in condensed matter: materials, phases and the measurements under them.

Kai: I'm Kai.

Mira: And I'm Mira, here with Lev.

Kai: We're looking at the paper: "Neural Object Detection for 4D STEM".

Mira: Here's the gist of the paper. High-throughput analysis of multidimensional transmission electron microscopy (TEM) datasets remains a significant challenge, restricting TEM’s broader applicability in strategic materials research<ref:2506.04477#pg2> In this work, we introduce an endto-end post-processing framework for large-scale fourdimensional scanning transmission electron microscopy (4D-STEM) datasets, built around a highly efficient neural network-based object detection model<ref:2506.04477#pg2> Central to our method is a sub-pixel accurate object center localization algorithm, which serves as the foundation for high-precision and high-throughput analysis of electron diffraction patterns<ref:2506.04477#pg2> We demonstrate a strain measurement precision of 5x10−4, quantified by the standard deviation of strain values within the strain-free Si substrate of a Si/SiGe multilayer TEM sample<ref:2506.04477#pg2> Furthermore, by implementing an asynchronous, non-blocking object detection workflow, we achieve speeds exceeding 100 frames per second (fps), substantially accelerating the crystallographic phase identification and strain mapping in complex multiphase metallic alloys<ref:2506.04477#pg2> The Core Framework The framework is designed to replace conventional peak-fitting and crosscorrelation algorithms with a learned regression approach for direct localization of electron diffraction patterns<ref:2506.04477#pg2> This end-to-end object detection-based framework is built around a highly efficient neural network architecture<ref:2506.04477#pg2> The model utilizes a fully convolutional framework, adopting the YOLOv8 architecture which directly predicts bounding box coordinates from feature maps<ref:2506.04477#pg2> This anchor box-free design enables a well-generalized object-center-aware feature augmentation where grid cells serve as reference points for predicting offsets to object centers<ref:2506.04477#pg2> The backbone of the network consists of a deep CNN responsible for extracting hierarchical feature maps from input images, consisting of convolutional blocks with residual connections denoted as C1→C2→C3→C4→C5<ref:2506.04477#pg2> How the training signal is built Model training begins with transfer learning using the pre-trained weights of the YOLOv8 Nano (YOLOv8n) model, which serves as the foundation for bounding box predictions<ref:2506.04477#pg2> This model is then fine-tuned using manually annotated images to adapt specifically for detecting the center coordinates of diffraction discs<ref:2506.04477#pg2> The YOLOv8n model is designed for lightweight and efficient object detection tasks, featuring the fewest convolutional layers, channels, and parameters (3.5 million parameters)<ref:2506.04477#pg2> For fine-tuning, we curated a dataset of 42 electron diffraction patterns obtained from a diverse range of materials<ref:2506.04477#pg2> Each diffraction pattern was manually annotated to generate ground truth bounding boxes, and the dataset was subsequently divided into training and validation subsets to facilitate model evaluation<ref:2506.04477#pg2> Where the gains come from The performance of our object detection-based peak localization workflow is evaluated on the task of disentangling complex interatomic displacement fields in a multiphase Ti-80Nb metallic alloy<ref:2506.04477#pg2> The model incorporates an efficient autocorrelation-based approach, leveraging the Wiener–Khinchin theorem to improve center localization<ref:2506.04477#pg2> An asynchronous function-calling strategy enables parallel processing of large 4D-STEM datasets, ensuring that I/O operations are non-blocking during AI model inference<ref:2506.04477#pg2> To further enhance performance, we integrated automatic mixed precision (AMP) during inference, which leverages lower precision arithmetic (e.g., FP16) to reduce memory usage and increase computation speed<ref:2506.04477#pg2> Experimental Validation and Results The workflow was tested on a Si/SiGe multilayer TEM sample to measure strain mapping fidelity<ref:2506.04477#pg2> The object detection model generates bounding box predictions, and from these predictions, we extract the center coordinates for each diffraction disc<ref:2506.04477#pg2> Strain maps εxx and εyy represent the out-of-plane and in-plane strain fields populated from two orthogonal reciprocal lattice vectors, ⟨100⟩ and ⟨110⟩, respectively<ref:2506.04477#pg2> The standard deviation representing the precision of the analysis is calculated using the Si substrate<ref:2506.04477#pg2> Computational benchmarking showed an effective throughput of 110 fps when processing a 97×40 pixels field of view dataset<ref:2506.04477#pg2> The performance on the Ti-80Nb alloy revealed small, spherical bccprecipitates (Phase I) coherently embedded within the parent β-bcc matrix<ref:2506.04477#pg2> In this work, we introduce a parallelized post-processing framework for large-scale 4D-STEM datasets, leveraging an efficient neural network-based object detection model<ref:2506.04477#pg2> Rapid and accurate object detection presents a promising avenue for the analysis of electron diffraction patterns and the localization of interatomic lattice distances<ref:2506.04477#pg2> Our approach enables sub-pixel precision in the detection of diffraction features, enabling high-accuracy crystallographic analysis<ref:2506.04477#pg2> The ability to rapidly and reliably extract crystallographic information from 4D-STEM datasets acquired under varying experimental conditions accelerates materials discovery<ref:2506.

Title and authors: Kai: So we're looking at this paper today called Neural Object Detection for 4D STEM: High-Throughput Sub-Pixel Electron Diffraction Pattern Recognition <ref:2506.04477#pg1,Neural Object Detection for 4D STEM: High-Throughput Sub-Pixel Electron Diffraction>. It sounds like they're tackling a huge headache in materials research, which is dealing with these massive four-dimensional scanning transmission electron microscopy datasets.

Mira: Exactly. The title itself tells you the core idea: using neural network object detection to handle those huge 4D STEM datasets and get sub-pixel accuracy on the electron diffraction patterns <ref:2506.04477#pg1>. It's about making high-throughput analysis possible where it was before, which is really limiting TEM’s use in strategic materials research <ref:2506.04477#pg1>.

Lev: From an error correction standpoint, that implies we are moving away from these slow, manual steps and replacing them with something automated and learned. It suggests a massive reduction in the time it takes to get meaningful crystallographic information out of the data <ref:2506.04477#pg1>.

Kai: Right. So what's actually in this paper? The summary says they introduce an end-to-end post-processing framework built around a neural network model specifically for detecting those diffraction patterns and localizing them accurately using a sub-pixel accurate center localization algorithm <ref:2506.04477#pg1>.

Mira: It's not just about finding the patterns; it’s about bridging the gap where conventional workflows fail because they don't record a diffraction pattern for every single probe position in 4D STEM, which is what this technique does by using high-speed detectors <ref:2506.04477#pg2>.

Lev: And they are using a fully convolutional framework based on the YOLOv8 architecture, which is designed to directly predict bounding box coordinates from feature maps without needing anchor boxes <ref:2506.04477#pg1>. That’s a big architectural move for speed and flexibility.

Kai: The training side seems pretty smart too. They start with transfer learning using the pre-trained weights of the YOLOv8 Nano model, which is lightweight, and then they fine-tune it on a dataset of forty-two electron diffraction patterns manually annotated to detect those centers <ref:2506.04477#pg2>.

Mira: That training signal is crucial because it adapts the general detection ability of the YOLOv8 model specifically to find the center coordinates of those diffraction discs, which is what you need for high-precision mapping <ref:2506.04477#pg2>.

Lev: From an engineering view, that three point five million parameter model size sounds efficient enough for practical use, and the design using grid cells as reference points for predicting offsets to object centers helps generalize the detection across different patterns <ref:2506.04477#pg1>.

Title and authors: Kai: So where does this actually show up in terms of performance? They evaluated it on a Si/SiGe multilayer TEM sample to measure strain mapping fidelity, and they report a strain measurement precision of five times ten to the minus four, which they quantify by the standard deviation of strain values within the Si substrate <ref:2506.04477#pg1>.

Mira: That level of precision is what matters when you're trying to disentangle complex interatomic displacement fields in a multiphase Ti-80Nb metallic alloy, which they used for testing <ref:2506.04477#pg2>.

Lev: And they achieved this by incorporating an efficient autocorrelation based approach, leveraging the Wiener–Khinchin theorem to improve the center localization, which is a clever mathematical trick <ref:2506.04477#pg3>.

Kai: They also used an asynchronous, non-blocking object detection workflow combined with automatic mixed precision during inference to handle the data flow efficiently and speed up the analysis substantially <ref:2506.04477#pg2>.

Mira: That combination of techniques is what gets them speeds exceeding one hundred frames per second, which significantly accelerates crystallographic phase identification and strain mapping in complex metallic alloys <ref:2506.04477#pg1>.

Lev: It's about making sure the AI inference doesn't get bogged down by the input/output operations, keeping things non-blocking while also using lower precision arithmetic like FP16 to cut down on memory usage and increase computation speed <ref:2506.04477#pg2>.

Kai: The experimental validation showed an effective throughput of one hundred ten frames per second when processing a dataset that was ninety seven by forty pixels in field of view, which is pretty solid performance for this kind of imaging task <ref:2506.04477#pg1>.

Mira: What’s interesting is that the results on the Ti-80Nb alloy revealed small, spherical bcc precipitates identified as phase one coherently embedded within the parent beta bcc matrix, which gives us physical insight into what we are mapping <ref:2506.04477#pg2>.

Lev: So, to summarize the improvements they mentioned: they replaced conventional peak-fitting and crosscorrelation algorithms with a learned regression approach for direct localization of electron diffraction patterns <ref:2506.04477#pg1>.

Kai: And they focused on the object detection model itself, specifically developing an object detection model based on YOLOv8 Nano fine-tuned for detecting the center coordinates of electron diffraction discs using that anchor box-free design <ref:2506.04477#pg1>.

Mira: They also emphasized the role of asynchronous function-calling strategy and automatic mixed precision during inference to get that fourfold increase in frames per second and reduce processing time interactively on standard desktop computers <ref:2506.04477#pg2>.

Title and authors: Lev: And they also mentioned an efficient autocorrelation based approach leveraging the Wiener–Khinchin theorem without applying a logarithmic transform during spectral decomposition, which helps enhance the extraction of interatomic distances <ref:2506.04477#pg3>.

Kai: So, to wrap up, this work on Neural Object Detection for 4D STEM is introducing a parallelized post-processing framework that uses an efficient neural network model to rapidly localize electron diffraction patterns with sub-pixel precision <ref:2506.04477#pg1>.

Mira: Rapid and accurate object detection presents a promising avenue for the analysis of electron diffraction patterns and the localization of interatomic lattice distances, enabling high-accuracy crystallographic analysis <ref:2506.04477#pg1>.

Lev: This ability to rapidly and reliably extract crystallographic information from 4D STEM datasets acquired under varying experimental conditions really accelerates materials discovery <ref:2506.04477#pg1>.

Kai: So, that’s the core idea: using AI to handle the sheer volume of data coming from 4D-STEM so we can get better, faster results on strain and phase mapping <ref:2506.04477#pg2>.

Mira: It moves us from slow, sequential processing steps to a dynamic, end-to-end framework that handles the complexity inherent in these large datasets <ref:2506.04477#pg3>.

Lev: The caveat they mention is that CNN architectures like U-Net often exhibit limited generalization with respect to variations in feature scale, meaning model performance is closely tied to the quantity and dive of training data <ref:2506.04477#pg3>.

Kai: It’s a fair point about generalization; it means we still need good datasets when we move beyond the specific materials they tested on <ref:2506.04477#pg1>.

Mira: And that ties back into how much curated data, like those forty-two patterns they used for fine-tuning, is actually necessary to make the model robust across different types of samples <ref:2506.04477#pg2>.

Lev: From a hardware perspective, getting that sub-pixel accuracy and high throughput running reliably on real hardware means we need to manage the computational overhead of that entire pipeline carefully <ref:2506.04477#pg1>.

Kai: We'll keep an eye on how these models scale as the data volumes get even larger, which is where this kind of end-to-end framework will really show its value in strategic materials research <ref:2506.04477#pg1>.

Mira: It’s about making sure that the incredible data acquisition from 4D STEM isn't wasted just because we can't process it fast enough to get those critical insights <ref:2506.04477#pg2>.

Lev: So, this paper on Neural Object Detection for 4D STEM is a significant step toward making high-throughput analysis practical for real-world materials science experiments <ref:2506.04477#pg1>.

The paper's summary: Kai: So, what we're hearing is that this paper is about building an entire system to process those huge four dimensional scanning transmission electron microscopy datasets in a way that's much faster than what we can do now.

Mira: Right, it’s basically replacing the old slow methods—the peak-fitting and crosscorrelation stuff—with something learned by the machine. It uses an end-to-end post processing framework built around a neural network to directly find and localize those electron diffraction patterns.

Lev: That means instead of a human having to look at every single spot in that massive four dimensional map, the AI does it all automatically and learns how to do it by itself.

Kai: It’s not just finding them; the authors are focusing on sub pixel accurate object center localization, which they claim gives us strain measurement precision of five times ten to the minus four. That’s a very specific number for accuracy in this field.

Mira: And that precision is what allows them to map out strain fields in things like Si/SiGe multilayer samples with a lot more detail than before. It’s about getting the fine details you need for crystallographic analysis out of the data without missing anything.

Lev: From an engineering standpoint, that level of accuracy is what makes it viable for real hardware. If you were to run this on a real system, you’re looking at a lot of overhead because you have to manage that AI inference process while keeping the data flow non blocking and using automatic mixed precision during the calculations.

Kai: They achieved speeds exceeding one hundred frames per second when processing a field of view dataset, which is what they call high throughput analysis. That speed is what really changes things for strategic materials research, because you can get faster feedback loops.

Mira: And that speed isn't just for the sake of being fast; it allows them to quickly identify crystallographic phases and map strain in complex metallic alloys like that Ti-80Nb sample they tested. It means you can see what's happening at a microscopic level much quicker than waiting for traditional methods.

Lev: The method itself is clever because they use an anchor box free design with the YOLOv8 architecture, which lets it predict those center coordinates directly from feature maps without needing those pre-defined boxes. That’s a significant architectural choice for efficiency.

Kai: It also mentions using an efficient autocorrelation based approach leveraging the Wiener–Khinchin theorem to improve the localization, which is a mathematical trick to clean up the data and reduce ambiguity caused by sample tilt or thickness variations.

Mira: And they fine-tuned this YOLOv8 model using a small dataset of forty two electron diffraction patterns that they manually annotated to get ground truth bounding boxes. That training signal is what makes it specific enough for their application.

Lev: So, the implication here is that you can get high fidelity strain mapping and phase identification on large 4D STEM datasets much faster, which accelerates the entire discovery cycle in materials science <ref:2506.04477#pg1>.

Kai: It’s about moving from slow manual analysis to rapid automated extraction of crystallographic information under varying experimental conditions.

Mira: This capability really means we can analyze complex samples with a level of detail that was previously just out of reach because the data volume was too large for conventional techniques to handle efficiently.

The paper's improvements: Tom: So we're looking at what the authors are suggesting to do next with this object detection framework for 4D STEM data <ref:2506.04477#pg1>.

Kai: They’re pointing toward making the model more robust across different materials, which means they need more diverse training data to ensure it doesn't just memorize the patterns from one sample. It suggests that the generalization capability is a key area for future work.

Mira: Exactly, and they mentioned that since CNN architectures like U-Net can struggle with feature scale variations, building better ways to handle those different scales of features will be important for making the model work on more varied samples. It’s about improving how the network sees things at different resolutions.

Lev: From an engineering standpoint, that translates to needing more sophisticated data augmentation strategies or perhaps a multi-scale network design to handle those feature scale differences effectively. It means we need better ways to train the AI to recognize patterns regardless of whether the feature is big or small in the image.

Kai: They also touched on how they can integrate this detection workflow into a broader, more adaptive framework, which could potentially allow for real-time, interactive analysis directly on standard desktop computers. That would be a big step for accessibility.

Mira: It’s about moving beyond just processing existing data and toward a system that can dynamically adjust its analysis based on what it sees in the moment. That opens up possibilities for more complex, automated experiments where the AI guides the next measurement or analysis step.

Lev: If they succeed in making it truly interactive and adaptive, that lowers the barrier significantly for running these kinds of high throughput analyses on standard computing hardware, which is a big hurdle right now when you're dealing with massive datasets.

Kai: So, what this suggests is a future where we don't just run one analysis pipeline, but have an adaptable system that can handle the inherent noise and variation in real world TEM data.

Mira: It means the next big step isn't just about getting faster detection; it’s about building intelligence into the analysis process itself, allowing it to handle more complex scientific questions without needing constant manual reprogramming.

Conclusion: Kai: So we're wrapping up this look at "Neural Object Detection for 4D STEM: High-Throughput Sub-Pixel Electron Diffraction Pattern Recognition" <ref:2506.04477#pg1,Neural Object Detection for 4D STEM: High-Throughput Sub-Pixel Electron Diffraction>. Basically, they've shown how you can use AI to process those huge four dimensional scanning transmission electron microscopy datasets much faster than before.

Mira: Right, the main point is that rapid and accurate object detection is a real path for analyzing electron diffraction patterns and figuring out where the atomic lattice distances are located with high accuracy. It means we can do much more detailed crystallographic analysis than we could manually.

Lev: I just want to say that being able to rapidly extract crystallographic information from 4D STEM datasets acquired under varying experimental conditions really accelerates materials discovery, which is a massive time saver for anyone trying to figure out new stuff <ref:2506.04477#pg1>.

Kai: It’s about taking the complexity of 4D data and making it manageable by using an efficient neural network model that learns to do the localization sub pixel level <ref:2506.04477#pg1>.

Mira: That ability to rapidly and reliably extract crystallographic information is what lets us see what's happening in complex metallic alloys much quicker than before, which is critical for understanding phase transitions and strain fields.

Lev: The method itself uses a parallelized post-processing framework that leverages an efficient neural network model to do the heavy lifting of localization without getting bogged down by slow I/O operations during inference.

Kai: It’s about moving away from those slow peak-fitting methods to a learned regression approach, which is much more direct for finding the pattern centers.

Mira: And that direct localization, when combined with sub pixel accuracy, allows for strain mapping precision of five times ten to the minus four in their test sample. That level of fidelity is what makes these results meaningful for materials science.

Lev: If you were to run this on real hardware, you’d need to manage the computational overhead from that entire pipeline carefully because it’s pushing speeds over one hundred frames per second.

Kai: So, in the end, this paper on "Neural Object Detection for 4D STEM: High-Throughput Sub-Pixel Electron Diffraction Pattern Recognition" gives us a framework where AI handles the massive data volume to enable high accuracy crystallographic analysis <ref:2506.04477#pg1,Neural Object Detection for 4D STEM: High-Throughput Sub-Pixel Electron Diffraction>.

Mira: It’s about making sure that we don't waste the incredible data acquisition from 4D STEM just because we can't process it fast enough to get those critical insights <ref:2506.04477#pg1>.

Lev: So, this is a solid piece of work for integrating AI into high fidelity imaging analysis, and it opens the door for using these techniques on even larger datasets down the line.

Arda Genc, Ravit Silverstein

University of California Santa Barbara · University of Florida

cond-mat.mtrl-sci

Submitted: 2025-06-04

Updated: 2025-07-15

Comments: Accepted to ICCV2025

DOI: 10.1109/ICCVW69036.2025.00378

Code: https://github.com/ultralytics/ultralytics

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 90/100

The gist: High-throughput analysis of multidimensional transmission electron microscopy (TEM) datasets remains a significant challenge…

Key concepts

End-to-end object detection framework
This is a complete system where the neural network handles everything from input image to output. Instead of using separate steps for finding patterns and measuring them, the model learns directly how to find the diffraction discs and determine their exact center coordinates in one integrated process, making it more efficient.
YOLOv8 Architecture
YOLOv8 is a specific type of neural network designed for fast object detection. It uses a fully convolutional structure that predicts bounding box coordinates directly from feature maps without needing pre-defined anchor boxes. This makes the model very flexible and capable of quickly identifying the precise location of diffraction features in images.
Sub-pixel accurate localization
This refers to achieving extremely fine spatial resolution when locating features, down to a fraction of a pixel. The method uses an efficient algorithm based on autocorrelation and the Wiener–Khinchin theorem to refine the initial detection, ensuring that the center coordinates of diffraction patterns are pinpointed with high precision.
Asynchronous, non-blocking workflow
This technique allows the computer to perform other tasks while waiting for a long process, such as AI model inference on large data. By making operations asynchronous and non-blocking, the system can process multiple frames simultaneously at high speeds, drastically improving overall throughput when handling massive 4D-STEM datasets.

Terminology

Summary

High-throughput analysis of multidimensional transmission electron microscopy (TEM) datasets remains a significant challenge, restricting TEM’s broader applicability in strategic materials research<ref:2506. p. 0

Where the paper says it

edu, rsilverstein@ufl. edu Abstract High-throughput analysis of multidimensional transmission electron microscopy (TEM) datasets remains a significant challenge, restricting TEM’s broader applicability in strategic materials research.

04477#pg2> In this work, we introduce an endto-end post-processing framework for large-scale fourdimensional scanning transmission electron microscopy (4D-STEM) datasets, built around a highly efficient neural network-based object detection model<ref:2506. p. 0
Where the paper says it

Conventional workflows typically involve sequential, modular processing steps that necessitate extensive manual intervention and offline parameter tuning. In this work, we introduce an endto-end post-processing framework for large-scale fourdimensional scanning transmission electron microscopy (4D-STEM) datasets, built around a highly efficient neural network-based object detection model.

04477#pg2> Central to our method is a sub-pixel accurate object center localization algorithm, which serves as the foundation for high-precision and high-throughput analysis of electron diffraction patterns<ref:2506. p. 0
Where the paper says it

In this work, we introduce an endto-end post-processing framework for large-scale fourdimensional scanning transmission electron microscopy (4D-STEM) datasets, built around a highly efficient neural network-based object detection model. Central to our method is a sub-pixel accurate object center localization algorithm, which serves as the foundation for high-precision and high-throughput analysis of electron diffraction patterns.

04477#pg2> We demonstrate a strain measurement precision of 5x10−4, quantified by the standard deviation of strain values within the strain-free Si substrate of a Si/SiGe multilayer TEM sample<ref:2506. p. 0
Where the paper says it

Central to our method is a sub-pixel accurate object center localization algorithm, which serves as the foundation for high-precision and high-throughput analysis of electron diffraction patterns. We demonstrate a strain measurement precision of 5x10−4 , quantified by the standard deviation of strain values within the strain-free Si substrate of a Si/SiGe multilayer TEM sample.

04477#pg2> Furthermore, by implementing an asynchronous, non-blocking object detection workflow, we achieve speeds exceeding 100 frames per second (fps), substantially accelerating the crystallographic phase identification and strain mapping in complex multiphase metallic alloys<ref:2506. p. 0
Where the paper says it

We demonstrate a strain measurement precision of 5x10−4 , quantified by the standard deviation of strain values within the strain-free Si substrate of a Si/SiGe multilayer TEM sample. Furthermore, by implementing an asynchronous, non-blocking object detection workflow, we achieve speeds exceeding 100 frames per second (fps), substantially accelerating the crystallographic phase identification and strain mapping in complex multiphase metallic alloys.

04477#pg2>

The Core Framework

The framework is designed to replace conventional peak-fitting and crosscorrelation algorithms with a learned regression approach for direct localization of electron diffraction patterns<ref:2506.04477#pg2> This end-to-end object detection-based framework is built around a highly efficient neural network architecture<ref:2506. p. 0

Where the paper says it

Conventional workflows typically involve sequential, modular processing steps that necessitate extensive manual intervention and offline parameter tuning. In this work, we introduce an endto-end post-processing framework for large-scale fourdimensional scanning transmission electron microscopy (4D-STEM) datasets, built around a highly efficient neural network-based object detection model.

04477#pg2> The model utilizes a fully convolutional framework, adopting the YOLOv8 architecture which directly predicts bounding box coordinates from feature maps<ref:2506. p. 2
Where the paper says it

Methodology 3. Bragg disc detection and localization Single-stage object detection models, such as YOLOv8, adopt a fully convolutional framework that directly predicts bounding box coordinates from feature maps [22, 54].

04477#pg2> This anchor box-free design enables a well-generalized object-center-aware feature augmentation where grid cells serve as reference points for predicting offsets to object centers<ref:2506. p. 2
Where the paper says it

Bragg disc detection and localization Single-stage object detection models, such as YOLOv8, adopt a fully convolutional framework that directly predicts bounding box coordinates from feature maps [22, 54]. This anchor box-free design enables a well-generalized [14], object-center-aware feature augmentation, wherein grid cells serve as reference points for predicting offsets to object centers.

04477#pg2> The backbone of the network consists of a deep CNN responsible for extracting hierarchical feature maps from input images, consisting of convolutional blocks with residual connections denoted as C1→C2→C3→C4→C5<ref:2506. p. 2
Where the paper says it

The backbone, composed of a deep CNN, is responsible for extracting hierarchical feature maps from input images. It consists of a sequence of convolutional blocks with residual connections, denoted as C1→C2→C3→C4→C5, corresponding to increasing stride levels of 2, 4, 8, 16, and 32.

04477#pg2>

How the training signal is built

Model training begins with transfer learning using the pre-trained weights of the YOLOv8 Nano (YOLOv8n) model, which serves as the foundation for bounding box predictions<ref:2506.04477#pg2> This model is then fine-tuned using manually annotated images to adapt specifically for detecting the center coordinates of diffraction discs<ref:2506. p. 4

Where the paper says it

Model training and optimization Model training for the object detection task begins with transfer learning using the pre-trained weights of the YOLOv8 Nano (YOLOv8n) model, which serves as the foundation for the bounding box predictions. This model is then fine-tuned using manually annotated images to adapt specifically for detecting the center coordinates of the diffraction discs.

04477#pg2> The YOLOv8n model is designed for lightweight and efficient object detection tasks, featuring the fewest convolutional layers, channels, and parameters (3. p. 4
Where the paper says it

Each bounding box prediction includes the center coordinates along with the box’s width and height (bx, by, bw, bh). The YOLOv8 Nano model is designed for lightweight and efficient object detection tasks, featuring the fewest convolutional layers, channels, and parameters (3.

5 million parameters)<ref:2506. p. 4
Where the paper says it

The YOLOv8 Nano model is designed for lightweight and efficient object detection tasks, featuring the fewest convolutional layers, channels, and parameters (3. 5 million parameters).

04477#pg2> For fine-tuning, we curated a dataset of 42 electron diffraction patterns obtained from a diverse range of materials<ref:2506. p. 4
Where the paper says it

The YOLOv8n model was initialized through transfer learning, leveraging pre-trained weights from the MS COCO (Microsoft Common Objects in Context) dataset, which comprises 330,000 annotated images across 80 object classes. For fine-tuning, we curated a dataset of 42 electron diffraction patterns obtained from a diverse range of materials, including metallic alloys and thin-film heterostructures.

04477#pg2> Each diffraction pattern was manually annotated to generate ground truth bounding boxes, and the dataset was subsequently divided into training and validation subsets to facilitate model evaluation<ref:2506. p. 4
Where the paper says it

For fine-tuning, we curated a dataset of 42 electron diffraction patterns obtained from a diverse range of materials, including metallic alloys and thin-film heterostructures. Each diffraction pattern was manually annotated to generate ground truth bounding boxes, and the dataset was subsequently divided 5

04477#pg2>

Where the gains come from

The performance of our object detection-based peak localization workflow is evaluated on the task of disentangling complex interatomic displacement fields in a multiphase Ti-80Nb metallic alloy<ref:2506.04477#pg2> The model incorporates an efficient autocorrelation-based approach, leveraging the Wiener–Khinchin theorem to improve center localization<ref:2506. p. 3

Where the paper says it

5 + dbottom − dtop 2 · s (3) Where cx and cy are the column and row indices of the grid cell, dn is the bounding box parameter, and s is the stride level. To improve the center localization in our object detection model, we incorporated an efficient autocorrelationbased approach, leveraging the Wiener–Khinchin theorem [17, 23, 57].

04477#pg2> An asynchronous function-calling strategy enables parallel processing of large 4D-STEM datasets, ensuring that I/O operations are non-blocking during AI model inference<ref:2506. p. 4
Where the paper says it

An asynchronous function-calling strategy enables parallel processing of large 4D-STEM datasets [15]. This design ensures that I/O operations are non-blocking during AI model inference, thereby eliminating GPU load/unload cycles and maximizing computational throughput, as shown in Fig.

04477#pg2> To further enhance performance, we integrated automatic mixed precision (AMP) during inference, which leverages lower precision arithmetic (e. p. 4
Where the paper says it

To further enhance performance, we integrated automatic mixed precision (AMP) during inference [31]. AMP leverages lower precision arithmetic (e.

g., FP16) to reduce memory usage and increase computation speed<ref:2506. p. 4
Where the paper says it

, FP16) where appropriate, reducing memory usage and increasing computation speed without compromising model accuracy. This combination of asynchronous processing and AMP resulted in a fourfold increase in fps and a substantial reduction in end-to-end processing time.

04477#pg2>

Experimental Validation and Results

The workflow was tested on a Si/SiGe multilayer TEM sample to measure strain mapping fidelity<ref:2506.04477#pg2> The object detection model generates bounding box predictions, and from these predictions, we extract the center coordinates for each diffraction disc<ref:2506. p. 5

Where the paper says it

6 illustrates a set of diffraction patterns from the Si substrate of the Si/SiGe TEM sample. The object detection model generates bounding box predictions, and from these predictions, we extract the center coordinates for each diffraction disc.

04477#pg2> Strain maps εxx and εyy represent the out-of-plane and in-plane strain fields populated from two orthogonal reciprocal lattice vectors, ⟨100⟩ and ⟨110⟩, respectively<ref:2506. p. 5
Where the paper says it

The alternating Si and SiGe layers provide distinct light and dark contrast, as shown in the virtual annular dark-field STEM image in Fig. Strain maps εxx and εyy represent the out-of-plane and in-plane strain fields populated from two orthogonal reciprocal lattice vectors, ⟨100⟩ and ⟨110⟩, respectively.

04477#pg2> The standard deviation representing the precision of the analysis is calculated using the Si substrate<ref:2506. p. 6
Where the paper says it

to minimum strain value of 0 % across the alternating Si layers sandwiched between the SiGe layers, contributing to a well-defined experimental profile. The standard deviation representing the precision of the analysis is calculated using the Si substrate.

04477#pg2> Computational benchmarking showed an effective throughput of 110 fps when processing a 97×40 pixels field of view dataset<ref:2506. p. 6
Where the paper says it

To evaluate the end-toend latency of the strain analysis, we utilized a 4D-STEM dataset acquired from an ROI scanned over a 97×40 pixels field of view. A total of 3880 diffraction patterns, each with an image resolution of 512×512 pixels, were processed asynchronously in 35 seconds, yielding an effective throughput of 110 fps.

04477#pg2> The performance on the Ti-80Nb alloy revealed small, spherical bccprecipitates (Phase I) coherently embedded within the parent β-bcc matrix<ref:2506. p. 7
Where the paper says it

8 presents example displacement field maps derived from two orthogonal ⟨110⟩ reciprocal lattice vectors, respectively. These maps reveal small, spherical bccprecipitates (Phase I), approximately 50 nm in size, that are coherently embedded within the parent β-bcc matrix.

04477#pg2>

Conclusion

In this work, we introduce a parallelized post-processing framework for large-scale 4D-STEM datasets, leveraging an efficient neural network-based object detection model<ref:2506.04477#pg2> Rapid and accurate object detection presents a promising avenue for the analysis of electron diffraction patterns and the localization of interatomic lattice distances<ref:2506. p. 7

Where the paper says it

Conclusion In this work, we introduce a parallelized post-processing framework for large-scale 4D-STEM datasets, leveraging an efficient neural network-based object detection model. Rapid and accurate object detection presents a promising avenue for the analysis of electron diffraction patterns and the localization of interatomic lattice distances.

04477#pg2> Our approach enables sub-pixel precision in the detection of diffraction features, enabling high-accuracy crystallographic analysis<ref:2506. p. 7
Where the paper says it

Rapid and accurate object detection presents a promising avenue for the analysis of electron diffraction patterns and the localization of interatomic lattice distances. Our approach enables sub-pixel precision in the detection of diffraction features, enabling high-accuracy crystallographic analysis.

04477#pg2> The ability to rapidly and reliably extract crystallographic information from 4D-STEM datasets acquired under varying experimental conditions accelerates materials discovery<ref:2506. p. 7
Where the paper says it

Our approach enables sub-pixel precision in the detection of diffraction features, enabling high-accuracy crystallographic analysis. The ability to rapidly and reliably extract crystallographic information from 4D-STEM datasets acquired under varying experimental conditions accelerates materials discovery, from semiconductor development to alloy design.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Accelerated 4D-STEM data analysis by replacing conventional peak-fitting and crosscorrelation algorithms with a learned regression approach for direct localization of electron diffraction patterns. This allows for real-time analysis of large 4D-STEM datasets, achieving speeds exceeding 100 frames per second (fps).

  2. Sub-pixel accurate object center localization using an endto-end post-processing framework built around a highly efficient neural network model. This enables strain measurement precision of 5x10-4, significantly reducing endto-end latency in crystallographic phase identification and strain mapping.

  3. Implementation of an asynchronous, non-blocking object detection workflow combined with automatic mixed precision (AMP) during inference. This combination results in a fourfold increase in fps and a substantial reduction in end-to-end processing time when analyzing large 4D-STEM datasets interactively on standard desktop computers.

  4. Development of an object detection model based on YOLOv8 Nano fine-tuned for detecting the center coordinates of electron diffraction discs, utilizing an anchor box-free design with center-based feature augmentation. This compact design enhances the model’s ability to rapidly and accurately detect centers in high-throughput analysis applications.

  5. Incorporation of an efficient autocorrelationbased approach leveraging the Wiener–Khinchin theorem without applying a logarithmic transform during spectral decomposition. This strategy enhances the extraction of interatomic distances while minimizing potential ambiguities in Bragg disc intensities caused by sample tilt and thickness variations.

Abstract

High-throughput analysis of multidimensional transmission electron microscopy (TEM) datasets remains a significant challenge, limiting the broader impact on strategic materials research. Conventional workflows typically involve sequential, modular processing steps that necessitate extensive manual intervention and offline parameter tuning. In this work, we introduce an end-to-end post-processing framework for large-scale four-dimensional scanning TEM (4D-STEM) datasets, built around a highly efficient neural network-based object detection model. Central to our method is a sub-pixel accurate object center localization algorithm, which serves as the foundation for high-precision and high-throughput analysis of electron diffraction patterns. We demonstrate a strain measurement precision of 5x 10-4, quantified by the standard deviation of strain values within the strain-free Si substrate of a Si/SiGe multilayer TEM sample. Furthermore, by implementing an asynchronous, non-blocking object detection workflow, we achieve speeds exceeding 100 frames per second (fps), substantially accelerating the crystallographic phase identification and strain mapping in complex multiphase metallic alloys.

Sources

Related papers