Multitask Scanning Probe Microscopy

The paper introduces multitask scanning probe microscopy (SPM), a live, closed-loop workflow in which a multitask Gaussian process learns spatial and cross-modal relationships and autonomously selects both the next measurement location and the next experimental protocol. The approach is implemented on an automated large-sample atomic force microscope and demonstrated on a composition-spread AlScN wafer using tapping-mode and Dual AC Resonance Tracking (DART) measurements.

The experimental problem contains two coupled decisions: the microscope must determine where an additional measurement will be informative and which measurement protocol should be performed at that location. Conventional automated and autonomous scanning probe workflows have largely focused on the first of these decisions, with the measurement protocol selected before the experiment and remaining fixed throughout the active-learning process. The paper states: Multimodal measurements are consequently acquired according to a predefined sequence or repeated at every selected location, irrespective of whether all modalities provide sufficient additional information. The authors note that Live, closed-loop selection and execution of both the measurement location and the instrument protocol on an operating microscope has, however, remained outstanding.

In the multitask framework, individual scanning probe modalities are treated as related prediction problems, referred to as tasks, each with its own spatial response landscape and measurement noise but sharing a common underlying material structure. A multitask Gaussian process learns the spatial structure within each task together with the correlations between tasks. The paper explains: "A measurement performed using one modality updates the model predictions for that modality and, through the learned statistical correlation between modalities, simultaneously refines the predictions for the unmeasured modality across the entire wafer."

The implementation uses the intrinsic coregionalization model (ICM). The augmented model input appends a task index to the spatial coordinate, and the covariance between two location-task pairs is K[(x, t), (x′, t′)] = Kx(x, x′) · Btt′, where Kx is the spatial covariance kernel and B is the T × T task-covariance matrix. The task-covariance matrix is represented as B = WWT + diag(κ), where W is a T × R matrix defining the low-rank shared latent representation and κ accounts for task-specific variance. The spatial kernel is a Matérn 5/2 kernel with automatic relevance determination, providing independent length scales along the two wafer spatial coordinates. The paper notes: The Matérn 5/2 kernel is preferred over the squared-exponential kernel because it permits less smooth spatial variation, which is appropriate for surface properties that may change sharply across the wafer.

At each active-learning step, the model predicts the posterior mean and uncertainty for every task at all candidate locations. The decision domain is the joint set of unmeasured location-task pairs. The acquisition function uses random-scalarization upper-confidence-bound, where task weights are sampled from a symmetric Dirichlet distribution and applied to a standardized UCB value computed separately for each task. The selected experiment is (x*, t*) = arg max wt(signt · μ̂t(x) + √β eff · σt(x)), where β eff is an exploration parameter annealed over the course of the experiment. To prevent spatial clustering and biased hyperparameter estimation, a random exploration step is inserted every three active-learning iterations, selecting the candidate location farthest from all previously measured positions with task assigned uniformly at random.

The experiment was conducted on an Oxford Instruments Asylum Research Jupiter AFM equipped with a motorized large-sample stage and software-accessible instrument controls, interfaced through the aespm automated-experiment software environment. The experimental domain was a circular region with a radius of 35 mm on an AlScN composition-spread wafer, with a hexagonal grid of 1,117 candidate measurement positions at 2 mm node spacing. Two tasks were defined: tapping-mode roughness and DART-mode roughness. For each acquired image, the scalar roughness descriptor was computed as the standard deviation of the height values after excluding pixels more than five standard deviations from the image mean.

Five seed positions were selected uniformly at random from the candidate grid, and both tapping and DART measurements were performed at each seed location, providing 10 initial location-task observations. These paired measurements establish an initial estimate of the task-covariance matrix B before decoupled active selection begins. Following initialization, only one task was measured at each selected location. The experiment ran for 25 active-learning steps following the 5 seed locations, for a total of 35 individual scans.

The learned task-correlation matrix yielded ρ = B12/√(B11B22) = 0.752, indicating substantial positive correlation between the two modes. The paper notes: "This is a property of the fitted model rather than of the raw data: the Pearson coefficient of the measured-versus-predicted cross-task scatter is r = 0.304, the difference reflecting the per-measurement noise and the small number of paired observations. The resulting posterior mean maps are strongly co-varying across the candidate grid (r = 0.984), though the paper cautions: because both maps derive from one joint model with a shared spatial kernel, this agreement is partly imposed by construction and should not be read as independent validation."

Over the 25 active steps, the controller selected DART 14 times and tapping 11 times, a difference within the range expected by chance, with 8 of the 25 assignments made by the forced-exploration rule rather than by the model. The learned per-task noise differs substantially (0.233 nm for tapping and 0.437 nm for DART). Averaged over the candidate grid, the posterior standard deviation after all 35 measurements is 0.176 nm for tapping and 0.235 nm for DART, against prior standard deviations of 0.925 and 1.098 nm respectively; the joint model therefore removes roughly 80% of the prior uncertainty in both landscapes while measuring only one modality at each location.

The paper discusses natural extensions of the framework to combinations of measurement protocols that differ substantially in acquisition cost and physical information content, including tapping-mode topography with contact current mapping, fixed-bias current mapping with local current-voltage spectroscopy, small-signal PFM imaging with local switching spectroscopy, and KPFM with conductive AFM. The authors note that The experimental value of multitask SPM emerges when an inexpensive, rapid, or weakly perturbative measurement can reduce the number of slower contact or spectroscopic measurements, and describe how explicit measurement cost can be incorporated into the acquisition function.

The paper concludes: "We have introduced and experimentally operationalized multitask scanning probe microscopy in a live closed loop, extending autonomous SPM from selection of spatial coordinates to autonomous selection and execution of combined location-measurement pairs. The framework uses a multitask Gaussian process with an intrinsic coregionalization model to learn the spatial response of individual AFM protocols together with their cross-task covariance. Measurements acquired using one protocol can therefore update predictions for the other protocols, allowing the instrument to distribute its measurement budget across both spatial and modality dimensions."

Improvements to AI systems based on this paper:

1. Joint spatial-modality active learning: The AI can now decide both where to measure and which sensor/protocol to use, rather than fixing the modality in advance. This enables adaptive resource allocation across heterogeneous data sources.

2. Cross-modal uncertainty propagation via multitask Gaussian processes: The AI learns a task-covariance matrix (e.g., via intrinsic coregionalization) that lets a measurement in one modality reduce predictive uncertainty in another, even at unmeasured locations. This is useful for any system with correlated but expensive-to-acquire signals.

3. Cost-aware acquisition with random scalarization: The AI can optimize a multi-objective acquisition function by sampling task weights from a Dirichlet distribution, allowing flexible trade-offs between exploration of different modalities without manual tuning.

4. Forced spatial exploration to prevent clustering: The AI periodically ignores the acquisition function and selects the most distant unmeasured location, preventing bias in hyperparameter estimation and ensuring global coverage—critical for non-stationary environments.

5. Annealed exploration parameter (β eff): The AI automatically reduces exploration over time, balancing early broad search with later exploitation of learned correlations.

6. Low-rank task representation (B = WWT + diag(κ)): The AI can scale to many modalities by sharing a low-dimensional latent structure, reducing the number of parameters needed to model inter-task correlations.

7. Matérn 5/2 kernel with ARD: The AI models spatial fields that can change sharply (non-smooth), with independent length scales per spatial axis, improving accuracy on heterogeneous surfaces.

What the improved AI system can do:

• Operate a physical instrument (e.g., AFM, robot, sensor network) in a live closed loop, autonomously selecting both measurement location and measurement type (e.g., fast vs. slow, invasive vs. non-invasive) to maximize information gain per unit cost.

• Learn cross-modal correlations online from sparse paired measurements, then use cheap or rapid measurements to infer expensive or slow-to-acquire quantities across an entire spatial domain.

• Reduce total measurement time/cost by up to 80% uncertainty reduction while measuring only one modality per location, as demonstrated (35 scans instead of 70 for two modalities).

• Adapt to heterogeneous noise across modalities (e.g., tapping vs. DART) and allocate more measurements to noisier tasks when beneficial.

• Generalize to multi-protocol experiments (e.g., topography + current, PFM + spectroscopy) where one modality is fast and another is slow, automatically deciding when the fast modality suffices and when the slow one is necessary.

• Provide calibrated uncertainty maps for each modality, enabling risk-aware decision-making in materials discovery, quality control, or environmental monitoring.

• Handle non-stationary spatial fields with sharp boundaries (e.g., phase transitions, defects) better than smooth-kernel methods, thanks to the Matérn 5/2 kernel.

Scanning probe microscopy provides nanoscale access to structural, electrical, electromechanical, magnetic, and mechanical properties of materials. Its increasing use for wafer-scale characterization and combinatorial materials exploration creates a need to distribute measurements efficiently across large spatial domains. This is particularly important when available modalities differ in acquisition time and potential for tip and sample damage, making exhaustive multimodal mapping over spatial grids impractical. Here, we demonstrate multitask scanning probe microscopy, a live, closed-loop workflow in which a multitask Gaussian process learns spatial and cross-modal relationships and autonomously selects both the next measurement location and the next experimental protocol. The approach is implemented on an automated large-sample atomic force microscope and demonstrated on a composition-spread AlScN wafer using tapping-mode and Dual AC Resonance Tracking (DART) measurements. Paired initial measurements establish the relation between the tasks, after which noncoincident measurements are used to update both response landscapes. The resulting workflow extends active learning in scanning probe microscopy from spatial sampling to autonomous allocation of measurement modalities and provides a basis for combining rapid, weakly perturbative imaging with slower contact, electrical, electromechanical, magnetic, or spectroscopic measurements.