High-speed volumetric amplitude-spectrum dynamic optical coherence tomography by neural network with multi-burst scanning

Dynamic optical coherence tomography (DOCT) enables label-free, three-dimensional (3D) assessment of tissue dynamics. However, it suffers from long acquisition times because conventional time-spectrum DOCT requires hundreds of repeated OCT frames per location. Here we present a neural network (NN) framework integrated with a non-uniform-time scanning protocol (multi-burst scan) to accelerate amplitude-spectrum DOCT (AS-DOCT). Combining 3D convolutional and long-short term memory (LSTM) layers with dual inputs (the temporal OCT sequence and its pseudo-amplitude spectrum), the model generates AS-DOCT images from only 16 frames per location. Validated on 29 cancer spheroids, the proposed method resolved distinct functional domain structures with high fidelity (structural similarity index metric (SSIM) > 0.8) and enabled full volumetric AS-DOCT acquisition in 26.2 seconds. This method will enable high-throughput 3D dynamic tissue screening.

Dynamic optical coherence tomography (DOCT) is a contrast-extension method of optical coherence tomography (OCT) that three-dimensionally assesses cellular activity and differentiates tissues based on their functions without using any exogenous agent. Under the assumption that cellular and intracellular activities cause scatterer motion and subsequent OCT signal fluctuations, DOCT extracts the temporal fluctuations of the OCT signal as a contrast source for assessing scatterer motion and imaging tissue dynamics. DOCT imaging consists of two steps: sequential acquisition of multiple repeated OCT frames at a single tissue location and extraction of dynamic information from the fluctuations using statistical analysis.

Several DOCT algorithms have been proposed, including time-variance-based or standard-deviation-based analysis, temporal auto-correlation analysis, and time-spectrum analysis. These DOCT algorithms have been successfully used to evaluate several properties of scatterer motion, including the occupancy of moving scatterers, the magnitude of scatterer motion, and the speed or frequency of scatterer motion.

Despite the success of DOCT, it requires acquiring a large number of OCT frames (tens to hundreds) at each tissue location, resulting in long acquisition times, especially for 3D imaging. For variance- or standard-deviation-based DOCT computations, Abd El-Sadek et al. and Morishita et al. used 32 OCT frames per location, whereas Apelian et al. used 100 frames per location. For temporal auto-correlation-based DOCT computations, Abd El-Sadek et al. used 32 frames per location, and Oldenburg et al. used 100 frames per location. For time-spectrum-based DOCT computations, Münter et al. used 100 to 150 frames per location, and Scholler et al. used 500 frames per location.

To reduce the number of OCT frames and decrease the acquisition time of DOCT, the authors previously trained a neural network to generate a time-variance-based DOCT image (logarithmic intensity variance, LIV) from only 4 OCT frames instead of the conventional 32 frames, reducing the volumetric acquisition time of LIV from 1 min to 6 s. However, LIV only represents the magnitude of the signal fluctuation and does not capture speed- or frequency-related information.

Amplitude-spectrum DOCT (AS-DOCT) is a type of time-spectrum DOCT sensitive to specific bands of the OCT signal's time spectrum. AS-DOCT typically requires a large number of frames per location (around 100 to 150 frames), and it makes an NN-based reduction in the required frame count highly beneficial.

Although the previous NN method successfully generated LIV images from a small number of OCT frames, it cannot be directly applied to frequency- or speed-sensitive DOCT generation because the previous NN primarily utilized two-dimensional spatial convolutional layers, which cannot properly handle the time-order information of OCT frames. In addition, the input OCT frames for LIV generation were acquired with uniform time intervals, preventing the effective transmission of a wide frequency range at a high signal-to-noise ratio (SNR). Heldt et al. used a neural gas method to enhance the image contrast of AS-DOCT computed from the time spectrum of a small number of OCT frames, but their approach suffers from high speckle noise.

To address these issues, the authors propose: (1) a new NN architecture using three-dimensional (3D) convolutional layers combined with a long short-term memory (LSTM) module to extract time-sequential information from the OCT sequence, where 3D refers to 2D spatial dimensions plus time; and (2) a new scanning protocol that captures a small number of OCT frames with non-uniform time separations, ensuring that the NN input effectively covers a wide range of frequency or speed information. The proposed method generates frequency-sensitive DOCT (AS-DOCT) from only 16 frames, enabling volumetric AS-DOCT measurement within 26 s.

AS-DOCT is a type of DOCT sensitive to the frequency of cellular motility. An AS-DOCT image consists of three channels corresponding to three frequency bands: low, middle, and high frequency. This method was originally proposed by the University of Lübeck and is typically computed from 100 to 150 OCT frames per location in conventional implementations. Specifically, a Fourier transform is performed on the temporal OCT amplitude signal to obtain the frequency spectrum. The absolute value of the frequency spectrum (the amplitude spectrum) is then split into low-, middle-, and high-frequency bands, which correspond to the low, middle, and high frequencies of tissue dynamics, respectively. Finally, the amplitude spectrum of each frequency band is integrated and logarithmized.

The proposed NN architecture is a modified U-Net consisting of three parts: an encoder, a decoder, and skip connection layers. The input consists of two parts: (1) a cross-sectional 16-frame OCT time sequence with non-uniform time spacing, and (2) the pseudo-amplitude spectrum, which is the amplitude of the fast Fourier transform (FFT) spectrum of the input 16 OCT frames. Here, pseudo is used because a standard FFT was directly applied to the non-uniformly spaced temporal sequence. This pseudo-amplitude spectrum input provides supplemental frequency information. Each input is five-dimensional, with the shape [batch size, time or pseudo-spectrum points, height, width, feature channels]. The NN outputs three images corresponding to the low-, middle-, and high-frequency channels of AS-DOCT.

The NN architecture uses two core modules to extract temporal information. One is the 3D convolutional layer, which convolves sequential images along both spatial and temporal axes. Temporal convolution enables the extraction of order-sensitive temporal features among neighboring frames. Specifically, the 3D convolutional layer consists of 3D convolution with a 2 × 2 × 2 kernel, rectified linear unit (ReLU) activation, and layer normalization. The other core module is the LSTM layer, which captures long-range temporal dependencies within the image sequence. It enables the extraction of temporal features between frames with long time intervals. The LSTM layer is implemented using the Conv2dLSTM function in Keras, with a kernel size of 2 × 2. The combination of 3D convolution and the LSTM layer (Conv3D-LSTM module) extracts sequential information across a wide range of frame separations.

In the encoder, feature representations extracted from the OCT image and pseudo-amplitude spectrum are concatenated along the feature channel dimension. Each pooling layer decreases the spatial image size to 1/4 (1/2 in height × 1/2 in width) using a combination of max pooling and average pooling. Throughout the encoder, pooling is performed four times, reducing the spatial dimensions to 1/16 of the original input in both height and width. The feature maps then pass from the encoder into the decoder, which increases the spatial resolution using transposed convolutional layers. Multi-scale features are preserved by concatenating encoder and decoder representations via skip connection layers. At the final layer of the decoder, the architecture branches into three output heads to separately reconstruct the low-, middle-, and high-frequency AS-DOCT images. Losses from the three frequency channels are computed separately and summed to form the final loss during model training.

To facilitate the extraction of various frequency components, the authors designed a non-uniform time-scanning protocol so that the 16-frame OCT sequence effectively conveys a wide frequency range information. This scanning protocol captures 16 OCT frames with both short and long inter-frame intervals. It first continuously captures 4 frames within a short duration of 4 Δt, referred to as a burst, where Δt represents the single-frame acquisition time (12.8 ms in the specific implementation). The burst provides high-frequency information with a high SNR. Four bursts are repeated while maintaining an idle interval (equivalent to 124 Δt) between adjacent bursts. This idle time intentionally extends the total acquisition duration of the frame sequence to effectively encode low-frequency information with a high SNR. This protocol is denoted as the multi-burst scanning protocol.

Furthermore, in volumetric imaging implementations, the idle interval is utilized for burst scanning of other spatial locations, enabling volumetric DOCT acquisition without dead time.

A total of 218 cancer spheroids were used for NN training and evaluation. They consisted of three types of cancer spheroids: 155 breast cancer (MCF-7 cell line) spheroids formed by seeding 1,000 cells, 58 colon cancer (HT-29 cell line) spheroids formed by seeding 1,000 cells, and 5 breast cancer (MCF-7 cell line) spheroids formed by seeding 5,000 cells.

The NN training process used only 1,000-cell spheroids, including 143 MCF-7 and 46 HT-29 spheroids. To increase sample variation for training, spheroids were treated with drugs under various conditions. MCF-7 spheroids were treated with paclitaxel (PTX) at 0.1-μM, 1-μM, and 10-μM concentrations; tamoxifen (TAM) at 0.1-μM, 1-μM, and 10-μM concentrations; doxorubicin (DOX) at 0.1-μM, 1-μM, and 10-μM concentrations; or left untreated. Treatment durations varied across 1, 3, and 6 days. HT-29 spheroids were subjected to treatments using SN-38 at concentrations of 0, 0.1, 1, and 10 μM for durations of 1, 3, or 6 days.

The remaining samples were reserved for NN evaluation. These consisted of 1,000-cell MCF-7 spheroids (n = 12), 1,000-cell HT-29 spheroids (n = 12), and 5,000-cell MCF-7 spheroids (n = 5). All 5,000-cell spheroids were untreated. The 12 MCF-7 evaluation spheroids were prepared under 12 distinct conditions, defined by four treatment groups times three treatment durations. Similarly, the 12 HT-29 evaluation spheroids were prepared under 12 distinct conditions combining untreated controls or SN-38 treatments (0.1 μM, 1 μM, 10 μM) across durations of 1, 3, and 6 days.

A Jones-matrix swept-source OCT (SS-OCT) system was utilized for acquiring data. The center wavelength of the probe beam is 1.3 μm, and the A-line rate is 50 kHz. The axial and lateral resolutions are 14 μm (in tissue) and 18.1 μm, respectively. Although this system possesses polarization sensitivity, only polarization-insensitive intensity images (obtained by intensity-averaging of four polarization channels) were used in this study.

To compute ground truth data for NN training, conventional AS-DOCT was implemented. For each spheroid, continuous 512 OCT frames were captured at a single location over a lateral field of view of 1 mm. Each frame comprised 512 A-lines, corresponding to a frame acquisition time of 12.8 ms and a Nyquist frequency of 40 Hz. Ground truth AS-DOCT was computed from the full 512-frame sequence using the conventional method. An FFT yielded an amplitude spectrum with 256 frequency bins. The spectrum was split into three frequency bands: low frequency [0–0.5 Hz], middle frequency [0.5–5 Hz], and high frequency [5–25 Hz]. The spectrum in each frequency band was integrated using the numpy.trapz function, logarithmized using numpy.log10, and designated as ground truth.

For the NN input, 16 frames were extracted from the same 512-frame sequence used for ground truth generation. To match the multi-burst scanning protocol, the extracted frames were the 1st–4th, 129th–132nd, 257th–260th, and 385th–388th frames of the 512-frame sequence. The short intra-burst frame interval was 12.8 ms, while the long inter-burst interval was 1.59 s (equivalent to 12.8 ms/frame × 124 frames).

The training set consisted of 118 MCF-7 spheroids and 34 HT-29 spheroids, while the validation set comprised 25 MCF-7 spheroids and 12 HT-29 spheroids. Cross-sectional 16-frame sequences for input and corresponding ground truth were paired. Image pairs were cropped into 64 × 64-pixel patches with an overlap of 32 pixels in both depth and fast-scan directions. Only patches containing the sample region were used for training, where sample regions were manually delineated using ImageJ. The training and validation datasets contained 5,362 and 1,350 patch pairs, respectively.

During training, the batch size was set to 1. Mean squared error (MSE) served as the loss function. The average MSE across all three channels was computed as the total loss. The Adam optimizer was used with learning rate decay: learning rate = (0.0005/epoch) + 0.0001. Training was stopped when validation loss failed to decrease for 5 consecutive epochs, and the checkpoint with the lowest validation loss was selected for inference. The NN was implemented using TensorFlow-GPU version 2.6 and trained on an NVIDIA RTX 4090 GPU with 24 GB of VRAM. Model training took 25.5 hours over 21 epochs.

The proposed NN method was evaluated via two studies. Study 1 evaluated NN performance by comparing contrast similarity between 16-frame NN-generated images and 512-frame conventional AS-DOCT images. Study 2 performed an ablation analysis to confirm the necessity of the multi-burst scanning protocol and the dual-input strategy.

Evaluation was conducted across four image types: low-, middle-, and high-frequency band images, and composite pseudo-color RGB AS-DOCT images. Composite pseudo-color RGB images were formed by combining the three single-channel outputs. Each channel was normalized to the range [0, 1], after which high-, middle-, and low-frequency images were assigned to the red, green, and blue channels, respectively.

Similarity between ground truth and NN-generated images was evaluated on three spheroid types: 1,000-cell MCF-7 spheroids (n = 12), 1,000-cell HT-29 spheroids (n = 12), and 5,000-cell MCF-7 spheroids (n = 5). Notably, because the NN model was trained exclusively on 1,000-cell spheroids, the five 5,000-cell spheroids evaluated the model's generalization capabilities.

Image contrast similarity was evaluated both subjectively and objectively. The structural similarity index (SSIM) within the sample region was computed to quantify contrast fidelity. SSIM values were calculated using the skimage.metrics.structural similarity function in scikit-image (version 0.25). The data range parameter was set to the maximum value of the ground truth image. The function evaluated local SSIM using a 7 × 7-pixel sliding window to produce a spatial SSIM map. This map was multiplied by a segmentation mask (manually segmented using ImageJ) to compute average SSIM strictly within the sample boundaries.

Quantitative metrics derived from DOCT images were also evaluated. Mean AS-DOCT values for each frequency band were measured across three regions of interest (ROIs): core, intermediate, and periphery ROIs. Untreated 5,000-cell MCF-7 spheroids (n = 5) were used for this evaluation.

In the ablation study, two additional NN models were trained. The first model accepted only 16 OCT frames and omitted the pseudo-amplitude spectrum input ("single-input burst NN m…

Improvements to AI systems:

1. Hybrid 3D-CNN + LSTM architecture for spatiotemporal sequence learning – Integrate 3D convolutional layers (capturing local spatial-temporal correlations) with LSTM layers (capturing long-range temporal dependencies) to process time-ordered image sequences more effectively than 2D CNNs alone. This enables the AI to learn both short- and long-term dynamics from limited frames.

2. Dual-input fusion of raw temporal sequence and pseudo-frequency spectrum – Feed both the raw OCT frame sequence and its fast Fourier transform (pseudo-amplitude spectrum) as parallel inputs, concatenating their feature maps. This provides complementary information: spatial-temporal details from the sequence and explicit frequency-band cues from the spectrum, improving reconstruction fidelity for frequency-sensitive outputs.

3. Non-uniform temporal sampling (multi-burst protocol) for frequency coverage – Replace uniform frame sampling with a burst-based non-uniform schedule (short bursts with idle gaps) to encode both high-frequency (short intervals) and low-frequency (long total duration) information at high SNR. This allows the AI to recover a wide frequency range from few frames, overcoming Nyquist limitations of uniform sampling.

4. Multi-head output for multi-band frequency reconstruction – Use a decoder with three parallel output heads, each dedicated to reconstructing a specific frequency band (low, middle, high). This decouples the learning task per frequency channel, improving per-band accuracy and enabling composite pseudo-color visualization.

5. Ablation-informed design – Validate the necessity of each component (dual inputs, non-uniform sampling) via systematic ablation studies. This ensures the AI system’s architecture is optimized for both accuracy and robustness, avoiding unnecessary complexity.

What the improved AI system can do:

• Generate high-fidelity amplitude-spectrum DOCT (AS-DOCT) images from only 16 OCT frames per location, achieving structural similarity index (SSIM) > 0.8–0.97 across frequency bands, compared to conventional 512-frame reconstructions.

• Resolve distinct functional tissue domains (e.g., necrotic core, quiescent intermediate zone, proliferating periphery) in 3D cancer spheroids, matching ground truth from 512-frame acquisitions.

• Perform full volumetric AS-DOCT imaging in 26.2 seconds using a standard 50-kHz OCT system, a 10–20× speedup over conventional methods, enabling high-throughput 3D dynamic tissue screening.

• Generalize to unseen sample sizes (e.g., 5,000-cell spheroids) despite training only on 1,000-cell spheroids, indicating robust feature learning.

• Provide quantitative AS-DOCT metrics (mean values per region) with small bias and narrow limits of agreement in most tissue regions, suitable for qualitative and semi-quantitative analysis.

• Enable label-free, non-invasive assessment of cellular motility and tissue dynamics at high speed, facilitating drug response screening, tumor biology studies, and potential clinical translation.

Dynamic optical coherence tomography (DOCT) enables label-free, three-dimensional (3D) assessment of tissue dynamics. However, it suffers from long acquisition times because conventional time-spectrum DOCT requires hundreds of repeated OCT frames per location. Here we present a neural network (NN) framework integrated with a non-uniform-time scanning protocol (multi-burst scan) to accelerate amplitude-spectrum DOCT (AS-DOCT). Combining 3D convolutional and long-short term memory (LSTM) layers with dual inputs (the temporal OCT sequence and its pseudo-amplitude spectrum), the model generates AS-DOCT images from only 16 frames per location. Validated on 29 cancer spheroids, the proposed method resolved distinct functional domain structures with high fidelity (structural similarity index metric (SSIM) > 0.8) and enabled full volumetric AS-DOCT acquisition in 26.2 seconds. This method will enable high-throughput 3D dynamic tissue screening.