AgriChrono: A Longitudinal Dataset and Benchmark for Novel-View Synthesis of Field-Grown Crops

arXiv:2508.18694 · cs.RO, cs.AI, cs.SY, eess.SY · Submitted 2025-08-26 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "AgriChrono: A Longitudinal Dataset and Benchmark for Novel-View Synthesis of Field-Grown Crops".

Paper summary: Rosa: So to recap, this paper presents AgriChrono as a modular robotic data collection platform and a multi-modal dataset specifically designed to capture dynamic farmland conditions for three dee reconstruction of crop structures <ref:2508.18694#pg1,a modular robotic data collection platform and a multi-modal dataset>.

Dev: The thesis is that while controlled indoor environments are good for high quality object level reconstruction, scaling those methods to complex in the wild agricultural settings is really hard due to the lack of dedicated datasets and benchmarks.

Taro: AgriChrono addresses this gap by introducing a comprehensive in-the-wild multimodal dataset and benchmark suite enabled by a dedicated robotic acquisition framework designed to capture environmental variations of real agricultural scenes.

Rosa: They developed this platform for remote, hands off operation via a customized web user interface, which served as the foundation for long term time synchronized collection of high quality multimodal data including RGB, Depth, LiDAR, IMU, and Pose.

Dev: This infrastructure allows for the precise time synchronized acquisition of four view RGB two view Depth LiDAR IMU and VIO Pose all managed via a custom web user interface to maintain rigorous data protocols even from the field.

Taro: The main contribution is releasing the eighteen terabyte AgriChrono dataset which captures comprehensively in the wild environments, documenting continuous morphological changes in a complex natural setting.

Rosa: They captured this data four times per day under different illuminations during the crop’s main growth period, which provides documentation of how things change over time in a natural setting.

Dev: Furthermore, they introduce a new benchmark for three dee reconstruction featuring seven scenarios that reflect complex in-the-wild agricultural environments to illustrate the significant challenges involved <ref:2508.18694#pg1,complex in-the-wild agricultural>.

Taro: This benchmark specifically highlights the crucial role that the AgriChrono dataset plays in advancing research because it shows where current models struggle when faced with these real world conditions.

Conclusion: Rosa: So looking at the AgriChrono paper, with its focus on longitudinal data and novel view synthesis, the implication is that we finally have a way to move beyond just indoor lab testing for three dee crop reconstruction <ref:2508.18694#pg1>.

Dev: It moves us toward having robust AI models that can handle dynamic environments in agriculture because this dataset and benchmark suite force researchers to test their methods under real world complexities like lighting shifts and growth stages.

Taro: Essentially, the paper shows that you need these types of large scale, time synchronized datasets and tough benchmarks to make sure any new reconstruction method is actually reliable when applied outside a controlled setting.

Rosa: The authors developed this modular platform to tackle the challenge of building three dee digital twins for expansive agricultural environments by capturing continuous growth in the field <ref:2508.18694#pg1>.

Dev: The work suggests that for agriculture automation to become widespread, we need these kinds of comprehensive data sources and standardized ways to evaluate performance across diverse real world conditions.

Jaehwan Jeong, Tuan-Anh Vu, Mohammad Jony, Shahab Ahmad, Md. Mukhlesur Rahman, Sangpil Kim, †M. Khalid Jawed

Korea University · University of California, Los Angeles · North Dakota State University

cs.RO, cs.AI, cs.SY, eess.SY

Submitted: 2025-08-26

Updated: 2026-11-14

Comments: Keywords: Agricultural robotics, Multimodal dataset, Longitudinal crop monitoring, Illumination variability, 3D Gaussian Splatting, Novel view synthesis, 3D Reconstruction

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 82/100

The gist: Accurate 3D reconstruction of crop structures is essential for geometryaware agricultural automation, yet remains challenging due to occlusion, thin plant topology, and dynamic motion.

Key concepts

AgriChrono
A modular robotic data-collection platform integrating multiple sensors like RGB, Depth, LiDAR, IMU, and Pose data. It is designed for remote, time-synchronized acquisition of dynamic farmland conditions to build a large dataset.
Multi-modal Dataset
A massive collection of diverse sensor data captured over time (18 TB) documenting the entire growth cycle of canola under different illumination. This dataset captures the complex, 'in-the-wild' environment necessary for robust 3D crop reconstruction.
Temporal Alignment Mechanism
A central software trigger that writes to a shared sync_time.txt file to ensure precise temporal alignment between independent sensors (RGB, Depth, LiDAR). This guarantees that all data streams are accurately synchronized to a single, defined start time.
Benchmark Scenarios
Seven distinct testing conditions used in the new benchmark designed to isolate specific environmental stressors like lighting variability and morphological changes in the crop. This tests how well 3D reconstruction methods perform under diverse real-world complexities.

Terminology

Summary

Accurate 3D reconstruction of crop structures is essential for geometryaware agricultural automation, yet remains challenging due to occlusion, thin plant topology, and dynamic motion. p. 0

Where the paper says it

Khalid Jawed2,† 1Korea University 2University of California, Los Angeles 3North Dakota State University Abstract. Accurate 3D reconstruction of crop structures is essential for geometryaware agricultural automation, yet remains challenging due to occlusion, thin plant topology, and dynamic motion.

This paper introduces AgriChrono, a modular robotic data-collection platform and a multi-modal dataset designed to capture dynamic farmland conditions. p. 0
Where the paper says it

This deficiency fundamentally limits the robustness of AI models, particularly for scene-level dynamic 3D reconstruction in dynamic environments. In this paper, we present AgriChrono, a modular robotic data-collection platform and a multi-modal dataset designed to capture dynamic farmland conditions.

Dataset Construction and Scale

The AgriChrono platform integrates multiple sensors, enabling remote, time-synchronized acquisition of RGB, Depth, LiDAR, IMU, and Pose data for efficient and repeatable long-term data collection in real-world agricultural environments. The researchers successfully collected 18 TB over one month documenting the entire growth cycle of Canola under diverse illumination conditions. p. 0

Where the paper says it

Our platform integrates multiple sensors, enabling remote, time-synchronized acquisition of RGB, Depth, LiDAR, IMU, and Pose data for efficient and repeatable long-term data collection in real-world agricultural environments. We successfully collected 18 TB over one month, documenting the entire growth cycle of Canola under diverse illumination conditions.

This dataset captures comprehensively the inthe-wild environment, documented four times per day during the crop’s main growth period. p. 2
Where the paper says it

– We release the 18 TB AgriChrono dataset, which captures comprehensively the inthe-wild environment. The data were captured four times per day (under different illuminations) during the crop’s main growth period, documenting continuous morphological changes in a complex natural setting.

System Architecture and Data Synchronization

The platform is built on the AgileX Scout 2.0 Unmanned Ground Vehicle (UGV), integrating a tiered sensor suite, an onboard compute unit, and a wireless communication module. p. 5

Where the paper says it

3, the system is built on the AgileX Scout 2. 0 Unmanned Ground Vehicle (UGV), integrating a tiered sensor suite, an onboard compute unit, and a wireless communication module.

The system ensures precise temporal alignment between independent sensors through a central software-trigger mechanism that writes to a shared sync time. p. 6
Where the paper says it

Sensor synchronization is ensured through a shared sync_time. txt file generated at the start of each session, guaranteeing consistent temporal alignment across all modalities.

txt file. This guarantees consistent temporal alignment across all modalities, ensuring all data streams are precisely aligned to a single, software-defined start time. p. 6
Where the paper says it

Sensor synchronization is ensured through a shared sync_time. txt file generated at the start of each session, guaranteeing consistent temporal alignment across all modalities.

Benchmark Design and Evaluation Metrics

A new benchmark for 3D reconstruction is introduced featuring seven scenarios that isolate specific environmental stressors, such as lighting variability and morphological changes. The evaluation of state-of-the-art methods reveals significant performance variance across these scenarios, highlighting the lack of uniform robustness in existing models when faced with diverse real-world complexities. p. 1

Where the paper says it

Building on this high-fidelity data, we introduce a new 3D reconstruction benchmark for non-rigid, in-the-wild scenes, featuring seven scenarios that isolate specific environmental stressors, such as lighting variability and morphological changes. Our evaluation of state-of-the-art methods from the last three years reveals significant performance variance across these scenarios, highlighting the lack of uniform robustness in existing models when faced with diverse real-world complexities.

The benchmark evaluates two core challenges: Lighting Variance and Growth Span. p. 9
Where the paper says it

As shown in Table 4, the benchmark evaluates two core challenges across seven scenarios. The first, Lighting Variance, consists of four scenarios using Day 19 captures at four distinct times (6:00 AM, 11:00 AM, 4:00 PM, 9:00 PM) to assess robustness to lighting changes.

Performance Analysis of Reconstruction Methods

The benchmark compares 16 state-of-the-art 3D reconstruction methods, including 12 Gaussian Splatting-based and 4 NeRF-based techniques across seven distinct conditions. Experimental results demonstrate that all baselines not only exhibit poor performance across scenarios but also fail to maintain consistent performance levels across them. p. 1

Where the paper says it

Building on this high-fidelity data, we introduce a new 3D reconstruction benchmark for non-rigid, in-the-wild scenes, featuring seven scenarios that isolate specific environmental stressors, such as lighting variability and morphological changes. Our evaluation of state-of-the-art methods from the last three years reveals significant performance variance across these scenarios, highlighting the lack of uniform robustness in existing models when faced with diverse real-world complexities.

Zip-NeRF achieves the highest overall scores, demonstrating that top-tier NeRF models remain a superior benchmark for rendering quality. p. 4
Where the paper says it

3DGS-based methods. The original Gaussian Splatting method [12] represents scenes as anisotropic 3D Gaussians rendered via screen-space splatting, achieving NeRF-level image quality with real-time rendering and fast training, and serves as our main explicit baseline.

The qualitative results show that all models exhibit notable deviations from the ground truth in delicate canopy reconstruction, regardless of illumination time or growth stage. p. 0
Where the paper says it

Scaling these capabilities to complex real-world agricultural settings necessitates robust reconstruction performance as a foundational prerequisite. For instance, beyond thin-canopy challenges, models must account for dynamic environmental factors, such as illumination shifts, wind-induced motion, and non-rigid morphological changes, throughout the growth cycle to transition from the laboratory to unpredictable real-world farmland.

Limitations and Future Directions

The data collection faced challenges, with Site 2 exhibiting lower FPS values due to thermal issues during prolonged operation. Immediate future work will involve refining the platform structure to mitigate thermal issues and developing software solutions for dynamic time synchronization using RTK-GNSS. The research efforts are directed toward scaling 3D Digital Twins from controlled indoor settings to expansive, in-the-wild environments. p. 0

Where the paper says it

Accurate 3D reconstruction of crop structures is essential for geometryaware agricultural automation, yet remains challenging due to occlusion, thin plant topology, and dynamic motion. While controlled indoor environments facilitate high-quality object-level 3D reconstruction, scaling these methods to complex "in-the-wild" agricultural settings remains a formidable challenge due to scarcity of dedicated datasets and benchmarks.

Acknowledgements

This work was supported in part by the US Department of Agriculture (grant numbers 2024-67021-42528 and 2022-67022-37021), the Culture, Sports, and Tourism R&D Program through the Korea Creative Content Agency grant funded by the Ministry of Culture, Sports and Tourism in 2024 (International Collaborative Research and Global Talent Development for the Development of Copyright Management and Protection Technologies for Generative AI, RS-2024-00345025), and the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2019-II190079, Artificial Intelligence Graduate School Program, Korea University). p. 0

Where the paper says it

Mukhlesur Rahman3 , Sangpil Kim1,† , and M. Khalid Jawed2,† 1Korea University 2University of California, Los Angeles 3North Dakota State University Abstract.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Develop robust 3D reconstruction models capable of handling dynamic perturbations by training them on AgriChrono, which includes continuous structural changes and color shifts caused by the changing position of the sun. This allows AI models to generalize beyond controlled indoor settings to complex 'in-the-wild' agricultural settings.

  2. Enhance model performance under illumination variability by using the AgriChrono benchmark, which features scenarios testing four distinct lighting conditions. This enables AI systems to maintain high fidelity even when facing intense, directional illumination versus softer lighting.

  3. Improve non-rigid scene understanding by training methods on the AgriChrono dataset, which documents continuous morphological changes in a complex natural setting. This directly addresses the challenge of reconstructing non-rigid, in-the-wild scenes.

  4. Increase pose estimation reliability by leveraging the multi-modal nature of AgriChrono data to derive Visual-Inertial Odometry (VIO) poses with consistent, low-variance inter-frame displacements, leading to highly robust robot trajectories.

  5. Create a modular robotic data collection system that enables remote, hands-off operation via a customized WebUI. This platform allows for the precise, time-synchronized acquisition of 4-view RGB, 2-view Depth, LiDAR, IMU, and VIO Pose over long distances.

Abstract

Reliable 3D reconstruction of field-grown crops is hindered by thin, overlapping structures, wind-driven foliage motion, and changing illumination. Yet longitudinal field datasets and standardized novel-view synthesis (NVS) benchmarks remain scarce. We present an integrated framework for robotic acquisition, dataset construction, NVS evaluation, and agricultural scene reconstruction. AgriChrobot enables web-based teleoperation and repeated multi-view capture along crop rows. Using this platform, we introduce AgriChrono, a longitudinal canola dataset collected over approximately one month, comprising 160 recording sessions, four RGB streams, two depth maps, and estimated camera poses. We define seven NVS scenarios spanning four times of day and three acquisition dates, and benchmark 15 NeRF- and Gaussian Splatting-based methods. In addition to whole-image metrics, we assess fine-detail reconstruction within manually annotated canola regions using gradient- and edge-based metrics. Our results show that strong whole-image scores do not guarantee faithful reconstruction of thin, overlapping crop structures. To address this gap, we propose AgriSplat, which relaxes visibility filtering during initialization and reallocates model capacity through contribution-based pruning and budgeted reinvestment in sub-threshold candidates ranked by existing gradient statistics. After reallocation, asymmetric directional supervision penalizes insufficient contrast along reference image-gradient directions. Relative to Octree-GS, AgriSplat improves mean crop-region gradient-energy ratio from 0.506 to 0.847 and edge F-score from 0.712 to 0.797 across seven scenarios, while also improving whole-image PSNR, SSIM, and LPIPS. These contributions provide a dataset, benchmark, and method for systematic crop reconstruction under field conditions.

Sources

Related papers