GrowSplat: Constructing Temporal Digital Twins of Plants with Gaussian Splats

1UC Berkeley

2Siemens Research Lab, Berkeley

3Netherlands Plant Eco-phenotyping Centre, Wageningen University and Research

CASE 2025

paper
Paper
code
</Code >
Coming Soon
data
Data
Coming Soon

Abstract

Accurate temporal reconstructions of plant growth are essential for plant phenotyping and breeding, yet remain challenging due to complex geometries, occlusions, and non-rigid deformations of plants. We present a novel framework for building temporal digital twins of plants by combining 3D Gaussian Splatting with a robust sample alignment pipeline. Our method begins by reconstructing Gaussian Splats from multi-view camera data, then leverages a two-stage registration approach: coarse alignment through feature-based matching and Fast Global Registration, followed by fine alignment with Iterative Closest Point. This pipeline yields a consistent 4D model of plant development in discrete time steps. We evaluate the approach on data from the Netherlands Plant Eco-phenotyping Center, demonstrating detailed temporal reconstructions of Sequoia and Quinoa species.

3D Gaussian Splats for Plants

GrowSplat takes in 1) single images from 15 static cameras and 2) corresponding masks and 3) a pointcloud . By using a custom version of 3D Gaussian Splatting using Splatfacto with the Markov Chain Monte Carlo strategy, we generate 3D reconstructions of plants.:

Input Marvin Images

Click and move the Quinoa Plants across dates! You can zoom in/out, move up/down, and move left/right. For each date, you can see above the 15 camera images from MaxiMarvin.

Click and move me!

Input Marvin Images

Click and move the Sequoia Plants across dates! You can zoom in/out, move up/down, and move left/right. For each date, you can see above the 15 camera images from MaxiMarvin.

Click and move me!

These 3D reconstructions are rendered in-browser! If you think that's cool, check out Viser!

Plant Registration


    We propose the algorithm framework as follows:
  1. Point Cloud Initialization and Preprocessing: Gaussian splats provide a dense point cloud with an associated uncertainty model; however, using the raw data directly leads to excessive computational overhead and introduces outliers stemming from the uncertainty. To address these challenges, we propose a point cloud downsampling pipeline for Gaussian splats that applies three principal filtering con- ditions with Gaussian splats properties to ensure physically valid and well-conditioned data. First, we discard splats with excessively large or small scales by inspecting their log-scale range, thus preventing skewed modeling. Second, we compute the scale ratio to remove overly elongated splats that do not faithfully represent plant geometry. Lastly, we validate the rotation parameters by checking the norm of the associated quaternion to reduce numerical errors. After this filtering process, we estimate surface normals to support the computation of Fast Point Feature Histograms (FPFH)[31]. These FPFH descriptors capture local geometry and serve as key features in subsequent alignment steps.

  2. Global Registration (Coarse Alignment): For an initial, coarse alignment of point clouds across different time steps, we adopt a feature-based matching approach inspired by Fast Global Registration (FGR). Specifically, we first compute FPFH for each point in both the reference point cloud Pref and the temporal point cloud Ptk . The FPFH descriptors encode local geometric properties (e.g., curvature, normal variation) that are invariant to rigid transformations, facilitating the discovery of putative correspondences be- tween the two point clouds.We then utilize a RANSAC- based scheme for robust outlier rejection. In each iteration of RANSAC, a minimal set of correspondences is randomly sampled, and an initial rigid transformation is estimated. The transform is evaluated against the entire set of FPFH correspondences to identify and reject outliers. We set the convergence condition to be (3) or the maximum number of iterations reach the threshold. The resulting transformation Ttk approximately aligns Ptk to Pref . Once we have a candidate set of correspondences that survive RANSAC, we refine them using the optimization procedure proposed in FGR, which employs a robust objective to handle remaining outliers more gracefully than pure least-squares approaches. By combining FPFH-based matching, RANSAC filtering, and FGR optimization, the final output of this global reg- istration step is a coarse but sufficiently accurate alignment of Ptk to Pref . This ensures a robust starting configuration for subsequent local (fine) registration.

  3. Fine Registration (Local Alignment): Following coarse alignment, we refine each temporally acquired point cloud using standard Iterative Closest Point (ICP), which iteratively refines the rigid transformation by minimizing the Euclidean distance between matched points. To further improve alignment, we employ Colored ICP, which incorporates photomet- ric consistency by minimizing color discrepancies between corresponding points. This color term helps disambiguate similar geometric features, particularly in organic structures with subtle variations.

View Rendering

We use the registered plant frame and the reconstructed Gaussian splats to render rotating viewing angles. The final output is a temporally consistent sequence of 3D point clouds and video that best captures the growth dynamics of the plant

Experiments

We evaluated GrowSplat with plant species Sequoia and Quinoa that were captured by NPEC Maxi-Marvin system with longest time series with 55 time points over 76 days

  1. Dataset:

  2. Results:
    Top row: registration result for Sequoia. Bottom row: registration result for Quinoa. We use the blue to indicate the point cloud in the earlier date and the green indicate the point cloud in the later date.

Citation

If you use this work or find it helpful, please consider citing: (bibtex)

@inproceedings{adebola2025growsplat,
         author={Adebola, Simeon and Xie, Shuangyu and Kim, Chung Min and Kerr, Justin and van Marrewijk, Bart M. and van Vlaardingen, Mieke 
                        and van Daalen, Tim and van Loo, E.N. and Susa Rincon, Jose Luis and Solowjow, Eugen and van Zedde, Rick and Goldberg, Ken},
         booktitle={2025 IEEE 21st International Conference on Automation Science and Engineering (CASE)}, 
         title={GrowSplat: Constructing Temporal Digital Twins of Plants with Gaussian Splats}, 
         year={2025},
         volume={},
         number={},
         pages={1766-1773},
         doi={10.1109/CASE58245.2025.11163998}}