Trees are an important component throughout the world. They form and function in natural ecosystems such as forests, and also in human-made environments for instance parks and gardens [1
]. Urban scenes without trees or plants are lifeless. Furthermore, satisfying environmental goals always require heavy reliance on vegetation mapping and monitoring [2
]. Models of trees, therefore, have a wide range of applications, including urban landscape design, ecological simulation, forestry management, and virtual entertainment. While applications such as landscape design and visualization only require modelling virtual trees, lots of other applications relevant for ecological modelling and forestry management require accurate estimation of tree parameters (e.g., height, stem thickness). Accurate tree modelling not only enhances the realism of a scene, but also provides promising approaches to scientifically manage vegetation and forests, which will in return contribute to ecosystem protection, resource preservation, preventing degradation, and many other human activities [3
]. Hence, obtaining accurate 3D tree models is necessary and of great importance to the modern society.
The traditional way of measuring trees is to manually conduct fieldwork, which is usually expensive and time-consuming [4
]. Since the last several decades, remote-sensing technology has been widely exploited in extracting different information on forests and plants [5
]. Both satellite sensors and airborne sensors can effectively acquire digital images with high spatial resolution, which provide viable data sources for forestry analysis at individual tree level [3
]. Moreover, with the development of digital image processing technologies, researchers considered reconstructing digital tree models from photographs [6
]. The work of [7
] utilized visual hulls of the original tree shape to approximate the main skeleton of the tree, based on which small twigs and leaves are synthesized to generate a plausible tree model. Reche-Martinez et al. [6
] described a volumetric approach to reconstruct trees from multiple views. By combining plant images with sparse point clouds obtained from Structure From Motion (SFM), Quan et al. [8
] reconstructed realistic plants with generic leaves incorporating user interaction. While the above studies can produce impressive modelling results, they do not aim to reconstruct explicit branch or leaf geometry. Reconstructing trees from photographs remains a challenging problem due to the complexity of the modelling process [9
Recently, Light Detection and Ranging (LiDAR) technology have been widely used in forestry-related analysis and studies. As measurements from LiDAR can achieve millimeter-level of details from objects, it has become possible to directly capture 3D information and rapidly estimate tree attributes [10
]. For example, LiDAR measurements are widely applied in researches such as tree height estimation [11
], tree canopy analysis [12
] and tree species classification [13
]. Moreover, by applying LiDAR technology we are capable of acquiring highly dense point clouds, which lays the foundation for accurate tree reconstruction and modelling.
To achieve accurate tree modelling from laser scans, both the branch geometry and the tree topological structure are required. Among most literature studies, the common approach to obtain the tree branch geometry is cylinder-fitting [14
]. When it comes to the reconstruction of tree topological structure, existing methods can be classified into two main categories: segmentation-based and skeleton-based. Segmentation-based approaches first segment the tree point cloud into small subsets and then connect them procedurally to reconstruct the topological structure of the tree. For example, Hackenberg et al. [15
] developed a hierarchical-cylinder structure that enables parent-child neighbor relations among branches, which can efficiently extract different tree components such as the tree stem or a single branch. Raumonen et al. [16
] proposed another tree modelling method based on a step-by-step collection of small connected surface patches. Bucksch et al. [17
] organized the input points into an octree structure and generated skeletal curves from the octree cells. Yan et al. [18
] applied a K-means clustering-based approach to extract the tree topological structure. However, these works highly rely on the quality of the input data and therefore may not be robust enough to data with quality issues such as outliers or missing data due to occlusions.
Unlike segmentation-based approaches that reconstruct the tree topological structure from small segments and subsets, skeleton-based methods directly extract skeleton curves from raw input point clouds. Some works employ a rule-based procedural modelling approach to synthesize branches [9
], which generate the tree skeleton with high quality but require prior knowledge as well as manual parameter adjustment. Some other works proposed purely data-driven methods to automatically extract the skeleton without requiring additional user interactions. The work of [20
] constructed the shortest-path map over the input point clouds to extract consecutive skeletal curves. Following this work, Delagrange et al. [21
] developed a tool PypeTree to reconstruct tree branch tissues from point clouds. As an alternative for the shortest-path approach, Dey and Sun [22
] utilized the medial axis to represent the skeletal structure of 3D tree-like objects. Livny et al. [23
] computed a minimum spanning graph over the point cloud to obtain an initial tree skeleton and applied several global optimization techniques to refine the tree branch structure. Following this work, we further improve the fidelity of the reconstructed tree models.
In this paper, we propose a skeleton-based approach to accurately reconstruct tree branches from individual tree point clouds. Our method employs a Minimum Spanning Tree (MST) algorithm to effectively extract the initial tree skeleton over input points. By iterative skeleton simplification and cylinder fitting, we obtain a tree model with reconstructed branches. Leaves and textures are added to enhance the realism of the tree model. One novelty of our work is that we construct the initial tree skeleton based on the intrinsic spatial distribution of input points. Furthermore, we develop a specific simplification strategy to maintain the natural topological structure of tree branches while collapsing redundant vertices and edges. Our experiments and various comparisons demonstrate both the geometrical correctness and the topological fidelity of the generated tree models.
4. Results and Discussion
In this section, we provide the result analysis, aiming to test if our modelling results have fulfilled the functional and user requirements proposed in Section 3
. First, a set of visual results are presented to evaluate the topological fidelity of the reconstructed tree models. Then, we compute the distance between input points and the output tree branch model to verify the geometrical accuracy of the modelling results. In Section 4.3
, we illustrate the robustness and applicability of our algorithm over various tree types and data sources, which enables our tree modelling process to be fully automatic. Some discussions considering comparisons with the state-of-the-art methods, limitations, and future applications are made at the end of this section.
4.1. Visual Evaluation
We reconstructed a variety of trees of different species, sizes, and branch structure. Figure 11
a shows a vertical and slim tree with relatively simple branch structure, while Figure 11
b gives an example of another tree with a tilted stem and complex branch structure. The reconstruction results of these two trees demonstrate that our method is capable of processing trees with different shapes and structures, which benefits from the skeleton-based approach that we adopt.
Besides, we also tested our method on scanned trees from various data sources, including mobile scanning, static scanning as well as airborne scanning. It is observed that point clouds collected by mobile scanning (Figure 11
c) or static scanning (Figure 11
d) have a high quality and thus were all accurately reconstructed. On the other hand, Figure 11
e gives an example of an input point cloud obtained by airborne scanning, which is poorly sampled and is quite sparse. Even for such a low-quality input, our approach is still able to produce a visually plausible 3D reconstruction.
4.2. Reconstruction Accuracy
We quantified the geometrical accuracy of the modelling results by computing the mean distance between the input points and the generated tree branch model [9
]. The reconstruction accuracy and standard deviation in Table 1
suggested that overall our approach can generate tree models that fit closely to the input point cloud data and thus ensures high geometrical accuracy. When it comes to the individual tree level, typically a short tree with highly dense points will have a more accurate modelling result. Also, compared to trees with irregular shapes (i.e., Figure 11
b), trees with a compact and standard shape usually enable higher reconstruction accuracy.
visualizes the per-point error distribution of the reconstructed 3D model shown in Figure 11
a. In the visualization, the blue colour indicates a low error value and the red colour indicates a high error value. From such a visualization, we can conclude that points lying within the main branches typically fit closer to the model, while points near the branch tips usually have high error values. This indicates that our method can generate highly precise main branch structure of the input tree (thanks to the non-least squares based branch fitting). However, points are getting sparser near the branch tips and thus not sufficient to reliably reconstruct these small features from the under-sampled data.
As described in Section 3
, the simplification threshold
is introduced during the tree skeleton simplification process, where we utilize an indicator to measure the proximity between adjacent vertices. This section discusses how different parameter values influence the modelling results, based on which, we choose the threshold values that best fit our methodology.
The simplification threshold
controls the similarity indicator
, which determines the relative proximity between adjacent vertices. We tested the value of
from 0.5 to 3 and the results are shown in Figure 13
. According to our experiments, a very small threshold value for
for the indicator makes it difficult to merge close-by vertices, while a very big
causes oversimplification. Therefore, we chose 1.5 as the threshold value. It is denoted that the parameter value is pre-fixed in our algorithm, which means that we used the same parameter setting for generating all the 3D models in this paper. As
is a relative value indicating the closeness among vertices, it is generally applicable for most trees. Users do not have to adjust the specific threshold value for specific input data, which makes our approach robust to various trees.
We compared our method to a few state-of-the-art approaches [15
In Figure 14
, we show a visual comparison of our results against the method described in [23
] as it is most related to our work. It can be seen that our reconstructed models have more faithful branch structure and also fit better to the input points. Given the same point cloud, our algorithm is capable of reconstructing a tree model with higher topological and geometrical accuracy. The performance gain benefits from two improvements. Firstly, we identify and centralize main-branch points, which in return generates topologically correct tree skeletons. Secondly, our cylinder fitting exploits a distance-weighted non-linear least squares fitting, which significantly improves the geometrical accuracy.
We also compare our modelling results to other tree modelling approaches publically accessible to us, such as PypeTree [21
], TreeQSM [16
], and SimpleTree [15
]. A visual comparison is demonstrated in Figure 15
. Among these approaches, PypeTree aims to only give a rough description of the tree topological structure, which is not intended to recover the branch geometry. In contrast, TreeQSM, SimpleTree, and our method can recover the geometry of tree branches. However, TreeQSM cannot ensure consistent recovery of the tree branches. The reconstruction using SimpleTree is quite tedious as it requires user input of the key parameters such as the radii of the branches. In our comparison, we also observed that SimpleTree requires nearly perfect input, i.e., complete point clouds, which is rarely satisfied in practice. In the test data, the tree was scanned from a single viewpoint by a terrestrial laser scanner and thus the surfaces of the branches were partially sampled. SimpleTree failed to detect the branch cylinders and to recover the branch geometry. In contrast, our method successfully and faithfully recovered both the topology and the geometry of the tree branches.
Our algorithm can successfully obtain accurate and detailed 3D tree models from point clouds. However, it still has some limitations. First of all, our approach depends on the quality of the input data. For poorly scanned data with sparse points, our method can reconstruct a plausible topological structure of the tree branches but is unable to achieve sufficient geometrical accuracy. Moreover, our work does not consider natural growing rules of tree branches (e.g., branch split angle, branch growing length). The incorporation of domain knowledge will further constrain the reconstructed models to be topologically correct and improve the fidelity of the models, improving both geometrical and topological accuracy.
4.6. Potential Applications
Our proposed approach enables accurate reconstruction of 3D trees from point clouds. The generated tree models can be employed in many applications. The accurate tree models with synthesized leaves and textures can be directly applied in applications such as urban landscape design and entertainment, to convey the realism of the scenes. Our method also opens up the opportunity for automatically obtaining precise tree attributes (e.g., the height of a tree, the thickness of the trunks, and the diameter at a specified height). It enables to save a significant amount of time and laboring efforts compared to the traditional manual measuring method. With accurate 3D tree models, implicit tree attributes, such as wood volume, biomass, the amount of carbon dioxide to be absorbed and oxygen to be emitted, can also be automatically derived or estimated.
5. Conclusions and Future Work
In this paper, we proposed an automatic approach to accurately reconstruct 3D tree branches from point clouds. During the reconstruction, both the geometrical accuracy and topological fidelity of the tree are taken into consideration. One novelty of our work is that we aid the skeleton construction process by the main-branch point centralization, which contributes to improving the quality of the generated tree branch structure. Moreover, an optimization-based approach is employed to accurately reconstruct the geometry of the tree branches. Experimental results revealed that our method is robust in dealing with various types and sizes of trees. As long as the input point clouds demonstrate clear branch structure, our method is capable of generating tree models of high quality.
In future work, we would like to perform automatic segmentation of trees. As our method only works for individual tree point clouds, involving existing automatic segmentation approaches will expand our algorithm to a broader range of applications. Besides, as there are many irregular shapes of tree branches in nature, we will further consider fitting free-form surfaces instead of cylinders to model the branch geometry more precisely.