Automated Segmentation of Individual Tree Structures Using Deep Learning over LiDAR Point Cloud Data

: Deep learning techniques have been widely applied to classify tree species and segment tree structures. However, most recent studies have focused on the canopy and trunk segmentation, neglecting the branch segmentation. In this study, we proposed a new approach involving the use of the PointNet++ model for segmenting the canopy, trunk, and branches of trees. We introduced a preprocessing method for training LiDAR point cloud data speciﬁc to trees and identiﬁed an optimal learning environment for the PointNet++ model. We created two learning environments with varying numbers of representative points (between 2048 and 8192) for the PointNet++ model. To validate the performance of our approach, we empirically evaluated the model using LiDAR point cloud data obtained from 435 tree samples scanned by terrestrial LiDAR. These tree samples comprised Korean red pine, Korean pine, and Japanese larch species. When segmenting the canopy, trunk, and branches using the PointNet++ model, we found that resampling 25,000–30,000 points was suitable. The best performance was achieved when the number of representative points was set to 4096.


Introduction
Reducing carbon emissions is important for mitigating the risk of climate change.One effective method to achieve this is through the appropriate management of forests, which act as carbon sinks.This process requires accurate data regarding forest resources, which can be obtained using remote-sensing technology and field surveys.Above-ground biomass (AGB) can be measured using destructive methods, such as cutting and weighing the plant material, whereas understory ground biomass is estimated using allometry equations.The allometry equation refers to the method of estimating biomass by applying the taper equation using the true biomass of a single tree, which is calculated using the fresh and dry weights of the trunk, branches, and leaves measured through field surveys.However, these methods require a considerable amount of labor and can be dangerous to workers.With the advancement in remote-sensing technology, estimating biomass in a nondestructive manner has become possible [1][2][3][4][5][6][7][8][9][10].This method is less labor-intensive and reduces the risk of injury to workers.
The structure of the trees needs to be segmented first.The trunk is used to estimate the wood volume, and the branch is utilized to calculate the amount of the fuel and biomass in forest fires [5,11,12].However, the degree of difficulty in dividing tree structures can vary depending on the tree species and stand structure.The biomass estimation from commonly used satellite imagery is affected by low spatial and spectral resolutions, making data acquisition of the trunk portion difficult.Recently, Light Detection and Ranging (LiDAR) systems have been used to overcome this limitation.LiDAR systems can be classified into

•
We proposed a new approach that leveraged PointNet++ for segmenting the canopy, trunk, and branches of trees.By applying PointNet++, we addressed the limitations of previous studies that have primarily focused on canopy and trunk segmentation, neglecting branch segmentation.

•
We introduced a preprocessing method for LiDAR point cloud data, which was tailored to handle the characteristics of tree-related LiDAR data, leading to improved accuracy in the segmentation results.

•
We identified an optimal learning environment for PointNet++ in the context of treestructure segmentation.We achieved superior segmentation results and enhanced the overall effectiveness of the PointNet++ model.
This paper is structured as follows: Section 2 describes previous research.Section 3 outlines the data acquisition, preparation, and model performance verification methods.Section 4 explains the model segmentation accuracy according to the learning environment, and Section 5 provides the discussion.Finally, Section 6 presents the conclusion.

Related Work
Machine-learning approaches can be categorized as supervised, which involves using labeled data to train a model, or unsupervised, where the system groups data into similar clusters [23][24][25][26][27][28][29][30][31].Each of these methods uses different techniques and approaches to segment the point cloud data (Table 1).
The graph-based method creates nodes using density-based spatial clustering of applications with noise (DBSCAN), mean shift, and K-mean clustering methods, and subsequently divides the canopy and trunk by arranging them in a topological network using the shortest path algorithm.One advantage of this method is that it is relatively easy to implement; however, it suffers from several limitations.When many nodes exist, the computational burden increases, and small branches may be misclassified as the trunk and canopy.Additionally, the structural differences among species can act as a limiting factor in selecting the parameters for the clustering process.

Hyperparameter
Tuning Test Description [23,24,31] Random Forest, Data clustering, CPS algorithm X X X A study that combines data clustering and shortest path algorithms to segment the canopy and trunk.
In the point cloud obtained from LiDAR, the canopy possesses a strong dispersive property, whereas the stem mainly has a linear or surface vector property.The branches possess attributes of both the canopy and trunk.These differences can be distinguished using a geometrical feature basis.Among the geometric feature-based methods, classificationperformance verification studies have been conducted on random forests, Gaussian mixture models, and support vector machines [32]; furthermore, 3D convolutional neural networks (CNNs), such as the PointNet and PointNet++ models are being reviewed [33].The supervised learning method requires preprocessing of training data and sophisticated labeling.Hence, the model performance may vary, but adjusting the parameters according to the structural differences based on the species is not necessary.To improve the performance of tree-structure segmentation using the geometric feature method, research on preprocessing of the learning data, learning environments, and classifier selections is required.
During the preprocessing stage of the training data, various characteristics of the point cloud such as red, green, and blue (RGB), intensity, surface normal, and scattering can be added to the x, y, and z values of the point cloud to enhance the segmentation of the tree structure.The canopy, which is rich in chlorophyll, appears green, whereas the trunk appears brown due to the presence of lignin.These contrasting features provide classifiers with clear information for segmenting the canopy and trunk.However, we need to note that the availability of the RGB information is dependent on the employed LiDAR equipment, and it may not be available for all parts due to occlusion.Combining information such as linear and surface vectors with geometric features can improve the segmentation performance [33,36].
However, most prior research has focused on mentioning the accuracy of the adopted algorithms or comparing the performance (accuracy, precision, recall, and F 1 score) of different models.For deep learning models, providing quantitative evaluations of the preprocessing effect of the training data and adjusting the training environment on the performance of the model are particularly important.

Data Preparation
The data used for learning and verification were collected from the Korean red pine (Pinus densiflora), which is an artificial forest located in the Backdudaegan National Arboretum (BNA) in Bonghwa-gun, Gyeongsangbuk-do, and from the Korean pine (Pinus koraiensis) and Japanese larch (Larix kaempferi), which are artificial forests in the Leading Forest Management Zone (LFMZ) in Chuncheon-si and Hongcheon-gun, Gangwon-do, respectively.
The data used in the test were obtained from the Korean red pine artificial forest located in Sangju-si, Gyeongsangbuk-do (Sangju Experimental Forest (SEF)) and the Korean pine and Japanese larch artificial forest located in the Gwangneung Experimental Forest, Namyangju-si, Gyeonggi-do (shown in Figure 1 and Table 2).
The data used in the test were obtained from the Korean red pine artificial forest l cated in Sangju-si, Gyeongsangbuk-do (Sangju Experimental Forest (SEF)) and the Kore pine and Japanese larch artificial forest located in the Gwangneung Experimental Fore Namyangju-si, Gyeonggi-do (shown in Figure 1 and Table 2).The plot was scanned onceusing a backpack laser scanner (BLS) (Libackpack D50, Greenvalley International, Berkeley, California, USA) and 18 times using a terrestrial laser scanner (TLS) (Leica RTC 360, Leica, Wetzlar, Hesse, Germany).To prevent data loss due to occlusion, the BLS data collection method involves passing through all individual trees.Conversely, the TLS data collection method selects a survey method based on the international benchmarking [37].
The scan data, which were taken multiple times, were combined into one scan data using the Leica Cyclone REGISTER 360.Despite the precautions taken, some occlusions remained in the data that were collected using the BLS and TLS methods.Incomplete data can lead to mis-segmentation as reported in studies such as [38,39].However, they may inevitably exist depending on the LiDAR data collection method and equipment performance.Therefore, all the data, including incomplete ones, were included in the training data for a rigorous test of part segmentation of PointNet++ at https://github.com/yanx27/Pointnet_Pointnet2_pytorch (accessed on 4 June 2023).
As shown in Figure 2, all scanned data underwent the following preprocessing steps.Down-sampling was performed to a 5 cm point resolution using Poisson sampling provided by the Point Data Abstraction Library (PDAL-2.5.2 version) (Figure 2a).( 1) The ground was flattened and removed (Figure 2b).( 2) Understory vegetation in the forest can affect the training, verification, and testing process.Therefore, the trunk and canopy regions were first separated using PDAL to remove the understory vegetation.The trunk region was divided based on the understory vegetation with the highest height in the plot without any special criteria.In the present study, most of the understory vegetation was in between 3.5 and 4.3 m high; thus, the trunk was set to 0-4.8 m, and the canopy was set to 4.5-100 m and cut down (Figure 2c).The TreeSeg [40] application was used to remove the understory vegetation and extract the trunk from the cut-trunk region [41].The TreeSeg application used Euclidean clustering to organize the unorganized point cloud and then separated the trunk and understory vegetation using region-based segmentation.(3) Next, the separation was performed by matching the cylinder shape using the random sample consensus and least median of square algorithms to extract only the trunk (Figure 2d).( 4) Finally, the trunk and canopy regions were merged using PDAL to obtain clean data without understory vegetation (Figure 2e).
The canopy, trunk, and branch were manually labeled using the CloudCompare program from preprocessed data from 435 trees (Table 3).The canopy, trunk, and branch were manually labeled using the CloudCompare program from preprocessed data from 435 trees (Table 3).Out of the 435 trees, 306 were used as learning data (102 Korean red pine, 102 Korean pine, and 102 Japanese larch), 72 as verification data (24 Korean red pine, 24 Korean pine, and 24 Japanese larch), and 57 as test data (19 Korean red pine, 19 Korean pine, and 19 Japanese larch).The canopy (green), trunk (blue), and branches (red) were labeled in individual tree data (Figure 3a).The labeling was performed by manually selecting the areas corresponding to the canopy, trunk, and branches using the editing tools supported by the CloudCompare program, which allows for mouse-based interaction.In the case of Korean red pine, the region of the branches and canopy was clearly distinguished, but distinguishing between the Korean pine and Japanese larch was difficult.Therefore, structural analysis and labeling of each tree required 7 days.Due to the tree growth process, distinguishing between the trunks and branches are challenging when offshoots are present.To segment them, the direction of the vector was used.Points that were formed in the z direction were segmented as trunks, whereas those in the x and y directions were segmented as branches.To accurately segment the branches, a detailed segmentation process was necessary because the branches were connected to the canopy.Although efforts were made to segment only the branches, instances may have occurred where points segmented as branches were also included as parts of the canopy because of practical limitations (Figure 3b).
were made to segment only the branches, instances may have occurred where points segmented as branches were also included as parts of the canopy because of practical limitations (Figure 3b).
This limitation was particularly noticeable in the Korean pine and Japanese larch, causing different numbers of the branch classes for each tree species.The final segmented training data included 306 canopies, 306 trunks, and 270 branches.The training, validation, and test data also included a similar number of each class with an average of 2,176,282 points for the canopies, 448,644 points for the trunks, and 68,094 points for the branches.This limitation was particularly noticeable in the Korean pine and Japanese larch, causing different numbers of the branch classes for each tree species.The final segmented training data included 306 canopies, 306 trunks, and 270 branches.The training, validation, and test data also included a similar number of each class with an average of 2,176,282 points for the canopies, 448,644 points for the trunks, and 68,094 points for the branches.

Experiment Environment
To segment the tree parts, we utilized PointNet++ [42], which achieves a high level of intersection over union (IoU) of 93.2 in the 3D part segmentation taxon of the state of the art.In the part segmentation of the canopy, stem, and branch classes, we did not use the RGB information because some LiDAR equipment does not support RGB and the colors of the canopy, trunk, and branches collected in this study were not clearly distinguished.Specifically, in the case of the Japanese larch, the branches had the same color as the canopy.Therefore, color was excluded from the learning factor.Only each point (x, y, and z), its surface normal vector (x, y, and z), and label values were used (Figure 4).
To find the optimal values that could accurately divide the three classes (canopy, trunk, and branch), we modified and learned the different hyperparameters (Table 4).
The basic architecture of PointNet++ extracted 1024 local and global features per sample.In addition to the correlation between the batch size and learning rate, the classification and segmentation accuracies were affected by the number of feature points extracted in the sampling layer and density-adaptive layer stages [43,44].
Therefore, the model was trained under two different conditions:

•
The learning environment was adjusted by extracting 2048, 4096, and 8196 local and global features at the sampling-layer stage.Setups 1-3 were used for nonresampling as the learning material, whereas Setups 4-6 were used for resampling as a learning environment.

Experiment Environment
To segment the tree parts, we utilized PointNet++ [42], which ach intersection over union (IoU) of 93.2 in the 3D part segmentation taxo art.In the part segmentation of the canopy, stem, and branch classes RGB information because some LiDAR equipment does not support of the canopy, trunk, and branches collected in this study were not cl Specifically, in the case of the Japanese larch, the branches had the sa opy.Therefore, color was excluded from the learning factor.Only eac its surface normal vector (x, y, and z), and label values were used (Fig To find the optimal values that could accurately divide the th trunk, and branch), we modified and learned the different hyperpara  Furthermore, the ball-tree method was used for grouping the feature points instead of the Euclidean method in which the radius value was set at 0.05 m.Successful detection was considered when the point cloud and ground-truth point cloud detected by the canopy,

Evaluation of the Model Performance
The performance of the PointNet++ model was evaluated using precision, recall, and F 1 scores after the learning process was completed.The calculations were made using Equations ( 1)-( 4), and the results were obtained by calculating the true positives (TPs), true negatives (TNs), false positives (FPs), and false negatives (FNs) using a confusion matrix.

Experiment A
The results of the class division based on the number of representative points (2048, 4096, and 8192) are listed in Table 5.The results of the class division of the Korean red pine, Korean pine, and Japanese larch based on the number of representative points (2048, 4096, and 8192) were as follows:

•
For the Korean red pine, precision was the highest for 8192 representative points (86.0%), recall was the highest for 2048 representative points (70.6%), and the best F 1 score was achieved for 2048 representative points (0.7), indicating that the segmentation of the Korean red pine without resampling performed well at these numbers of representative points.

•
The Korean pine also performed well for 2048 representative points with an F 1 score of 0.7.

•
The Japanese larch had a lower F 1 score of 0.5 in all conditions, which was lower than that of the Korean red pine and Korean pine.
The model performance results by class for 2048 representative points were as follows (Table 6).The results of the segmentation of the canopy and trunk of the tree species (Korean red pine, Korean pine, and Japanese larch) showed a high average F 1 score of 0.85, but the score for the branches was very low at 0.16.In all tree species, most branches were incorrectly segmented as canopy.Not all branches of the Japanese larch could be identified and were mis-segmented as canopy (Figure 5c) The reason could be that the PointNet++ model, which used the ball-tree algorithm to extract the feature points based on the metric space at the sampling-layer stage, did not sufficiently extract the branches because the distance between the canopy and branches did not meet the threshold of 0.05 m set for segmentation.The results of the segmentation of the Korean red pine and Korean pine demonstrated that although the branches located in the trunk region were properly segmented, those located in the same region as the canopy was not properly segmented (orange) (Figure 6a).Additionally, even when the trunk was located in a region separate from the canopy, many cases of proper segmentation occurred.However, mis-segmentation (purple) also occurred toward the treetop.This result was particularly prominent in the case of the Korean pine and Japanese larch.In the case of the Korean red pine, the canopy, trunk, and the regions between branches were separated, whereas in the Korean pine and Japanese larch, the trunks were often covered by the canopy (Figure 6b,c).Hence, the trunk recall value was found to be lower than that of the Korean red pine.
In summary, in Environment A where resampling was not performed, when 2048 representative points were present (Setup 1), we found that the performance of the model segmentation was high on average.Furthermore, we observed that in the case of Point-Net++, which extracted representative points using metric space, the segmentation performance was poor when the space between the canopy, trunk, and branches was not sufficiently segmented.The results of the segmentation of the Korean red pine and Korean pine demonstrated that although the branches located in the trunk region were properly segmented, those located in the same region as the canopy was not properly segmented (orange) (Figure 6a).Additionally, even when the trunk was located in a region separate from the canopy, many cases of proper segmentation occurred.However, mis-segmentation (purple) also occurred toward the treetop.This result was particularly prominent in the case of the Korean pine and Japanese larch.In the case of the Korean red pine, the canopy, trunk, and the regions between branches were separated, whereas in the Korean pine and Japanese larch, the trunks were often covered by the canopy (Figure 6b,c).Hence, the trunk recall value was found to be lower than that of the Korean red pine.

Experiment B
The results of the class division based on the number of representative points (2048, 4096, and 8192) are listed in Table 7.When resampling was conducted, the model performance improved with an increase in the number of representative points; the highest performance was observed when 4096 or 8192 points were used.Specifically, the best performance was achieved when 4096 points were used for the Korean red pine (F1 score = 0.9), Korean pine (F1 score = 0.9), and Japanese larch (F1 score = 0.8).
The model performance results by class for 4096 representative points are listed in Table 8.In summary, in Environment A where resampling was not performed, when 2048 representative points were present (Setup 1), we found that the performance of the model segmentation was high on average.Furthermore, we observed that in the case of PointNet++, which extracted representative points using metric space, the segmentation performance was poor when the space between the canopy, trunk, and branches was not sufficiently segmented.

Experiment B
The results of the class division based on the number of representative points (2048, 4096, and 8192) are listed in Table 7.When resampling was conducted, the model performance improved with an increase in the number of representative points; the highest performance was observed when 4096 or 8192 points were used.Specifically, the best performance was achieved when 4096 points were used for the Korean red pine (F 1 score = 0.9), Korean pine (F 1 score = 0.9), and Japanese larch (F 1 score = 0.8).
The model performance results by class for 4096 representative points are listed in Table 8.The results of the canopy and trunk segmentation yielded a high average F 1 score of 0.95, and the segmentation of the branches exhibited an even higher score of 0.67 than that in the A environment.Specifically, the Korean red pine and Korean pine demonstrated exceptional segmentation performance and showed a reduced tendency for misclassification of branches compared with the previous setup (as shown in Figure 7a,b).The accuracy of the branch segmentation of the Japanese larch slightly improved compared with that in environment A, but it remained low with a recall of 29.1%.The results of the canopy and trunk segmentation yielded a high average F1 score of 0.95, and the segmentation of the branches exhibited an even higher score of 0.67 than that in the A environment.Specifically, the Korean red pine and Korean pine demonstrated exceptional segmentation performance and showed a reduced tendency for misclassification of branches compared with the previous setup (as shown in Figure 7a,b).The accuracy of the branch segmentation of the Japanese larch slightly improved compared with that in environment A, but it remained low with a recall of 29.1%.Resampling was used to create a space between the canopy, trunk, and branch regions, as shown in Figure 8.However, Figure 6c shows that in the case of the Japanese larch, the region between the branch and canopy considerably overlapped and the distance space was not sufficient, causing similar results as those shown in Figure 7c.
As a result, when resampling was performed, the segmentation performance of the model was found to be high, on average, when 4096 representative points were used (Setup 5).This was because the resampling ensured a sufficient distance space between the class regions, which was beneficial for extracting representative points using a ball tree.Therefore, when segmenting a tree with similar characteristics as the Japanese larch, securing sufficient distance space between classes was necessary by lowering the ball-tree threshold to 0.05 m or less or by increasing the strength of the resampling.Resampling was used to create a space between the canopy, trunk, and branch regions, as shown in Figure 8.However, Figure 6c shows that in the case of the Japanese larch, the region between the branch and canopy considerably overlapped and the distance space was not sufficient, causing similar results as those shown in Figure 7c.
As a result, when resampling was performed, the segmentation performance of the model was found to be high, on average, when 4096 representative points were used (Setup 5).This was because the resampling ensured a sufficient distance space between the class regions, which was beneficial for extracting representative points using a ball tree.Therefore, when segmenting a tree with similar characteristics as the Japanese larch, securing sufficient distance space between classes was necessary by lowering the ball-tree threshold to 0.05 m or less or by increasing the strength of the resampling.

Comparative Analysis of Part Segmentation Results in Related Studies
Recent studies have largely employed supervised learning methods using CNN structures such as PointNet and PointCNN for segmenting the canopy, trunk, and ground [34] as well as for distinguishing individual trees within forested areas [45,46].Unsupervised techniques such as DBSCAN and mean shift have also been used to differentiate between the canopy and trunk regions.However, in contrast to these previous studies that focused primarily on segmenting the canopy and trunk, this study specifically targeted the extraction of the canopy, trunk, and branches.To evaluate the performance of our approach based on PointNet++, we provided precision and F1 scores in Table 9, while noting that the accuracy of the branch segmentation was not presented, as the previous studies listed in Table 9 only address the canopy and trunk segmentation.

Comparative Analysis of Part Segmentation Results in Related Studies
Recent studies have largely employed supervised learning methods using CNN structures such as PointNet and PointCNN for segmenting the canopy, trunk, and ground [34] as well as for distinguishing individual trees within forested areas [45,46].Unsupervised techniques such as DBSCAN and mean shift have also been used to differentiate between the canopy and trunk regions.However, in contrast to these previous studies that focused primarily on segmenting the canopy and trunk, this study specifically targeted the extraction of the canopy, trunk, and branches.To evaluate the performance of our approach based on PointNet++, we provided precision and F 1 scores in Table 9, while noting that the accuracy of the branch segmentation was not presented, as the previous studies listed in Table 9 only address the canopy and trunk segmentation.When comparing the segmentation results, using the F 1 score is an accurate approach even when the number of test data samples differs.However, in similar studies where the F 1 score was not reported, the comparison was solely based on the precision values.In terms of trunk-segmentation performance, Table 9 indicates that the segmentation results achieved by the PointNet++ model outperformed those obtained by both PointCNN and PointNet.
Notably, the canopy-segmentation accuracy in this study was relatively low, reaching 90.3%.This decrease could be attributed to the additional segmentation of the branches, which impacted the precision analysis.An effective method using mean shift and Dijkstra's algorithm was proposed for classifying the canopy and wood (trunk and branches) [31].Unsupervised methods offer the advantage of not requiring labor-intensive labeling work and reducing pre-processing time.However, determining the appropriate parameter values for clustering tasks, particularly when processing point cloud data for trees, can be challenging owing to the variability among species.In this study, we employed resampling strength and varied the number of representative points to ensure consistent learning data.The proposed environment in this study presented advantages for accurately segmenting complex tree structures, including the canopy, trunk, and branches.

Discussion
The high segmentation accuracy was demonstrated through the resampling of training data to approximately 25,000-30,000 points using Poisson sampling and extracting 4096 representative points.However, the performance of the PointNet++ model deteriorated when dealing with areas where the distinction between the canopy, trunk, and branches was unclear, such as in case of Japanese larch.This limitation can be attributed to the hierarchical feature-learning approach of PointNet++ that relies on a metric space.Although the performance of the model improved with resampling compared with nonresampling scenarios (Experiment A: accuracy = 79.8% and Experiment B: accuracy = 92.2%),both precise labeling and methods are required to increase the amount of training data and enhance the performance of the model.

Conclusions
This study evaluated the performance of part segmentation for three coniferous species, i.e., Korean red pine, Korean pine, and Japanese larch, using the PointNet++ model.By adjusting two learning environments, we segmented the canopy, trunk, and branches.Comparing the empirical results, we observed an 11% increase in the accuracy when resampling was applied compared to when it was not.In the resampling environment, we achieved an average F 1 score of 0.86 with 4096 representative points.This implies that for optimal segmentation of the canopy, trunk, and branches using PointNet++, resampling of approximately 25,000-30,000 points is recommended and the model performs well with 4096 representative points.In our future work, we intend to explore the accurate estimation of AGB while simultaneously automating the segmentation of the canopy, trunk, and branches.This will be achieved by leveraging PointNet++ and incorporating biomass coefficients derived from comprehensive field surveys.

Figure 1 .
Figure 1.Location of the study site.The data collection was performed using BLS (backpack las scanner) equipment for the Korean red pine and the TLS (terrestrial laser scanner) equipment the Korean pine and Japanese larch.The point cloud shown in Figure 1 is a representation of only part of the employed data.

Figure 1 .
Figure 1.Location of the study site.The data collection was performed using BLS (backpack laser scanner) equipment for the Korean red pine and the TLS (terrestrial laser scanner) equipment for the Korean pine and Japanese larch.The point cloud shown in Figure 1 is a representation of only a part of the employed data.

Figure 2 .
Figure 2. Data preprocessing.(a) Point clouds down-sampled to 5 cm resolution using Poisson sampling.(b) Results of the normalizing and removing ground points.(c) Tree trunk area is segmented considering the height of understory vegetation.(d) Trunk point clouds were extracted using the TreeSet application.(e) Stand point clouds with the understory vegetation was removed.

Figure 2 .
Figure 2. Data preprocessing.(a) Point clouds down-sampled to 5 cm resolution using Poisson sampling.(b) Results of the normalizing and removing ground points.(c) Tree trunk area is segmented considering the height of understory vegetation.(d) Trunk point clouds were extracted using the TreeSet application.(e) Stand point clouds with the understory vegetation was removed.

Figure 3 .
Figure 3. Manually labeled canopy, trunk, and branch point clouds.(a) Canopy, branch, and trunk point clouds were labeled and manually segmented, including offset offshoots.(b) Manually segmented Korean pine trunk and branch point clouds.

Figure 3 .
Figure 3. Manually labeled canopy, trunk, and branch point clouds.(a) Canopy, branch, and trunk point clouds were labeled and manually segmented, including offset offshoots.(b) Manually segmented Korean pine trunk and branch point clouds.

Figure 4 .
Figure 4. Tree data that included RGB.Because the branch and trunk have si circle) (left), the RGB values were removed and surface normal vector (x, y, z) the data (right).

Figure 4 .
Figure 4. Tree data that included RGB.Because the branch and trunk have similar color (in the red circle) (left), the RGB values were removed and surface normal vector (x, y, z) values were added to the data (right).

ForestsFigure 5 .
Figure 5. Results of the part segmentation by label (canopy, trunk, and branch) of Setup 1 (representative point = 2048) using PointNet++.(a) Confusion matrix of the Korean red pine; (b) confusion matrix of the Korean pine; and (c) confusion matrix of the Japanese larch.In all segmentation results, the branch-segmentation performance was low, and in the case of the Japanese larch, most of the branches were estimated as canopy.

Figure 5 .
Figure 5. Results of the part segmentation by label (canopy, trunk, and branch) of Setup 1 (representative point = 2048) using PointNet++.(a) Confusion matrix of the Korean red pine; (b) confusion matrix of the Korean pine; and (c) confusion matrix of the Japanese larch.In all segmentation results, the branch-segmentation performance was low, and in the case of the Japanese larch, most of the branches were estimated as canopy.

Figure 6 .
Figure 6.Results of the class (canopy, trunk, and branch) part segmentation in Setup 1.The purple color indicated the result of the trunk being mis-segmented as a canopy, and the orange color indicated the result of the branch being mis-segmented as a canopy.(a) Results of the mis-segmentation of the Korean red pine; (b) results of the mis-segmentation of the Korean pine; and (c) results of the mis-segmentation of the Japanese larch.

Figure 6 .
Figure 6.Results of the class (canopy, trunk, and branch) part segmentation in Setup 1.The purple color indicated the result of the trunk being mis-segmented as a canopy, and the orange color indicated the result of the branch being mis-segmented as a canopy.(a) Results of the mis-segmentation of the Korean red pine; (b) results of the mis-segmentation of the Korean pine; and (c) results of the mis-segmentation of the Japanese larch.

Figure 7 .
Figure 7. Results of part segmentation by label (canopy, trunk, and branch) in Setup 5 (representative points = 4096) using PointNet++.(a) Confusion matrix of the Korean red pine; (b) confusion matrix of the Korean pine; and (c) confusion matrix of the Japanese larch.In all segmentation results, the branch-segmentation performance was low, and in the case of Japanese larch, most of the branches were estimated as canopy.

Figure 7 .
Figure 7. Results of part segmentation by label (canopy, trunk, and branch) in Setup 5 (representative points = 4096) using PointNet++.(a) Confusion matrix of the Korean red pine; (b) confusion matrix of the Korean pine; and (c) confusion matrix of the Japanese larch.In all segmentation results, the branch-segmentation performance was low, and in the case of Japanese larch, most of the branches were estimated as canopy.

Figure 8 .
Figure 8. Results of class (canopy, trunk, and branch) part segmentation in Setup 5.The purple color indicated the result of the trunk being mis-segmented as canopy, the yellow color represented the result of the branch being mis-segmented as canopy, and the white color showed the result of the branch being mis-segmented as trunk.(a) Results of the segmentation of the Korean red pine; (b) results of the mis-segmentation of the Korean pine; and (c) results of the mis-segmentation of the Japanese larch.

Figure 8 .
Figure 8. Results of class (canopy, trunk, and branch) part segmentation in Setup 5.The purple color indicated the result of the trunk being mis-segmented as canopy, the yellow color represented the result of the branch being mis-segmented as canopy, and the white color showed the result of the branch being mis-segmented as trunk.(a) Results of the segmentation of the Korean red pine; (b) results of the mis-segmentation of the Korean pine; and (c) results of the mis-segmentation of the Japanese larch.

Table 1 .
Comparison of existing works for segmenting individual tree structures.

Table 3 .
Details of tree data labeled using the CloudCompare program from the preprocessed data.

Table 3 .
Details of tree data labeled using the CloudCompare program from the preprocessed data.

Table 4 .
Hyperparameter values of PointNet++ to segment the canopy, trunk, and branch.

Table 4 .
Hyperparameter values of PointNet++ to segment the canopy, trunk

Table 5 .
Comparison of the accuracy, precision, recall, and F 1 score of the tree species in Experiment A setup.

Table 6 .
Comparison of the accuracy, precision, recall, and F 1 score by label (canopy, trunk, and branch) in Setup 1 (2048 representative points).

Table 7 .
Comparison of the overall accuracy, precision, recall, and F1 score of the tree species in the Experiment B setup.

Table 7 .
Comparison of the overall accuracy, precision, recall, and F 1 score of the tree species in the Experiment B setup.

Table 8 .
Comparison of the accuracy, precision, recall, and F 1 score by label (canopy, trunk, and branch) in Setup 5 (representative points = 4096).

Table 8 .
Comparison of the accuracy, precision, recall, and F1 score by label (canopy, trunk, and branch) in Setup 5 (representative points = 4096).

Table 9 .
Comparison of the part segmentation results with similar studies.The part marked with * refers to the results of segmenting the trunk and branch by treating them as the same region.Unsupervised learning contains the mean shift and Dijkstra methods.

Table 9 .
Comparison of the part segmentation results with similar studies.The part marked with * refers to the results of segmenting the trunk and branch by treating them as the same region.Unsupervised learning contains the mean shift and Dijkstra methods.Norway spruce (Picea abies),Lodgepole pine (Pinus contorata) etc.