Individual Tree Species Classification Using the Pointwise MLP-Based Point Cloud Deep Learning Method ^†

Abstract

Tree species is a critical factor in the practice of forest resource field sample surveys. Light detection and ranging (LiDAR) can obtain three-dimensional structural information about forests and trees and is increasingly being used in forest resource surveys. We used three pointwise multi-layer perceptron (MLP)-based deep learning methods (PointNet, PointNet++, and PointMLP) to identify individual tree point clouds of seven different tree species to explore the effectiveness of point cloud deep learning in classifying individual tree point clouds. Experiment results were extremely exciting. Higher classification accuracy can be attained in trials utilizing 2048 points. The tree classification accuracies of PointMLP and PointNet++ on the test set were 0.9474 and 0.9483, respectively, in classification experiments with a balanced sample size. PointMLP, the current state-of-the-art pointwise MLP-based model, is faster to train and performs better.

1. Introduction

In the practice of forest resource field sample surveys, tree species is an essential survey factor. Tree species information is also an important parameter for ecosystem modeling and forest resource management [1]. With the continuous development of light detection and ranging (LiDAR) technology, the 3D structural parameters of forest trees can be obtained quickly and accurately by using ground-based LiDAR systems for sample plot scanning. The focus and difficulty of current research is how to accurately identify tree species information from laser point clouds of individual trees.

Traditional machine learning methods necessitate the manual extraction of copious amounts of 3D structural information for modeling, and recognition accuracy is low [2,3,4,5,6]. A breakthrough in computer vision has been made in classifying 3D object shapes using point cloud deep learning techniques, opening up a new practical direction for tree species classification.

Some previous studies converted LiDAR data into images with various classification features [7,8,9] and then used image deep learning methods to study the classification of tree species. With the development of point-based deep learning methods such as PointNet [10] and PointNet++ [11], scholars have started to use point-by-point deep learning models for tree species classification research. Seidel et al. [12] used PointNet to classify seven tree species, but the classification accuracy was very low and then shifted the focus of the study to a picture convolutional neural network (CNN) approach, which finally achieved a high classification accuracy. In other studies, using point cloud deep learning models [13,14,15,16,17,18,19], good accuracy of tree classification was achieved.

To explore the potential of point cloud deep learning for individual tree species point cloud classification, we used three pointwise MLP-based point cloud deep learning methods (PointNet, PointNet++, PointMLP) to classify tree species from individual tree point clouds of four and seven tree species. We used the farthest point sampling method to reduce the number of points in each individual tree point cloud to 1024 and 2048. We achieved extremely exciting experimental results. In the classification experiments with a balanced sample size, PointMLP and PointNet++ achieved high accuracy in tree classification on the test set.

2. Materials and Methods

2.1. Individual Tree Point Cloud Data

In this study, we used a publicly available dataset for our experiments [20]. The name of the dataset is “Single tree point clouds from terrestrial laser scanning”. It contains individual tree point cloud data for seven tree species (

Table 1. Information on tree species and number of samples.

Species	Number	Species	Number
Beech 1	104	Oak	31
Douglas Fir 1	116	Ash	27
Spruce 1	127	Pine	21
Red oak 1	100

¹ To balance the sample data, we conducted a comparison experiment using four datasets with similar numbers of samples from Beech, Douglas Fir, Spruce, and Red oak.

). Detailed information about the data acquisition and sensors can be found in the article [12].

2.2. Data Preprocessing

We referred to the [12] process for data preprocessing with some modifications. We performed the following four preprocessing operations on the point cloud data.

• Manual selection: The individual tree data were screened artificially to control the quality of the individual tree samples.
• Deleted: Deleted the point of 30% density at the bottom of the tree.
• Downsampling: The farthest point sampling method was used to sample the points of the individual tree as 1024 and 2048.
• Data organization: The data organization of the ModelNet40 [21] dataset was used to create the sample database for this study experiment.

2.3. Point Cloud Deep Learning Models

PointNet [10] is the pioneer of point-based deep learning. PointNet uses a shared MLP to directly process unordered point sets as input. PointNet contains three key modules: a maximum pooling layer as a symmetric function to aggregate information from all points, a combined local and global information structure, and two joint alignment networks for aligning input points and point features.

PointNet is unable to capture the local structure generated by metric space points, thus limiting its ability to recognize fine-grained patterns and generalize to complex scenes. The proposed PointNet++ [11] method bridges this gap. PointNet++ is a hierarchical neural network that addresses two core problems: how to generate partitions of point sets and how to abstract point sets or local features by local feature learners. The position and scale of the prime points are used to describe each partition of the click, and the FPS algorithm is used to select the prime points. Features extracted within each partition using PointNet are used as descriptions of local features. The model structure contains two ensemble abstraction layers (SAs). Each SA layer consists of three parts: a sampling layer, a grouping layer, and a PointNet layer.

The PointMLP [22] model follows the design philosophy of PointNet, a simpler but deeper network architecture. PointMLP learns the point cloud representation by a simple feed-forward residual MLP network that hierarchically aggregates the local features extracted by the MLP. PointMLP introduces a lightweight local geometric affine module that adaptively transforms point features in local regions. PointMLP is similar to PointNet and PointNet++, but it is more general and exhibits better performance.

The hyperparameters of PointNet, PointNet++, and PointMLP model training are summarized in

Table 2. Summary of hyperparameters for deep learning models.

Model	PointNet	PointNet++	PointMLP
Batch Size	12	12	12
Number of Points	1024/2048	1024/2048	1024/2048
Categories	4/7	4/7	4/7
Epochs	200	200	300
Optimizer	Adam	Adam	SGD
Learning Rate	0.001	0.001	0.1
Weight Decay	0.0001	0.0001	0.0002
Momentum	—	—	0.9

. PyTorch (1.10.0 + CUDA 11.3) is the framework used by three deep learning methods. The graphics card used in this study was an NVIDIA GeForce RTX 3070 (8 GB).

2.4. Model Accuracy Evaluation Metrics

In this experiment, balanced accuracy (BAcc) and kappa coefficients were used to compare and analyze the results of tree species classification in terms of accuracy evaluation and models.

$$\mathit{kappa}=\frac{ \mathit{Acc} -p_e}{1-p_e}$$

(1)

The kappa coefficient is calculated from the confusion matrix of the classification results, and its value ranges from −1 to 1. Usually, the kappa coefficient is greater than zero, and its absolute value is larger, indicating better classification results.

3. Results

The tree classification results for the six sets of experiments using the three deep learning models are summarized in

Table 3. Accuracy of tree species classification results for three deep learning models.

Model	Number of Categories	Points	Train		Test
			BAcc	kappa	BAcc	kappa
PointNet	4	1024	0.4865	0.3246	0.5205	0.3663
		2048	0.5135	0.3598	0.4954	0.3412
	7	1024	0.2672	0.2473	0.2923	0.2879
		2048	0.2638	0.2309	0.2776	0.2598
PointNet++	4	1024	0.7663	0.6910	0.8787	0.8298
		2048	0.8109	0.7430	0.9483	0.9297
	7	1024	0.6630	0.6686	0.7263	0.7927
		2048	0.7791	0.7762	0.8849	0.9205
PointMLP	4	1024	0.9176	0.8885	0.9074	0.8694
		2048	0.9782	0.9700	0.9474	0.9296
	7	1024	0.7654	0.8444	0.7061	0.7839
		2048	0.7852	0.8073	0.8460	0.8803

. As seen from the table, the accuracy of the experiments using the four tree species with balanced sample data for classification is relatively high. The experiments with 2048 sampling points achieved a higher accuracy of tree species classification. A comparison of the classification accuracies of the test dataset shows that the results of both models, PointNet++ and PointMLP, are increasingly similar.

By mapping the training accuracy of each epoch during model training (Figure 1), we portrayed the training process of the three deep learning models. As seen from the figure, the classification accuracy of the PointNet model is extremely low, and the classification accuracy of the test dataset does not increase with the increasing training accuracy of the model on the test dataset. During the operation of the PointNet++ model, the classification accuracy of the test set was higher than that of the training set and has been growing, finally obtaining a high classification accuracy. The test accuracy of the PointMLP model increases with the training accuracy and finally achieves a classification accuracy similar to that of the PointNet++ model.

We summarized and plotted the time trained for the six sets of experiments with 2048 sampling points (Figure 2). It can be seen from the figure that the PointNet++ model is more time-consuming, and it takes a longer time to train to obtain the optimal model parameters, while the PointMLP model can achieve a classification accuracy similar to that of the PointNet++ model in a brief period. The PointNet model saturates the model performance shortly after starting training, and the final classification accuracy is extremely low.

4. Discussion

In this study, three pointwise MLP-based point cloud deep learning methods were used to investigate the classification of individual tree point clouds. According to the findings, two deep learning models, PointNet++ and PointMLP, can better identify the tree species information in individual tree point clouds.

The tree species classification obtained by the PointNet model in our study was exceptionally low, similar to the results obtained by Seidel et al. [12]. This is because PointNet cannot capture local features of 3D objects, which limits its ability to classify and recognize similar objects. However, both the PointNet++ and PointMLP models introduce a local feature extraction module, which can extract the fine-grained local features of 3D objects well and thus achieve good classification accuracy. A related study [19] also demonstrated that PointNet++ can achieve good classification accuracy for tree species classification problems.

In this study, the experiments to classify the four tree species achieved a high classification accuracy. This is because the four tree species used have a larger number of samples and are more balanced in number. The other three tree species have a smaller number of samples, which limits the further learning of classification features by deep learning. From this, we suggest that when using deep learning methods for object classification in the future, researchers try to choose a substantial number of samples and keep the distribution of the number of samples consistent.

The classification accuracy is higher when the number of sampling points of an individual tree is 2048, which is consistent with the findings of [19]. This indicates that 1024 points are not a good representation of the accurate 3D structural information of individual trees.

Our experiments show that the two models, PointNet++ and PointMLP, achieve comparable tree classification accuracy. However, the PointMLP model can be trained in less time to obtain the optimal deep learning model parameters. This is due to PointMLP’s simpler and deeper network architecture, which is achieved by learning the point cloud representation using a simple feed-forward residual MLP network. PointMLP has exceptional model performance.

5. Conclusions

Point cloud deep learning models of the MLP type with local feature extraction are proven to be accurate for tree species classification of individual tree point clouds. PointMLP, the current state-of-the-art (SOTA) MLP-based point cloud deep learning method, has promising applications in tree species classification.