YOLO-SegNet: A Method for Individual Street Tree Segmentation Based on the Improved YOLOv8 and the SegFormer Network

Abstract

In urban forest management, individual street tree segmentation is a fundamental method to obtain tree phenotypes, which is especially critical. Most existing tree image segmentation models have been evaluated on smaller datasets and lack experimental verification on larger, publicly available datasets. Therefore, this paper, based on a large, publicly available urban street tree dataset, proposes YOLO-SegNet for individual street tree segmentation. In the first stage of the street tree object detection task, the BiFormer attention mechanism was introduced into the YOLOv8 network to increase the contextual information extraction and improve the ability of the network to detect multiscale and multishaped targets. In the second-stage street tree segmentation task, the SegFormer network was proposed to obtain street tree edge information more efficiently. The experimental results indicate that our proposed YOLO-SegNet method, which combines YOLOv8+BiFormer and SegFormer, achieved a 92.0% mean intersection over union (mIoU), 95.9% mean pixel accuracy (mPA), and 97.4% accuracy on a large, publicly available urban street tree dataset. Compared with those of the fully convolutional neural network (FCN), lite-reduced atrous spatial pyramid pooling (LR-ASPP), pyramid scene parsing network (PSPNet), UNet, DeepLabv3+, and HRNet, the mIoUs of our YOLO-SegNet increased by 10.5, 9.7, 5.0, 6.8, 4.5, and 2.7 percentage points, respectively. The proposed method can effectively support smart agroforestry development.

1. Introduction

A significant portion of city afforestation consists of street trees. Trees enhance environmental comfort by improving the air quality [1], providing shade [2], and storing carbon [3,4].

The parameters of standing street trees, such as the tree height, trunk diameter at breast height, crown width, and tree species, are important components of urban forest inventory systems [5,6]. Traditional field measurements are time-consuming and expensive and unable to reliably and promptly capture dynamic changes in street trees [7]. Computer vision is a new but increasingly popular technology for urban street tree intelligent research. Street tree images are usually captured by consumer phones, iPads, or cameras, and tree parameters are collected from these images via visual interpretation. Wu et al. [8] first used a feature-adaptive mean-shift algorithm to segment images and extract tree contours [9]. Then, a passive measurement method for the tree height and crown diameter based on a smartphone was proposed.

Segmenting individual trees from complex backgrounds is the crucial first step of street tree inventories. Studies on tree segmentation include two main styles: studies based on point clouds [10,11,12] and studies based on images [13,14]. The morphological parameters of street trees extracted from light detection and ranging (LiDAR) point cloud data are close to the measured values [15]. Mobile laser scanning (MLS) can collect high-precision point clouds of trees [16,17,18]. One of the purposes of these three-dimensional 3D point clouds is to measure the morphological parameters of street trees, and two-dimensional 2D images can also achieve this purpose. Moreover, these data acquisition devices are very expensive and difficult to carry, which is problematic for some researchers without professional equipment who wish to carry out intelligent tree research. Furthermore, compared to 2D images, point clouds require more attempts to prepare and annotate a dataset.

Another type of tree segmentation research is based on low-cost 2D images. Many convolutional neural networks (CNNs) have been used in these studies [19,20]. A depth camera was used to acquire RGB and depth images. A branch segmentation method based on a region convolutional neural network was developed for apple trees [21]. The efficiency and accuracy of this method still need to be greatly improved. The parts of the tree should be recognized, including the trunk and whole tree. A SegNet-based network architecture [22] based on a Kinect V2 camera was used for the segmentation of the branch, trunk, and trellis wire [19]. This method obtained the foreground RGB based on the depth camera to improve the IoU value, but it still needs to be improved. A method based on UNet++ for tree branch localization [23] was implemented to achieve pixelwise segmentation on an RGB image [24]. It is an improved version of UNet [25], where the skip paths are revised with denser skip connections. Most of the existing tree segmentation methods are focused on local information extraction, such as tree branches and canopies, and there is a lack of research on whole trees covered with leaves. Moreover, these experimental data are customized, and the effectiveness of these algorithms has not been verified in relevant public datasets.

To address these problems, we use larger, publicly available urban street tree datasets to verify our segmentation method. The data and number of tree species used in this paper far exceed those of previous related studies. Tree image object detection was first conducted to extract tree positions in images, and then the image segmentation step was performed to improve the street tree segmentation accuracy. Therefore, in the object detection stage, the BiFormer attention mechanism [26] was introduced into the YOLOv8 network [27] to extract regions of interest and reduce the impact of the background on target segmentation. Based on the detection results, the SegFormer [28] network was introduced to segment street trees. The multilayer perceptron (MLP) decoder aggregates information from different layers and thus combines both local and global attention to render powerful representations. Therefore, it is very suitable for street tree objects with large differences in global and local features.

The main contributions of this study are as follows.

(1)

Street tree datasets were reconstructed. The original images [13] were re-annotated and re-transformed according to the YOLO dataset format to produce a new street tree dataset for object detection. Our method was applied to this public street tree segmentation dataset.

(2)

The BiFormer module was introduced into YOLOv8 to reduce the background interference in street tree segmentation tasks, enabling the backbone network to capture long-distance dependencies effectively. The experimental results also show that the street tree object detection results markedly affect the image segmentation task.

(3)

This new YOLO-SegNet model, combined with the improved YOLO + BiFormer and SegFormer models, was used to accurately segment individual street trees of different species.

2. Materials and Methods

2.1. Street Tree Dataset

2.1.1. Image Acquisition

In previous research, we published a large public dataset on urban street tree image segmentation [13]. The details of the street tree dataset are available at https://ytt917251944.github.io/dataset_jekyll (accessed on 10 October 2023). These street tree images were obtained using different consumer-grade equipment, such as an iPhone 12, iPhone 13, and iPad 9.

The street tree images were collected in natural environments. A total of 3949 street tree images were produced, and the numbers of different tree species are shown in Figure 1A. A tree image object detection task was first conducted to extract tree positions from the image to improve the accuracy of street tree segmentation. Then, an image segmentation step was performed. Based on public street tree images [13], the LabelMe rectangle tool was used to carry out object detection labeling, as shown in Figure 1B.

2.1.2. Image Annotation

In our object detection task, we further used rectangular boxes to label the original street tree images. In the instance segmentation task, this public street tree dataset provides fine annotations at the pixel level, as Figure 2 shows. The rectangular boxes are the object detection labels, and the segmentation masks with different colors are the segmentation labels. Intuitively, there are great differences in the contour information of different tree species. Some tree crowns are nearly flat–spherical, such as those of Cinnamomum camphora (Linn) Presl. Some are nearly conical, such as Cedrus deodara. Some are nearly umbrella-shaped, such as Ginkgo biloba. Some are nearly weeping, such as Salix babylonica. We also divided the street tree dataset. See

Table 1. Street tree dataset.

Street Tree Dataset	Ratio	Number	Dataset Format (Object Detection)	Dataset Format (Segmentation)
Train set	8	3168	YOLO [27]	VOC2012 [29]
Validation set	1	395
Test set	1	386

for details.

2.2. YOLO-SegNet Model

Figure 3 shows the structure of our method, YOLO-SegNet, including the object detection module and image segmentation module. Based on the street tree object detection dataset, the raw tree images were fed into the YOLOv8 module to obtain the background-removed images. The background region was set to black. Based on the street tree image segmentation dataset, the background-removed images were then input into the SegFormer network to obtain the final segmentation images.

2.2.1. YOLOv8 with the BiFormer Attention Mechanism

Researchers have conducted multiple updates and iterations on YOLO [30,31,32]. YOLOv8 [27], an improvement on the previous versions, further enhances the object detection performance. See the left of Figure 3. YOLOv8 is an anchor-free model that can directly predict the center of an object.

A street tree is an irregular larger object. The distance between the tree top and bottom is large, and the difference in features is large. Therefore, this study introduces the BiFormer in the backbone part [26,33,34], which is used to capture long-range dependencies in street tree images, as shown in Figure 4. BiFormer uses bilevel routing to enable more flexible computation allocation with content awareness, and it benefits the training and inference of large models [35,36,37,38].

Several studies [39,40,41,42,43] have proposed different sparse attention mechanisms. Vanilla attention is the most basic attention mechanism, with high computational complexity and a heavy memory footprint. Local attention limits the attention operations to local windows, but may sacrifice some global context information. Axial attention is a pattern of sparse attention along a specific axis. Dilated attention is calculated by using an expanded window pattern over a local area. Deformable attention allows the attention window to be adjusted adaptively according to the image content, thus capturing image features more flexibly. Observing Figure 5b–d, these existing studies try to reduce the complexity via sparse attention.

Therefore, a dynamic, query-aware bilevel routing attention (BRA) was proposed, as shown in Figure 5f. The upper-layer router interacts with all image blocks through a global self-attention mechanism and generates a global image representation. The low-level router uses a local self-attention mechanism to interact with each image block and generate a local image representation. Through this dual-layer routing attention mechanism, BiFormer is able to capture both global and local feature information, thereby improving the accuracy of the model.

BRA: A feature map $\mathrm{X}\in R^{H\times W\times C}$ is separated into S × S nonoverlapping regions such that each region contains ${\displaystyle \frac{H\times W}{S^2}}$ vectors. This part is carried out by reshaping X as ${\mathrm{X}}^r\in S^2\times {\displaystyle \frac{H\times W}{S^2}}\times C$. The query Q, key K, and value V tensor, where $\mathrm{Q}, \mathrm{K}, \mathrm{V}\in {\mathrm{R}}^{S^2\times {\displaystyle \frac{H\times W}{S^2}}\times C}$, are as follows:

$$\mathrm{Q}={\mathrm{X}}^r{\mathrm{W}}^q, \mathrm{K}={\mathrm{X}}^r{\mathrm{W}}^k, \mathrm{V}={\mathrm{X}}^r{\mathrm{W}}^v$$

(1)

where ${\mathrm{W}}^q{, \mathrm{W}}^k, {\mathrm{W}}^v\in R^{C\times C}$ are the projection weights, respectively.

Each query token in region i will attend to all key–value pairs residing in the union of k routed regions indexed with ${\mathrm{I}}_{(i,1)}^r,{\mathrm{I}}_{(i,2)}^r{,...,\mathrm{I}}_{(i,k)}^r$. The key and value tensors are as follows:

$${\mathrm{K}}^g=\mathrm{g}\mathrm{a}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\left({\mathrm{K},\mathrm{I}}^r\right), {\mathrm{V}}^g=\mathrm{g}\mathrm{a}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\left({\mathrm{V},\mathrm{I}}^r\right), {\mathrm{I}}^r=\mathrm{t}\mathrm{o}\mathrm{p}\mathrm{k}\mathrm{I}\mathrm{n}\mathrm{d}\mathrm{e}\mathrm{x}\left({Q^r{(\mathrm{K}}^r)}^T\right)$$

(2)

where ${\mathrm{K}}^g, {\mathrm{V}}^g\in {\mathrm{R}}^{S^2\times {\displaystyle \frac{k\times H\times W}{S^2}}\times C}$ are the gathered key and value tensors, respectively. $Q^r,{ K}^r\in R^{S^2\times C}$. See Figure 6.

The attention:

$$O=\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}({\mathrm{Q},\mathrm{K}}^g,{\mathrm{V}}^g)+\mathrm{LCE}\left(\mathrm{V}\right)$$

(3)

Here, the function LCE(·) was parameterized with depthwise convolution.

2.2.2. SegFormer Network

SegFormer [28] is an efficient and reliable semantic segmentation model that combines transformers with lightweight multilayer perceptron (MLP) decoding. SegFormer includes a hierarchically structured transformer encoder. The output feature size of each transformer layer decreases layer by layer. In this way, feature information of different scales is captured, i.e., coarse-grained large-scale and fine-grained small-scale features are extracted. Instead of using complex decoders, SegFormer is a simple and lightweight decoder. The proposed MLP decoder merges information from different layers and thus combines local and global attention to provide forceful representations. Therefore, it is very suitable for street tree objects with large differences in global and local features.

Heads A, B, and D have the same dimensions. N(N = H × W) × C. The self-attention is as follows:

$$\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(A, B, D)=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\displaystyle \frac{A{B}^{\mathrm{T}}}{\sqrt{d_{head}}}}\right)B$$

(4)

In the attention calculation, we use the downsampling rate R to process B as follows:

$$\begin{array}{c}\widehat{B}=\mathrm{Reshape}({\displaystyle \frac{N}{R}},C·R)(B)\\ B^{\prime }=\mathrm{Linear}(C·R, C)(\widehat{B})\end{array}$$

(5)

where $B$ is the mapping feature of the input, $\widehat{B}$ is the feature after $B$ dimension transformation, and $B^{\prime }$ is the feature after dimension reduction.

The Mix-FFN (feed-forward network) is as follows:

$$X_{out}=MLP\left(GELU\right({Conv}_{3\times 3}\left(MLP\right(x_{in}\left)\right)\left)\right)+x_{in}$$

(6)

where x_in is the feature from the self-attention module. The Mix-FFN mixes a 3 × 3 convolution and an MLP into each FFN.

In the lightweight ALL-MLP decoder part, first, we unify the channel dimension:

$$\widehat{F_i}=\mathrm{L}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{r}(C_i,C)(F_i),{\forall }_i$$

(7)

where i∈{1, 2, 3, 4}, and F_i denotes multilevel features.

Then, the features are upsampled and concatenated:

$$\widehat{F_i}=\mathrm{U}\mathrm{p}\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}({\displaystyle \frac{W}{4}}\times {\displaystyle \frac{W}{4}})(\widehat{F_i}),{\forall }_i$$

(8)

Next, the concatenated features F are

$$F=\mathrm{L}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{r}(4C,C)(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}(\widehat{F_i})),{\forall }_i$$

(9)

Finally, we predict the segmentation mask M:

$$M=\mathrm{L}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{r}(C,N_{cls})(F)$$

(10)

where M has a ${\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}\times N_{cls}$ resolution, and N_cls is the number of categories.

3. Results and Analysis

3.1. Experimental Environment and Evaluation Indicators

The experimental environment consists of an Intel Core i7-12700KF CPU, 64 GB RAM, 10 GB of video memory, an Nvidia GeForce RTX3090Ti GPU, CUDNN V11.7, the CUDA Toolkit 9.0, the Ubuntu 22.04 operating system, and Python 3.7.12.

The precision, recall, and mean average precision (mAP) are used as the object detection evaluation indices [27]. The intersection over union (IoU), mean intersection over union (mIoU), pixel accuracy (PA), and mean pixel accuracy (mPA) are used as segmentation evaluation indices [28]. Specific definitions are provided in the corresponding research literature.

3.2. YOLOv8m+BiFormer Model Results and Analysis

The YOLOv8 series are YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x. For the urban street tree image object detection task, the hyperparameters were set to 200 epochs, a batch size of 32, an initial learning rate of 0.01, momentum of 0.937, and a weight attenuation coefficient of 0.0005. Then, the trained model was evaluated on the validation and test sets.

Figure 7A–D exhibit the loss function curves on the train and validation sets. These loss function curves decrease gradually to flatness, meaning that the models converge. Figure 7E–H display the results of the object detection evaluation indices on the validation set. These curves show that the network converges quickly and tends to be stable.

In this work, five models of YOLOv8 were trained and verified using a large public urban street tree dataset. The details are provided in

Table 2. Different scale models of the YOLOv8 series.

Model	Size (Pixels)	Params (M)	FLOPs (B)	Layers	Preprocess (ms)	Inference (ms)	Postprocess (ms)
YOLOv8n	640	3.0	8.2	225	1.6	0.6	1.6
YOLOv8s	640	11.1	28.7	225	0.5	1.0	0.8
YOLOv8m	640	25.8	79.1	295	1.3	2.1	1.1
YOLOv8l	640	43.6	165.5	365	0.5	3.7	1.3
YOLOv8x	640	68.2	257.5	365	0.6	5.5	0.3
YOLOv8m+BiFormer	640	36.3	178.5	306	0.2	2.3	0.5

and

Table 3. Object detection performance of the YOLOv8 series on the validation set.

Model	Recall	Precision	mAPval50	mAPval50-95
YOLOv8n	96.9	97.1	99.1	91.8
YOLOv8s	97.1	98.1	99.3	93.2
YOLOv8m	98.2	97.6	99.4	93.9
YOLOv8l	99.4	97.5	99.5	94.3
YOLOv8x	97.5	98.4	99.5	94.1
YOLOv8m+BiFormer	98.9	97.5	99.4	94.6

. The results show that the YOLOv8m model values for the params (M), layers, preprocessing (ms), inference (ms), and postprocessing (ms) parameters were superior to those of YOLOv8l and YOLOv8x, and the object detection evaluation indicators were comparable. Compared with those of the smaller YOLOv8n and YOLOv8s models, the preprocessing and postprocessing values of YOLOv8m decreased rather than increasing, and the mAP^val50 and mAP^val50-95 improved, as detailed in Table 3.

Table 2 shows that the params, FLOPs, layers, preprocessing, inference and postprocessing values of the YOLOv8m+BiFormer model were much lower than those of YOLOv8l and YOLOv8x. As shown in Table 3, the mAP^val50-95 values also increased by 0.3 and 0.5 percentage points, respectively. The experimental results demonstrate that the YOLOv8m+BiFormer model can still achieve high object detection performance while reducing the number of parameters and layers. Moreover, this approach greatly reduces the preprocessing, inference, and postprocessing speeds.

Therefore, we specifically selected YOLOv8m, which has a moderate model size but relatively high detection accuracy, as the main network of the object detection module. The loss function of YOLOv8m+BiFormer on the training and validation sets and the change curves of the object detection indicators on the validation set are shown in Figure 7.

To display the performance of the object detection models more directly, Figure 8A shows thermal map examples of the YOLOv8 series models and YOLOv8m+BiFormer in the training process to reveal the intensity distribution of the detected targets. According to the thermal images of layer 9 in Figure 8A, most of the YOLOv8 series models mistakenly captured the left target, while the YOLOv8m+BiFormer model was better able to capture the target region. In the thermal images of layer 10, the color highlight region of the YOLOv8m+BiFormer model is larger and more closely fits the tree crown region than the other models, indicating that the model is more accurate in locating the targets of interest in the image. In the thermal images of layer 12, there are multiple scattered thermal regions, while our model focuses more on capturing the slender trunk region. The thermal region of our YOLOv8m+BiFormer model is larger and closer to the target region in the images; in other words, the region that attracts the attention of the network is relatively more accurate.

Figure 8B displays examples of the results of the different object detection models on the street tree test set. Combined with Figure 8A,B, it is found that the trunk is an important false detection region. The thermal value of the trunk in the thermal map is relatively low, so the trunk region object detection effect in the sample images is not ideal. However, after the BiFormer attention mechanism is fused, the trunk region detection performance significantly improves.

3.3. Results and Analysis of the Different Segmentation Models

In this paper, an approach for urban street tree image segmentation based on the improved YOLO-SegNet is proposed. Firstly, the YOLOv8m+BiFormer model was introduced to conduct the object detection task, and then the object detection results were fed into the SegFormer model for the segmentation task. To confirm the segmentation performance of our YOLO-SegNet method, we carried out FCN [44], LR-ASPP [45], PSPNet [46], UNet [25], DeepLabv3+ [47], HRNet [48], and SegFormer [28] experiments. For the urban street tree segmentation task, the hyperparameters were set as follows: 200 epochs, 8 batches, an initial learning rate of 0.01, momentum of 0.9, and a weight attenuation coefficient of 0.0001. Then, the trained model was evaluated on the street tree validation and test datasets. Figure 9A shows the training loss function curves of the segmentation models without the object detection module. Figure 9B shows the training loss function curves of the segmentation models with the object detection module. The loss function curves of these models decline steadily and flatten, indicating that these models converge.

These results prove that the segmentation performance of almost all of these models based on the YOLOv8m+BiFormer module is better than that of the original models on the street tree segmentation dataset. The specific comparison results are shown in

Table 4. Segmentation results of different models on the validation set.

	Model	mIoU	mPA	f_Score
Not based on YOLOv8m+BiFormer module	FCN [44]	81.5	-	-
	LR-ASPP [45]	82.3	-	-
	PSPNet [46]	87.0	92.7	95.8
	UNet [25]	85.2	91.7	85.3
	DeepLabv3+ [47]	87.5	92.5	97.0
	HRNet [48]	89.3	94.7	97.0
	SegFormer [28]	91.0	95.4	97.8
Based on YOLOv8m+BiFormer module	YOLO-FCN	82.8	-	-
	YOLO-LR-ASPP	84.8	-	-
	YOLO-PSPNet	87.9	93.3	97.0
	YOLO-UNet	84.7	91.4	88.6
	YOLO-DeepLabv3+	88.3	93.4	97.6
	YOLO-HRNet	90.5	95.3	97.4
	Ours: YOLO-SegNet	92.0	95.9	98.0

. Compared to these original models without the YOLOv8m+BiFormer module, the mIoUs of these models based on YOLOv8m+BiFormer, e.g., YOLO-FCN, YOLO-LR-ASPP, YOLO-PSPNet, YOLO-DeepLabv3+, YOLO-HRNet, and YOLO-SegNet, increased by 1.3, 2.5, 0.9, 0.8, 1.2, and 1.0 percentage points, respectively. Additionally, compared with those of the FCN, LR-ASPP, PSPNet, UNet, DeepLabv3+, and HRNet models, the mIoUs of YOLO-SegNetSegFormer increased by 10.5, 9.7, 5.0, 6.8, 4.5, and 2.7 percentage points, respectively. Moreover, the values of mPA and the f_score also improved. This also indicates that the performance of our YOLO-SegNet model is significantly better than that of other models.

Compared with those of the original SegFormer model, the mIoU, mPA, and f_score indices of our YOLO-SegNet model are 1.0, 0.5, and 0.2 percentage points greater, respectively. The important index value mIoU is increased by 1.0 percentage points, which also indicates that the YOLOv8m+BiFormer module introduced is beneficial to the SegFormer model for the segmentation of street trees. In addition, we also conducted experiments on the street tree test set.

Table 5. Segmentation results of different models on the test set.

	Model	mIoU	mPA
Not based on YOLOv8m+BiFormer module	FCN	71.3	-
	LR-ASPP	76.6	-
	PSPNet	80.6	88.6
	UNet	81.2	89.6
	DeepLabv3+	83.2	90.7
	HRNet	88.1	93.5
	SegFormer	87.3	93.1
Based on YOLOv8m+BiFormer module	YOLO-FCN	79.3	-
	YOLO-LR-ASPP	83.4	-
	YOLO-PSPNet	81.1	88.9
	YOLO-UNet	79.7	88.5
	YOLO-DeepLabv3+	86.0	92.6
	YOLO-HRNet	87.7	94.0
	Ours: YOLO-SegNet	88.4	93.7

shows the performance of the different segmentation models on the test set. The segmentation model based on the YOLOv8m+BiFormer module is significantly improved compared with the original models.

4. Discussion

4.1. YOLOv8m+BiFormer Module

This part further analyzes the the street tree object detection module and image segmentation module. In the object detection task, YOLOv8m, which has a moderate model size and good detection performance, is selected as the main network [27,49,50]. As shown in Table 2, the parameters, layers, preprocessing, inference, and postprocessing values of the YOLOv8m model are superior to those of YOLOv8l and YOLOv8x, and the obtained object detection evaluation indices are comparable. Compared with those of the smaller YOLOv8n and YOLOv8s models, the preprocessing and postprocessing speeds of YOLOv8m decreased rather than increased, and the mAP^val50 and mAP^val50-95 improved. On this basis, we introduce the BiFormer attention mechanism to further improve the object detection accuracy [26,33,34]; this can help networks to capture long-distance context dependencies.

To confirm the effectiveness of the YOLOv8m+BiFormer model further [51], the following object detection experiments were conducted: YOLOv8n+BiFormer, YOLOv8s+BiFormer, YOLOv8l+BiFormer, and YOLOv8x+BiFormer. The experimental results show that the YOLOv8m+BiFormer model is superior to the YOLOv8 series of different scale models based on the BiFormer, as detailed in

Table 6. Comparison of the YOLOv8 series models based on the BiFormer attention mechanism.

Model	Size (Pixels)	Params (M)	FLOPs (B)	Layers	Preprocess (ms)	Inference (ms)	Postprocess (ms)
YOLOv8m [27]	640	25.8	79.1	295	0.1	1.6	0.9
YOLOv8n+BiFormer	640	3.3	18.6	236	1.8	1.2	2.3
YOLOv8s+BiFormer	640	12.2	70.0	236	0.6	1.2	0.7
YOLOv8l+BiFormer	640	80.9	357.4	376	0.4	4.5	1.1
YOLOv8x+BiFormer	640	126.4	557.9	376	0.2	6.3	0.7
Ours: YOLOv8m+BiFormer	640	36.3	178.5	306	0.2	2.4	0.5

and

Table 7. Comparison of the detection performance on the validation set of YOLOv8 series models based on the BiFormer attention mechanism.

Model	Recall	Precision	mAPval50	mAPval50-95
YOLOv8m [27]	98.2	97.6	99.4	93.9
YOLOv8n+BiFormer	98.1	97.1	99.4	92.5
YOLOv8s+BiFormer	97.9	96.9	99.3	93.4
YOLOv8l+BiFormer	98.6	98.9	99.5	94.9
YOLOv8x+BiFormer	98.4	98.9	99.5	94.9
Ours: YOLOv8m+BiFormer	98.9	97.5	99.4	94.6

. The YOLOv8m+BiFormer model’s size is moderate, the preprocessing and postprocessing models are the fastest, and the difference between the important index values mAP^val50 and mAP^val50-95 is small compared with those of YOLOv8l+BiFormer and YOLOv8x+BiFormer. Moreover, compared with that of the original BiFormer model, the mAP^val50-95 of our YOLOv8m+BiFormer model increased by 0.7 percentage points.

4.2. YOLO-SegNet Network

Regarding the segmentation task, Section 3.3 analyzes the performance of the different segmentation models overall. However, to some extent, these average indices cannot directly reflect the segmentation results for different street tree species. Therefore, in this paper, we analyze the segmentation results of different methods on the validation set and the test set to evaluate the segmentation performance of the YOLO-SegNet model more comprehensively.

Figure 10 depicts the segmentation performance of different models in detail with a radar diagram, which can intuitively show the segmentation differences of different models for the same street tree species. Figure 10(A₁) shows the distribution of the IoU index values of the different segmentation models on the validation set. The IoU values of the YOLO-SegNet model almost exceed those of the other segmentation models. Compared with the other models, the YOLO-SegNet model significantly improved the IoU values of the Acer palmatum, Elaeocarpus decipiens, Koelreuteria paniculata, Liriodendron chinense, Platanus, and Sapindus saponaria street tree species. Figure 10(A₁,A₂) shows that the segmentation performance of the classical PSPNet, UNet, and DeepLabv3+ models listed in this paper are poor for street tree species such as Acer palmatum, Elaeocarpus decipiens, and Liriodendron chinense. However, the IoU and PA values of our method are significantly improved. In addition, compared with those of the original models, the mIoU values of the YOLO-HRNet model improved, and the IoU values of tree species such as Acer palmatum, flowering cherry, and Zelkova serrata trees significantly increased. In conclusion, the mIoU and mPA overall segmentation index values of our YOLO-SegNet model on the validation set and the individual IoU, PA, and accuracy values of different tree species demonstrate the effectiveness of the model, especially for some tree species that are not easy to segment, such as Liriodendron chinense. This is because the canopy of this tree species is relatively sparse, and the texture is more complex.

Figure 11 shows the results of different segmentation models on the test set, where (A) and (C) are evergreen street trees with dense canopies, (B) is deciduous trees with relatively sparse canopies, and (D) is deciduous trees with multiple branches. The original classical segmentation model has a good segmentation effect for street trees with dense canopies but a relatively poor segmentation effect for trees with sparse canopies. Additionally, the segmentation effect for trunk parts is poor. The classical models based on YOLOv8m+BiFormer also improve the segmentation indices of the street tree trunk, as shown in Figure 11(A₅,A₁₁,C₂,C₈,D₂,D₈). This is because the object detection module conducts the preliminary detection of the trunk and eliminates non-target regions, especially the white region of the trunk, which is not considered in this paper. Moreover, the segmentation models based on the object detection module can improve the segmentation of street trees with sparse canopies to a certain extent, as shown in Figure 11(B₄,B₁₀).

5. Conclusions

In this work, the object detection model based on the YOLOv8m+BiFormer was introduced to remove the image background. On this basis, a SegFormer network tree segmentation method was proposed to obtain tree edge information more efficiently. Combined with the proposed YOLOv8m+BiFormer and SegFormer networks (YOLO-SegNet), a new model was proposed to effectively eliminate complex backgrounds and was very beneficial for urban street tree segmentation. The experimental results show that our method obtains an mIoU of 92.0% on the validation set and an mIoU of 88.4% on the test set. Our approach displays higher accuracy than other models. Therefore, our method accurately segments single trees, providing support for the calculation of the tree height and crown length and tree species identification. In subsequent studies, we will expand the tree dataset to verify the robustness of our method.