Deep Learning for Urban Tree Canopy Coverage Analysis: A Comparison and Case Study

Abstract

Urban tree canopy (UTC) coverage, or area, is an important metric for monitoring changes in UTC over large areas within a municipality. Several methods have been used to obtain these data, but remote sensing image classification is one of the fastest and most reliable over large areas. However, most studies have tested only one or two classification methods to accomplish this while using costly satellite imagery or LiDAR data. This study seeks to compare three urban tree canopy cover classifiers by testing a deep learning U-Net convolutional neural network (CNN), support vector machine learning classifier (SVM) and a random forests machine learning classifier (RF) on cost-free 2012 aerial imagery over a small southern USA city and midsize, growing southern USA city. The results of the experiment are then used to decide the best classifier and apply it to more recent aerial imagery to determine canopy changes over a 10-year period. The changes are subsequently compared visually and statistically with recent urban heat maps derived from thermal Landsat 9 satellite data to compare the means of temperatures within areas of UTC loss and no change. The U-Net CNN classifier proved to provide the best overall accuracy for both cities (89.8% and 91.4%), while also requiring the most training and classification time. When compared spatially with city heat maps, city periphery regions were most impacted by substantial changes in UTC area as cities grow and the outer regions get warmer. Furthermore, areas of UTC loss had higher temperatures than those areas with no canopy change. The broader impacts of this study reach the urban forestry managers at the local, state/province, and national levels as they seek to provide data-driven decisions for policy makers.

1. Introduction

As urban land in the United States continues to expand, urban forests will continue to play a substantial role in the overall health of urban and rural areas alike [1,2]. Urban forests, defined as individual or clusters of trees within or adjacent to an urban setting, provide several well-documented economic, ecological, and health benefits for residents of the urban environments [3,4]. For example, urban trees provide shade and mitigate the effects of the urban heat island effect [5,6]. They also combat particle air pollution and CO₂ emissions by sequestering carbon [7,8], are linked to positive effects in human psychology in urban environments [9,10,11], and reduce stormwater runoff [12,13]. Despite their enormous benefits, urban tree canopy (UTC) coverage is in decline as considerable urban growth in some regions has necessitated the removal of urban forest, leaving some cities struggling with the urban heat island (UHI) effect and other challenges [2]. Managing UTC necessitates accurate measurement and inventory of urban trees within a city, and many municipalities within the Unites States of America (USA) have created organizations or branches of government to oversee the changing urban landscapes [14].

While tagging and inventorying trees in the field creates a valuable and accurate catalog, the process consumes significant time and monetary resources. Many have turned to less time-intensive methods, such as using remote sensing technologies, to provide a review of the urban tree status [15,16]. Remote sensing technologies, such as light detection and ranging (LiDAR), unoccupied aerial vehicles (UAVs), and aerial and satellite imagery, have been used to inventory individual trees or tree canopy coverage in towns and cities [17,18]. LiDAR data provide accurate representation of tree canopies, but can be costly and require significant computational resources for processing large datasets [19,20,21]. UAV data are captured in very high spatial resolutions that is effective in identifying trees and tree canopy, but it is difficult to capture data large spatial extents, i.e., an entire municipality [22,23,24]. Satellite and aerial imagery are being increasingly used in the context of mapping urban tree canopies because of the increasingly high spatial resolution and wide availability [25,26,27,28].

Recent advancements in computer science have increased the applicability of both machine learning and deep learning image classifiers for remote sensing applications [29,30,31]. State-of-the-art machine and deep learning models are available to UTC managers to quickly obtain accurate information regarding tree canopy coverage and how the coverage is changing over time. Accuracy is of the utmost importance as these data are used for legislative decision making and funding [14]. These models are now being offered as tools in accessible and easy-to-use ways through popular GIS software, such as ArcGIS Pro. Recently, scientists have found it possible to detect individual trees or calculate individual tree canopies within UTC coverage for large areal extents with both machine learning and deep learning classifiers and detectors. Yan et al. [32] compared convolutional neural networks (CNNs), support vector machine (SVM) and random forests (RF) models for UTC mapping and found CNNs performed the best. However, they used expensive satellite imagery and investigated delineating individual tree canopies in a city instead of total UTC area within an urban environment. Lv et al. [33] compared several CNNs, including one developed by their own team, but once again sought individual tree segmentation and failed to include machine learning classifiers in their comparison. Zamboni et al. [34] likewise surveyed a large number of deep learning models for individual tree detection but did not use machine learning classifiers. Wang et al. [35] compared a U-Net deep learning model to object-based classification and found U-Net to be a superior classifier for UTC coverage area, but again did not compare the results to machine learning classifiers. Despite the successes of machine and deep learning classifiers on their own, further investigation on the worth of deep learning and machine learning from a practical perspective is important. Deep learning classifiers can require more time to train and classify than machine learning classifiers, suggesting that small differences in overall accuracy may not be worth the longer processing times [36]. Application-based studies, like [37], have created a comprehensive set of UTC data for the entire state of Wisconsin using machine learning algorithms, but the results were not compared with deep learning algorithm results. In many of the application studies, multispectral satellite data or LiDAR data are combined with aerial imagery to enhance the classification [27,28,38,39]. Therefore, comparisons with less complex and still reliable machine learning classification models, like SVM and RF, are needed on simple optical imagery readily available from a management and policy-making perspective.

The present literature on urban tree canopy mapping and UHI have focused on highly populated urban centers at the cost of neglecting smaller or mid-sized urban centers. This study is the first of its kind to compare three machine and deep learning classifiers, U-Net deep learning, support vector machine, and random forests machine learning, for mapping UTC coverage in two cities of different size and shape in the USA from the perspective of an urban tree forestry manager. From this perspective, cost, computational time, and accuracy are the most important considerations. For this purpose, free National Agriculture Imagery Program (NAIP) imagery datasets are used to provide an alternative to the costly satellite, LiDAR, and UAV data that many others use for UTC managers. Commonly used ArcGIS software and desktop computer specifications are used for analysis to simulate the resources available to urban forestry managers. Following the comparison, a change detection analysis is presented as a case study to reveal where and how tree canopies in the two diverse cities of interest, Laurel, Mississippi (MS) and Georgetown, Texas (TX), USA, have changed over a 10-year period. Temperature maps derived from Landsat 9 thermal bands are also used to reveal the location of high temperatures in relation to tree canopy gain or loss. Such land surface temperature mapping and subsequent comparisons with tree canopy coverage is common among research projects as many seek to apply the technology to identify or solve many concerns in urban areas. In fact, several studies are now working to model changes of LST in the future using data from the past now that we have sufficiently large data histories [40,41]. Many are working hard on specific applications in large cities around the world, but few are investigating smaller cities [42,43,44]. We aim to expand the literature in this area as well by looking at smaller regions and compare with deep learning data, which has not yet been conducted.

We hypothesize that deep learning, as available through accessible and widely used GIS software packages, will provide the best overall accuracy results and suggest that NAIP data alone is sufficient to provide accuracies similar to other studies that use multiple datasets. Furthermore, we suggest that tree canopy loss will be more prevalent over a 10-year period for Georgetown, TX because of its rapid growth and it will be visually correlated with high temperatures in the regions of canopy loss. Section 2 of this article will introduce the study areas that make this study unique and discuss methods for classifying data and generating temperature maps. Section 3 will showcase the results of this study, while Section 4 will discuss the strengths of our approach as well as limitations. Finally, Section 5 concludes by sharing the most important information.

2. Materials and Methods

2.1. Study Areas

This study identified two cities of interest to investigate urban tree canopy coverage in the United States (Figure 1). The first of these two cities, Laurel, Mississippi, USA, represents an older and smaller urban environment. Laurel is a town of just over 17,000 people in southern Mississippi and was established as a lumber town in 1882 [45]. Situated in a humid subtropical climate, mean high temperatures can rise as high as 36 degrees Celsius in July and August and fall as low as 23.8 degrees Celsius in January [46]. Recently, Laurel has been the host of the popular home renovation television show Home Town on Home and Garden Television [47]. It has also been a designated “Tree City, USA” for over 40 years now, the first city in the state of Mississippi with that title [48]. Laurel’s geographic boundaries are fairly uniform in nature, without fragmented annexes and appendages, and encircle 40.8 km² of city, airport, and surrounding industrial zones and forest.

Georgetown, Texas, a suburb of Austin, is representative of a mid-sized city. It has a population of 75,420. In 2022, Georgetown was the fastest-growing city by percent change with an impressive population increase rate of 14.4% [49]. Known for being a warm city, Georgetown is also in a humid subtropical climate where mean high temperatures range from 25.5 degrees Celsius to 36 degrees Celsius. The city area is a fragmented, complex, and unique 156.96 km². Within city boundaries, special trees titled “Heritage trees” are protected and require a permit to prune or remove [50]. However, this is a small portion of the total UTC within the city and unlike Laurel, Georgetown is not designated as a “Tree City, USA”.

2.2. Remote Sensing Data

2.2.1. NAIP Imagery Data

The National Agriculture Imagery Program (NAIP) began in 2002 and is administered by the U.S. Department of Agriculture (USDA) Farm Service Agency. The goal of the program is to collect aerial imagery during leaf-on conditions. NAIP imageries are collected using digital sensors placed on aircraft that meet rigid calibration specifications [51]. NAIP imagery is generally collected at a 1 m spatial resolution and as true color (RGB) imagery, though more recent imagery has increased spatial resolutions (50–60 cm) and is additionally presented as color-infrared (CIR) imagery. NAIP imagery flights are conducted at variable altitudes with a variety of aircraft and minimum side overlap of 30%. Data management, georegistration, and quality control are all managed by the imagery program. NAIP imagery has been used for urban tree canopy in several other studies [14,37,52].

For this study, four NAIP imagery datasets were acquired and masked to each municipality’s boundaries for further processing. Laurel, Mississippi datasets were acquired for 2012 and 2023, while Georgetown, TX data were gathered for 2012 and 2022. Images for Laurel were transformed into NAD83 (2011) Universal Transverse Mercator (UTM) Zone 16N, and Georgetown images were transformed into NAD83 (2011) UTM Zone 14N for further processing (

Table 1. NAIP image data characteristics.

	2012 Georgetown Image	2022 Georgetown Image	2012 Laurel Image	2023 Laurel Image
Dates	25 June 2012	11 June 2022	22 August 2012	11 October 2023
Pixel size	1 m	60 cm	1 m	60 cm
Flight altitude	9052 m	4875 m	9144 m	4470 m
Aircraft	Cessna 441	Cessna Conquest and Cessna 414	Cessna Conquest	Cessna 441
Sensor	Leica Geosystems ADS80/SH82 digital sensors	Leica ADS100 pushbroom sensor	Integraph Digital Mapping Camera	Leica Geosystem’s ContentMapper digital sensors
Masked image size	1.95 gb	30.15 gb	251 mb	698 mb

).

2.2.2. LANDSAT Data

Landsat 9 is the most recent iteration of the LANDSAT earth observation missions from NASA, having launched 27 September 2021. Landsat 9 collects a new image at each location on Earth every 16 days, resulting in the potential for cloud-free imagery at regular intervals. Cloud free Landsat tiles were collected for each city of interest on a date as close to the NAIP acquisition as possible. Unfortunately, cloudy images were often found in the image acquisition dates directly adjacent to the NAIP data acquisition dates. Nevertheless, imagery captured at dates as close to ideal were gathered and processed. The Georgetown image was gathered from 28 May 2022 and the Laurel LANDSAT image was captured 09/23/2023. The Landsat 9 thermal band 10 was predominately used, along with bands 4 (red) and 5 (NIR) for NDVI and data processing. Band 10 is collected by the Thermal Infrared Sensor 2 (TIRS-2), whose center wavelength is 10,800 nm and spatial resolution is 100 m. The band 10 image was resampled to 30 m for the purpose of this study.

After acquiring the Landsat 9 tiles from the Unites States Geological Survey EarthExplorer website (https://earthexplorer.usgs.gov, accessed on 1 April 2024), the large images were clipped to the extents of each city. Further processing was needed to create surface temperature maps using bands 4, 5, and 10. To convert the raw values into surface temperature, several steps were required. First, raw values were converted to top of atmosphere (TOA) spectral radiance. This was achieved using Equation (1):

TOA (L) = M_L * Q_cal + A_L

(1)

where M_L is the radiance multiplication factor for Band 10 found in the metadata of Landsat 9 images, Q_cal is the band itself, and A_L is the radiance addition factor found in the metadata as well.

After calculating TOA, we can calculate the brightness temperature from the following Equation (2):

BT = (K2/(ln(K1/L) + 1)) − 273.15

(2)

where K1 is calibration constant 1 provided in the metadata, K2 is calibration constant 2 which is also provided in the metadata, and L is the spectral radiance or TOA layer in (Watts/(m²∗sr∗μm)). Subtracting 273.15 converts the results from degrees kelvin to degrees Celsius.

To improve the temperature map, further steps are needed. First, the normalized difference vegetation index (NDVI) is required to continue and is calculated from Bands 4 and 5 using the traditional equation for NDVI, found here as Equation (3):

NDVI = float(Band 5 − Band 4)/float(Band 5 + Band 4)

(3)

where Band 5 is the NIR band and band 4 is the red band on Landsat 9. The NDVI is then used as an input to find the proportion of vegetation (Pv). Pv is calculated using the previously calculated NDVI band in Equation (4):

Pv = ((NDVI − NDVImin)/(NDVImax + NDVImin))

(4)

Emissivity is then calculated based on the Pv just obtained using the equation:

E = 0.004 * Pv + 0.986

(5)

Finally, land surface temperature (LST) can be calculated using several of the variables just obtained for our area of interest using the Landsat 9 bands. The final LST band was calculated using Equation (6):

LST = (BT/(1 + (0.00115 * BT/1.4388) * ln(E)))

(6)

where BT is the brightness temperature layer, and E is the emissivity layer. The final result was a LST layer in degrees Celsius where we could identify spatial correlations between heat and tree canopy loss.

To assess the quality of the LST products, quality assessment (QA) data were included in the surface temperature data download from EarthExplorer. In this raster band, each pixel represents the level of uncertainty in the temperature values. This band was overlayed on the temperature raster results to indicate where higher levels of temperature uncertainty exist. In Figure 2, the areas of higher uncertainty have a pink tint and stands out against the LST map background. We found that the centers of the cities were the least affected, and the most affected regions are where water bodies existed. The water regions were removed from analysis prior to classification and case study analysis. Upon assessment, the temperature looked to be fairly underestimated, but spatial patterns were consistent with what was expected so the maps were used for analysis.

2.3. Classification and Accuracy Assessment

A general workflow of the experiment and case study analysis is presented in Figure 3. Three remote sensing image classifiers are compared, and the best performing classifier is used to classify more recent imagery, eventually resulting in a post-classification change detection and surface heat and tree canopy spatial correlation analysis.

2.3.1. Classifiers

Two traditional machine learning classifiers and a commonly used deep learning classifier are compared in this study by applying each to the same datasets with the same training data on the 2012 imagery collected for both Laurel and Georgetown. The U-Net CNN is one of the most commonly used deep learning image classifiers found in GIScience and is easy to access for UTC mangers through ESRI ArcGIS Pro 3.1.3 [53]. Originally designed for biomedical image segmentation, its applications in remote sensing image analysis are broad but now include pixel-level image classification [54]. U-Net gets its name from its U-shaped design—the architecture of the network is divided into an encoding and decoding side. U-Net has been previously used for individual tree crown extraction [35,55] and with LiDAR Data and multispectral imagery for UTC coverage mapping from UAV data [23]. It has also shown success in modified versions of U-Net for mapping UTC [25] and other pixel-based classifications in a variety of environmental applications [36,56,57].

The two traditional machine learning-based classifiers used for this study are support vector machine (SVM) and random forest (RF) classifiers. They are frequently found in the literature as high-performing image classification models ([58], p. 337). SVM is based on statistical learning theory and can be used for both linear and nonlinear classification, regression, and a series of other tasks. In its simplest form, SVM strives to find a hyperplane between the classes (it can handle many) in the target feature to separate them. Additionally, SVM classifiers do not need to have normally distributed or large training samples to perform well [59]. For this study, we used the support vector machine classifier with the maximum number of samples per class set as 1000. The segment attributes used for classification were active chromaticity color, mean digital number, and compactness.

Random forests (also called random trees in ArcGIS Pro) is also a supervised classification method that relies on a series of decisions being made based on the statistical characteristics of the training dataset and image. The decisions all stem from past decisions to make what looks like a tree when represented graphically. During the process of classification, many decision trees are made (i.e., a forest is grown) and the most frequently created output is used for the final classification [58]. Of the three classifiers used for this comparative study, the ‘best’ classifier will be determined by overall accuracy (OA), kappa coefficient, processing time, and ease of use. For the RF classifier, we used 150 as the maximum numbers of trees, 60 maximum tree depth, and 1000 maximum number of samples per class. The segment attributes used for classification once again were active chromaticity color, mean digital number, and compactness.

2.3.2. Classification

The same 2012 NAIP orthomosaic images were classified three times by each classifier for both Laurel and Georgetown. The NAIP imagery for 2012 clipped was to the municipality boundaries (Figure 4). Training samples and validation samples were first collected using ArcGIS Pro 3.1.3 for each of the study areas using the high resolution NAIP imagery and prior knowledge of each city. In order to increase our focus on obtaining the most accurate results of UTC area and keep classifications simple, only three classes were specified for the classifications. After testing a variety of classes, the most effective and efficient classification occurred with a tree class, grass class, and urban/barren class. Prior to classification, water bodies within municipality boundaries were masked and removed. Because NAIP imagery was clipped to municipality boundaries, only the three classes were present and relevant to the study at hand. A total of 156 total training polygons were collected for Laurel before training the classifier, representing a total area of 1.06 km². A total of 267 samples were obtained for Georgetown as well, resulting in a total area of 1.41 km². All samples were randomly obtained in a variety of regions around the study areas in order to fairly represent the diversity of grasses, urban structures, and trees found throughout the cities.

The collected training polygons were once again used in ArcGIS Pro 3.1.3 to train each of the U-Net CNN, SVM, and RF classifiers. Each classification was conducted within the ArcGIS Pro graphical user interface environment to replicate the experience urban forestry managers would have when attempting similar remote-sensing based inventories.

2.3.3. Accuracy Assessment

Accuracy was assessed for the 2012 and 2022/23 classifications using 182 validation samples for Laurel and 282 polygons for Georgetown collected separately from the training polygons. No overlap between the two samples were used to ensure fair assessment (Figure 4). Four remote sensing image classification accuracy statistics, users’ accuracy, producer’s accuracy, overall accuracy (OA), and kappa coefficient, were used to assess each classification. The producer’s accuracy is the total number of pixels classified correctly for a class based on the validation data, divided by the total pixels for that class. The user’s accuracy is the number of correctly classified pixels divided by the number of pixels classified into any particular class. All four statistics were seamlessly calculated using the ArcGIS Pro 3.1.3 compute confusion matrix tool.

The confusion matrix tool was used to first generate a stratified random sample of points dispersed equally throughout the validation polygons. Then fields in the accuracy assessment points were populated from the ground truth, or validation data, as well as the classified raster. Once the fields were complete, the tool computed the OA using the following equation:

$$\mathrm{OA}={\displaystyle \frac{{\sum }_{i=1}^{K}xii}{N}}$$

(7)

where x_ii represents a pixel classified correctly, N is the total number of pixels [60]. The kappa coefficient was likewise computed using the following equation:

$$\widehat{K}={\displaystyle \frac{N{\sum }_{i=1}^{K}xii-{\sum }_{i=1}^K(x_{i+}\times x_{+j})}{N^2-{\sum }_{i=1}^k(x_{i+}\times x_{+j})}}$$

(8)

where N is the total number of samples, k is the number of rows in the confusion matrix, x_ii is the number of observations in row i and column i, and x_i+ and x_+j are the marginal totals for row i and column j [61]. The results were presented in tables for the 2012 images and assessed to determine the best classifier. The best classifier was then used on the 2022 and 2023 images.

2.4. Case Study and Heat Mapping

In order to assess the tree canopy changes and potential impact on urban heat, the most accurate classifier was used to create an urban tree canopy map of Georgetown and Laurel in the years 2022 and 2023, respectively. New training and validation samples were digitized in ArcGIS Pro 3.1.3 by scientists familiar with the urban areas using the NAIP imagery. Once again, three classes, urban, tree canopy, and grass, were used for classification because of our specific interest in the tree canopy area. The classified images were used in a post-classification change detection, one of the most common change detection methods [62,63]. This is where two remote sensing images, representing T1 (Time 1) and T2 (Time 2), are classified into land cover classes first and one is subsequently subtracted from the other to show a change in a particular class over time. For this study, we were interested in the canopy change over the 10–11 year period between NAIP data collection dates.

The pixels representing changes in canopy, either from canopy to grass or canopy to urban/barren, were symbolized and overlaid onto the Landsat 9 heat maps generated previously. Using visual techniques described in [60], spatial correlations were identified and will be presented hereafter. To quantitatively assess temperature within the areas of canopy loss and the rest of the city boundaries, 1000 stratified random points were generated for both the areas of canopy loss and no or little change (500 points in each). Using the extract values to points ArcGIS tool, we obtained temperature values for each of the points and then compared the means of the two groups using SPSS software. The results of a independent samples t-test indicates if there is a statistically significant difference between the temperatures found in the areas of canopy change and the general areas of the area otherwise specified.

3. Results

3.1. Laurel, MS Classification Experiment

The Laurel, MS image map from 2012 was classified successfully using the U-Net CNN, SVM, and RF classifiers. The U-Net classifier took the most time, totaling 53 min and 11 s for both model training and classification. The SVM required a total of 8 min and 17 s, while the RF classifier only needed 4 min and 54 s. This was completed on a Dell Inspiron 5680 6-core Intel i7 central processing unit (CPU) desktop with 16 GB RAM, and a Nvidia GTX 1060 3 GB graphical processing unit (GPU). Classification times, OA, and kappa coefficient are reported in

Table 2. Laurel, MS classification accuracy assessment.

	U-Net	SVM	RF
Overall accuracy	91.4%	84.2%	76.2%
Kappa coefficient	0.871	0.763	0.644
Classifier training time	20 m 7 s	2 m 4 s	1 m 30 s
Classification time	33 m 4 s	6 m 13 s	3 m 24 s

.

Despite taking longer, the overall accuracy and kappa coefficients of the U-Net classifier outperformed the same metrics from the SVM and RF classifiers. Overall accuracy of the U-Net classifier reached 91%, while SVM was adequate at 84% and RF came in third at 76%.

Table 3. U-Net classifier confusion matrix for Laurel, MS.

	Grass	Developed	Forest	Total	User’s Accuracy
Grass	382	3	0	385	99%
Developed	18	397	82	497	80%
Forest	0	0	318	318	100%
Total	400	400	400	1200
Producer’s accuracy	96%	99%	80%

,

Table 4. SVM classifier confusion matrix for Laurel, MS.

	Grass	Developed	Forest	Total	User’s Accuracy
Grass	286	1	13	300	95%
Developed	0	396	59	455	87%
Forest	114	3	328	445	74%
Total	400	400	400	1200
Producer’s accuracy	71.5%	99%	82%

and

Table 5. RF classifier confusion matrix for Laurel, MS.

	Grass	Developed	Forest	Total	User’s Accuracy
Grass	205	1	15	221	93%
Developed	97	392	68	557	70%
Forest	98	7	317	422	75%
Total	400	400	400	1200
Producer’s accuracy	51%	98%	79%

reveal more detailed results pertaining to the users and producers’ accuracy for each classifier.

After close examination, the U-Net also performed the best overall between the user’s accuracy and producer’s accuracy for the forest class especially. Interestingly, the forest class was more often confused with the developed class than the grass class, despite the grass class having more similar RGB values to the forest class than the developed class. Similar to the overall accuracies and kappa coefficient, the SVM and RF classes followed U-Net in terms of forest class accuracy, in that order. Figure 5 visualizes the differences between the three classifier results and the original NAIP image within Laurel boundaries. There are some inconsistencies within each classified image, but the U-Net classifier resulted in tree canopy without the well-documented salt and pepper effect, or large areas of misclassification.

3.2. Georgetown, TX Classification Experiment

The Georgetown, TX classification required considerably more time for training and classification due to its complex geographic extent. The U-Net CNN once again required the most time at 5 h 5 min and 7 s, the SVM classifier required 13 min and two seconds, and the RF classifier required 14 min and 1 s. We hypothesize that the U-Net CNN required significant more time because of the complexity of the image segmentation process needed to generate individual tiles and process each individually. Classification times, OA, and Kappa coefficient are reported in

Table 6. Georgetown, TX classification accuracy assessment.

	U-Net	SVM	RF
Overall accuracy	89.8%	68.6%	71.3%
Kappa coefficient	0.8475	0.529	0.57
Classifier training time	13 m 53 s	4 m 16 s	3 m 48 s
Classification time	3 h 51 m 17 s	8 m 46 s	10 m 13 s

. We discovered that the U-Net classifier required a significantly longer time for classification, but only 9 min longer to train the classifier. The U-Net proved the time investment to be well worth it as the overall accuracy and kappa coefficients were well above the results reported for SVM and RF.

More specifically, each classifier performed differently for each class, as seen in

Table 7. U-Net classifier confusion matrix for Georgetown, TX.

	Grass	Developed	Forest	Total	User’s Accuracy
Grass	335	0	7	342	98%
Developed	44	376	26	446	84%
Forest	21	24	367	412	89%
Total	400	400	400	1200
Producer’s accuracy	84%	94%	92%

,

Table 8. SVM classifier confusion matrix for Georgetown, TX.

	Grass	Developed	Forest	Total	User’s Accuracy
Grass	207	2	5	214	97%
Developed	168	368	147	683	54%
Forest	25	30	248	303	82%
Total	400	400	400	1200
Producer’s accuracy	52%	92%	62%

and

Table 9. Random forest classifier confusion matrix for Georgetown, TX.

	Grass	Developed	Forest	Total	User’s Accuracy
Grass	212	0	10	222	95%
Developed	182	385	131	698	55%
Forest	6	15	259	422	93%
Total	400	400	400	1200
Producer’s accuracy	53%	96%	65%

.

Despite their rapid training and classification, it is apparent that the SVM and RF classifiers struggled with the geographically complex Georgetown, TX imagery. The U-Net classifier likewise was not as successful in classifying Georgetown tree canopy as it was for Laurel, MS, but it significantly outperformed the other two machine learning classifiers. Once again the developed class was most confusing for the forest class, but the U-Net classifier was able to navigate the difference between the two the best. This is visualized in Figure 6, where each classifier result is compared against the original NAIP image from 2012.

3.3. Case Study Results

Following the experiment results, we determined to use the U-Net CNN classifier for the 2022 and 2023 images for the study areas. The classifications were successful, and each required more time to complete to the classifications performed on the 2012 images (

Table 10. Case study classification results.

	Time	User’s Accuracy (Forest Class)	Producer’s Accuracy (Forest Class)	OA	Kappa Coefficient
Laurel, MS	1 h 45 m 32 s	91%	95%	95%	0.92
Georgetown, TX	42 h 15 m 30 s	81%	92%	83%	0.75

). However, they also resulted in similar accuracies for the forest classes, overall accuracies, and kappa coefficients. We suspect the longer processing times are related to the higher spatial resolution size (60 cm vs. 1 m) since the computer used for processing remained the same.

After the post-classification change detection was performed, the canopy changes were analyzed with the urban heat surface maps (Figure 7). Georgetown, the growing southern city, showed extensive tree canopy changes from 2012 to 2022. Laurel, on the other hand, showed less significant changes. Many of the smaller UTC changes may be considered false positive changes. These occur where tree canopy was misclassified during either the 2012 or 2022/2023 classifications and therefore considered a change when there was simply a mistake in classification. Small changes were ignored in the qualitative assessment of tree canopy changes.

Overall, Laurel experienced far less canopy change than Georgetown, but still experienced a small amount of loss connected with heat exposure. For example, Figure 8 shows the 2012 NAIP image (A), 2023 NAIP image (B), heat map (C), and heat map showing detected canopy loss (D) in a small area in the northwest portion of the city limits. The removal of trees for a small development of duplex homes may seem insignificant, but the higher heat has a strong spatial correlation with the loss of tree canopy in that region. The surrounding areas remain cool with the unchanging urban forest. Generally, Laurel has remained fairly stable with urban tree canopy area. Nevertheless, the difference between the temperatures in the small areas of canopy loss around the city and the rest of the city were statistically significant at the p < 0.001 level, indicating high confidence.

Georgetown, TX was very different with the significant loss of canopy between the two data collection dates. Between that time, Georgetown grew significantly, and this is represented in the several new neighborhood developments. Although there were several examples of this, we represent just two in Figure 8 and Figure 9. In Figure 9, we can see heat especially where the newly developed subdivision or neighborhood was built, but it decreases in the area to the east where urban trees were still present. Just like Laurel, the difference in land surface temperatures in the tree canopy loss areas and the rest of the city limits was statistically significant. In fact, the tree canopy loss areas had temperatures that were higher and statistically significant at the p < 0.001 level.

Figure 10 displays a second example of this phenomenon in the northeastern region of the city where a small development had not been quite completed yet in 2012 but was finished by 2022. There are small pockets of urban forest that reduce heat to a small degree, but the new development overwhelms much of the good the urban forest is doing by making the urbanized region much warmer.

4. Discussion

This study examined the effectiveness of two machine learning and one deep learning models adapted to remote sensing imagery classification for calculating UTC area within a small southern town and a quickly growing southern town in the United States of America. All three classifiers performed more quickly when processing the smaller town data from Laurel as the data size was smaller and more geographically compact. The overall accuracy for each classifier was highest for the small, more simple Laurel imagery, but for both Georgetown and Laurel, the U-Net classifier performed the best overall (89.8% and 91.4%, respectively), better than the SVM and RF classifiers. However, the SVM and RF classifiers were much faster than the U-Net classifiers in the ArcGIS Pro 3.1.3 setting. This may be caused by the computer hardware’s limitations.

The findings from the experiment reflect well among similar experiments performed with other classifiers for determining UTC area coverage (

Table 11. Relevant studies using deep and machine learning to classify tree canopy area.

Data Sources and Study Areas	Imagery Dates	Methods	Highest Overall Accuracies Achieved	Authors
33 imagery patches of Vaihingen, Germany	2013	U-Net and OBIA	U-Net; 99%	[35]
Google Earth Satellite Imagery and LiDAR of Tasmania, Australia	2005; 2015	Object based CNN	2015 data; 98%	[38]
UAS LiDAR/Multispectral data over Toronto, Ontario, Canada	2020	Deep CNN, SVM, Maximum likelihood	Deep CNN, RGB and LiDAR; 97%	[23]
Gaofen-2 Satellite images over Beijing, China	2021	Improved Double Branch U-Net; other U-Net variants	Improved Double Branch U-Net; 95.8%	[25]
High-resolution satellite images for 888 cities across South America	2018–2020	Deeplab3+ deep learning	Deeplab3+; exceeds 90% for all 888 cities	[26]
NAIP over Columbia, SC, USA	2005; 2015	SVM	SVM; 94%	[52]
Planet SkySat satellite imagery over Chattanooga, TN, USA	2021–2022	SVM	SVM; 91%	[27]
Gaofen-2 satellite imagery over Hui’an county, Fujian Province, China	2021	Superpixel-enhanced deep neural forests (SDNF); RF	SDNF; 95%	[64]
Worldview-2 satellite imagery over Miami-Dade County, Florida, USA	2019; 2014	RF	RF; 87%	[28]
Worldview-3 satellite and LiDAR data over Brussels, Belgium	2015; 2017	Geospatial OBIA	GEOBIA; 71%	[39]
NAIP and SPOT satellite imagery over Wisconsin, USA	2013	RF, SVM, boosted regression trees	No significant differences between three classifiers	[37]

). In their comparative study of different spatial scales for mapping accurate UTC using U-Net and object-based image analysis (OBIA), Wang et al. [35] found their U-Net model implemented with Python outperformed OBIA in every measure, reaching a result of 99% overall accuracy. Using both Google Earth imagery and LiDAR datasets, Timilsina et al. [38] obtained overall accuracies of 96% and 98% for 2005 and 2015, respectively. Most studies have relied solely on aerial or satellite imagery to detect and classify tree canopy and have resulted in high overall accuracies when processed with a deep learning classifier. The weakest results were still impressive because of the intent to not only classify tree canopy but the species of tree [39]. Overall, the U-Net classification overall accuracy obtained in this study (89.8% and 91.4%) are comparable to other results presented in Table 11. The SVM and RF classifiers, however, performed poorly compared to the studies presented in Table 11. The SVM and RF classifiers repeatedly performed at about 70–80% overall accuracy, while the results shown below reach upwards of 94% [27,28,52]. This reveals some potential improvements to be made to the SVM and RF methods presented here. Because SVM is known to run well on fewer training samples, reducing the input for training this classifier may have improved the results. Nevertheless, the results of this study suggest that free-to-use NAIP RGB imagery is an accessible and comparable resource to use for obtaining UTC area coverage over small and growing cities. The NAIP imagery and robust tools available in ArcGIS Pro, especially the U-Net deep learning classifier, are able to provide accurate results for urban forestry managers without needing programming experience.

The case study presented UTC change results between the 2012 and 2022/2023 NAIP imagery sets for each city and compared the spatial distribution of UTC change to the urban heat maps derived from the Landsat 9 thermal bands. The patterns found in large amounts of growth in Georgetown, TX, and to a smaller extent in Laurel, were similar to the areas of growth in Columbia, SC in [52]. They are also consistent with findings by Tamaskani Esfehankalateh et al. [6] and Loughner et al. [5] that suggest a strong correlation between UTC and urban heat. Spatially, the city centers of both cities experienced the least change in UTC while the periphery experienced the most urban growth and UTC decline. This suggests a stable city, like Laurel, is likely to experience little change to the UHI effect within its borders as long as trees are protected like they are in Laurel. It also suggests that it is more difficult to mitigate UHI effect in the center of cities as there is less room for trees to be planted and grow once they are already removed for urban growth. Very little, if any, tree growth was reported in either city during the 10-year period. We also found that the differences between the temperatures in the areas tree canopy loss and the temperatures in the areas of no change in both cities were statistically significant, indicating the role UTC plays in mitigating heat. The temperatures of the tree canopy loss areas were higher than those throughout the rest of the city. Future policies should prevent significant loss of tree canopy within city limits to mitigate further heat increases.

Several limitations were encountered during the course of this study. NAIP data are collected at times and dates that are beyond the control of any application. For example, despite being collected during leaf on conditions, the window for data collection is from march until October in most southern US cities, meaning that each year may be more or less easily classified depending on the tree conditions and greenness. Additionally, technological limitations with the computer hardware can impact data classification times. More modern computers, supercomputers, and cloud computing with ArcGIS can remove much of the frustration with processing time that we encountered in this study. Future work should focus on adding additional deep learning models to compare with U-Net to determine if another model may perform faster and provide a more accurate classification. Studies focusing on the identification and counting of individual urban trees have assessed multiple deep learning models, but it has yet to be investigated for mapping UTC area coverage [32,33,34,65,66,67].

5. Conclusions

This study tested three remote sensing image classifiers for mapping UTC area in both a small southern USA city and a midsized, growing USA city. This study determined that by using available tools from industry-leading ESRI ArcGIS Pro and free aerial imagery through the NAIP, overall accuracies similar to those obtained using expensive LiDAR and satellite data can be achieved. Of the three classifiers tested, the U-Net CNN classifier within ArcGIS Pro performed the best but required the longest processing time. The U-Net CNN overall accuracies of 89.8% and 91.4% compare favorably with other studies conducted with similar and more costly resources. Furthermore, insights into UTC changes and spatial correlations between urban heat and those UTC changes occurring in small and midsize cities in the southern US were obtained. For example, the peripheries of even midsized growing cities like Georgetown, TX are getting warmer as more UTC is removed for suburban neighborhoods. The broader impacts of this study reach the urban forestry managers at the local, state/province, and national levels as they seek to provide data-driven decisions for policy makers.