Monitoring of Chestnut Trees Using Machine Learning Techniques Applied to UAV-Based Multispectral Data

Abstract

Phytosanitary conditions can hamper the normal development of trees and significantly impact their yield. The phytosanitary condition of chestnut stands is usually evaluated by sampling trees followed by a statistical extrapolation process, making it a challenging task, as it is labor-intensive and requires skill. In this study, a novel methodology that enables multi-temporal analysis of chestnut stands using multispectral imagery acquired from unmanned aerial vehicles is presented. Data were collected in different flight campaigns along with field surveys to identify the phytosanitary issues affecting each tree. A random forest classifier was trained with sections of each tree crown using vegetation indices and spectral bands. These were first categorized into two classes: (i) absence or (ii) presence of phytosanitary issues. Subsequently, the class with phytosanitary issues was used to identify and classify either biotic or abiotic factors. The comparison between the classification results, obtained by the presented methodology, with ground-truth data, allowed us to conclude that phytosanitary problems were detected with an accuracy rate between 86% and 91%. As for determining the specific phytosanitary issue, rates between 80% and 85% were achieved. Higher accuracy rates were attained in the last flight campaigns, the stage when symptoms are more prevalent. The proposed methodology proved to be effective in automatically detecting and classifying phytosanitary issues in chestnut trees throughout the growing season. Moreover, it is also able to identify decline or expansion situations. It may be of help as part of decision support systems that further improve on the efficient and sustainable management practices of chestnut stands.

1. Introduction

Chestnut trees (Castanea sativa Mill.) are one of the most important species in Portugal for both forestry and agricultural purposes. In an agricultural context, in 2018, this species represented a surface of 38,728 ha with 33,929 tons of chestnuts produced. It is especially relevant in the northern region of the country, where it represents 89% of planted surface (34,504 ha) and 88% of yield (29,908 tons) [1]. Chestnut trees can be affected by several phytosanitary issues due to both biotic or abiotic factors. These issues can significantly impact the chestnut development and yield [2]. Chestnut ink disease (Phytophthora cinnamomi Rands) [3], chestnut blight (Cryphonectria parasitica (Murr.) Barr.) [4], nutritional deficiencies [5] and, more recently (June 2014), the chestnut gall wasp (Dryocosmus kuriphilus Yasumatsu) [6], are among the most meaningful biotic and abiotic factors that can affect chestnut trees. The phytosanitary condition of chestnut stands is usually evaluated by in-field observations, which are time-consuming, laborious, demand specialized human resources, and are based on small samples. Using currently available methods and tools, all based on manual and laborious measurements, phytosanitary conditions monitoring over a longer period of time is even more challenging.

Remote sensing can be considered as a viable approach to help in monitoring and managing chestnut stands regarding phytosanitary issues caused by either biotic or abiotic factors. There are several studies published using remote sensing platforms coupled with different sensors that have chestnut trees as their research subject. Small format aerial photography via manned aircrafts was used to assess chestnut ink disease [7,8,9] and chestnut blight [8]. The same aerial imagery format was also used to assess chestnut ink disease and blight spread through the use of geostatistical methods [10,11] and to discriminate different phytosanitary statuses [12]. Airborne low-density LiDAR (Light Detection and Ranging) data were also used for biomass estimation [13]. However, these initial attempts to use aerial images to assess chestnut trees’ health status revealed low correlation. More recently, Marchetti et al. [14] proposed an approach for mapping chestnut stands using WorldView satellite multispectral imagery, enabling the classification of chestnut stands within a heterogeneous landscape.

Meanwhile, unmanned aerial vehicles (UAVs) have been established as a versatile remote sensing platform capable of being coupled to an array of different sensors and to operate under diverse flight and terrain conditions. Furthermore, they are also able to adapt to specific requirements for monitoring different crops (both temporal and spatial). Precision agriculture and forestry have greatly benefited from this remote sensing platform in the last few years, with many advances being published and already in use. As for studies related to chestnut trees, orthophoto mosaics obtained through photogrammetric processing of high-resolution imagery acquired from UAVs were used by Martins et al. [15] to monitor 231 ha of chestnut trees. By comparing these data with aerial imagery acquired almost ten years earlier, it was possible to measure areas of new plantations and to assess the decline of chestnut trees. The main conclusion drawn by this study was that the decline was very significant along that time period. The study was extended to subsequent years, and the results are presented in Pádua el al. [16], where a decline from 2014 to 2015 was confirmed. A novel method based on image processing was proposed in Marques et al. [17], enabling the automatic monitoring of chestnut trees through estimation of some of the main parameters, such as tree height and crown diameter and area. By applying this methodology, multi-temporal analysis was possible both at the tree and plantation level. Di Gennaro et al. [18] used a similar method to estimate the pruning wood biomass of chestnut trees. Finally, in Pádua et al. [19], the impact of different flight heights in the estimation of tree height and crown diameter was evaluated using UAV-based RGB imagery. It was concluded that flight altitudes of 60 and 120 m (corresponding to a spatial resolution of 2.65 and 5.25 cm, respectively) presented the best overall results.

Nonetheless, despite the numerous advances in monitoring chestnut trees provided by the use of UAV-based high-resolution aerial imagery, little progress has been made in both automatic detection and classification of the biotic or abiotic factors that can affect them. The ability to act (or react) in the timely detection of factors that can negatively affect the phytosanitary condition of a chestnut stand will be essential to improve management practices and, therefore, have a significant social and economic impact. In this study, we explore UAV-based multispectral imagery with high spatial and temporal resolutions [20] of chestnut stands to detect potential phytosanitary problems.

There are studies with comparable objectives for vineyards [21] to detect Flavescence dorée, for olive groves [22] to detect symptoms of Xylella fastidiosa, and for oil palm plantations [23] to assess health status and disease detection. Balasundram et al. [24] provided insights into the deployment of site- and time-specific approaches to manage plant disease problems. However, and according to the authors’ best knowledge, there are no similar studies or approaches applied to chestnut trees. Indeed, the challenge is even greater when dealing with chestnut trees as there are several biotic and abiotic factors that can cause similar symptoms, with very different mitigation treatments or methods.

As such, seeking to determine precisely which factors are affecting a given tree, the proposed methodology also includes an incremental approach based on machine learning methods. Several flight campaigns were accomplished to acquire multispectral imagery over a chestnut stand located in northeastern Portugal. Furthermore, field surveys were also conducted, by an expert, to obtain the phytosanitary characterization of every individual chestnut tree within the monitored area. The proposed methodology begins by applying photogrammetric processing to the acquired high-resolution aerial imagery. Then, tree crowns are detected using the outcomes of the first step. A random forest (RF) classifier is then applied to distinguish healthy trees from those affected by any biotic or abiotic factor. This process is repeated to determine which phytosanitary problem is affecting each tree. Multi-temporal analysis comes into play by applying the proposed methodology to data acquired in different flight campaigns that occur in the same growing season. It is a contributing factor to improve this methodology’ precision as some chestnut trees were asymptomatic or showed a low incidence of phytosanitary issues early in the growing season. The proposed methodology is able to distinguish healthy chestnut trees, and it can also identify which is the specific limiting factor affecting the development of each tree.

2. Materials and Methods

2.1. Study Area Characterization

Research involving trees, in general, and chestnut trees, in particular, requires keen knowledge of the area under study. The studied chestnut stand is a laboratory area located in northeastern Portugal (Figure 1a, 41°22–42.8 N, 7°35–01.4 W, altitude 760 m) within one of the main chestnut production regions in Portugal [25]. It has an area of approximately 0.4 ha and is composed of 52 trees from which 46 are chestnut trees (marked in Figure 1b). This area was rigorously characterized by experts from the University of Trás-os-Montes e Alto Douro (Vila Real, Portugal), and to ensure the representativeness of this area, the most common cultural practices were also used [17].

In-field observations were carried out during the 2018 growing season, on the same dates as the flight campaigns, to assess the phytosanitary condition of the chestnut stand. Issues such as chestnut ink disease and potential nutritional deficiencies were evaluated using a discrete scale ranging from zero (absence) to four (strong incidence). Furthermore, the condition of each chestnut tree was evaluated using a five-level scale ranging from 1 = very bad condition to 5 = excellent condition, considering the severity of the phytosanitary issues along with the overall tree status. This qualitative classification was performed by an expert and based on the severity of visible symptoms [15]. Dendrometric measurements of each tree were also acquired. Soil analyses were also conducted to assess the nutrient levels.

2.2. UAV-Based Data Acquisition

A DJI Phantom 4 (DJI, Shenzhen, China) was used to acquire the aerial imagery used in this study. This multi-rotor UAV comes equipped out-of-the-box with a complementary metal oxide semiconductor (CMOS) sensor—mounted in a 3-axis gimbal—capable of acquiring georeferenced RGB imagery with 12.4 MP resolution (details about UAV and sensor specifications can be found at [26]). The UAV was modified to support multispectral imagery acquisition (Figure 1f) using the Parrot Sequoia (Parrot SA, Paris, France). This sensor is composed of a camera array responsible for acquiring green, red, red-edge (RE), and near-infrared (NIR) single-band images. Each band has a 1.2 MP resolution. For radiometric calibration of the multispectral imagery, irradiance data are acquired during flight (from a sensor positioned at the top of the UAV—see Figure 1f) and, prior to each flight, reflectance data are acquired using a calibration target.

Flight campaigns were carried out throughout the growing season of 2018 to acquire multispectral aerial imagery. A flight mission was planned to provide a complete overview of the area, in a double-grid pattern, with 80% overlap between images and 70% overlap between flight lines. Flight height from the take-off point was set to 60 m, and the total area covered by flights was approximately 2 ha. Considering the planned imagery overlap and the flight height, the sensor was set up to acquire images at each 11 m travelled. The same flight plan was used for all flight campaigns.

The selected dates—27 May, 24 June, 8 July, 8 August, 25 September, and 16 October—allowed spanning across the most important stages of chestnut tree development: sprouting, flowering, fruiting, and defoliation. The vegetative dormancy of chestnut trees usually spans from November to March [27].

2.3. Data Processing

Figure 2 presents the main steps of the proposed methodology as applied to each flight campaign’s data. Outcomes generated from the initial photogrammetric processing enable output of a crop height model (CHM) and several vegetation indices. The latter are computed from different combinations of the four acquired bands and can be used for different purposes: (1) individual tree crown detection; (2) object-based image analysis (OBIA); and (3) dataset features. Training and prediction processes are the same to (1) classify the presence of phytosanitary issues, and (2) to identify the specific phytosanitary issue (if any). Only the number of classes varies. The proposed methodology remains functional, regardless of the sensor used.

2.3.1. Photogrammetric Processing and Vegetation Indices Computation

Pix4DMapper Pro (Pix4D SA, Lausanne, Switzerland) was used to achieve the photogrammetric processing of the acquired aerial imagery. It provides a complete processing pipeline dealing with imagery correction, alignment, and radiometric calibration, producing dense point clouds.

Within this study, RGB orthophoto mosaics were computed for visualization purposes only. Despite the flight campaigns being carried out at the same height with both sensors, their spatial resolution differs. Indeed, due to different sensor resolutions, the ground sample distance (GSD) of the RGB imagery is approximately double that of the multispectral imagery (2.6 and 6 cm, respectively). Point cloud density, per m³, was, respectively, ~500 and ~40 points. Data from both sensors were aligned relative to each other by using points that are clearly identifiable in the imagery and then merged, and geometric correction was applied using ground control points (GCPs) using both natural features and artificial targets.

In projects using multispectral imagery, a radiometric calibration is performed. Reflectance maps are generated for each band, and the most relevant vegetation indices—suggested in the literature—to monitor spatiotemporal variations in biomass and yield and to estimate leaf pigments [21] are computed (

Table 1. Computed vegetation indices found in the literature and their respective equations.

Name	Equation	Ref.
Normalized Difference Vegetation Index	$\mathrm{NDVI}=\frac{\mathrm{N}-\mathrm{R}}{\mathrm{N}+\mathrm{R}}$	[28]
Green Normalized Difference Vegetation Index	$\mathrm{GNDVI}=\frac{\mathrm{N}-\mathrm{G}}{\mathrm{N}+\mathrm{G}}$	[29]
Green Red Vegetation Index	$\mathrm{GRVI}=\frac{\mathrm{G}-\mathrm{R}}{\mathrm{G}+\mathrm{R}}$	[30]
Normalized Difference Red Edge	$\mathrm{NDRE}=\frac{\mathrm{N}-\mathrm{RE}}{\mathrm{N}+\mathrm{RE}}$	[31]
Soil Adjusted Vegetation Index	$\mathrm{SAVI}=\frac{\mathrm{N}-\mathrm{R}}{\mathrm{N}+\mathrm{R}+\mathrm{L}}\times 1+\mathrm{L}$	[32]
Renormalized Difference Vegetation Index	$\mathrm{RDVI}=\frac{\mathrm{N}-\mathrm{R}}{\sqrt{\mathrm{N}+\mathrm{R}}}$	[33]
Simple Ratio	$\mathrm{SR}=\frac{\mathrm{N}}{\mathrm{R}}$	[34]
Transformed Chlorophyll Absorption Reflectance Index	$\mathrm{TCARI}=3 \left[\left(\mathrm{RE}-\mathrm{R}\right)-0.2\left(\mathrm{RE}-\mathrm{G}\right)\times \frac{\mathrm{RE}}{\mathrm{R}}\right]$	[35]

G: Green; R: Red; N: NIR; RE: Red edge; L = 0.5.

). A digital surface model (DSM) and a digital terrain model (DTM) are also generated, and a CHM is computed. This process was accomplished in QGIS software by subtracting the DTM to the DSM.

In addition to the vegetation indices shown in Table 1, new ones are proposed in this study. In fact, knowledge about the typical spectral signature of symptomatic and asymptomatic chestnut trees, allowed to conclude the relevance of the red-edge (RE) and near-infrared (NIR) regions. Figure 3 presents the spectral signatures for the main issues identified in the chestnut stand (chestnut ink disease and nutritional deficiencies) and for chestnut trees with no visible symptoms obtained from UAV-based hyperspectral data using the Nano-Hyperspec® VNIR (400–1000 nm) imaging sensor (Headwall Photonics, Inc., MA, USA). Significant differences among them are observed along spectrum; in the visible part (400–690 nm), a higher reflectance is achieved in trees with nutritional deficiencies, followed by trees with no visible symptoms, while in the RE and NIR parts (690–900 nm), the opposite is verified. Trees affected by the ink disease always presented the lowest reflectance.

Therefore, customized vegetation indices were developed considering the strong influence of the RE and NIR bands. These vegetation indices are inspired by the Excess Green Index (ExG) [36] that showed effectiveness in weed discrimination [36], crop identification [37,38] and quantification [39], early-season crop monitoring [40], and multi-temporal mapping of vegetation fractions [41] using both close-range and UAV-based imagery. Thus, the assumption that added weight of both RE and NIR bands would improve the detection of phytosanitary problems was made (Figure 3). Two new vegetation indices are proposed and were named Excess NIR (ExNIR) and Excess RE (ExRE) and are represented by the following equations:

$$\mathrm{ExNIR}=2\times {\mathrm{N}}_{\mathrm{n}}-{\mathrm{G}}_{\mathrm{n}}-{\mathrm{R}}_{\mathrm{n}}-{\mathrm{RE}}_{\mathrm{n}},$$

(1)

$$\mathrm{ExRE}=2\times {\mathrm{RE}}_{\mathrm{n}}-{\mathrm{G}}_{\mathrm{n}}-{\mathrm{R}}_{\mathrm{n}}-{\mathrm{N}}_{\mathrm{n}},$$

(2)

where G_n, R_n, NIR_n and RE_n corresponds to the division of, respectively, green, red, NIR and RE bands by the sum of the four bands. Normalized difference versions of the two proposed indices, the Normalized Difference Excess NIR (NDExNIR) and the Normalized Difference Excess RE (NDExRE), were also computed as follows:

$$\mathrm{NDExNIR}= \frac{2\times {\mathrm{N}}_{\mathrm{n}}-{\mathrm{G}}_{\mathrm{n}}-{\mathrm{R}}_{\mathrm{n}}-{\mathrm{RE}}_{\mathrm{n}}}{2\times {\mathrm{N}}_{\mathrm{n}}+{\mathrm{G}}_{\mathrm{n}}+{\mathrm{R}}_{\mathrm{n}}+{\mathrm{RE}}_{\mathrm{n}}}$$

(3)

$$\mathrm{NDExRE}= \frac{2\times {\mathrm{RE}}_{\mathrm{n}}-{\mathrm{G}}_{\mathrm{n}}-{\mathrm{R}}_{\mathrm{n}}-{\mathrm{N}}_{\mathrm{n}}}{2\times {\mathrm{RE}}_{\mathrm{n}}+{\mathrm{G}}_{\mathrm{n}}+{\mathrm{R}}_{\mathrm{n}}+{\mathrm{N}}_{\mathrm{n}}}$$

(4)

2.3.2. Individual Tree Crown Detection and Multi-Temporal Analysis

For extraction of individual tree parameters (Step 1 from Figure 2), each tree must be isolated from its surrounding environment (soil, vegetation, and other trees). However, given its planting distance and crown size, chestnut trees tend to be too close from each other, giving rise to the need for their segmentation and isolation. In this way, the orthorectified outcomes can be used as input in an image processing method for individual tree crown detection. For this purpose, the principles enunciated in Marques et al. [17] were used with slight modifications to encompass multispectral imagery. The method was developed for chestnut plantation monitoring with the scope of performing multi-temporal analysis. It relies on the combination of photogrammetric outcomes in a raster format which, in turn, is automatically binarized. Some changes were implemented to ensure that all monitored chestnut trees within the study area were included for analysis. Taking both the NIR band and the CHM as inputs, a locally adaptive threshold [42] is used in the binarization of the stand. A visual analysis allowed us to conclude that apart from trees of significant size (chestnuts and other trees) the amount of green vegetation in the study area was low or almost absent (depending on the flight campaign). For that reason, a value of 0.20 m was selected for CHM thresholding. Both binary images were then concatenated.

In the output, most of the pixels in the binary image (Figure 4b) belong to the crowns of chestnut trees. Still, some clusters of pixels can eventually represent more than one tree, leading to the need for a cluster isolation step (see Figure 4). The inverse of the binary imagery is used to compute a distance transform (Figure 4c) based on the Euclidean distance transform [43], where a value is assigned for each pixel corresponding to the distance to the nearest pixel with a zero value. In turn, the complement image is used in the watershed transform [44]. This way, in an ideal scenario, clusters representing multiple trees are separated into individual clusters representing a single tree (Figure 4d). However, given the high spatial resolution, there can be cases where small parts can be erroneously separated. The process is reversed by analyzing the bounding boxes overlap ratio, namely, if it is higher than 90% relative to another. Binary images as presented in Figure 4b,d were used to mask the color–infrared image.

After cluster isolation, it is possible to obtain individual tree parameters. Several parameters can be driven by the analysis of each cluster, such as the crown diameter, perimeter, and area. Moreover, values retrieved from remotely sensed data, such as the CHM (tree height), vegetation indices, or spectral bands, can be obtained by matching each cluster to the raster data. This information can be presented as geospatial data in vector format (shapefile) to be analyzed in a geographic information system (GIS) or in a table format.

Finally, multi-temporal analysis can be carried out using the values extracted for each flight campaign by comparison with the subsequent campaign. This way, the extracted parameters can be used for individual tree monitoring or to obtain an overview of the chestnut stand at the time of each flight campaign. In this study, the tree crown area and the mean NDVI value are analyzed in a multi-temporal perspective, focusing on the overall stand development and on trees affected/non-affected by phytosanitary issues.

2.4. Detection of Phytosanitary Issues Using a Random Forest Classifier

Apart from the possibility of doing multi-temporal analysis using the extracted parameters, they can be used in a machine learning (ML) approach to distinguish chestnut trees in different phytosanitary conditions. Then, it is possible to (1) classify healthy chestnut trees and chestnut trees with phytosanitary issues and (2) distinguish among phytosanitary issues. The clusters resulting from the automatic individual tree detection were labeled in two ways according to their phytosanitary status: in two classes—with or without phytosanitary issues; and in three classes—to distinguish the different major phytosanitary problems (no visible symptoms, ink disease, and nutritional deficiencies).

An RF algorithm was used to carry out these classifications. It is a type of ensemble classifier that generates several decision trees using a random subset of training samples capable of handling high data dimensionality and multicollinearity and is insensitive to overfitting [45]. This method is widely used in remote sensing applications [46,47,48], including tree species classification [49,50,51].

2.4.1. Data Augmentation from Object-Based Image Analysis

As mentioned in Section 2.1, the use of a well-characterized area with well-known behavior is essential for validation of results. However, the fact that the stand used in the study is composed of a relatively small number of samples is a challenge for ML techniques. In fact, the essence of ML is based on using a high number of observations/samples. To overcome this limitation, the number of available samples was substantially increased using an OBIA approach (Step 2 from Figure 2). This was done using large-scale segmentation based on the mean shift algorithm [52] from the Orfeo ToolBox (OTB) [53]. It requires a raster as input and results in a set of objects in vector format with a similar spectral similarity. To better discriminate tree crowns, the NIR, green, and red bands (NGR, example in Figure 4a) were concatenated and rasterized to produce a three-band false-color image. This combination of bands was revealed to be the best compromise for this specific task. To increase the number of objects produced in this procedure, its sensitivity was augmented. Therefore, the spectral radius was set to 10 while the spatial radius and minimum segment size were kept at five and 50, respectively. The originated objects which intercept the detected tree crowns inherit the same classification as their correspondent tree, being classified according to its predominant phytosanitary issue that was observed in the field. Figure 5a presents part of the output obtained from the OBIA step; the objects matching tree crowns (Figure 5b) are then used for further model training and testing (Figure 5c).

2.4.2. Feature Selection, Training, Validation, and Prediction

The created dataset is composed of the mean values of 16 features: the eight vegetation indices presented in Table 1; the green, red, RE and NIR bands; and the last four corresponding to the vegetation indices proposed in this study (see Section 2.3.1.). As such, the database connected to the objects representing the tree crowns include a column with the mean value of these features. However, given the number of features to discriminate, those may behave differently by class. Hence, to decrease the number of features, an intermediate step was introduced. For this purpose, recursive feature elimination (RFE) [54] was used (Step 3, from Figure 2). This method ranks features recursively based on their respective importance [55].

For training and evaluation of the RF models (Step 3, from Figure 2), a hold-out strategy was used by randomly performing train-test splits (70% to train and 30% to test). To avoid possible discrepancies in the solution, an average of 10 repetitions was used. The data split operation is made by considering the area of each object within each tree and using 70% of the tree crown area for training and the remaining 30% for testing. This step is applied to the datasets from each flight campaign.

To evaluate the classification procedure in the different flight campaigns, the resultant confusion matrices were analyzed. For this purpose, the following metrics were used: precision—the number of objects correctly classified for a given class divided by its total number of samples; recall—the number of correct classifications for a given class divided by its row total; and F1score—the harmonic mean of precision and recall measures. The overall accuracy and the Cohen’s kappa coefficient (K) [56]—a statistic used to measure inter- or intra-rater reliability for qualitative items—were also analyzed for a general perspective of the models’ behavior. While the overall accuracy indicates the proportion of correct classifications in the total number of samples, the kappa coefficient evaluates the performed classification while considering the possibility of the agreement occurring by chance.

To predict potential phytosanitary issues in the analyzed stand (Step 4, from Figure 2), the mean value of each tree is used. The mean feature value differs from the training values since the mean value of the whole tree crown is different from the mean value of their objects (used from training and testing). To evaluate the performance, the overall accuracy and the classification errors for each class were assessed. The predictions were made for two classes (with or without phytosanitary problems) and categorized according to the detected issue (no visible symptoms, ink disease, and nutritional deficiencies).

3. Results

3.1. Phytosanitary Characterization of the Study Area

The phytosanitary issues detected in affected trees were mostly ink disease and nutritional deficiencies. Both show symptoms on tree crown and foliage: while for chestnut ink disease, the dieback can be observed by low-density foliage or even its absence in some parts of the canopy (Figure 1d), nutritional deficits are noticeable by leaf discoloration and stress symptoms (Figure 1e). From the 46 chestnut trees assessed, 16 presented nutritional deficiencies (Figure 6a), eight had a higher predominance of ink disease symptoms (Figure 6b), and the remaining 22 were considered to be without symptoms. The latter had 6.5 m mean height and a mean crown diameter of 6.5 m. Those presenting ink disease symptoms had 6.2 m mean height and 6.1 m mean crown diameter. Chestnut trees with symptoms of nutritional deficiencies had a mean height of 4.5 m and a mean crown diameter of 4.0 m. The overall mean chestnut tree height was 5.8 m, and the overall mean crown diameter was 5.6 m. The global condition of each evaluated chestnut tree is presented in Figure 6c.

3.2. Multi-Temporal Analysis

The estimated individual parameters of the chestnut trees from the different flight campaigns allowed for an understanding of the overall evolution of the stand. From these, the crown area and vegetation indices are the foremost parameters that can support multi-temporal analysis. Figure 7a presents the overall area occupied by chestnut trees, while Figure 8 depicts the individual crown area for each chestnut tree. A growth trend from the first (late May) to the fourth flight campaign (August) can be observed. From the fourth up to the last flight campaign, an overall decline occurred. The first three flight campaigns (May to June and June to July) presented a growth in area of 9% and 5%, respectively (from 767 to 877 m²), while from the third to the fourth flight campaign, a growth of 13% was verified (991 m²). In the last two flight campaigns, a decline was registered (16% and 6%, respectively), resulting in a final chestnut tree crown area of 783 m².

Considering trees with no visible symptoms (22 trees, ~48% of the total number of trees) and trees otherwise affected by phytosanitary issues (24 trees affected by ink disease or/and nutritional deficiencies), the former represents between 63% to 67% of the crown area along the flight campaigns. Figure 7a presents the crown area of the chestnut trees (i) that had no visible phytosanitary issues detected in the in-field characterization; (ii) with phytosanitary issues, regardless of which (24 trees); (iii) affected by ink disease (8 trees); and (iv) with nutritional deficiencies (16 trees). In general, the various curves fit well in their behavior, presenting an almost linear increase in crown area until the third flight. Maximum crown area was reached in the fourth flight. Crown area decline for trees affected by phytosanitary issues was 29% (−98 m², from 341 to 244 m²), with data acquired in the two last flight campaigns. As for chestnut trees with no visible symptoms, the area decline was 17% (−108 m², from 626 to 519 m²). Crown areas of chestnut trees affected by ink disease or/and with nutritional deficiencies have a 30 m² mean difference, representing 14% and 18% of the overall crown area, respectively.

The distribution of tree crown area is presented in Figure 7b–d. While some trees present a tree crown area higher than 40 m², others present an area lower than 1 m² (Figure 7b). Such discrepancies can be justified by the fact that the trees of smaller area represent recent plantations, carried out to replace dead trees. Considering all flight campaigns, the mean chestnut tree crown area is 18 m². Whereas chestnut trees with no visible phytosanitary issues had a higher mean crown area (26 m²), trees affected by ink disease had 15 m², and trees with nutritional deficiencies presented a mean crown area value of 10 m².

The mean NDVI value of each chestnut tree is presented in Figure 9. Slight variations can be detected in the first two flight campaigns. However, a constant decline was verified in the remaining campaigns. Chestnut trees with symptoms of phytosanitary issues presented a lower NDVI value in all flight campaigns when compared to healthy trees. Indeed, the lowest mean values were presented by trees with nutritional deficiencies. This difference increased throughout the flight campaigns. While for the first four campaigns, the mean difference—tree crown growth—was −0.06 for trees affected by ink disease and −0.17 for those that showed nutritional deficiencies, for the last two flight campaigns—tree crown decline—these were −0.09 and −0.32, respectively.

3.3. Detection of Trees with Phytosanitary Symptoms

The individual tree crown projections obtained with each flight campaign were subjected to an OBIA procedure to output a set of objects (step 2, in Figure 2). Naturally, each set has a different number of objects due to canopy area evolution and appearance over time. An average of 1650 objects was obtained throughout all flight campaigns. While 1527 objects were identified in May, that number grew in the following two flight campaigns—1668 in June and 1720 objects identified in July—and decreased in August (1389 objects). Then, it grew again in the last two flight campaigns, with 1452 objects identified in September and 2165 in October. These variations can be explained by changes in the canopy appearance over time, which result in reflectance alteration. The latter can be justified either by the dieback observed in the overall leaf discoloration and/or by the presence of some chestnut fruits, which results in higher spectral differences among each tree crown. As for the distribution of objects per class (considering the average of all epochs), the class of healthy trees has a higher number (67%), while the class with phytosanitary issues is left with the remainder (33%). Ink disease represents 15% and nutritional deficiencies 19% of the latter class.

3.3.1. Dataset Description and Feature Selection

Figure 10 presents the trends for vegetation indices throughout the flight campaigns. Excluding RVI—for the first campaign—the majority of the vegetation indices present values disposed in a shorter interval. However, in the last flight campaigns, values are spanned on a larger interval. An example is NDVI: it tends to decrease in value but presents an increased value span in the last two flight campaigns. GNDVI shows a different trend, increasing in the first four flight campaigns and decreasing in the last ones. As for GRVI, it presents an overall decline along the flight campaigns. ExNIR, NDExNIR, NDRE, ExRE, and NDExRE present a similar behavior, increasing in value from the first to the second flight campaigns, followed by a small decrease in the third flight campaign. Hereinafter, values increase again in the fourth campaign and decrease in the last two flight campaigns. Lastly, RVI presents values spread over a larger interval in almost all flight campaigns, being lower in the last two.

When analyzing the values of the vegetation indices automatically extracted from the detected tree crowns—considering trees with or without phytosanitary symptoms (Figure A1)—both classes are distinguishable by their interquartile range (IQR). Trees presenting ink disease and/or nutritional deficiencies can be clearly distinguished from those that are healthy in all flight campaigns and by vegetation indices, with the exception of ExRE (see distribution in Figure A2). Moreover, ExRE is again the exception when comparing values between healthy trees and those affected by chestnut ink: in the remaining vegetation indices, the latter presented lower values. As for nutritional deficiencies, only the third flight campaign of ExRE presents a higher value when compared to healthy trees. It should also be noted that trees affected by chestnut ink disease had higher values in comparison with those affected by nutritional deficiencies.

Feature selection based on RFE allowed for understanding the influence of features extracted from each object on the RF classifier. These results are presented in

Table A1. Recursive feature elimination results for each flight campaign, considering two classes (C2) and three classes (C3), and its overall rank. Top ten features are highlighted.

Feature	May		Jun		Jul		Aug		Sep		Oct		Overall
	C2	C3	C2	C3	C2	C3	C2	C3	C2	C3	C2	C3	C2	C3
NDExNIR	1	2	2	4	2	2	1	1	4	4	2	2	1	1
EXNIR	3	3	3	3	3	3	3	3	2	3	3	3	2	2
GNDVI	4	1	1	1	4	5	2	4	3	2	4	5	3	3
NDRE	8	6	7	10	1	1	6	7	1	1	1	1	4	4
RVI	6	7	4	2	5	6	4	2	5	5	5	9	5	5
NDVI	2	4	5	5	6	9	5	5	6	9	9	10	6	7
RED	5	5	11	7	8	10	8	6	7	6	6	4	7	6
NDExRE	7	9	6	6	9	11	9	10	9	11	11	11	8	10
GRVI	11	11	12	12	7	4	10	8	10	7	8	6	9	8
EXRE	10	8	10	8	11	8	13	11	8	8	7	7	10	9
TCARI	9	12	9	9	12	12	7	9	13	10	10	14	11	12
SAVI	16	14	8	14	14	15	11	15	16	15	13	15	12	15
GREEN	13	10	13	11	13	7	14	13	11	12	16	8	13	11
NIR	12	13	14	13	10	13	15	12	14	13	15	13	14	13
RDVI	14	16	16	16	15	16	12	14	12	16	12	16	15	16
RE	15	15	15	15	16	14	16	16	15	14	14	12	16	14

. By analyzing the overall results—achieved by adding all ranks and sorting the features by their lower value—when considering two and three classes, the top ten features are the same in both situations (highlighted in bold in Table A1). As such, those features were selected to be used in the subsequent analysis.

3.3.2. Random Forest Classifier and Dataset Performance Evaluation

The model was trained by using ten random selections of 70% of each tree crown area per epoch. It was then tested using the remaining 30%. The mean accuracy of the ten random splits and their standard deviations were used to evaluate the model performance.

Table 2. Performance evaluation results (and its standard deviation) of OBIA objects considering two classes (1: no visual symptoms; 2: phytosanitary issues) for each flight campaign.

Month	Class	Precision	Recall	F1-score	Kappa Index	Overall Accuracy
May	1	0.89 (0.02)	0.92 (0.01)	0.90 (0.01)	0.65 (0.04)	0.86 (0.01)
	2	0.79 (0.04)	0.72 (0.04)	0.75 (0.03)
Jun.	1	0.88 (0.01)	0.90 (0.01)	0.89 (0.01)	0.66 (0.02)	0.85 (0.01)
	2	0.79 (0.02)	0.75 (0.02)	0.77 (0.02)
Jul.	1	0.90 (0.01)	0.92 (0.01)	0.91 (0.01)	0.72 (0.02)	0.88 (0.01)
	2	0.83 (0.02)	0.80 (0.02)	0.81 (0.02)
Aug.	1	0.91 (0.02)	0.89 (0.01)	0.90 (0.01)	0.71 (0.02)	0.87 (0.01)
	2	0.79 (0.02)	0.83 (0.02)	0.81 (0.02)
Sep.	1	0.93 (0.01)	0.94 (0.02)	0.94 (0.01)	0.80 (0.02)	0.91 (0.01)
	2	0.88 (0.03)	0.85 (0.03)	0.86 (0.02)
Oct.	1	0.91 (0.01)	0.92 (0.01)	0.91 (0.01)	0.72 (0.03)	0.88 (0.01)
	2	0.82 (0.02)	0.81 (0.02)	0.81 (0.02)

presents the results when considering only two classes: absence or presence of phytosanitary issues. Datasets acquired from all flight campaigns obtained an overall accuracy equal or higher than 85%. The highest value (91%) was achieved in September’s flight campaign. As for accuracy statistics (kappa index), a substantial agreement (kappa > 0.65) was obtained in all flight campaigns. From July onward, kappa was always equal or higher than 0.71. Regarding metrics, when each class is analyzed individually—precision, recall, and F1score—the healthy tree class achieved better results. When comparing each flight campaign, June and May showed similar results. However, a higher standard deviation was observed in May. Similarly, July, August, and October presented similar results. Still, results from August are slightly lower.

Table 3. Performance evaluation results (and its standard deviation) from OBIA objects considering three classes (1: no visual symptoms; 2: ink disease; 3: nutritional deficiencies) for each flight campaign.

Month	Class	Precision	Recall	F1-Score	Kappa Index (STD)	Overall Accuracy (STD)
May	1	0.88 (0.01)	0.93 (0.01)	0.91 (0.01)	0.55 (0.01)	0.80 (0.01)
	2	0.58 (0.08)	0.53 (0.04)	0.55 (0.04)
	3	0.58 (0.04)	0.49 (0.03)	0.53 (0.02)
Jun.	1	0.88 (0.01)	0.92 (0.02)	0.90 (0.01)	0.60 (0.02)	0.81 (0.01)
	2	0.64 (0.04)	0.55 (0.06)	0.59 (0.03)
	3	0.66 (0.03)	0.61 (0.04)	0.64 (0.03)
Jul.	1	0.89 (0.01)	0.94 (0.01)	0.91 (0.01)	0.65 (0.03)	0.83 (0.02)
	2	0.66 (0.04)	0.63 (0.07)	0.64 (0.05)
	3	0.74 (0.06)	0.62 (0.04)	0.67 (0.03)
Aug.	1	0.89 (0.02)	0.91 (0.01)	0.90 (0.01)	0.60 (0.03)	0.80 (0.01)
	2	0.58 (0.06)	0.58 (0.07)	0.58 (0.05)
	3	0.64 (0.06)	0.60 (0.05)	0.62 (0.02)
Sep.	1	0.92 (0.02)	0.94 (0.01)	0.93 (0.01)	0.69 (0.02)	0.85 (0.01)
	2	0.60 (0.03)	0.57 (0.05)	0.58 (0.03)
	3	0.77 (0.05)	0.73 (0.05)	0.75 (0.03)
Oct.	1	0.90 (0.01)	0.94 (0.01)	0.92 (0.01)	0.67 (0.03)	0.85 (0.01)
	2	0.62 (0.04)	0.60 (0.04)	0.61 (0.03)
	3	0.83 (0.04)	0.71 (0.04)	0.76 (0.03)

presents the results obtained when distinguishing between specific phytosanitary issues of ink disease and/or nutritional deficiencies. The minimum overall accuracy is 80% (May and August flight campaigns) and the highest (85%) was achieved in the September and October flight campaigns. However, the statistical significance of the results differs: a moderate agreement (kappa > 0.55) was registered in May, while in the remaining epochs, the value increased. Indeed, the highest value was in September (0.69). July and October also registered a kappa value of at least 0.65. Regarding each class classification, F1-score was always higher than 0.90 for trees without phytosanitary problems. As for the other two classes—affected by ink disease and/or by nutritional deficiencies—the F1-score was lower. Nonetheless, the class containing trees affected by nutritional deficiencies had better results.

3.3.3. Detection of Chestnut Trees Affected by Phytosanitary Issues

The mean value of each tree crown feature was used to assess whether it was affected by phytosanitary issues. Results are presented in Figure 11 and Figure A3a. The overall accuracy is equal to or higher than 85%. The lowest value was achieved in May (85%) and the highest in the last two flight campaigns (96%). In the remaining flight campaigns, the overall accuracy is 91% in both June and August, and 94% in July. Indeed, the earliest flight campaign in the season (May) had the most misclassifications—seven chestnut trees, representing about 15% of the total number of chestnut trees monitored (46): two healthy trees were classified as being affected by phytosanitary issues and five the exact opposite. The number of misclassified chestnut trees without visible symptoms was consistently low in the remaining flight campaigns: one in June, and two in both July and August. As for misclassified chestnut trees with phytosanitary issues, there were three in June, one in July, and two in the remaining flight campaigns.

Figure 12 presents the assessment results when using three classes (no visual symptoms, ink disease, and/or nutritional deficiencies). The higher overall accuracy value is achieved in September (91%) and the lowest in May (78%); see Figure A3b. The remaining flight campaigns present a relatively stable overall accuracy value, ranging between 83% and 87%. Chestnut trees without visible symptoms present the lowest misclassification values (5% overall). No misclassifications were observed in both September and October. Moreover, in July, there were no misclassifications in chestnut trees affected by ink disease. Affected trees were mainly misclassified as having no phytosanitary issues: there were two misclassifications on average (August and October had three). Regarding chestnut trees affected by nutritional deficiencies—an average of four misclassified trees considering all flight campaigns—they were misclassified in both of the other two classes: 10 in healthy trees and 12 in trees affected by ink disease.

4. Discussion

The multi-temporal data analysis enabled characterization of both spatial and temporal variability of the studied chestnut stand. Studies on chestnut trees management rely only on yearly flight campaigns [16] to monitor the overall condition and to study vegetation decline [15], limiting the intra-seasonal monitoring of potential issues. Indeed, no intra-seasonal multi-temporal studies were found for chestnut trees, and these can be fundamental for detecting potential phytosanitary issues earlier on, which will enable timely mitigation actions. Furthermore, each tree can automatically be classified regarding its phytosanitary status as affected, ink disease or nutritional deficiencies, or healthy.

Regarding the crown area for the monitored chestnut trees throughout the season, it is of note that it increased from May to August and decreased hereinafter. This trend is verified more often in healthy chestnut trees (see Figure 7 and Figure 8). Those affected by phytosanitary issues presented a smaller crown area growth in the first three flight campaigns. The west side area of the stand had higher NDVI values throughout the analyzed period (Figure 9) while the opposite was verified in the east. Whereas the size of the chestnut trees—smaller precisely in the east area (Figure 8)—can explain this because it usually means lower foliage density, trees affected by phytosanitary issues are mostly located in that area (see Figure 6). A clear distinction between trees with and without phytosanitary problems (Figure 7c), and between phytosanitary problems can be observed.

The crown area of chestnut trees affected by ink disease are usually larger when compared to the ones from trees affected by nutritional deficiencies (Figure 7d). The latter present a small increase in crown area in August. As for NDVI, while chestnut trees affected by phytosanitary issues presented decreasing values throughout the season, this trend is less clear-cut in chestnut trees affected by ink disease. With reference to the feature selection procedure, the proposed vegetation indices were among the ones with best discrimination performance. This can be explained by the fact that spectral differences are more significant when addressing symptoms caused by the studied phytosanitary issues (Figure 3). Spectral bands can also be considered less relevant features than vegetation indices. Indeed, green, NIR, and RE bands did not perform well when compared to the VIs, which was not verified in studies using RGB-based vegetation indices [57].

The employed methodology can be regarded as accurate not only when classifying chestnut trees as affected (or not) by phytosanitary issues (Figure 7c), but also (when affected) in distinguishing which phytosanitary issue is present in each case. The crowns detection for individual trees employed in this study allow for discarding most outliers unrelated to chestnut trees, such as soil and low-height vegetation, while other studies relied on OBIA with more steps [58]. As such, an ML classification step to detect trees is not a requirement.

When considering the possibility of having a chestnut tree affected by a phytosanitary issue (Table 2, Figure 11 and Figure A3a), the obtained results show that, both in testing and detection, September’s flight campaign data had the best accuracy rates. While similar results were achieved in October, the kappa value was slightly lower. Remaining campaigns also achieved good accuracy values. May corresponds to an early phase of the chestnut’s phenological cycle, when most symptoms caused by phytosanitary issues are not yet clearly noticeable. This justifies the higher standard deviation observed in May. However, when three classes are considered—healthy, ink disease, and nutritional deficiencies—the overall accuracy generally decreases (Table 3, Figure 12 and Figure A3b). Again, September’s data outperform those of the remaining flight campaigns. Whilst the October flight campaign presented better results when distinguishing ink disease, this can be explained by environmental factors: chestnut trees were exposed to longer periods of low/no precipitation (causing low soil moisture) and higher temperatures than those registered in the summer, which result in trees having more stress and, therefore, to manifest a higher incidence of phytosanitary symptoms [59]. On the other hand, this decrease in classification can also be related to the chestnut harvesting season (and, therefore, trees start their senescence). Currently, to mitigate the occurrence of chestnut ink disease, hybrid chestnut trees are being used with good results [60].

It should be noted that some chestnut trees did not present symptoms uniformly. Indeed, some parts had similar spectral responses than healthy trees. Despite values in Figure A1 and Figure A2 having a different separation between classes in the training phase, objects formed from the OBIA procedure have similar objects in the classes, since the whole tree was considered as being affected by only one issue. In other works, different types of classes were classified as different tree species [50,61,62,63] or distinguished completely different types of classes [47,57,64,65]. Moreover, in Gini et al. [66], multispectral imagery was combined with texture features for tree species classification, increasing the overall accuracy. In this study, only Castanea sativa Mill. trees were evaluated using UAV-based multispectral data to automatically distinguish the presence (or absence) of phytosanitary issues. Therefore, it was more challenging when considering data classification into three classes since there were more spectral similarities.

When compared to traditional in-field approaches that require several days of field surveys/measurements, the whole pipeline proposed in this study can deliver the final results in a single day. Future developments would rely on data processing and results being delivered on the fly, similar to what was demonstrated from tree counting [67]. Data from sensing payloads other than multispectral imagery can help improve the differentiation between the phytosanitary issues analyzed in this study. Indeed, thermal infrared, hyperspectral, and fluorescence data [22,68,69] are options to be considered (e.g., allowing the creation of narrow-band vegetation indices). However, hyperspectral data require more complex data processing and with higher computational and financial costs compared to multispectral data. The proposed method can also be explored in other contexts such as arid and semi-arid land vegetation monitoring [70].

5. Conclusions

This study shows the suitability of image analysis and processing to automatically detect phytosanitary issues in individual chestnut trees within a chestnut stand using UAV-based multispectral data. The results demonstrate the effectiveness of the RF classifier in discriminating trees with and without phytosanitary issues and to classify according to the issue affecting the trees (ink disease and nutrient deficiency). In addition, new vegetation indices were proposed, which helped to improve the results. The obtained results also allowed us to conclude that the latter stages of the season is the optimal time (less misclassifications) for the application of the proposed methodology. This way, the dormancy period can be used to apply corrective treatments on the trees identified as having phytosanitary issues (e.g., soil nutrient corrections, biomass pruning tree optimization, tree replacement). However, the results from early and mid season (May to June) are also promising—phytosanitary issues can be detected even in cases when symptoms are not significant—and can be used to optimize field inspections, reducing the amount of work/time needed compared to manual/visual inspections. Some treatments can be directed to those trees to prevent the further development of issues. Moreover, the usage of multi-temporal data enabled the monitoring of the chestnut stand along the season.

In the near future, the proposed methodology can be applied to monitor chestnut trees at a larger scale, providing a cost-effective and less laborious alternative to field surveys to assess overall phytosanitary condition. Moreover, it can also be used in the long-term monitoring of damage caused by the chestnut gall wasp in both phytosanitary and development status of individual chestnut trees. Lastly, the greater spatial resolution provided by UAV-based data when compared to other remote sensing platforms can allow for yield estimation by automatically detecting chestnut clusters, since they tend to grow in tree branch tips and are therefore visible from an aerial perspective. Other types of sensors should also be evaluated, such as thermal and hyperspectral, increasing the variety of features that can be used for analysis and in promoting efficient and sustainable management practices.