Comparative Assessment of Pixel and Object-Based Approaches for Mapping of Olive Tree Crowns Based on UAV Multispectral Imagery

Abstract

Pixel-based (PB) and geographic-object-based (GEOBIA) classification approaches allow the extraction of different objects from multispectral images (MS). The primary goal of this research was the analysis of UAV imagery applicability and accuracy assessment of MLC and SVM classification algorithms within PB and GEOBIA classification approaches. The secondary goal was to use different accuracy assessment metrics to determine which of the two tested classification algorithms (SVM and MLC) most reliably distinguishes olive tree crowns and which approach is more accurate (PB or GEOBIA). The third goal was to add false polygon samples for Correctness (COR), Completeness (COM) and Overall Quality (OQ) metrics and use them to calculate the Total Accuracy (TA). The methodology can be divided into six steps, from data acquisition to selection of the best classification algorithm after accuracy assessment. High-quality DOP (digital orthophoto) and UAV_MS were generated. A new accuracy metric, called Total Accuracy (TA), combined both false and true positive polygon samples, thus providing a more comprehensive insight into the assessed classification accuracy. The SVM (GEOBIA) was the most reliable classification algorithm for extracting olive tree crowns from UAV_MS imagery. The assessment carried out indicated that application of GEOBIA-SVM achieved a TA_COR of 0.527, TA_COM of 0.811, TA_OQ of 0.745, Overall Accuracy (OA) of 0.926 or 0.980 and Area Under Curve (AUC) value of 0.904 or 0.929. The calculated accuracy metrics confirmed that the GEOBIA approach (SVM and MLC) achieved more accurate olive tree crown extraction than the PB approach (SVM and MLC) if applied to classifying VHR UAV_MS imagery. The SVM classification algorithm extracted olive tree crowns more accurately than MLC in both approaches. However, the accuracy assessment has proven that PB classification algorithms can also achieve satisfactory accuracy.

1. Introduction and Background

The olive is one of the oldest and most widely cultivated species in the Mediterranean [1,2,3]. Over the centuries, the distribution, spread, and finally the dominance of olives through the centuries have shaped the Mediterranean landscape’s recognizable character [1]. Over 70% of the olives in the world are grown in the Mediterranean countries of the European Union [4]. Olives are well adapted to sloping and poorly fertile soils, thus providing an ecological, economic and social benefit to the areas in which they are cultivated [5]. Preservation of olive groves as an element of Mediterranean identity [6] and a strategic economic resource [7] depends on sustainable environmental management, based on the improvement of the competitiveness of the agricultural sector and on precision agricultural methodology and technology [8,9]. The development of geospatial technologies has enabled precise mapping and inventorying of olives [10,11]. The application of unmanned aerial vehicles (UAV) in aerial photogrammetry is widely accepted as a reliable and accurate remote sensing method for environmental protection and mapping of vegetation species [10,12,13]. Compared to satellite images, UAV multispectral images (UAV_MS) have significantly better spatial resolution and greater flexibility in the selection of the appropriate spatio-temporal resolution [14]. In comparison to airborne photogrammetric systems, UAV-based systems have lower operating costs and the possibility of lower flight altitudes, thus flying closer to treetops and providing a very detailed insight into vegetation dynamics [15]. Various pixel-based (PB) and geographic-object-based (GEOBIA) classification approaches allow the extraction of different objects from MS, and, depending on the set parameters and the quality of the input data, they can achieve different results [16]. PB methods use the individual pixels of an MS as a minimum mapping unit, thus allowing a very detailed classification of different objects, which in some cases can potentially lead to the formation of various types of “noise” or the “salt and pepper” effect [17,18]. On the other hand, GEOBIA allows pixels of given images to be grouped into meaningful homogeneous “superpixels” of different shapes and sizes, according to their common spectral, spatial and geometric features [18,19,20]. A highly accurate land cover model (LCM) can be generated using the MS and PB or GEOBIA approach, together with selected test samples and an appropriate classification algorithm [18]. However, classifying data collected by remote sensing methods into a meaningful and accurate thematic map remains a challenge. The classification outcome is still influenced by many factors such as the complexity of the landscape within the study area, selected test patterns and chosen approaches to image processing and classification [21,22,23]. Therefore, the constant emergence of new classification algorithms and methods [24] in recent years requires a critical approach. Different classification algorithms should be compared and evaluated to facilitate the selection of the most accurate and suitable one for particular research [16].

Many authors have used UAVs for aerial photogrammetric surveys over areas of olive groves to collect the data required to extract olive trees crowns via the GEOBIA approach. In [25], olive and citrus tree crowns were extracted from UAV_MS imagery using a multiscale object-based approach. In [26], photogrammetric point clouds were generated by UAV technology and analyzed using GEOBIA. In [27] and [28], olive trees were also extracted from the UAV_MS imagery using the GEOBIA approach to assess olive tree characteristics. In [29], the authors developed and tested the performance of a method based on low-cost UAV imagery to estimate olive crown parameters (tree height and crown diameter). Different authors have used different classification algorithms in their research; for example, those in [25] used the Assign Class algorithm, those in [30] used Random Forest by applying PB and GEOBIA approaches and those in [31] compared the results of two deep learning (Fully Convolutional Networks and patch-based Deep Convolutional Neural Networks) and two conventional (Support Vector Machine (SVM) and Random Forest) classification algorithms. In [32], the Maximum Likelihood Classifier (MLC) was used, while in [29] classification was performed using the Classification and Regression Trees algorithm. Furthermore, researchers used different methods to assess the accuracy of the models; for example, those in [25] used Recall, Precision, F-score and Branching Factor metrics. In [27] and [32], the Overall Accuracy (OA) [33] metric was used. The authors in [34] used Correctness (COR) [35], Completeness (COM) [35] and Overall Quality (OQ) [35] metrics. In [27], an OA value of 0.93 was obtained. The authors in [30] also used OA, as well as Producer’s Accuracy (PA) [33] and User’s Accuracy (UA) [33], which were also used by [31]. Using these metrics, [30] proved that GEOBIA separates tree species more reliably than the PB approach.

The primary goal of this research is the analysis of UAV imagery applicability and accuracy assessment of the most commonly used classification algorithms (MLC and SVM) [34,36,37,38,39] within PB and GEOBIA classification approaches. The secondary goal is to use different accuracy assessment metrics to determine which of the two tested classification algorithms (SVM and MLC) most reliably distinguishes olive tree crowns, and which approach is more accurate (PB or GEOBIA). The third goal is to add false polygon samples for COR, COM and OQ metrics, and use them to calculate Total Accuracy (TA).

The paper includes the following sections. Section 1. Introduction and Background and Section 2. Materials and Methods, which consists of Section 2.1. Study Area, Section 2.2. The Methodological Framework of the Research, Section 2.3. Field Research, Section 2.3.1. Data Acquisition, Section 2.4. UAV Imagery Processing, Section 2.5. Segmentation, Section 2.6. Adding Test Samples, Section 2.7. Classification and Section 2.8. Accuracy Assessment. Then follows Section 3. Results and Discussion, which consists of Section 3.1. Derivation of DSM and DOP, Section 3.2. Derivation of UAV_MS, Section 3.3. Segmentation, Section 3.4. Adding Test Samples, Section 3.5. Results of Classification Algorithms and Section 3.6. Accuracy of Classification Algorithms. The last section is Section 4. Conclusions.

2. Materials and Methods

2.1. Study Area

The study area for the extraction of olive tree crowns is the topographic basin of the settlement of Sali (Figure 1). It largely includes the Saljsko polje olive grove (202.1 ha), which protects the Natura 2000 ecological network as a special botanical reserve. The area is located in the south-eastern part of Dugi Otok, and contains olive trees up to 700 years old which, along with the size of the trees, makes this a unique ecological area within this area [40].

2.2. The Methodological Framework of the Research

The methodology is divided into the following steps: field research (1); development of digital orthophoto (DOP) and UAV_MS (2); UAV_MS segmentation (3); adding test samples (4); PB classification using two classification algorithms (MLC and SVM) (5.1); GEOBIA classification using two classification algorithms (MLC and SVM) (5.2); estimation of the accuracy of the model and selection of the best classification algorithm (6) (Figure 2).

2.3. Field Research

The field research (Figure 2(1)) was conducted on June 27 and 28, 2020. This period is most suitable for aerial photogrammetric imaging of olive trees due to the fact that pruning is already done, the culmination of vegetation and the minimal presence of shadows [41]. Two aerial photogrammetric surveys of the topographic basin of the Sali settlement were performed. The first survey used a camera that records the visible part of the spectrum (RGB), and the second used a multispectral camera. In the process of collecting RGB images, a UAV DJI Matrice 210 RTK V2 was used, on which a Zenmuse X7 camera was mounted (Figure 2(1)), while a DJI Matrice 600 Pro on which a MicaSense RedEdge-MX (Figure 2(1)) multispectral camera was mounted was used to collect multispectral images. An RTK-GPS Stonex S10 (Figure 2(1)) was used to collect ground control points (GCPs) and checkpoints (CPs).

2.3.1. Data Acquisition

The first step was to mark and collect points on a local geodetic basis. GCPs were collected to achieve a better absolute orientation of the model. A total of ten GCPs and five CPs were marked and measured in the official projection coordinate reference system of the transverse Mercator projection (HTRS96/TM). The points were collected at different altitudes, considering the rules for the spatial arrangement of placement of landmarks in photogrammetry (distribution throughout the entire survey plan) [42] (Figure 1). The next step was to create an optimal flight plan. This included selecting mission types considering terrain morphology, the research object, and the distribution of GCPs. The development of missions via UAV was performed in the DJI GS Pro software. Seven double grid missions with front and side overlap of 80% were planned for the RGB camera’s data acquisition in the topographic basin settlement of Sali, while seven single grid missions with front and side overlap of 80% were planned for data acquisition by the multispectral camera. Considering the comprehensiveness of the terrain and the desired spatial resolution (ground sampling distance: GSD) for DOP (GSD = 5 cm), the average flight or acquisition altitude was about 200 m, and for UAV_MS (GSD = 18 cm), it was 260 m. The compasses and inertial measurement systems (IMUs) of both UAVs were then calibrated, and a multispectral camera radiometric calibration was performed using a reflection calibration panel (CRP2). The multispectral camera was calibrated before and after each mission, in order to accurately display the light conditions during flight. The calibration process was performed in such a way that the CP2 was lowered onto a flat surface. Then, the camera was connected to the configuration page via a Wi-Fi network. Special attention was given to ensuring that the panel was not in the shade when taking a calibration photo, and that the sensor was at least 1 m above the CP2. Finally, the aerial photogrammetric recording was performed.

2.4. UAV Imagery Processing

The second step in extracting olive tree crowns was the production of DOP and UAV_MS images (Figure 2(2)). The collected RGB and multispectral images were processed using Agisoft Metashape Professional 1.5.1., one of the most commonly used software packages for photogrammetric image processing [43,44]. Thanks to implemented structure-from-motion (SfM) and multi-view algorithms, Agisoft enables 3D modelling based on reconstructing 3D structures from overlapping 2D images [45,46,47]. First, connection points were generated, and models were oriented with the help of collected GCPs. Then, dense point clouds and polygon networks were created, from which digital surface models (DSM) (Figure 3C), DOP (Figure 3A) and UAV_MS (Figure 3B) were derived.

2.5. Segmentation

The third step was the segmentation of UAV_MS (Figure 2(3)) based on the Mean shift approach [19] within ArcGIS. The Segment Mean Shift tool within ArcGIS identifies features or segments in imagery by grouping neighboring pixels together that have similar spectral, spatial, and geometric characteristics [19]. Since the characteristics of the image segments depend on the spectral detail, spatial detail and the minimum segment size, the optimization of the values of stated parameters was achieved. An iterative process (n = 64) was performed. The best combination of parameter values was selected based on the visual interpretation of the UAV_MS segmentation results.

2.6. Adding Test Samples

The fourth step was to add test samples (Figure 2(4), Supplementary Material Figure S1). This refers to the process of collecting test samples and verifying them. Samples were collected for the following 18 classes: Young Olive Trees, Old Olive Trees, Older Olive Trees, Oldest Olive Trees, Tile, Sheet Metal, Sheet Metal 2, Asphalt, Concrete, Shadow, High Vegetation, Macadam, Bare Stone, Quarry, Sea, Grass, Shrubby Vegetation, Brown Soil. Due to the possible input distortions of the sharpened multispectral image, a larger number of samples were marked. The collected test samples were checked by the cross-validation method. In total ≈ 1200 samples for all classes were collected.

2.7. Classification

The fifth step was classification (Figure 2(5)). Within the PB (Figure 2(5.1)) and GEOBIA (Figure 2(5.2)) classification approaches, MLC and SVM classification algorithms were tested. Classifications were based on selected samples. For this purpose, a tool called PvO-ACP (pixel vs. object automated classification process) was used (Supplementary Material Figure S2). This was created in Model Builder within ArcGIS. PvO-ACP allows simultaneous generation of PB and GEOBIA models via access based on selected parameters. The generated models were then reclassified into 12 classes (Figure 4) and finally into two classes: Olive Tree and Other, as in [28] (Figure 7).

2.8. Accuracy Assessment

The sixth step was to assess the accuracy of the models and select the best classification algorithm (Figure 2(6)). Estimating the accuracy of the generated models was based on COR, COM and OQ metric indicators. These metrics quantify the relationship between reference objects (reference olive trees, R_OT) and derived objects (classified olive trees, C_OT) and examine the accuracy of the executed classification [48]. The accuracy assessment was performed based on 13 polygon features of R_OTs (R_OT1–R_OT13) (Figure 4A2,B2,C2,D2 and Figure 7). R_OTs were selected using the Create Accuracy Assessment Points tool within ArcGIS. Olive trees on which a pixel was located, or was closest to them, were vectorized at a scale of 1:25 from the produced DOP with a spatial resolution of 5 cm. The authors in [26] also manually delineated tree crowns in their research, but they vectorized all the trees. The overlap area (A_O) of all four C_OTs was calculated. The values were used to assess the accuracy of the classification algorithms according to the following formulas:

$$\mathrm{COR}=\frac{{\mathrm{A}}_{\mathrm{o}}}{ {\mathrm{A}}_{{\mathrm{C}}_{\mathrm{O}\mathrm{T}}}}$$

(1)

where ${\mathrm{A}}_{\mathrm{o}}$ is the overlap area of R_OT and C_OT and ${\mathrm{A}}_{{\mathrm{C}}_{\mathrm{O}\mathrm{T}}}$ is the total area of C_OT

$$\mathrm{COM}=\frac{{\mathrm{A}}_{\mathrm{o}}}{{\mathrm{A}}_{{\mathrm{R}}_{\mathrm{O}\mathrm{T}}}}$$

(2)

where ${\mathrm{A}}_{\mathrm{o}}$ is the overlap area of R_OT and C_OT and ${\mathrm{A}}_{{\mathrm{R}}_{\mathrm{O}\mathrm{T}}}$ is the total area of R_OT

$$\mathrm{OQ}=\frac{{\mathrm{A}}_{\mathrm{o}}}{{\mathrm{A}}_{{\mathrm{R}}_{\mathrm{O}\mathrm{T}}}+{\mathrm{A}}_{{\mathrm{C}}_{\mathrm{O}\mathrm{T}}}-{\mathrm{A}}_{\mathrm{o}}}$$

(3)

where ${\mathrm{A}}_{\mathrm{o}}$ is the overlap area of R_OT and C_OT, ${\mathrm{A}}_{{\mathrm{R}}_{\mathrm{O}\mathrm{T}}}$ is the total area of R_OT and ${\mathrm{A}}_{{\mathrm{C}}_{\mathrm{O}\mathrm{T}}}$ is the total area of C_OT.

The COR, COM and OQ metrics values vary in the range of 0–1. Higher values indicate a higher match between reference and classified objects, i.e., higher accuracy of the classification algorithm [49].

Since it can be seen from the generated models that the classification algorithms in the PB models overestimated the area of olive tree crowns, it was necessary to add another 13 polygon features, representing false olive trees (F_OT). This is the first time, to the best of our knowledge, that false polygon samples have been added to these metrics. F_OTs were added on the same principle as R_OTs. Then, COR, COM and OQ values for F_OTs were calculated. The same methodology was applied as for R_OTs, except that in this case a lower value represents higher accuracy. The Total Accuracy (TA) was calculated by subtracting the F_OT indicator value from the R_OT indicator value according to the formulas below:

$$\mathrm{T}\mathrm{A}=\mathrm{C}\mathrm{O}{\mathrm{R}}_{\mathrm{R}\mathrm{O}\mathrm{T}}-\mathrm{C}\mathrm{O}{\mathrm{R}}_{\mathrm{F}\mathrm{O}\mathrm{T}}$$

(4)

$$\mathrm{T}\mathrm{A}=\mathrm{C}\mathrm{O}{\mathrm{M}}_{\mathrm{R}\mathrm{O}\mathrm{T}}-\mathrm{C}\mathrm{O}{\mathrm{M}}_{\mathrm{F}\mathrm{O}\mathrm{T}}$$

(5)

$$\mathrm{T}\mathrm{A}=\mathrm{O}{\mathrm{Q}}_{\mathrm{R}\mathrm{O}\mathrm{T}}-\mathrm{O}{\mathrm{Q}}_{\mathrm{F}\mathrm{O}\mathrm{T}}$$

(6)

TA values can vary between −1 and 1. A higher value represents higher accuracy.

The second accuracy assessment was performed using Producer Accuracy (PA) and User Accuracy (UA) for both classes. PA represents the probability that a reference pixel was classified correctly while UA represents the probability that a classified pixel represents that class on the ground [50,51]. Overall Accuracy (OA) was also calculated for estimation of classification rate correctness. It represents the quotient of the total number of correct pixels and the total number of pixels in the error matrix [50,51]. The accuracy assessment using these indicators was calculated in two ways. The first way was to randomly add 500 points using the Create Accuracy Assessment tool. Each generated point received ground truth data and classification data (for each of the four generated models). The values of PA, UA and OA were then calculated using the Create Confusion Matrix tool within ArcGIS, based on the formulas below:

$$\mathrm{P}\mathrm{A}\mathrm{i}=\frac{{\mathrm{p}}_{\mathrm{i}\mathrm{i}}}{{\mathrm{p}}_{+\mathrm{i}}}$$

(7)

$$\mathrm{U}\mathrm{A}\mathrm{i}=\frac{{\mathrm{p}}_{\mathrm{i}\mathrm{i}}}{{\mathrm{p}}_{\mathrm{i}+}}$$

(8)

$$\mathrm{O}\mathrm{A}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{m}}{\mathrm{p}}_{\mathrm{i}\mathrm{i}}$$

(9)

where ${\mathrm{p}}_{\mathrm{i}\mathrm{i}}$ is the major diagonal element for class I, ${\mathrm{p}}_{+\mathrm{i}}$ is the total number of observations in column i (bottom margin), ${\mathrm{p}}_{\mathrm{i}+}$ is the total number of observations in row i (right margin) and m is the number of rows, columns in the error matrix. PA, UA and OA values can vary between 0 and 1. A higher value represents higher accuracy.

Another way to calculate PA, UA and OA values was to create a point feature for each pixel based on the generated R_OT and F_OT polygons. Considering the spatial resolution of the UAV_MS, as described in [34], the Create Fishnet tool within ArcGIS generated a grid of polygons of 18 ∗ 18 cm within the R_OTs and F_OTs, with centroids that were used as input data to assess the accuracy of the models. Then, in the same way, the pixel value for each generated model was added for all point features, and PA, UA and OA were calculated using the Create Confusion Matrix tool.

Tertiary accuracy assessment was performed using the receiver operating characteristics curve (ROC) and the calculation of the area under the curve (AUC), which are widely used methods for estimating the accuracy of classification algorithms [52,53,54,55]. AUC values vary in the range of 0–1, with higher values representing greater accuracy of the model. Values <0.6 represent poor accuracy; 0.6–0.7 average accuracy; 0.7–0.8 good accuracy; 0.8–0.9 very good accuracy; and > 0.9 excellent accuracy [56,57]. The creation of ROC curves was automated with the Calculate ROC Curves and AUC Values tools within ArcGIS. The same 500 random points as those created within R_OTs and F_OTs for calculating PA, UA and OA were used as input data to assess the accuracy of the models and select the best classification algorithm.

3. Results and Discussion

3.1. Derivation of DSM and DOP

A total of 6587 high-quality images were collected in the planned seven double grid missions. The total error in five CPs used for the accuracy assessment of the generated model CPs was 5.83 cm. Suppose the spatial resolution of the aerial photogrammetric images and the purpose of the model are considered. In that case, this accuracy of the developed models satisfies the needs of the analyses for which the created models were used. By interpolating 1.5 ∗ 10⁹ dots within a dense cloud, a 5 cm spatial resolution DSM was generated (Figure 3C). Based on the generated DSM and point clouds, a 5 cm spatial resolution DOP was created (Figure 3A), containing three visible channels (RGB).

3.2. Derivation of UAV_MS

A total of 7245 high-quality images were collected in the planned seven missions. Five images were collected at each recording location, one for each multispectral camera channel (red, green, blue, red edge, near-infrared). It follows that a total of 7245 × 5 = 36,280 images were taken. By processing the images within Agisoft Metashape 1.5.1., very-high-resolution UAV_MS were generated. The combination of channels determined that the differences between vegetation species are best observed in the layout of channels 5-4-1 (Figure 3B). The UAV_MS, with this channel order, served as the basis for extracting classes with the PB approach, while for the GEOBIA approach, segmentation of the same image was performed.

3.3. Segmentation

As in [29], the iterative process of testing segmentation parameters based on trial-error procedure yielded optimal values for olive tree crowns extraction by the GEOBIA approach. Of the 64 derived models, a model containing the following parameter values was selected: (a) spectral detail = 18/20; (b) spatial detail = 15/20; (c) minimum segment size = 10/20 (Figure 3D). Given that the spectral detail is conditioned by the spectral resolution of UAV_MS [58], which affects the differentiation of different vegetation species, the selected value of the spectral detail allowed the separation of olive trees from other vegetation. The value of the spatial detail parameter, which determines the importance of feature proximity in the multispectral model, enabled the isolation of individual trees and reduced the influence of generalization during classification. The minimum segment size was set at 10 pixels which, given the spatial resolution of the UAV_MS of 18 cm, allowed mapping of all olive tree crowns larger than 0.324 m².

3.4. Adding Test Samples

Numerous factors such as shadows caused by tall objects or terrain morphology, sunlight angle, vegetation physiology, etc., affect differences in the spectral characteristics of elements of the same class within the same image [59]. For this reason, a total of ≈1200 samples were collected within 18 classes. Four classes related to olive trees whose spectral differences are influenced by ecological conditions, tree age, agronomic processes, variety, leaf growth rate and other abiotic and biotic stress factors [60].

3.5. Results of Classification Algorithms

SVM and MLC models for the PB and GEOBIA approaches (Figure 4A–D) were generated by the developed PvO-ACP tool. The PB approach has been observed to overestimate olive tree crown areas, especially in areas where pine forests predominate (Figure 4A1,B1). This is because the pixel as the minimum mapping unit is responsible for the “salt and pepper” effect. The GEOBIA-SVM model recognized the fewest olive trees in the pine forest (Figure 4C1), while the GEOBIA-MLC model had slightly lower results (Figure 4D1). The algorithms recognized olive trees in the shadows within all LCMs, but mostly in those generated by the PB approach, particularly the MLC classification algorithm. PB models are “grainy” (Figure 4A2,B2), while GEOBIA models are more compact (Figure 4C2,D2), especially the SVM model (Figure 4C2).

3.6. Accuracy of Classification Algorithms

Using COR, COM and OQ metrics on R_OTs, the results obtained showed that the GEOBIA-SVM model had the highest COR value (0.9064), which is similar to the results in [34] (Figure 5C). The PB-SVM model had the highest value of the COM (0.8772), as well as of the OQ (0.7767) (Figure 5A), which proves it to be the most accurate model. However, using F_OTs, it has been proven that the GEOBIA-SVM model overestimates the surface area of the Olive Tree class by the least amount. GEOBIA-SVM had the best index values (COR = 0.3796; COM = 0.0159; OQ = 0.0157) (

Table 1. Results of COR, COM and OQ metrics for (A) PB-SVM; (B) PB-MLC; (C) GEOBIA-SVM; (D) GEOBIA-MLC on F_OTs.

A	Test Area	COR PB-SVM	COM PB-SVM	OQ PB-SVM	B	Test Area	COR PB-MLC	COM PB-MLC	OQ PB-MLC
	FOT1	0.0337	0.9868	0.0337		FOT1	0.8858	0.3399	0.3257
	FOT2	0.3274	0.9510	0.3219		FOT2	0.8777	0.2543	0.2456
	FOT3	0.0564	0.9912	0.0564		FOT3	0.3164	0.0075	0.0073
	FOT4	0.0527	0.9838	0.0527		FOT4	0.4259	0.0203	0.0198
	FOT5	0.0876	0.9809	0.0875		FOT5	0.8458	0.1304	0.1274
	FOT6	0.0158	0.9911	0.0158		FOT6	0.6000	0.0126	0.0125
	FOT7	0.1362	0.9900	0.1360		FOT7	0.9134	0.1649	0.1624
	FOT8	0.0500	0.9791	0.0500		FOT8	0.2491	0.0150	0.0143
	FOT9	0.0068	0.9983	0.0068		FOT9	0.7076	0.0057	0.0057
	FOT10	0.1948	0.9937	0.1945		FOT10	0.9185	0.1961	0.1928
	FOT11	0.0397	0.9879	0.0397		FOT11	0.3581	0.0042	0.0041
	FOT12	0.0743	0.9824	0.0742		FOT12	0.6395	0.0771	0.0739
	FOT13	0.0347	0.9991	0.0347		FOT13	0.7583	0.0188	0.0187
	Total	0.7595	0.0728	0.0712		Total	0.6535	0.0959	0.0931
C	Test Area	COR GEOBIA-SVM	COM GEOBIA-SVM	OQ GEOBIA-SVM	D	Test Area	COR GEOBIA-MLC	COM GEOBIA-MLC	OQ GEOBIA-MLC
	FOT1	0.8343	0.0196	0.0196		FOT1	0.9367	0.6308	0.605
	FOT2	0.8622	0.1161	0.114		FOT2	0.8105	0.1945	0.186
	FOT3	0	0	0		FOT3	0.0476	0.0006	0.0006
	FOT4	0	0	0		FOT4	0.5009	0.009	0.0089
	FOT5	1.1588	0.0005	0.0005		FOT5	0.882	0.0201	0.0201
	FOT6	0.2821	0.0005	0.0005		FOT6	0.5186	0.0016	0.0016
	FOT7	0.8524	0.0169	0.0169		FOT7	0.8818	0.0833	0.0824
	FOT8	0	0	0		FOT8	0.3238	0.0094	0.0092
	FOT9	0	0	0		FOT9	0.6445	0.0018	0.0018
	FOT10	0.9445	0.0532	0.0531		FOT10	0.8923	0.1276	0.1256
	FOT11	0	0	0		FOT11	0.3753	0.0109	0.0107
	FOT12	0	0	0		FOT12	0.7343	0.0073	0.0073
	FOT13	0	0	0		FOT13	0.4043	0.0032	0.0031
	Total	0.3796	0.0159	0.0157		Total	0.6117	0.0846	0.0817

C). By calculating the TA, the GEOBIA-SVM model was shown to be the most reliable (TA COR = 0.5269; TA COM = 0.8110; TA OQ = 0.7450) (

Table 2. TA for PB-SVM, PB-MLC, GEOBIA-SVM and GEOBIA-MLC models.

	COR	COM	OQ
TAPBSVM	0.111	0.804	0.705
TAPBMLC	0.240	0.725	0.648
TAGEOBIASVM	0.527	0.811	0.745
TAGEOBIAMLC	0.234	0.778	0.658

). In addition, it has been proven that false samples in these metrics affect the change in accuracy results.

The confusion matrix results showed that the GEOBIA-SVM model had by far the largest UA and OA values, regardless of the number and arrangement of points at which accuracy was tested (

Table 3. PA, UA and OA for: (A-A1) PB-SVM; (B-B1) PB-MLC; (C-C1) GEOBIA-SVM; (D-D1) GEOBIA-MLC.

A	Class Value	Other	Olive Tree	Total	UA	OA	A1	Class Value	Other	Olive Tree	Total	UA	OA
	Other	351	10	361	0.972			Other	762662	2412	765074	0.997
	Olive tree	69	70	139	0.504			Olive tree	108345	15360	123705	0.124
	Total	420	80	500				Total	871007	17772	888779
	PA	0.836	0.875					PA	0.876	0.864
	OA					0.842		OA					0.875
B	Class Value	Other	Olive Tree	Total	UA	OA	B1	Class Value	Other	Olive Tree	Total	UA	OA
	Other	350	16	366	0.956			Other	731779	3505	735284	0.995
	Olive tree	70	64	134	0.478			Olive tree	139228	14267	153495	0.093
	Total	420	80	500				Total	871007	17772	888779
	PA	0.833	0.800					PA	0.840	0.803
	OA					0.828		OA					0.839
C	Class Value	Other	Olive Tree	Total	UA	OA	C1	Class Value	Other	Olive Tree	Total	UA	OA
	Other	400	17	417	0.959			Other	856246	3177	859423	0.996
	Olive tree	20	63	83	0.759			Olive tree	14761	14595	29356	0.497
	Total	420	80	500				Total	871007	17772	888779
	PA	0.952	0.788					PA	0.983	0.821
	OA					0.926		OA					0.980
D	Class Value	Other	Olive Tree	Total	UA	OA	D1	Class Value	Other	Olive Tree	Total	UA	OA
	Other	369	14	383	0.963			Other	799475	2600	802075	0.997
	Olive tree	51	66	117	0.564			Olive tree	71532	15172	86704	0.175
	Total	420	80	500				Total	871007	17772	888779
	PA	0.879	0.825					PA	0.918	0.854
	OA					0.870		OA					0.917

C, C1). The OA values of 0.926 and 0.980 are similar to the results in [27], where an OA value of 0.93 was obtained after certain manual corrections to the final model. Both MLC models had slightly lower accuracy but were still better than results obtained in [32] where an OA value of 0.69 was obtained for an MLC model generated by supervised classification. It has been proven that an increase in the number of test points increases the model’s accuracy according to the OA indicator, but the value of UA decreases significantly for the Olive Tree class. Furthermore, with an increase in the number of test points, the value of PA increased, except in the case of PB-SVM, where it decreased slightly but was still the highest (Table 3A, A1). As in [30], the OA proved that GEOBIA more reliably separates tree species than the PB approach. From the results, it can be seen that there are some general features in the results of confusion matrices derived based on a larger number of test samples. These refer to relatively low values of the UA metric, ranging from 0.093 (Table 3B1) to 0.497 (Table 3C1). This may indicate the problem of overestimating the number of olive trees in a specific area. Therefore, this approach must be applied with caution, and additional ground validation data used in addition to MS.

According to the derived ROC curves and AUC values, SVM (GEOBIA) (Figure 4C and Figure 7) is the most reliable classification algorithm for extracting olive tree crowns from UAV_MS imagery (Figure 6). Its accuracy has excellent values of 0.904 when using 500 random points (Figure 6A) and 0.929 when using points within R_OTs and F_OTs (Figure 6B). The GEOBIA-MLC model also has excellent accuracy (0.901) in points within R_OTs and F_OTs and very good accuracy (0.864) within 500 random points. SVM (PB) has very good accuracy within both approaches (0.833 and 0.874), while the less accurate model is PB-MLC, also with very good accuracy, but much lower at 0.826 and 0.813. The AUC indicator derived for 13 R_OTs and 13 F_OTs indicates the problem of shadows, which classification algorithms, especially in PB analyses, recognize as an olive tree, mostly in areas of dense pine forest and tall buildings. The SVM classification algorithm has proven to be an excellent solution for reducing shadow problems in GEOBIA analyses, while in the case of PB analyses, SVM is also a better solution than MLC as the latter produces “noise” (Figure 7).

4. Conclusions

SVM and MLC LCMs of the topographic area of the Sali settlement on the island of Dugi Otok were generated using the PB and GEOBIA approaches. All generated LCMs were reclassified into two classes: Olive Tree and Other. Accuracy assessment metrics showed that the GEOBIA approach generated more accurate models than the PB approach. Therefore, it was found that the applied classification algorithms achieved better results on the segmented image. Since GEOBIA groups pixels into homogeneous meaningful “superpixels”, various types of “noise” within the shaded areas (forests and tall buildings) occur much less frequently. These problems can often occur in the PB approach because classification algorithms within the PB approach classify each pixel separately. However, PB classification algorithms can produce results with satisfactory accuracy.

Although the accuracy assessment indicated that SVM (GEOBIA) was the best algorithm for extracting olive tree canopies, it is necessary to point out certain limitations. As the results indicate (Table 3), there may be a problem of overestimation (low value of UA) of the number of olive trees in a specific area. Therefore, this approach must be applied with caution, and ground validation data should be used.

The COR, COM and OQ metrics proved that all classification algorithms overestimated the area of olive trees (high false-positive rate), except the SVM (GEOBIA), which does so rarely, as 7 out of 13 false samples did not recognize the Olive Tree class. For this reason, the TA results indicated that the GEOBIA-SVM model was the most accurate (TA_COR of 0.527, TA_COM of 0.811, TA_OQ of 0.745). Furthermore, other metrics used also showed that GEOBIA-SVM was the most accurate classification method. More precisely, GEOBIA-SVM achieved an OA of 0.926 or 0.980 and an AUC value of 0.904 or 0.929, depending on the number and arrangement of accuracy assessment points. Limitations on the software used and the lack of licenses for other software, such as eCognition, limited the choice of algorithms. In future research, the accuracy of several segmentation methods (Multiresolution, Spectral Difference, etc.) and other classification algorithms (Hierarchical Classification, Random Forest, Fully Convolutional Networks, Deep Convolutional Neural Networks, etc.) will be examined. Furthermore, the influence of spectral resolution and UAV flight settings (flight altitude and camera calibration) on the accuracy of GEOBIA and PB classification algorithms will be determined.