Multi-Temporal Sentinel-1 and Sentinel-2 Data for Orchards Discrimination in Khairpur District, Pakistan Using Spectral Separability Analysis and Machine Learning Classification

Abstract

Generating orchards spatial distribution maps within a heterogeneous landscape is challenging and requires fine spatial and temporal resolution images. This study examines the effectiveness of Sentinel-1 (S1) and Sentinel-2 (S2) satellite data of relatively high spatial and temporal resolutions for discriminating major orchards in the Khairpur district of the Sindh province, Pakistan using machine learning methods such as random forest (RF) and a support vector machine. A Multicollinearity test (MCT) was performed among the multi-temporal S1 and S2 variables to remove those with high correlations. Six different feature combination schemes were tested, with the fusion of multi-temporal S1 and S2 (scheme-6) outperforming all other combination schemes. The spectral separability between orchards pairs was assessed using Jeffries-Matusita (JM) distance, revealing that orchard pairs were completely separable in the multi-temporal fusion of both sensors, especially the indistinguishable pair of dates-mango. The performance difference between RF and SVM was not significant, SVM showed a slightly higher accuracy, except for scheme-4 where RF performed better. This study concludes that multi-temporal fusion of S1 and S2 data, coupled with robust ML methods, offers a reliable approach for orchard classification. Prospectively, these findings will be helpful for orchard monitoring, improvement of yield estimation and precision based agricultural practices.

1. Introduction

Fruit orchards play a vital role in global and regional economies, serving as cash crops with direct economic impacts. Accurate mapping of fruit orchards is crucial for research on their disease prevention, pest control, yield estimation, sustainable agricultural management and other socio-economic concerns [1]. Traditionally, orchard mapping requires time-consuming, labor-intensive and costly field surveys [2,3]. However, the development of remote sensing technologies allows for efficient mapping at low cost and at different geographic scales [4,5]. Remote sensing images, taking into account the spectral and temporal characteristics, are widely used to discriminate vegetation types based on their spectral signatures at different time periods [6,7]. Multi-temporal remote sensing images provide crucial time-based spectral information, particularly significant for orchard classification.

Compared to Landsat and MODIS satellites, the Sentinel-1 (S1) satellite, Synthetic Aperture Radar (SAR) of its C-band dataset and Sentinel-2 (S2) optical satellites have the advantage of their high spatial, temporal and spectral resolution, offering unique opportunities for global vegetation mapping [8]. Fusion of multi-temporal and multi sensor (MTMS) remote sensing images provides more appropriate datasets for vegetation types mapping [9,10]. Considering spectral similarity between vegetation types, single-date imagery may be unreliable [11,12]. Numerous studies have attempted to classify tree species using MTMS images [13,14,15,16], especially combining optical and SAR datasets [17,18]. The effective combination of optical and SAR datasets has shown advantages in vegetation classification [19]. Pham et al. [20] tested S1 alone, fusion of S1 and S2 datasets and only S2 to assess the land cover (LC) classification in a complicated landscape of agricultural in the Vietnam coastal area utilizing RF and OBIA algorithms. The findings indicated that using S1 alone had a lower accuracy result, with an accuracy rate of 60% against 63% and 72% accuracy rate, for S2 alone, as well as fused S1 and S2, respectively. The same conclusion was confirmed by [21]; both individual S1 and S2 produce lower accuracies as compared to the fusion of S1, S2 and Planet-Scope data, producing higher accuracy with a kappa coefficient greater than 0.8%. Similarly, Steinhausen et al. [22], gathered the optical and SAR data for high classification outputs. The utilization of optical and SAR datasets for vegetation mapping depends on the EM spectrum which is highly sensitive to the leaf structure, water and chlorophyll contents. SAR backscattering affects the roughness, structure, texture and the plant canopy dielectric properties [23]. Thus, combining these two datasets provides complementary information and improves accuracy in LC classification.

Accurate LC classification relies on advanced machine learning (ML) methods [24]. Thus, the selection of the finest classification method still remains a topic of ongoing debate within the remote sensing community [25]. Supervised ML methods, like Random Forest (RF) and Support Vector Machine (SVM) have shown high reliability and accuracy in remote sensing applications, especially in tree species classification tasks [26,27,28,29,30]. However, the performance variation among ML methods is often attributed to different combinations of remote sensing variables. While MTMS datasets enhance classification, they can be high-dimensional, redundant and multicollinear, leading to an increase in computational time and require more storage [31]. The inclusion of highly correlated variables can make the model sensitive to multicollinearity which affect the performance of the classification model, as well as reduce the precision and accuracy. Selecting the most useful variables from MTMS datasets yields more accurate results than using all the available variables. This was supported by [32,33], showing that images acquired in peak growth seasons are more correlated than images acquired in low growth seasons. Variable selection techniques, such as multicollinearity tests (MCT), help optimize the selection of variables for classification by removing highly correlated variables [31]. A threshold value of 0.7 is often considered appropriate for variable removal [34,35].

Pakistan is a rich country in terms of food production, particularly in major orchards contributing significantly to global exports. Notably, bananas, dates and mangoes play a pivotal role. Bananas hold a key position in the agriculture economy, covering an area of 34,800 hectares with a production of 154,800 tons [36]. Sindh province, where the soil and climatic conditions are favorable for banana cultivation, contributes 82%, while Punjab, Khyber Pakhtunkhwa and Baluchistan share 5%, 9.9% and 3%, respectively [37]. Mango, ranking high in both area and production, covers 156,000 hectares and 1,753,000 tons. Pakistan stands as the fourth largest mango producing and exporting country in the world [38]. Mango cultivation is prominent in Punjab and Sindh, recognized for the best quality mangoes in the world [39,40,41]. In dates production, Pakistan is the seventh largest producer with 557,279 metric tons in 2011, also leading in exports [42,43]. Dates are cultivated in all four provinces of Pakistan, with major producing districts including Khairpur and Sukkar (Sindh), Panjgur (Balochistan), Jhang, Muzaffar Garh and Multan (Punjab) and Dera Ismail Khan (Khyber Pakhtunkhwa). Sindh, particularly the Khairpur district, contributes significantly, accounting for approximately 85% of the total production with an area of 24,425 hectares yielding 164,653 tons [44,45].

Generally, several studies have been conducted using MTMS datasets for tree species classification. However, in Pakistan, no study has been conducted on orchards classification using MTMS images. Therefore, this study aims to utilize fine resolution multi-temporal S1 SAR and S2 optical images for orchard classification, focusing on the Khairpur district of Pakistan. Two well-known ML algorithms, RF and SVM, were applied to classify major orchard types including bananas, dates and mangoes. This study provides a relevant basis for applying fine spatial resolution, multi-temporal SAR and optical datasets for orchard classification, as well as supporting planting subsidies and production layout planning. The key objectives include (1) orchard discrimination with single and multi-temporal SAR and optical datasets using the JM Distance test, (2) comparative assessment of orchard classification abilities using RF and SVM methods, (3) evaluation of the significance of combining multi-temporal SAR and optical variables, along with vegetation indices, to improve orchard classification accuracy. The research outcomes are expected to provide a foundation for future applications of multi-temporal SAR and optical datasets in orchard classification.

2. Materials and Methods

2.1. Research Area

The research area of the Khairpur district is situated in the Southeast of Pakistan, bounded by Shikarpur and Sukkur districts in the North, Sanghar and Nawabshah districts in the South, with Larkana and the Indus River in the West and India to the East (Figure 1). The reason for selecting the Khairpur as a research area is that all the major orchards including bananas, dates and mangoes can be found in this region.

The topography of Khairpur can be divided into a plain cultivated area in the Northern and Western parts of the district and desert area to the East and Southeast. This area is very rich and fertile in the vicinity of the Indus River and its irrigation canals. The Eastern part of Khairpur lies in the Nara desert, which is part of the great Indian desert. This area is characterized by extreme temperatures and severe droughts. The district has two well defined seasons: hot summers which begin after March and continue until October as well as cold winters with December, January and February as the coldest months. In addition, dust storms are frequent during April, May and June while the cooling Southern winds during summer nights are the only soothing element. Mean annual rainfall in this area is 120 to 230 mm mostly occurring during July and August. Based on its natural conditions, the orchards in this focused study area have been planted for decades; hence, making it one of the major fruit production areas in Pakistan. The primary orchards here include dates, mango and banana plantations, while cotton, wheat and sugarcane are the other main agricultural crops.

2.2. Training and Validation Data

To train the ML algorithm for classification tasks, ground samples representing different LC types were needed. For this study, the training data were collected on a random samples basis, using android-based GPS application through a field survey conducted in late July 2022. Due to the harsh weather conditions and the high flood alert in this area during the survey period (July 2022), 70% of the samples were collected during the field survey while the remaining 30% were collected from high-resolution google earth images. The total 180 sample polygons representing four categories of LC (banana (50), dates (40), mango (30) and other LC (60)) were used as the training dataset (Figure 1B) for classification of LC. Other LC includes the mosaic of agriculture land, barren land, built-up area and water bodies. For accuracy assessment of the classified maps, overall, 625 systematic grid points were generated at a distance of 200 × 200 m throughout the area and validated during the survey and from Google Earth high resolution imagery (Figure 1A).

2.3. Data Processing

2.3.1. Sentinel-2 Data

In the present study, monthly based S2 Level-2 (L2A) images from January to December 2022, with cloud cover of less than 10% were downloaded from the website https://scihub.copernicus.eu/ assessed on 4 March 2023. L2A products have been pre-processed for atmospheric and geometric correction. S2 mission consists of the Sentinel 2A and 2B satellites equipped with a multispectral instrument, which provides global coverage at 5-days’ time intervals [46].

The S2 has 13 bands with different spectral resolutions ranging from 0.433 to 2.19µm. In visible and NIR regions, it has four 10 m bands. Six 20 m red-edge, NIR and SWIR bands as well and three 60 m bands for characterizing aerosol, cirrus cloud and water vapor [47]. In this study, 60 m bands including band-1, band-9 and band-10 were eliminated because these were specified for characterization of atmosphere and not for land surface observation. The 20 m spatial resolutions bands were resampled to 10 m spatial resolution utilizing an S2 resampling operator with nearest neighborhood techniques in SNAP 9.0 software.

2.3.2. Vegetation Indices

To measure the vegetation growth and health conditions, remote sensing indices are the main indicators. In this study, seven vegetation indices were used, calculated for all multi-temporal S2 images including Normalized Difference Vegetation Index (NDVI), Green Normalized Difference Vegetation Index (GNDVI), Transformed Normalized Difference Vegetation Index (TNDVI, Soil Adjusted Vegetation Index (SAVI), S2 Red-Edge Position (S2REP), Inverted Red-Edge Chlorophyll Index (IRECI) and Pigment Specific Simple Ratio (PSSRa). The formulas of these indices are given in

Table 1. Vegetation indices derived from the Sentinel-2 images.

Full Name	Index	Formula
Normalized Difference Vegetation Index	NDVI	$\frac{NIR - Red}{NIR + Red}$
Green Normalized Difference Vegetation Index	GNDVI	$\frac{NIR - Green}{NIR + Green}$
Transformed Normalized Difference Vegetation Index	TNDVI	$\sqrt{\left(\frac{NIR - Red}{NIR + Red}+0.5\right)}$
Soil Adjusted Vegetation Index	SAVI	$\left(1+L\right)\times \frac{\left(NIR - Red\right)}{\left(NIR + Red + L\right)}$
S2 Red-Edge Position	S2REP	$705+35\times \left(\right(Red+NIR)/2-Red2)/(Red3-Red2))$
Inverted Red-Edge Chlorophyll Index	IRECI	$(NIR-Red1)/\left(\frac{Red2}{Red3}\right)$
Pigment Specific Simple Ratio	PSSRa	$\frac{NIR}{Red}$

.

NDVI is the common vegetation index used to monitor vegetation density, intensity, health and growth [48]. It exploits the vigor and strength of the earth’s surface vegetation. The GNDVI developed by [49] was more reliable than NDVI in identifying the different concentration rates of chlorophyll. A green spectral band was shown to be more effective than a red band for assessing vegetation chlorophyll variability [49]. TNDVI is the square root of NDVI and indicates the relationship between the amount of green biomass in a pixel [50]. The SAVI introduced by [51] aims to be a cross between the ratio-based indices and the perpendicular indices. A soil adjustment factor (L) is included in SAVI to distinguish the impact of soil brightness. The factor ranges from 0 to 1 where the maximum value is used in regions with more visible soil. For intermediate vegetation densities, 0.5 is the conventional value that is mostly used in most applications [52]. The S2REP index based on linear interpolation was presented by [53]. Red-edge, as the point of strong red absorption to near infrared reflectance, includes the information on growth status and chlorophyl content of crops. The reflectance at the inflexion point was determined and REP was obtained through interpolation of S2 red-edge band-5 and -6, respectively. The IRECI incorporates four spectral bands to estimate canopy chlorophyll content [54]. The PSSRa, introduced by [55], investigates the range potential of spectral bands for quantifying pigment at the scale of the whole plant canopy. It has the strongest and most linear relationships with canopy concentration per unit area of chlorophyll and carotenoids.

2.3.3. Sentinel-1 Data

With the development of S1 mission, the SAR applications have gained increased usage due to the global data availability at free cost and their high temporal resolution. The S1 mission is a constellation of polar-orbiting Sentienl-1A and -1B satellites, prepared with C-band SAR sensors enabling data acquirement in all weather conditions [17]. S1 data are acquired in dual polarization with vertical transmit and horizontal receive (VH) and vertical transmit as well as (VV) backscattering, polarized vertical for transmission and horizontal/vertical for reception [56]. To examine the sensitivity of SAR backscatter for orchards classification, in the present study, multi-temporal S1 Level-1 Ground Range Detected (GRD) interferometric wide swath (IW) products were obtained with a temporal range of January to December 2022 (one image per month) from the website https://scihub.copernicus.eu/ assessed on 4 March 2023. The S1 images at GRD level are dual polarization VH and VV products which need to be preprocessed before being used for analysis. However, it required less processing time as compared to the Single Look Complex products.

The preprocessing of S1 GRD products includes four major steps: radiometric calibration, speckle filtering, radiometric terrain flattening and geometric correction. Radiometric calibration converts the pixels values to backscatter coefficients [57]. Before performing analysis in SAR, data speckle noise removal is necessary, as this degrades the quality of the image, making it difficult to understand the backscatter response from different LC [58]. To reduce the speckle noise effect, we applied a Lee filter with 3 × 3 window size to all multi-temporal images. The geometric correction of SAR images is also important because of the topographic variations in the scene and inclinations of the sensor, which distort the acquired images. As geometric correction is not considered in the GRD images, therefore, it is essential to perform and increase the Geolocation accuracy. The Range Doppler Terrain correction method was used for correction; we applied the Range Doppler Terrian Correction method by using a 30 m Copernicus Digital Elevation Model (DEM) to correct the topographic deformations. The Copernicus DEM was acquired automatically in SNAP software [59]. After the preprocessing, two polarization ratios were also calculated for all the multi-temporal images. The simple ratio of VV and VH (Equation (1)) and normalized ratio between the VV and VH bands (Equation (2)) [60] was calculated using the following formulas.

$$\mathrm{simple}\mathrm{ratio}=\frac{VH}{VV}$$

(1)

$$\mathrm{normalized}\mathrm{ratio}=\frac{VV-VH}{VV+VH}$$

(2)

2.4. Multicollinearity Test (MCT)

Utilization of MTMS data requires investigation of the correlation between MTMS variables. Using highly correlated variables can influence the model performance, using too many variables is also time consuming. Therefore, it was desirable to exclude the highly correlated variables and produce a model that can contribute to high accuracy and performance. In the present approach, based on the assumption that orchards may have the same spectral response at different time periods leads to correlation of variables, the MCT among all the multi-temporal S1 and S2 variables was performed using a psych package in R 2023.03 software, in order to eliminate the highly correlated variables. The analysis revealed that multi-temporal S2 spectral bands, vegetation indices and S1 bands and ratios were highly correlated. Hence, the threshold value of correlation >0.7, suggested in previous studies [34,35], was set to remove all the variables with a correlation greater than this threshold value. Figure 2 depicts variables used for classification after resolving the issue of multicollinearity.

2.5. Feature Combination Schemes

The total multi-temporal S1 and S2 variables were 252. However, after the MCT, only 56 multi-temporal variables were left among which 13 were S2 spectral bands, 16 vegetation indices, 5 S1 bands and 22 S1 bands ratios. To quantify the multi-temporal S1 and S2, both individually and combined impacts on the classification of orchard types, six different feature combination schemes were developed. These schemes were used as predictors for the classification of banana, dates, mango and other LC, in the study area. The details of each scheme are given in

Table 2. Feature combination schemes.

Schemes	Description	Number of Bands
Scheme-1	Spectral bands	13
Scheme-2	Spectral bands + Vegetation Indices	29
Scheme-3	SAR variables	27
Scheme-4	SAR variables + Vegetation Indices	43
Scheme-5	SAR variables + Spectral bands	40
Scheme-6	SAR variables + Spectral bands + Vegetation Indices	56

.

2.6. Spectral Separability Analysis

Spectral separability refers to quantifying the degree of separation between LC classes in different remote sensing images. Jeffries-Matusita (JM) distance is the indicator of separability commonly used in the applications of remote sensing [61,62,63,64,65]. JM distance was recommended to be more reliable for assessing separability [66] and demonstrated increased reliability for classes within homogenous distributions [67]. Therefore, we choose JM distance to examine the spectral separability among orchard types in multi-temporal S1 and S2 images.

JM distance [68] was calculated as

$$JM=2\left(1-e^{-B}\right)$$

(3)

where, Bhattacharyya distance is represented with B as given below:

$$B=\frac{1}{8}(x-y{)}^t{\left(\frac{{\Sigma }_x+{\Sigma }_y}{2}\right)}^{-1}\left(x-y\right)+\frac{1}{2}\ln \left(\frac{\left|\frac{{\Sigma }_x+{\Sigma }_y}{2}\right|}{\sqrt{\left|{\Sigma }_x\right|\cdot \left|{\Sigma }_y\right|}}\right)$$

(4)

where, x is the spectral response of the first object vector and y is the spectral response of the second object vector, $\Sigma$x and $\Sigma$y is the covariance matrix of samples x and y, alternatively.

JM distance value ranges from 0 to 2, where 0 value represents complete inseparability and a value of 2 represents completely separable [69]. We performed the JM distance analysis by using the training samples statistics at each class based on probability distribution.

2.7. Classification Algorithms

A variety of classification algorithms are available for LC mapping, but the ML-based RF and SVM methods are widely recognized for their reliable performance across a range of remote sensing applications [70,71,72,73]. RF is an ensemble learning method based on decision trees [74,75]. It predicts the classification class by constructing numerous decision trees, each trained on randomly selected samples [76]. Final classification is determined by majority votes across all trees [77]. Two parameters, namely the number of predictors (mtry) and the number of trees (ntree), must be chosen before training the RF model. For the presented dataset, mtry was set to 56 and ntree to 500. SVM suggested by [78], is a ML method based on a statistical theory that performs nonlinear classification using kernel functions [79]. Input vectors are mapped into a high-dimensional feature space through nonlinear transformation, and the SVM model identifies the best classification hyperplane to maximize the separation between classes in this space [80]. SVM primarily achieves fitting nonlinear functions by choosing a kernel function and other parameters, including gamma, which controls the smoothness of the hyperplane, and the cost, which controls the penalty for misclassification. In this study, we opted for the polynomial kernel function with gamma and cost values both set to 1.

2.8. Accuracy Assessment

To measure the performance of classification methods, accuracy assessment is very important. Commonly used accuracy assessment indicators are user accuracy (UA), producer accuracy (PA), overall accuracy (OA) and kappa coefficient (KC) [81,82], and are obtain through a confusion matrix [83]. A total of 625 systematic points, generated at a fixed distance of 200 × 200 m throughout the study area (Figure 1B), were used to calculate the confusion matrix and obtain the accuracy assessment indicators.

2.9. Feature Importance Measurement

Feature importance measurement aims to rank the relative importance of input multiple variables in ML models [84] and can be measured in several ways. The RF algorithms have benefits over other ML algorithms by having a built-in feature important measurement. The primary benefit of the RF method is that all the important input features can be calculated during the training of the RF model. In this study, Gini Index (GI) [85] was used for the feature importance measurement. The Gini index described the whole reduction in impurity of all nodes average and whole decision trees [86,87]. GI computes to reduce different samples through the whole nodes fragmented by a predictor variable in RF model. Gini examines the advantage of each feature, applying the information of all the features in the decision trees. However, all the features of the six classification schemes were input into the SVM without additional feature selection.

3. Results

3.1. Spectral Separability Analysis

The JM distance between each pair of orchard classes were calculated for single-date, multi-temporal and fusion of S1 and S2 datasets. The detailed results are shown in (Figure 3a) representing three pairwise orchards combinations. The horizontal axes represent datasets and vertical time scale individually. The color of each grid signifies the JM distance value of the resultant datasets and time. The redder the grid cell, the higher the JM distance value and more independent the two classes at time scale specifically. Banana was highly distinguishable from dates in blue, red, red-edge bands, NIR bands, NDVI, TNDVI, GNDVI, SAVI, PSSRA and IRECI index during the summer months (May, June, July and August), because these are the peak growing months (vegetation density is high) and SWIR-1 and SWIR-2 from January and February. The single date S1 variables throughout the year were not found suitable for separation of banana from dates. Because of the degree of lushness during the summer months, the mango and banana were highly distinguished in Red-Edge and NIR bands, GNDVI, SAVI, IRECI and S2REP and also in Red, SWIR bands and all vegetation indices expect S2REP from the winter months (January, February). However, no separation was detected between mango and banana in blue, green bands and S1 variables throughout the year. Dates and mango were least distinguishable from each other in all the S1 and S2 variables except for the summer months blue band, as well as September, October and November NDVI and TNDVI. However, different SAR and optical variables have different capacities for vegetation discrimination. All in all, red-edge, NIR bands and vegetation indices were superior to the other spectral bands and S1 variables. During the peak growing period (May to Aug), red-edge and NIR bands were slightly better than the vegetation indices in distinguishing banana from the other orchards.

Consequently, the JM distance of multi-temporal S1 and S2 variables was advance calculated as shown in (Figure 3b). The horizontal axes represent S1 and S2 variables, while the vertical axes represent the time scale in unit of two months, i.e., the two-month composite period. For the whole year, six temporal combinations were developed to calculate the JM distance. The results show that an increase in time series length increased the JM distance meaningfully. Once the time series length was greater than 4 months, banana-dates and mango-banana classes become almost completely separable, the JM distance reached 2.0, except for the simple ratio of VH/VV. However, the JM distance among dates-mango increased gradually with increasing the time series length. Once half year of time series composite was used, dates-mango were moderately separable. Furthermore, when the whole year composite was used, the JM distance reached 2.0 between dates-mango in blue band, red band, SWIR-1 band, NDVI, TNDVI and PSSRA only. For all the S1 variables, the dates-mango discrimination ability was lower even when the whole year composite was used. However, it can be observed that spectral bands and vegetation indices perform better than S1 variables. It is remarkable that for the most indistinguishable pair of dates-mango, multi-temporal S2 variables outperformed S1 variables.

The JM distance of the multi-temporal fusion of S1 and S2 features was also calculated, as shown in (Figure 3c). The results show that orchard classes were completely separable when multi-temporal optical and SAR features were combined while only date-mango was less separable in multi-temporal SAR features. In the fusion of multi-temporal S1 and S2 features, the JM distance value approached 2.0 for all the pairs of orchards including banana-dates, mango-banana and dates-mango. These results were in parallel with the classification results as, combining multi-temporal S1 and S2 features, achieved highest classification accuracy.

3.2. Assessment of Different Classification Schemes

Classification schemes with different features combinations were classified using RF and SVM methods. The overall accuracy and kappa value of each classification scheme are shown in (Figure 4). It was observed that RF and SVM algorithms achieved the highest accuracies of 93% and 94%, respectively, with scheme-6 which is the fusion of all multi-temporal S1 and S2 variables. Scheme-2 (optical bands + vegetation indices) ranked second with an accuracy of 91% and 93% for RF and SVM, respectively. Using only multi-temporal SAR variables (scheme-3) produces the relatively least reliable results with an accuracy of less than 80% in both RF and SVM methods. Compared with scheme-3, the combination of SAR variables with vegetation indices and optical bands (scheme-4 and scheme-5) significantly improved the accuracy of orchards classification. However, compared to other classification schemes, scheme-6 retrieved the best classification results in both RF and SVM methods. Hence, it revealed that a combination of multi-temporal optical and SAR features increased the classification accuracy.

3.3. Comparison of the Classification Algorithms

The comparative analysis revealed that the SVM method slightly outperformed the RF method in terms of accuracy across all classification schemes as shown in Figure 4. Across scheme-1 to -6, the OA and KC for SVM were consistently higher compared to RF, except in scheme-4 where RF showed a slight advantage with a marginally higher OA and KC value by 1% and 0.7%, respectively.

The PA and UA of each orchard class, classified by using different classification schemes with RF and SVM algorithms, are shown in (

Table 3. Confusion matrix of RF with different feature combination schemes.

Random Forest (RF)
Scheme-1	Others	Mango	Banana	Dates	User accuracy
Others	353	1	2	1	98.88
Mango	1	147	3	41	76.56
Banana	7	2	187	1	94.92
Dates	25	31	10	198	75
Producer accuracy	91.45	81.22	92.57	82.16
Scheme-2	Others	Mango	Banana	Dates	User accuracy
Others	354	0	1	3	98.88
Mango	6	154	5	3	91.67
Banana	11	0	191	6	91.83
Dates	15	27	5	229	82.97
Producer accuracy	91.71	85.08	94.55	95.02
Scheme-3	Others	Mango	Banana	Dates	User accuracy
Others	320	14	4	51	82.26
Mango	10	78	2	27	66.67
Banana	6	9	186	19	84.54
Dates	52	76	16	157	52.16
Producer accuracy	82.47	44.07	89.42	61.81
Scheme-4	Others	Mango	Banana	Dates	User accuracy
Others	363	1	14	5	94.78
Mango	2	143	0	6	94.70
Banana	2	1	177	5	95.67
Dates	19	36	11	225	77.32
Producer accuracy	94.04	79.00	87.62	93.36
Scheme-5	Others	Mango	Banana	Dates	User accuracy
Others	372	0	0	3	99.2
Mango	1	142	2	28	82.08
Banana	0	0	186	1	99.46
Dates	13	39	14	209	76
Producer accuracy	96.37	78.45	92.08	86.72
Scheme-6	Others	Mango	Banana	Dates	User accuracy
Others	368	0	0	3	99.19
Mango	4	150	1	5	93.75
Banana	3	1	193	1	97.47
Dates	11	30	8	232	82.56
Producer accuracy	95.34	82.87	95.54	96.26

and

Table 4. Confusion matrix of SVM with different feature combination schemes.

Support Vector Machine (SVM)
Scheme-1	Others	Mango	Banana	Dates	User accuracy
Others	357	0	3	0	99.17
Mango	0	161	2	11	92.53
Banana	2	1	188	0	98.43
Dates	27	19	9	230	80.70
Producer accuracy	92.49	88.96	93.07	95.44
Scheme-2	Others	Mango	Banana	Dates	User accuracy
Others	365	0	2	2	98.91
Mango	1	163	4	14	89.56
Banana	2	0	191	0	98.96
Dates	18	18	5	225	84.59
Producer accuracy	94.56	90.05	94.55	93.36
Scheme-3	Others	Mango	Banana	Dates	User accuracy
Others	324	2	1	28	91.27
Mango	3	121	5	49	67.98
Banana	1	1	190	4	96.94
Dates	60	53	12	173	58.05
Producer accuracy	83.50	68.36	91.35	68.11
Scheme-4	Others	Mango	Banana	Dates	User accuracy
Others	336	0	7	0	97.96
Mango	0	159	0	12	92.98
Banana	2	0	176	0	98.88
Dates	48	22	19	229	72.01
Producer accuracy	87.05	87.84	87.13	95.02
Scheme-5	Others	Mango	Banana	Dates	User accuracy
Others	357	1	3	0	98.89
Mango	0	154	2	19	88
Banana	2	1	190	0	98.44
Dates	27	25	7	222	79.00
Producer accuracy	92.49	85.08	94.06	92.12
Scheme-6	Others	Mango	Banana	Dates	User accuracy
Others	363	0	2	1	99.18
Mango	4	166	5	1	94.32
Banana	2	1	191	0	98.45
Dates	17	14	4	239	87.23
Producer accuracy	94.04	91.71	94.55	99.17

). The UA and PA of the mango class was relatively low with the SVM method and much lower with the RF method using multi-temporal SAR features only. In all other classification schemes, mango achieves greater than 70% UA and PA with RF and SVM methods, respectively. For the dates class, the highest PA with RF and SVM was reached by 96% and 99%, respectively, in scheme-6. However, the same approach produces the lower UA for dates as 82% and 87%, respectively, mainly due to confusion with mango class. The banana class with RF and SVM approaches achieves PA and UA higher than 90% in all the classification schemes except for scheme-3, while RF produced less than 90% PA and less than 86% UA in this scheme. Overall, the lowest UA (<70%) was obtained by mango and dates with both RF and SVM methods using scheme-3 (multi-temporal SAR features only). While in the same classification scheme, banana achieved a good UA (>80%) with both classification methods. The UA and PA of mango, dates and banana were relatively high with both classification algorithms using scheme-6. These findings were consistent with the JM distance separability analysis (as discussed in Section 3.1). This is easy to comprehend because the JM distance for all the orchard classes approaches 2.0 when all the multi-temporal features (scheme-6) were used.

3.4. Overall Classification Results and Accuracy Assessment

The spatial distribution maps of the classification from different feature combination schemes using RF and SVM are depicted in (Figure 5). It can be spatially observed that dates are largely distributed in the East and Southeast of the research area while mangos are in the middle and Southeast of the area. Bananas cover a large area and are distributed throughout the focused area. Other LCs, such as agricultural land, built-up areas and water bodies, collectively contribute a small amount of the total area. The classified maps obtained from the RF and SVM methods, using different features combinations, are consistent in most regions of the area. However, through the comparative analysis of the confusion matrixes results from RF and SVM methods (Table 3 and Table 4), a high confusion of mango and dates was found in all the classification schemes which leads to the misclassification of two classes. The classification results of RF and SVM algorithms for banana and other classes are relatively consistent among all the feature combination schemes. In addition, the speckled effect observed over the LC classes with the RF method as shown in Figure 5(b1,c1,d1,f1) was resolved with the SVM method (Figure 5(b2,c2,d2,f2)). Moreover, the RF produced relatively little “salt and pepper” maps as compared to SVM maps in all the schemes (Figure 5). Overall findings illustrate the synthesis of multi-temporal SAR and show that optical features have a significant role in the LC classification improvement.

3.5. Evaluating Features Importance for Orchards Classification

The feature significance scores were examined by the average reduction in the Gini Index (GI), using the RF model (Figure 6). Based on the top 20 important features in GI, multi-temporal SWIR and NIR bands, S2REP and NDVI indices are the most important features for orchards classification. Particularly, the multi-temporal S2REP index was more relevant, and in GI, the top 20 features has 7 multi-temporal S2REP indices, respectively.

When assessing the relevance of time periods, March and August were the most important months for the discrimination of orchard classes as three features from each month were ranked in the top 20 important features. In addition, March to October also appeared to be the critical time period as there was at least one high ranked feature from each month. February was the least important month as no single feature was ranked in the top 20 important features. Among the multi-temporal S1 features, October-VH was among the top 15 essential features. However, all other features were ranked much lower in GI scores.

4. Discussion

4.1. Multi-Temporal Imagery

We applied two approaches to examine the orchards judgment capacity of multi-temporal S1 and S2 datasets. In the first approach, JM distance was used to specify the separability among each orchard pair in single date images, multi-temporal and fusion of multi-temporal S1 and S2 images. The whole output illustrates that the multi-temporal data show superiority to single-date data. During the summer months (May to August), red-edge bands and NIR bands perform better to distinguish banana-dates and mango-banana. However, the performance for dates-mango in the same months was relatively poor. Date-mango only showed dissimilarity in the blue band during the growing season which may be linked to the blue band sensitivity to the water content in vegetation [88,89] and vegetation indices during monsoon season. In addition, the SWIR bands also showed good performance in January and February for discriminating banana-dates and mango-banana. These results correlate with the results obtained in other studies. Nelson and Immitzer et al. [90,91] reported that the NIR, SWIR and red-edge bands were among the most important for discriminating different LC. On the other hand, orchard types were difficult to differentiate from each other in all the single-date S1 variables. Also, the multi-temporal VH/VV ratio showed no advantage in discriminating orchard pairs on all dates. The VV and VH backscatter are more useful than VH/VV for discriminating orchard types [92]. Also, in the dates-mango discrimination, all the S1 multi-temporal variables showed almost no discrimination or very low JM distance value. Furthermore, the JM distance of the S1 and S2 variables mostly approached 2.0 when more than half year multi-temporal images were combined (Figure 3b). Hence, it can be concluded that S1 data could not be used alone due to the limited number of channels for discriminating the orchard types [93]. However, the fusion of multi-temporal S1 and S2 images demonstrate exceptional performance in the separation of orchards pairs. Similar to previous studies on fusion of S1 and S2 [94,95,96], from the combination of multi-temporal SAR and optical imageries, high classification accuracy was attained, compared with the result of multi-temporal SAR and optical alone.

In the second approach, we implemented RF and SVM algorithms to classify orchard categories in the study area by using different multi-temporal features combination. The best feature combination scheme was made up of a fusion of multi-temporal S1 and S2 features (scheme-6) with an overall accuracy of 93% and 94% with SVM and RF methods, correspondingly. This indicates that multi-temporal remote sensing imagery complements the information in single-date images and enhances the predictive power of the classification models. Moreover, these outcomes support previous studies concerning LC classification using multi-temporal data [90,97,98]. Hill et al. [99] utilized multi-temporal high-resolution Airborne Thematic Mapper images for the classification of the deciduous tree species. It was found that a single image provides satisfactory results with an accuracy of 71%, but combined multi-dates images notably achieves a higher accuracy of 84%. Kollert et al. [100] applied multi-temporal S2 images for the classification of tree species and their output illustrated that classification output attained with S2 temporal images was 10% higher than the single S2 images. Additionally, Zhu and Liu [101] achieved the highest accuracy of 88% for forest type classification using dense time series imagery. In our research, the multi-temporal optical bands (scheme-1) performed well independently but the overall accuracy plateaued when additional multi-temporal features were combined. Notably, this could be due to the spectral variation in orchard types being different in different time periods. Moreover, the multi-temporal SAR images (scheme-3) poorly performed, likely due to the small variations in backscatter coefficients of the orchard types. However, together with either optical bands or vegetation indices, it made up good classification results. These findings were similar to the findings of [102], who assessed the potential of S1 alone, S2 alone and combination of both S1 and S2 data for mapping forest-agricultural mosaic over Cantabrian Range, Spain and Paragominas in Brazil. The results obtained from S2 alone were more accurate than S1 alone. However, the highest accuracy was achieved by the fusion of S1 and S2. This may be due to the fact that SAR sensors are sensitive to vegetation structure while optical sensors are sensitive to the concentration of vegetation chlorophyll. Consequently, upon examination of a 10m spatial resolution, orchard classes can be easily distinguished based on their physiology and physical structure. Patel et al. [92] found that both polarizations VV and VH and their ratio VH/VV contributed equally but to a low importance in classifying the forest-agricultural mosaic. Thus, this leads to the conclusion that only the C-band SAR dataset is not appropriate to classify orchard types. Generally, C-band SAR has a faster single saturation rate and lower penetration depth than L-band SAR, it is less effective for monitoring trees [103]. Furthermore, Patel et al. [92] also highlighted that L-band is more sensitive to a variation in plant density than the C-band, which interacts mostly with the primary branches of the canopy, while the L-band is within the vegetation canopy. However, in the case of bare soil and artificial surfaces, the SAR data sometimes discriminate better than the optical data as the SAR single reacts differently to these two LC classes [104].

4.2. Performance on Orchard Types

In this study, three major orchard types in the focused study area including dates, mango and banana were classified with higher accuracy through the fusion of S1 and S2 multi-temporal data. Among these orchards, mango-dates were highly confused with each other in all the multi-temporal feature combination schemes. This was probably due to the reason that these two orchards mostly have a heterogenous pattern inside the study area shown in (Figure 5). Mango has the lowest PA of 44% and 68% in SAR data only, using RF and SVM, respectively. Similarly, using SAR data only, the dates retrieved the lowest accuracy of 61% and 68% with RF and SVM, respectively. While for both mango and dates, the PA reached 91% and 99% in a combination of multi-temporal SAR and optical datasets. As noticed, RF showed a substantial underestimation of orchard types, which produced low PA as relate to SVM method. The difference in the confusion matrices, from scheme-1 to scheme-6, is that the PA of mango and dates greatly improved. The reason is that the combination of multi-temporal S1 and S2 data increased the two classes separability. As compared to mango and dates, banana was less confused with these and vice versa in all the combination schemes. It could be due to the fact that bananas have a homogenous pattern in the research area. The research findings indicated that in similar prospective studies, bananas can be separated by using multi-temporal S1 and S2 data only. The class specific UA and PA is reported in (Table 3 and Table 4). However, for illustrative purposes, thematic maps (Figure 5) have also been included. For similar prospective studies, thematic mapping focusing on this study area may suggestively incorporate more LC classes, such as agricultural crop types, barren land and built-up area. The outputs of the present research are encouraging; homogenous areas of orchard types instead of mixed tree species, which vary greatly on spectral basis and are generally more challenging to classify [105] have been explored.

4.3. Feature Importance

In the present study, the MCT excluded 196 of the 252 features, presumably because they were highly correlated (R² > 0.7) with the 56 features that were selected. Among these 56 features, the temporal SWIR band, NDVI and S2REP indices were ranked highly in the Gini Index, as well as typically from the growing month which strengthened the position of these images. Furthermore, a majority of their S2REP indices ranked among the top 10 features. However, discrepancies occur regarding the importance of specific features. The study’s results from [90,91] reports that the blue and red bands were more critical. However, in this study, these bands were removed due to their high correlation with other bands.

4.4. Performance of Classification Algorithms

In this study, the classification performance of RF and SVM algorithms was evaluated. These algorithms demonstrate less sensitivity to the choice of input features [18] and are equally reliable [106,107,108]. The maps in (Figure 5), obtained through RF and SVM methods with different features combinations, were quite similar, except for differences in accuracies. Indeed, the kappa values and overall accuracy revealed discrepancies in the output classifications. Overall, SVM performance is superior to the RF algorithm which is consistent with previous studies [109,110]. However, other research studies comparing SVM and RF have reported contrasting results [111,112,113] possibly due to the application of a small training dataset. The RF model used a huge number of decision trees for the training data of high dimensional tree selection [85]; hence, it is crucial to train the RF model with a small dataset [114]. The SVM model can escape a small training dataset, as well as high-dimensional decision trees selection [79]. Moreover, other studies of SVM and RF comparisons propose the advantages of SVM method [115,116,117]. The details of the kappa coefficient and the overall accuracy attained by the SVM and RF methods using different classification schemes is shown in (Table 3 and Table 4).

5. Conclusions

This research examined the effectiveness of S1 and S2 multi-temporal SAR and optical data for orchard classification using two ML algorithms in the sub-arid region of Pakistan. The use of multi-temporal SAR data proved unreliable for the orchard discrimination, due to insufficient spectral dissimilarity. Though the multi-temporal S2 optical data performed relatively better than the SAR data, the fusion of multi-temporal S1 and S2 showed that high classification accuracy can be achieved regarding the accuracies with each sensor independently. In addition, the spectral dissimilarity in the fused datasets is high as compared to the single-date or temporal S1 and S2 datasets. Particularly, for the most indistinguishable pair dates-mango, fused data achieve a relatively high JM distance value and overall accuracy. However, the results of the present research suggested most multi-temporal variables are highly correlated; hence, there is need to conduct the MCT and remove highly correlated variables to reduce overfitting, as well as the time needed for analysis.

Although SVM was superior to RF in orchard classification, the differences in their performance were not significant, suggesting RF can also be considered a viable option. The important features ranked by the Gini Index were multi-temporal SWIR bands, NIR bands, S2REP and NDVI indices. Considering the S1 bands and ratios derived from these bands, multi-temporal S2 perform best in the orchard discrimination. The findings demonstrated that the multi-temporal fusion of SAR and optical is well-suited for orchard mapping using ML methods and can be applied to future research such as vegetation mapping, change detection and monitoring in the heterogeneous landscapes. The outcomes provide baseline information for the spatial-distribution-based mapping of orchards, yield estimations and agricultural practices. Future research should consider advanced deep learning and fusion techniques for improving LC classification and include more LC classes for detailed spatial distribution-based maps.