A Copernicus Sentinel-1 and Sentinel-2 Classification Framework for the 2020+ European Common Agricultural Policy: A Case Study in València (Spain)

Abstract

This paper proposes a methodology for deriving an agreement map between the Spanish Land Parcel Information System (LPIS), also known as SIGPAC, and a classification map obtained from multitemporal Sentinel-1 and Sentinel-2 data. The study area comprises the province of València (Spain). The approach exploits predictions and class probabilities obtained from an ensemble method of decision trees (boosting trees). The overall accuracy reaches 91.18% when using only Sentinel-2 data and increases up to 93.96% when Sentinel-1 data are added in the training process. Blending both Setninel-1 and Sentinel-2 data causes a remarkable classification improvement ranging from 3.6 to 8.7 percentage points over shrubs, forest, and pasture with trees, which are the most confusing classes in the optical domain as demonstrated by a spectral separability analysis. The derived agreement map is built upon combining per pixel classifications, their probabilities, and the Spanish LPIS. This map can be exploited into the decision-making chain for subsidies payment to cope with the 2020+ European Common Agricultural Policy (CAP).

1. Introduction

The Common Agricultural Policy (CAP) is one of the most major European Commission (EC) policies. With a budget of €385 billion [1] (28.5% of the overall European budget in the period 2021–2027), the CAP provides support to European farmers with the aim of ensuring the provision of affordable food while also guarantying its quality on European markets. In this context, €65.2 billion (68.9% of the total CAP budget) is for direct payments. CAP requirements for payments are bound to the agricultural land cover and land use. Every member state authority must verify farmers’ declarations in order to comply with legal cross-compliance control mechanisms [2]. If farmers’ declarations do not fit the CAP requirements and standards, associated payments could be canceled. Paying agencies of every member state are responsible of supervising at least 5% of the received declarations. This is achieved by means of in situ field inspections and photo interpretation of very high–resolution images acquired from airborne or satellite platforms. Field inspections are expensive mainly when large areas have to be assessed. In addition, photo interpretation is a time-consuming task, which may lead to different results depending on both inspector visual skills and the quality of the images. On 22 May 2018, the EC adopted a new regulation for 2020+ CAP payments that include the possibility of using Earth Observation (EO) data from the Copernicus Sentinel-1 and Sentinel-2 programs for monitoring farmers’ parcels and assess cross-compliance [3]. This action will be applied to all declarations whereas only 5% of the declarations concerned by eligibility criteria, commitments and other obligations that cannot be monitored by EO data, will be supervised carrying out in situ checks. This oncoming regulation also attempts to alleviate the administrative red tape in order to facilitate the interaction between farmers and paying agencies.

Some of the most widely used remote sensing (cost-free) data in land cover and land use studies are provided by the National Aeronautics and Space Administration (NASA) (e.g., Moderate Resolution Imaging Spectroradiometer (MODIS) data) [4] and the United States Geological Survey (USGS) (e.g., Landsat data) [5,6]). They deliver spectral information of land surface at global coverage with a per pixel spatial resolutions ranging from 250 m–1 km, in the case of MODIS, to 30 m in the case of Landsat. Temporal resolution varies from 16 days for Landsat to 1–16 days for MODIS (depending on the temporal composite period of the product). These features make them suitable for classification and vegetation monitoring over large areas. However, this spatial resolution can be a limitation for classification mainly over very heterogeneous areas characterized by small landholdings. In the frame of the European Space Agency (ESA) Copernicus program, data from the Sentinel-1 [7] and Sentinel-2 [8] are freely disseminated. These data provide global information at decametric spatial resolution every 5–6 days, offering promising possibilities for land use classification. As a result, several classification studies have recently sprung up in the literature with the aim of classifying crop types, tree species, and grassland habitats [9,10,11,12,13].

In general, crop classification approaches can be faced at pixel level or at parcel level (i.e., object-based) depending on both the spatial resolution of available images and the characteristics of the target area. In object-based approaches, the unit to be classified is the so-called object, which is built up from the image segmentation, whereas in pixel-based approaches, pixels are classified individually without taking into account spatial aggregation. Irrespective of the classification unit, EO techniques for deriving classification maps are usually based on supervised learning. Among the most widely used supervised classification algorithms we can find decision trees (DT) [14], k-nearest neighbor (k-NN) [15], support vector machines (SVM) [16], and random forests (RF) [17]. SVM and RF usually trend to outperform DT and k-NN [18,19,20]. In addition, we can find several studies in the literature combining classifiers and multitemporal remote sensing data [21,22], leading to higher accuracies in the classification.

Although many studies have been undertaken automatic vegetation classification from remote sensing data, to date of writing, only a few of them are set up to cope with the context of European CAP subsidies [23,24,25,26]. However, none of them deals with the need for updating the Land Parcel Identification System (LPIS), which is key for both improving LPIS reliability and distribution of CAP payments while also reducing administrative costs [27]. In this context, this paper describes a classification framework from Sentinel-1 and Sentinel-2 developed for improving subsidies control for the CAP in the Valencian Community (Spain) and assessing the LPIS update. The classification exploits a time series of both optical and microwave data, vegetation indices (VIs), as well as features provided by assembling decision trees. The main novelties are two-fold. First, in this paper, we benefit from the full capabilities of ensemble classifiers’ outputs using in both class predictions and the class probability obtained from the ensemble of decision trees, which is not very often exploited in most of remote sensing classification studies. Second, we combine both the prediction and its probability with the Spanish LPIS to derive an agreement map between derived classification and Spanish LPIS. This achievement demonstrates its applicability for decision making in the framework of CAP subsidies.

2. Data Collection

2.1. Study Area

The study area selected in this work belongs to the land portion of an entire Sentinel-2 tile covering part of the Valencian Community (see red square in Figure 1). The Sentinel-2 tile is the T30SYJ embracing 110 km × 110 km. The selected study area falls within a Sentinel-1 tile in descending mode that allows for the combination of both sources of information. The study area is located over the province of Valencia (East of Spain), which has a typical Mediterranean climate, with an average annual temperature and humidity of 17 °C and 65%, respectively. The dominant non-urban classes over the study area are shrubs (SH), fruits (FR), citrus (CI), forest (FO), rice crops (RI), olive grove (OL), pasture with trees (PAT), vineyards (VI), dried fruit (DFR), and pasture (PA), which altogether cover the majority of the study area.

2.2. Sentinel-1 and Sentinel-2 Data

ESA provides open access to Copernicus Sentinel-1 and Sentinel-2 data from the Sentinels Scientific Data Hub (https://scihub.copernicus.eu/). In this study, we used the Sentinel-1 Level-1 Ground Range Detected (GRD) product that uses the Wide swath (IW) mode providing data in VV and VH polarizations. The data were calibrated to compute backscatter coefficient (sigma nought, σ⁰), and subsequently, a multilooking was applied in order to mitigate the speckle noise effect with a 2 × 2 window size. Eventually, in order to correct geometric distortions caused by slant range, layover, shadows, and foreshortening, a geometric correction was performed using the digital elevation model from the Shuttle Radar Topography Mission at 1″ resolution (SRTM, 1sec Grid). These processing were undertaken using the Sentinel Application Platform (SNAP).

Sentinel-2 top-of-atmosphere reflectances (Level 1C) were downloaded and atmospherically corrected using the Sentinel-2 atmospheric Correction (Sen2Cor) toolbox [28] to obtain top-of-canopy corrected reflectance for all bands excluding the B10 band (cirrus band), which does not contain surface information over the study area.

Sentinel-2 Level 1C data were downloaded jointly with the Sentinel-1 GRD data from April 2017 to March 2018, embracing 30 Sentinel-1 and 11 Sentinel-2 images for the study area (see

Table 1. Sentinel-1 and Sentinel-2 acquisition dates in 2017 and 2018.

	2017	2018
Sentinel-1	12 April, 24 April, 6 May, 18 May, 30 May, 5 June, 17 June, 29 June, 11 July, 23 July, 4 August, 16 August, 28 August, 9 September, 21 September, 3 October, 15 October, 27 October, 8 November, 20 November, 2 December, 14 December, 26 December.	7 January, 31 January, 19 January, 12 February, 24 February, 20 March, 8 March.
Sentinel-2	6 May, 26 May, 15 June, 13 September, 13 October, 2 December, 17 December.	21 January, 26 May 26, 15 February, 7 March, 27 March.

). Only completely cloud-free Sentinel-2 images were considered throughout the temporal window from April 2017 to March 2018, which was selected to account for a complete vegetation cycle of all classes within the study area.

2.3. Sistema de Información Geográfica de Parcelas Agrícolas (SIGPAC)

Several European countries, including Spain, have their own Land Parcel Identification System (LPIS), which is annually updated [23,24,25]. In Spain the LPIS is given by the Sistema de Información Geográfica de Parcelas Agrícolas (SIGPAC), which assigns to every agricultural parcel a unique land use (SIGPAC use). SIGPAC allows the geographic identification of parcels declared by farmers and livestock farmers, under any subsidy regime relating to the area which is cultivated or used for livestock. It was built upon an orthophotography mosaic embracing the Spanish territory, in which cadastral information collected by authorities is over-imposed (https://www.fega.es).

SIGPAC classifies the uses into categories including natural vegetation, urban zones and mixed classes. In this study we have selected the main 10 classes that cover the vast majority of agricultural parcels over the study area (listed in Section 2.1. and Section 2.3.). We have excluded urban areas and buildings, mixed classes, water bodies, and very-low representative classes. Figure 2 shows the considered SIGPAC classes over the study area in 2017 and the percentage coverage of every class.

2.4. Ground Truth

The Valencian regional government, specifically its Department of Agriculture, Environment, Climate Change, and Rural Development (http://www.agroambient.gva.es/va/), provided us with an extensive ground truth regarding SIGPAC classes (see

Table 2. Ground truth: number of pixels for every class.

Class	Shrubs	Dried Fruit	Fruits	Pasture	Rice	Forest	Vineyard	Pasture with Trees	Olive Groove	Citrus
No. Pixels	20,382	20,703	19,800	20,058	20,157	20,478	20,295	18,846	19,434	20,385

). It was obtained through several field campaigns and parcel inspections as well as thanks to the expert knowledge provided by regional authorities’ technical agronomists.

3. Methodology

The methodology is outlined in Figure 3 which includes the main following steps: (1) data collection of ground truth, and processing selected Sentinel-1 and Sentinel-2 images as described in Section 2.2; (2) feature selection and spectral separability analysis; (3) assessment of different classifiers accuracy over a test set never used during the training process; (4) selection of the best classification method and derivation of both a classification and a class probability map over the study area (masking out the sea and non-interest areas) and (5) generation of an agreement map between the derived classification map and SIGPAC.

3.1. Feature Selection

Although the downloaded Sentinel-2 images are cloud free, it may occur that the atmospheric correction provided artifacts or slight changes in reflectance over the same classes in different dates (changes not related to vegetation phenology), which would introduce noise in the training process, leading to worse classification results. To avoid non-optimal Sentinel-2 images, as well as to alleviate the Hughes phenomenon [29] related with the decrease of the classification performance as increasing the number of features, a Sentinel-2 image selection was made using the Jeffries-Matusita (JM) [30] and Bhattacharyya (BH) [31] distances. The JM distance is a measure of the average distance between a pair of two classes defined as

$$J_{ij}=2\left(1-e^{-B_{ij}}\right)$$

(1)

where B_ij is the Bhattacharyya distance given by

$$B_{ij}=\frac{1}{8} {\left({\mu }_i-{\mu }_j\right)}^t {\left[\frac{{{{\displaystyle \sum }}^{}}_i-{{{\displaystyle \sum }}^{}}_j}{2}\right]}^{-1} {\left({\mu }_i-{\mu }_j\right)}^t+\frac{1}{2}\ln \left[\frac{|{{{\displaystyle \sum }}^{}}_i-{{{\displaystyle \sum }}^{}}_j|/2}{\sqrt{|{{{\displaystyle \sum }}^{}}_i\left||{{{\displaystyle \sum }}^{}}_j\right|}}\right]$$

(2)

where µ_i and µ_j are the mean vectors of the two considered classes and Σ_i and Σ_j are the corresponding covariance matrices. The JM distance ranges from 0 to 2 as increasing separability between classes, whereas a B_ij larger value corresponds to higher average distance between classes. In the case of having normally distributed classes, J_ij becomes B_ij [30].

In addition to the selected Sentinel-2 images, a set of VIs (see

Table 3. Vegetation indices used in the feature space. Reflectance is denoted by ${\rho }_{\lambda }$ (λ in nm).

Vegetation Index	Equation
NDVI	$\frac{{\rho }_{842}-{\rho }_{665}}{{\rho }_{842}+{\rho }_{665}}$
OSAVI	$\frac{{\rho }_{842}-{\rho }_{665}}{{\rho }_{842}+{\rho }_{665}+0.16}$
NDVI705	$\frac{{\rho }_{842}-{\rho }_{705}}{{\rho }_{842}+{\rho }_{705}}$
OSAVI705	$\frac{{\rho }_{842}-{\rho }_{705}}{{\rho }_{842}+{\rho }_{705}+0.16}$
MCARI	$\left[\left({\rho }_{705}-{\rho }_{665}\right)-0.2\left({\rho }_{705}-{\rho }_{560}\right)\right]\left(\frac{{\rho }_{705}}{{\rho }_{665}}\right)$
PSRI	$\frac{{\rho }_{665}-{\rho }_{560}}{{\rho }_{740}}$

) were stacked to the Sentinel-2 images feature space (i.e., used as predictors during the training). In particular, we considered the wide-used Normalized Difference Vegetation Index (NDVI) [32] and its red-edge version (NDVI₇₀₅) [33], the Modified Chlorophyll Absorption in Reflectance Index (MCARI) [34], which accounts for chlorophyll changes, the Plant Senescence Reflectance Index (PSRI) [35] that is sensitive to plant senescence, and in order to minimize soil background noise, both the Optimized Soil Adjusted Vegetation Index (OSAVI) [36] and its red-edge version (OSAVI₇₀₅).

The selected Sentinel-2 images and VIs were jointly stacked with the available Sentinel-1 images to build the remote sensing data set (see Figure 4) used for the classification. Hence, the final training feature space was formed by both optical and SAR data. From the microwave domain, the VV and VH polarizations were used, as well as the VH/VV ratio, which is employed in crop classification because it is considered to be a good indicator of vegetation status [10,37]. In addition, to assess the impact of Sentinel-1 in the classification, an experiment was undertaken by removing Sentinel-1 data to the six selected Sentinel-2 images and VIs. In this case, the training feature space was formed only by optical data.

3.2. Classifiers

The ground truth was associated to the remote sensing data and was then split into two sets, training and test set, comprising 2/3 and 1/3 of the pixels, respectively. The big effort made by the Valencian authorities allowed us to build a huge training set with balanced classes, thus avoiding issues introduced by imbalanced training samples [12].

Several classification algorithms based on supervised classification were trained using the training set, and their respective accuracies were assessed over the test set (see results in Section 4.2). For instance, we used a linear discriminant analysis (LDA), which is a classification algorithm based on a linear transformation of variables that reduces an original feature space to a lower one that maximizes separability among class distributions [38]. In addition, non-linear approaches such as the quadratic discriminant analysis (QDA) and the k-NN classifier have been considered. The k-NN algorithm is based on neighbor similarity features where the prediction is done taking into account the k-nearest neighbors, where k is a real number typically ranging from 1 to a given number depending on image spatial features. We also considered the use of SVM, which is a kernel method (i.e., a method that samples input data into a Hilbert space) that divides the training data using hyperplanes maximizing the margin [16].

Finally, for having a more comprehensive sense, we decided to use ensembles of classifiers based on decision trees. For the ensemble, different techniques such as bagging, boosting, and random forest [39] were used. In the bagging ensemble, every decision tree is trained on a random subset of training samples. The same sample can be selected for training all, several, or even none of the decision trees composing the ensemble. In the case of RF, the approach is similar to bagging, but the difference relies in the split of the nodes of every tree. The bagging method uses all input features for splitting the nodes, whereas in the RF approach, the split is done considering a number of features (typically its square root), which are selected randomly. In the boosting method, trees are trained using all training samples in an iterative approach that increases the weights for samples classified incorrectly in previous training rounds. The boosting approach uses multiple iterations to generate a single composite strong learner. In this study, the Adaboost.M2 algorithm [40] was used for boosting decision trees. The algorithm takes a bootstrap sample in every boosting iteration that is used for building a weak classifier. Then the pseudo-loss is computed and used for updating the sampling weights to be used for the next iteration. The final assignation for every pixel is given by the most frequent class, obtained from all decision trees in the ensemble.

3.3. Agreement Map

An agreement map between the classification map derived from Sentinel-1 and Sentinel-2, and SIGPAC was calculated. For this purpose, a series of logical rules have been proposed in order to discriminate eight levels of agreement. The levels of agreement were obtained blending the information provided by SIGPAC, and both the derived classification and confidence maps as described in

Table 4. Agreement levels between classifications and SIGPAC.

Classification Map and SIGPAC		Classification Confidence	Level of Agreement
Same class	AND	>95%	Very high agreement
Same class	AND	70–95%	High agreement
Same class	AND	50–70%	Significant agreement
Same class	AND	<50%	Low agreement
Different classes	AND	<50%	Low discrepancy
Different classes	AND	50–70%	Significant discrepancy
Different classes	AND	50–70%	High discrepancy
Different classes	AND	>95%	Very high discrepancy

.

4. Results and Analysis

4.1. Sentinel-2 Spectral Separability Analysis

Sentinel-2 image selection was made using the aforementioned JM and BH distances over the optical domain (i.e., Sentinel-2 bands). Only the images that maximize the class separability were used, which leaded to the choice of 6 out 11 available and cloud free Sentinel-2 images.

Table 5. Jeffries–Matusita (JM) and Bhattacharyya (BH) distances for Sentinel-2 image acquired on 6 May 2017. Classes: shrubs (SH), fruits (FR), citrus (CI), forest (FO), rice crops (RI), olive grove (OL), pasture with trees (PAT), vineyards (VI), dried fruit (DFR), and pasture (PA).

	JM	SH	DFR	FR	PA	RI	FO	VI	PAT	OL	CI
BH
SH			1.750	1.597	1.508	1.996	0.636	1.870	0.253	1.497	1.879
DFR		2.078		0.392	0.698	1.983	1.863	0.466	1.717	0.402	1.439
FR		1.603	0.218		0.621	1.985	1.760	0.672	1.538	0.250	1.472
PA		1.403	0.429	0.371		1.986	1.653	1.202	1.410	0.505	1.123
RI		6.102	4.776	4.880	4.939		1.989	1.989	1.986	1.988	1.995
FO		0.382	2.679	2.120	1.752	5.228		1.951	0.296	1.713	1.899
VI		2.731	0.265	0.409	0.919	5.249	3.714		1.872	0.792	1.692
PAT		0.135	1.955	1.465	1.221	4.991	0.160	2.752		1.446	1.837
OL		1.381	0.225	0.133	0.291	5.135	1.942	0.504	1.283		1.382
CI		2.802	1.271	1.331	0.825	6.033	2.988	1.870	2.509	1.174

shows the JM (upper diagonal) and BH (lower diagonal) distances obtained for the first selected Sentinel-2 image, which was acquired on 6 May 2017 over the study area (for the sake of brevity, we show the rest of JM/BH tables in Supplementary Material (Tables S1–S5). It can be observed that several classes reach JM values close to 2, which highlights the good separability between pair of classes. It is worth mentioning rice is the most distinguishable class, exhibiting JM values very close to 2 and the highest BH values. On the other hand, forest, pasture with trees, and shrubs are the classes presenting the lowest JM and BH values (<0.5), which denotes their respective significant spectral confusion, being thus more difficult to classify, a priori. Similarly, fruits and olive grove are also classes with short distances between them and are difficult to discriminate. In addition, the JM and BH distances were computed taking into account all six selected Sentinel-2 images jointly (see

Table 6. Jeffries–Matusita (JM) and Bhattacharyya (BH) distances taking into account all the six selected Sentinel-2 images. Classes: shrubs (SH), fruits (FR), citrus (CI), forest (FO), rice crops (RI), olive grove (OL), pasture with trees (PAT), vineyards (VI), dried fruit (DFR), and pasture (PA).

	JM	SH	DFR	FR	PA	RI	FO	VI	PAT	OL	CI
BH
SH			1.750	1.597	1.508	1.996	0.636	1.870	0.253	1.497	1.879
DFR		2.078		0.392	0.698	1.983	1.863	0.466	1.717	0.402	1.439
FR		1.603	0.218		0.621	1.985	1.760	0.672	1.538	0.250	1.472
PA		1.403	0.429	0.371		1.986	1.653	1.202	1.410	0.505	1.123
RI		6.102	4.776	4.880	4.939		1.989	1.989	1.986	1.988	1.995
FO		0.382	2.679	2.120	1.752	5.228		1.951	0.296	1.713	1.899
VI		2.731	0.265	0.409	0.919	5.249	3.714		1.872	0.792	1.692
PAT		0.135	1.955	1.465	1.221	4.991	0.160	2.752		1.446	1.837
OL		1.381	0.225	0.133	0.291	5.135	1.942	0.504	1.283		1.382
CI		2.802	1.271	1.331	0.825	6.033	2.988	1.870	2.509	1.174

). In this case, the results are better than the ones obtained individually for each of the six selected Sentinel-2 images, which evince the use of multitemporal features improves spectral separability among classes. However, for forest, pasture with trees, and shrubs, although their spectral separability increase, remain the lowest.

4.2. Accuracy Assessment

Different classifiers were trained using the training set and their respective accuracies were assessed over the test set (constituted by one third of the pixels never used during the training).

Table 7. Overall accuracy (OA) and kappa index (κ) obtained in the test set for every classifier.

Multitemporal Features (Optical + SAR)	Overall Accuracy (%) (κ)
	LDA	QDA	k-NN	SVM	RF	Bagging Trees	Boosting Trees
6 × Sentinel-2 (12 bands) + NDVI 30 × Sentinel-1 (3 bands)	69.82 (0.68)	80.36 (0.79)	84.93 (0.83)	86.80 (0.85)	88.69 (0.87)	88.30 (0.87)	92.66 (0.90)
6 × Sentinel-2 (12 bands) + OSAVI 30 × Sentinel-1 (3 bands)	70.58 (0.69)	80.45 (0.79)	85.68 (0.84)	86.97 (0.85)	88.86 (0.87)	88.48 (0.87)	93.58 (0.91)
6 × Sentinel-2 (12 bands) + NDVI705 30 × Sentinel-1 (3 bands)	67.54 (0.66)	76.41 (0.75)	83.29 (0.82)	85.29 (0.84)	87.64 (0.86)	87.23 (0.86)	90.75 (0.89)
6 × Sentinel-2 (12 bands) + OSAVI705 30 × Sentinel-1 (3 bands)	70.95 (0.70)	80.50 (0.79)	85.94 (0.85)	86.99 (0.84)	89.05 (0.88)	88.76 (0.87)	93.96 (0.91)
6 × Sentinel-2 (12 bands) + MCARI 30 × Sentinel-1 (3 bands)	68.93 (0.68)	77.83 (0.76)	83.15 (0.82)	85.04 (0.84)	88.15 (0.87)	87.51 (0.86)	90.15 (0.89)
6 × Sentinel-2 (12 bands) + PSRI 30 × Sentinel-1 (3 bands)	66.94 (0.66)	71.60 (0.69)	76.83 (0.75)	84.07 (0.83)	87.10 (0.86)	87.08 (0.86)	88.96 (0.88)
6 × Sentinel-2 (12 bands) 30 × Sentinel-1 (3 bands)	66.73 (0.65)	70.23 (0.69)	75.84 (0.74)	85.59 (0.84)	86.80 (0.85)	86.25 (0.85)	88.19 (0.87)

shows the overall accuracy (OA) and kappa index (κ) [41] of the results taking into account the different features used in the classifications, i.e., if any vegetation index was stacked to Sentinel-1 and Sentinel-2 data. Among all evaluated classifiers and vegetation indices, the ensembles of decision trees revealed the highest accuracies being the boosting approach the most accurate using the temporal information of OSAVI₇₀₅. It is interesting to highlight that the boosting approach outperforms both the rest of ensemble approaches and the rest of classifiers irrespective of the VI. If no VI is used in addition to the selected images, the classification performance is reduced (i.e., OA = 88.19% and κ = 0.87 obtained without VIs vs OA = 93.96% and κ = 0.91 obtained in the best case).

Table 8. Confusion matrix of the classification obtained for the boosting approach using as predictors Sentinel-1 and Sentinel-2 imagery, and OSAVI₇₀₅. Classes: shrubs (SH), fruits (FR), citrus (CI), forest (FO), rice crops (RI), olive grove (OL), pasture with trees (PAT), vineyards (VI), dried fruit (DFR), and pasture (PA).

		Ground Truth										Total	UA (%)
		SH	DFR	FR	PA	RI	FO	VI	PAT	OL	CI
Classified	SH	5560	19	19	1	0	259	1	329	69	7	6264	88.8
	DFR	24	6619	10	1	0	7	5	12	72	12	6762	97.9
	FR	70	63	6450	4	1	36	20	36	87	16	6783	95.1
	PA	3	8	1	6661	0	3	2	1	10	3	6692	99.5
	RI	1	1	0	2	6717	1	0	1	1	0	6724	99.9
	FO	455	11	15	5	0	5845	1	349	38	12	6731	86.8
	VI	3	11	18	1	0	0	6719	1	35	3	6791	98.9
	PAT	500	36	27	2	1	569	0	5443	50	10	6638	82.0
	OL	175	118	53	7	0	105	14	104	6101	36	6713	90.9
	CI	3	15	7	2	0	1	3	6	15	6696	6748	99.2
Total PA (%)		6794	6901	6600	6686	6719	6826	6765	6282	6478	6795	OA = 93.96% κ = 0.91
		81.8	95.9	97.7	99.6	99.9	85.6	99.3	86.6	94.2	98.5

shows the corresponding confusion matrix (i.e., for the best result in Table 6). In general, the results are good, and the majority of classes reveal both user’s accuracy (UA) and producer’s accuracy (PA) higher than 90% and 94%, respectively. However, shrubs, forest and pasture with trees presented lower accuracies ranging 82–88% with confusion among them. It is worth mentioning that this behavior is in accordance with the a priori analysis carried out using the JM and BH distances. The best result is obtained over rice where both UA and PA reach 99.9%, whereas pasture with trees is the class with worst UA (82%), and shrubs is the class with worst PA (81.8%).

As highlighted by the aforementioned results, the family of ensemble of trees provided the highest accuracies. The sensitivity of the ensembles of decision trees to the number of trees was analyzed by executing different ensembles with increasing number of trees (see Figure 4). OA was calculated from 1 to 100 trees in steps of 10, and from 100 to 1000 in steps of 100. Results show that 1000 decision trees are sufficient for obtaining stable and accurate predictions. Belgiu and Drăguţ [42] suggested that 500 trees are acceptable for remote sensing studies, however other studies use a number of trees ranging from 10 up to 5000 depending on the number of features and spectral-temporal features of the remote sensing data used [43,44].

We also assessed the effect of removing Sentinel-1 data in the classification. In this case, the training feature space was formed only by optical data. The removing of Sentinel-1 images (microwave data) worsen classification accuracies in all cases (see

Table 9. Overall accuracy (OA) and kappa index (κ) obtained in the test set for every classifier using as predictors only Sentinel-2 data and VIs. Best result highlighted in bold.

Multitemporal Features (Optical + SAR)	Overall Accuracy (%) (κ)
	LDA	QDA	k-NN	SVM	RF	Bagging Trees	Boosting Trees
6 × Sentinel-2 (12 bands) + NDVI 30 × Sentinel-1 (3 bands)	69.82 (0.68)	80.36 (0.79)	84.93 (0.83)	86.80 (0.85)	88.69 (0.87)	88.30 (0.87)	92.66 (0.90)
6 × Sentinel-2 (12 bands) + OSAVI 30 × Sentinel-1 (3 bands)	70.58 (0.69)	80.45 (0.79)	85.68 (0.84)	86.97 (0.85)	88.86 (0.87)	88.48 (0.87)	93.58 (0.91)
6 × Sentinel-2 (12 bands) + NDVI705 30 × Sentinel-1 (3 bands)	67.54 (0.66)	76.41 (0.75)	83.29 (0.82)	85.29 (0.84)	87.64 (0.86)	87.23 (0.86)	90.75 (0.89)
6 × Sentinel-2 (12 bands) + OSAVI705 30 × Sentinel-1 (3 bands)	70.95 (0.70)	80.50 (0.79)	85.94 (0.85)	86.99 (0.84)	89.05 (0.88)	88.76 (0.87)	93.96 (0.91)
6 × Sentinel-2 (12 bands) + MCARI 30 × Sentinel-1 (3 bands)	68.93 (0.68)	77.83 (0.76)	83.15 (0.82)	85.04 (0.84)	88.15 (0.87)	87.51 (0.86)	90.15 (0.89)
6 × Sentinel-2 (12 bands) + PSRI 30 × Sentinel-1 (3 bands)	66.94 (0.66)	71.60 (0.69)	76.83 (0.75)	84.07 (0.83)	87.10 (0.86)	87.08 (0.86)	88.96 (0.88)
6 × Sentinel-2 (12 bands) 30 × Sentinel-1 (3 bands)	66.73 (0.65)	70.23 (0.69)	75.84 (0.74)	85.59 (0.84)	86.80 (0.85)	86.25 (0.85)	88.19 (0.87)

) compared to the respective results in Table 7. In this case, the best results were obtained when fusing the selected six Sentinel-2 multispectral images and the OSAVI₇₀₅. Again, the boosting approach outperforms the rest of classifiers (OA = 91.18% and κ = 0.88).

Table 10. Confusion matrix of the classification obtained for the boosting approach using as predictors only the six selected Sentinel-2 images, and OSAVI705. Classes: shrubs (SH), fruits (FR), citrus (CI), forest (FO), rice crops (RI), olive grove (OL), pasture with trees (PAT), vineyards (VI), dried fruit (DFR), and pasture (PA).

		Ground Truth										Total	UA(%)
		SH	DFR	FR	PA	RI	FO	VI	PAT	OL	CI
Classified	SH	5050	20	32	2	0	409	2	475	67	9	6066	83.3
	DFR	16	6519	9	16	0	27	19	16	72	30	6724	97.0
	FR	101	111	6349	17	3	40	70	25	87	40	6843	92.8
	PA	3	8	1	6566	0	21	2	4	37	4	6646	98.8
	RI	2	3	0	3	6714	2	0	3	2	0	6729	99.8
	FO	955	11	17	31	0	5515	1	474	39	14	7057	78.1
	VI	13	20	55	2	0	0	6593	8	97	4	6792	97.1
	PAT	519	34	51	14	2	689	1	5160	99	11	6580	78.4
	OL	129	155	78	32	0	109	62	103	5962	160	6790	87.8
	CI	6	20	8	3	0	14	15	14	16	6523	6619	98.5
Total PA (%)		6794	6901	6600	6686	6719	6826	6765	6282	6478	6795	OA = 91.18% κ = 0.88
		74.3	94.5	96.2	98.2	99.9	80.8	97.5	82.1	92.0	96.0

shows the confusion matrix over the test set obtained in this case. The results are not so accurate than the previous case when using both Sentinel-1 and Sentinel-2 data (see Table 8) revealing a general decrease in accuracy for every class. It is worth mentioning that in terms of percentage points (pp) in UA this loss is greater over shrubs (5.5 pp), forest (8.7 pp), and pasture with trees (3.6 pp), and in PA, the loss is 7.5 pp for shrubs, 5.6 pp for forest, and 4.5 pp for pasture with trees. In addition, the confusion among these classes was also increased. This confirms that the combined use of Sentinel-1 and Sentinel-2 is useful for improving classification results.

For completeness, an experiment taking into account all 11 available Sentinel-2 images in the training was conducted and revealed worse results (OA = 86.21% and κ = 0.82 obtained in the best case. For the sake of brevity, we show the results in Table S6 of Supplementary Material). This confirms that the previous Sentinel-2 spectral separability analysis was useful for improving classification results.

4.3. Classification, Class Probability and Agreement Maps

After the accuracy assessment over the test set, the best classifier (i.e., the ensemble of boosting trees) was executed (using both optical and SAR data as predictors) over the study area to obtain a classification map (see Figure 5a). In addition, the ensemble approach also provides a per pixel probability of belonging to every class taking into account the predictions of every tree forming the ensemble of 1000 boosting tress. The per pixel probability of the class with the maximum probability, which is indeed the predicted class, was used for deriving a map that can be interpreted as the confidence of the classification (see Figure 5b). The 80% of the pixels were classified by a confidence higher than 50% (see greenish colored areas in Figure 5, right, and cumulative histogram in Figure 6). The 50% of all pixels were classified by a confidence greater than 77% (see Figure 6). The maximum confidence was achieved over rice pixels (reaching almost 100% of confidence) that, jointly with the spectral analysis undertaken in Section 4.1, corroborates the high reliability in rice classification from multitemporal Sentinel-1 and -2 data.

Taking into account Table 4 described in Section 3.3, we derived the agreement between the predictions derived from Sentinel-1 and Sentinel-2 data and the SIGPAC (see Figure 7a). Most of the pixels show bluish colors, which indicate the consistency between the classifications performed in this study and the official land parcel information system of Valencia (see Figure 7a). The highest agreement is found over rice crops, whereas significant, high, and very high discrepancies are found over less than 3.5% of the total classified area over different classes (see Figure 7b). In addition, only 0.1% of the pixels are labeled as very high discrepancy, which highlights the robustness of the classifier.

4.4. Utility of the Derived Agreement Map

We provide two examples of the utility of the derived agreement map. The first example is shown in Figure 8, which displays a zoomed area of the three maps mentioned above, containing a very high discrepancy area (highlighted in red in western part of the displayed maps). It is classified as forest, while the SIGPAC classifies it as citrus. Since the probability of the classification map over this zone is >95%, following rules in Table 4, the agreement map assigns to it a very high discrepancy. If this polygon is superimposed over the official orthophoto of 2017 (provided by Institut Cartogràfic Valencià, València, Spain) over the zone, it can be observed that the class corresponds to forest instead of citrus. This means that in this case SIGPAC should be updated.

The second example is exhibited in Figure 9, which shows an example of high discrepancy between the derived classification and SIGPAC. The area highlighted in red is classified as fruits, while the SIGPAC classifies it as citrus. The agreement map assigns to it a high discrepancy. If this polygon is superimposed over the official orthophoto of 2017 over the zone (Figure 9 (middle)), no conclusion can be drawn, so a field inspection is recommended to check out the correct class. If the class is identified as citrus, the corresponding pixels should be labeled as citrus and incorporated in future trainings to avoid similar errors. If the field check verifies that the class is fruits, the SIGPAC should be updated.

5. Discussion

It is recognized that object-based approaches can be useful to delineate polygons over agricultural areas, but their effectiveness for delineating and classifying agricultural parcels using Sentinel-2 time series is reduced [12,45]. In addition, other studies using Sentinel-2 for crop mapping did not found advantages regarding the use of object-based or pixel-based approaches, also highlighting the impossibility of using textural features for small objects [9,46]. In this study, the classification was carried out at pixel-level since both the classification of and object/polygon, which can be actually done aggregating the result of a pixel-level classification, and the report, if there exist different classes over a polygon, are relevant for CAP payments.

A key aspect in every classification process is related with the training samples selection. The close collaboration with the regional authorities, particularly with its technicians, allowed obtaining a very high number of training samples per class. Our results derived from the JM-BH measures revealed that the highest spectral separabilities among classes were obtained using only six out of the available and cloud-free Sentinel-2 images. The inclusion of the remainder images would decrease accuracy. This is mainly due to the non-optimal spectral separability among classes in these dates and the addition of redundant information, which may introduce noise in the features space of the training samples. This type of feature selection can be useful in order to avoid Hughes phenomenon and improve classification accuracy. However, this process must be taken carefully since, if not properly addressed, relevant information can be lost [20].

The classification results reveal the joint use of Sentinel-1, Sentinel-2, and derived vegetation indices improves classification accuracies irrespective of the classification algorithm. Zhong et al. [47] recently reported that the use of a single multitemporal vegetation index outperformed the use of only available Landsat reflectances. However, unlike the present study, in their study, the classes were not spectrally separable. It is interesting to note that when stacking the OSAVI versions (i.e., using either the red (ρ₆₆₅) or the red-edge band (ρ₇₀₅)) the best results were obtained. This could be partly due to the OSAVI capabilities of minimizing the noise introduced by soil both at sparse and dense vegetation canopies [35]. In addition, the experiments reveal that the use of red-edge does not always contributes to improve classification accuracies. OSAVI₇₀₅ seems to capture information provided by the red-edge in the classification process and slightly outperforms the OSAVI. Conversely, classification accuracy decreases when using NDVI₇₀₅ if compared with cases using NDVI. Furthermore, the results confirmed that the combination of Sentinel-1 and Sentinel-2 leads to improved accuracy in the classification, which highlights the complementary information conveyed by Sentinel-1 and Sentinel-2. In particular, our results demonstrate that the inclusion of Sentinel-1 data is able to better discriminate classes such as shrubs, forest and pasture with trees that presented rather similar features among them in the optical domain of Sentinel-2.

Classification accuracy depends on a number of factors including the number of training and test samples, the number of features and its relevance, the number of classes and its similarity, and the classification algorithm itself. For instance, Schmedtmann and Campagnolo [24] proposed a reliable crop identification (12 classes, OA = 68%) over Portuguese parcels covering 1057 km², Sitokonstantinou et al. [25] developed a parcel-based crop identification (nine classes, OA = 91.3% and k = 0.87) scheme for CAP and greening obligations over a Spanish area of 215 km². Kanjir et al. [48] developed a change detection method to support land use control within CAP activities over three areas of 7 km² in Slovenia, and Blaes et al. [23] developed a framework for area-based subsidies control in Belgium. The present study reported overall accuracy reaching ≈ 94%, which is similar to the aforementioned studies. A remarkably high accuracy was found over rice highlighting that its multitemporal features are extremely informative for rice phenology and classification, which is also reported in other studies [47,49,50,51].

The number of available Sentinel-2 images proved to be sufficient for obtaining accurate and satisfactory classification results in most of the classes. This means that the temporal behavior of the majority of classes is captured with as few as six Sentinel-2 multispectral images. However, there was a lack of cloud free Sentinel-2 images during the summer period in 2017 whose data could be of relevance for our study. The inclusion of Sentinel-1 data tried to mitigate this effect and also incorporate information of vegetation structure that is not provided by optical sensors such as the MSI onboard Sentinel-2. Data provided by Sentinel-1 VV and VH polarizations include information on the vegetation status, structure, and their interaction with soil background [52,53]. In addition, the ratio VH/VV can provide time series information on vegetation and soil properties such as soil moisture also reducing SAR acquisition system errors [10,54,55].

With the implementation of the new 2020+ CAP regulation, field inspections are earmarked for only 5% of the territory. Therefore, the identification of zones with well-founded suspicions of incompatibility between farmers’ declarations and SIGPAC is key. The derived agreement map is intended to help in this process by highlighting agreement and discrepancies between the classification map retrieved with Sentinels data and SIGPAC. Certainly, the definition of the logical rules for deriving the agreement map influences the final result. The rules proposed in Table 4 are fairly reasonable. When some level of discrepancy is found between maps, a photo interpretation using very high–resolution imagery such as orthophotography is recommended. Then, if the interpretation tips the scale in favor of the reported classification, SIGPAC should be updated with this class. On the contrary, those pixels should be labeled as SIGPAC does and can be incorporated into the training process to avoid similar misclassifications in the future. If no conclusion can be drawn, a field inspection is needed. In this study, the agreement map reported less than 3.5% of the pixels as significant, high, and very high discrepancies, hence, in the extreme case of impossibility of establish a class by photo interpretation, the number of field visits would fall within the 2020+ CAP regulation allowed range. If the level of discrepancy is low, a field inspection is not recommended, just photo interpretation, since the confidence of the derived classifications is not very high (<50% according to the established rules in Table 4) and the classifier may not be performing well over those pixels.

6. Conclusions

A classification framework based on the predictions provided by an ensemble of decision trees using a boosting approach has been developed. Ten classes were classified covering an area of 3568 km², which is much bigger than the previous studies dealing with CAP framework. In addition, we also benefit from the per class probability given by the ensemble in order to develop an agreement map between derived classifications and the official LPIS of Spain (SIGPAC).

The results revealed an overall accuracy of 93.96% and κ = 0.9 when using both Sentinel-1 and Sentinel-2 data, and the boosting approach in an ensemble of decision trees. The addition of Sentinel-1 to Sentinel-2 data improved classification accuracy with regard to using only Sentinel-2 data. The accuracies were improved in all classes, and the inter-class confusions were reduced mainly for shrubs, forest, and pasture with trees, stressing the usefulness of blending Sentinel-1 and Sentinel-2 data. The use of VIs as predictors improved classification accuracies in all cases, being the OSAVI₇₀₅, the VI with which best results were obtained.

The derived agreement map reported less than 3.5% of the pixels as significant, high, and very high discrepancies with respect to the classification obtained from Sentinels data. In the case of needing a field inspection for checking the class, the number of field recognitions would fall within the 2020+ CAP regulation allowed range. The agreement map is a clear example of the utility of the proposed methodology and is intended to help in the CAP subsidies payment process. The derivation of this map is a major feature of this work, which is suitable to update Spanish SIGPAC as suggested by the European Court of Auditors also following with the 2020+ CAP regulation.