The Importance of Agronomic Knowledge for Crop Detection by Sentinel-2 in the CAP Controls Framework: A Possible Rule-Based Classification Approach

Abstract

Farmers are supported by European Union (EU) through contributions related to the common agricultural policy (CAP). To obtain grants, farmers have to apply every year according to the national/regional procedure that, presently, relies on the Geo-Spatial Aid Application (GSAA). To ensure the properness of applications, national/regional payment agencies (PA) operate random controls through in-field surveys. EU regulation n. 809/2014 has introduced a new approach to CAP controls based on Copernicus Sentinel-2 (S2) data. These are expected to better address PA checks on the field, suggesting eventual inconsistencies between satellite-based deductions and farmers’ declarations. Within this framework, this work proposed a hierarchical (HI) approach to the classification of crops (soya, corn, wheat, rice, and meadow) explicitly aimed at supporting CAP controls in agriculture, with special concerns about the Piemonte Region (NW Italy) agricultural situation. To demonstrate the effectiveness of the proposed approach, a comparison is made between HI and other, more ordinary approaches. In particular, two algorithms were considered as references: the minimum distance (MD) and the random forest (RF). Tests were operated in a study area located in the southern part of the Vercelli province (Piemonte), which is mainly devoted to agriculture. Training and validation steps were performed for all the classification approaches (HI, MD, RF) using the same ground data. MD and RF were based on S2-derived NDVI image time series (TS) for the 2020 year. Differently, HI was built according to a rule-based approach developing according to the following steps: (a) TS standard deviation analysis in the time domain for meadows mapping; (b) MD classification of winter part of TS in the time domain for wheat detection; (c) MD classification of summer part of TS in the time domain for corn classification; (d) selection of a proper summer multi-spectral image (SMSI) useful for separating rice from soya with MD operated in the spectral domain. To separate crops of interest from other classes, MD-based classifications belonging to HI were thresholded by Otsu’s method. Overall accuracy for MD, RF, and HI were found to be 63%, 80%, and 89%, respectively. It is worth remarking that thanks to the SMSI-based approach of HI, a significant improvement was obtained in soya and rice classification.

1. Introduction

1.1. EU CAP

The common agricultural policy (CAP) is managed and funded by the European Union (EU) to support farmers and ensure food security. CAP’s main goals can be summarized in: (a) supporting farmers and improving agricultural productivity, ensuring a stable supply of affordable food [1]; (b) safeguarding EU farmers by guaranteeing a reasonable living; (c) helping tackle climate change [2] and the sustainable management of natural resources [3]; (d) maintaining rural areas and landscapes across EU [4]; (e) keeping the rural economy alive by promoting jobs in farming, agri-foods industries, and associated sectors (ec.europa.eu). CAP operates through two funds that are used to support main agricultural activities: the first and second pillars. These are respectively named EAGF (European Agricultural Guarantee Fund) and EAFRD (European Agricultural Fund for Rural Development) [5]. The former (EAGF) is linked to the Common Market Organization (CMO), which supports farmers with direct payments that reward actions promoting market stability, agricultural production increases, and environmental sustainability. The latter (EAFRD) promotes sustainable rural development and, in particular, climate change adaptation, fostering agricultural competitiveness, ensuring sustainable management of natural resources, and developing rural economies and communities [6].

Since CAP operates in all EU member states, it is hardly conceivable that payments should be centralized. Therefore, for efficiency and simplicity, payment management is delegated to the member states. In Italy, funded with a total of 41.5 billion euros in the 2014–2020 CAP, several regional payment agencies (PA) exist and refer to the national one (AGEA–Italian Agency for Payments in Agriculture). According to Art. 17 of Reg. (EU) n. 809/2014 [7] application for CAP subsidies must be presented by the farmer to the competent PA through the Geo-Spatial Aid Application (GSAA). GSAA contains information about managed fields (land use, location, and size) for which CAP contribution is requested. Additionally, it allows the unique identification of agricultural parcels populating databases useful for systematic checks [8]. GSAA data are managed by farmers through a GIS (geographic information system)-based system. PA verifies application compliance with requirements through the Integrated Management and Control System (IACS), relying on administrative (AC) and spot controls (SC) [9]. AC is automatically carried out by IACS for all the applications (100%) with the aim of verifying their compliance: with eligibility criteria and maintenance of long-term commitments, with due deadlines, and with required documentation. Moreover, IACS verifies that no other EU grant is financing the same field [10]. As far as SCs are concerned, they are performed on a sample subset of GSAAs corresponding to 5% of the total. SCs are intended for: (i) checking the truthfulness of declared areas; (ii) verifying compliance of the application with the eligibility criteria; (iii) testing commitments and obligations from farmers according to the declared crop. Field selection by SCs is managed for 80% according to risk-based criteria and for the 20% randomly. SCs are presently performed by photo-interpretation of high-resolution satellite/aerial images and/or, in specific and rare cases, by direct ground controls (GC).

1.2. Supporting CAP Controls by Copernicus Satellite Data

Copernicus is the EU’s Earth Observation (EO) Programme aimed at monitoring the Earth and its environment. Specifically, Sentinel-1 (S1) and Sentinel-2 (S2) missions are extensively used to support several human activities, including agricultural and forestry ones [11,12]. In particular, S2 images can be used to obtain several spectral indices (e.g., the Normalized Difference Vegetation Index, NDVI) that can be used as predictors for some vegetation properties [13,14,15] such as phenology [16,17], ecosystems characterization [18], crop production estimation [19,20,21], tree stability assessment [22], crop damage estimation for insurance purposes [23,24]. EU Reg. No. 809/2014 Art. 40 is amended by EU Reg. No. 746/2018, proposing the adoption of EO satellites to support SCs [10,25]. Several EU countries are working to include Copernicus data in their monitoring procedures [26]. A pilot project was experienced in Italy in 2018; AGEA proceeded to verify CAP applications for the Titles III and V of EU Regulation no. 1307/2013 (Basic Payment and Small Farmers Scheme) based on markers derived from satellite data in the province of Foggia (SE-Italy). Specifically, AGEA’s markers allow checking for each parcel the presence of: plowing, crop development, mowing of meadow, and checking the existence of vegetation and crop harvest. Afterward, they are used to establish different control phases, such as financial impact analysis of the farmer’s application or a parcel-specific assessment carried out by a qualified operator with other data sources (orthophotos, geotagged images). Other provinces enter the same project in the following years.

Within this framework, in this work, a prototype procedure is proposed to support CAP controls by the regional Agency for Payments in Agriculture (ARPEA) of the Piemonte Region (NW, Italy). Procedure development was achieved in the framework of a collaboration between ARPEA and the Department of Agricultural, Forest and Food Sciences (DISAFA) of the University of Torino that started in 2019. The procedure, mainly relying on the S2 image time series (TS), is aimed at classifying the main regional crops that CAP actions take into consideration to make it possible to better address controls at the ground. The procedure was tested within the province of Vercelli in the Piedmont (NW) region.

The authors’ philosophical approach for crop recognition by satellite imagery appears to be different from the mostly adopted ones, mainly relying on the identification of proper “markers” along a spectral index time series [27,28]. The idea is to exploit most of the spectral information that can be directly related to ordinary agronomic practices and crop behavior, making possible an immediate interpretation of spectral predictors from an agricultural point of view. This permits to better investigate back eventual failures of the recognition process and better control/correct system deductions in a reasonable way. We, in fact, strongly believe that, in such contexts, expert systems based on human knowledge (possibly related to the considered application domain) are mandatory and preferable to other approaches based, for example, on artificial intelligence (AI) [29,30]. Biotic issues and targets are difficult to be completely described by trained algorithms underlying AI since they continuously change in time and space, making practically impossible a complete generalization capability. Consequently, forecasted services based on digital technology, designed to support and address decision makers and controllers, must preserve a significant quote of controllability by human experts of the domain that can interpret and overcome possible paradoxes/errors from the system.

For example, one of the biggest challenges in the literature is the separation between winter and summer crops by using a multi-temporal NDVI profile [31,32,33]. The agronomic knowledge about crop development in such periods allows for defining a proper temporal window in order to focus the analysis on vegetation active phases. Without this refinement, high classification errors could persist [34]. Despite the improvement in such differentiation, similar NDVI temporal behavior could occur. For example, in Italy, soya and rice were poorly separable while working with NDVI temporal profile [10]. To avoid this issue, the agronomic knowledge about the phenological stage that maximizes the spectral differences between these crops allow for improving classification accuracy [35,36,37]. Therefore, a priori selection of temporal window and the maximum spectral separability moment should be locally calibrated and involved in the workflow before applying any classification algorithms.

With these premises, agronomic information concerning crop calendars phenology and management actions have been used to develop a rule-based procedure that operates hierarchically, trying to extract single crops of interest, step by step, with customized rules. After describing the proposed methodology, its performances were compared with the ones from more ordinary classification approaches, namely minimum distance and random forest operating on NDVI temporal profiles.

2. Materials and Methods

2.1. Study Area

The test area (AOI) is located in the southern part of the Vercelli province in the Piemonte Region (NW, Italy). The area develops at 130 m a.s.l. and a size of about 1295 km². In the area, the climate is temperate with a continental character; the yearly average rainfall gauge and temperature are 930 mm and 11.9 °C, respectively. AOI well fits its role of test site hosting crops of interest for CAP over more than 75% of its area. It is characterized by intensive agricultural management with a prevalence of submerged crops (Figure 1).

2.2. Crops of Interest

In Italy, CAP applications for AC are ordinarily verified by PAs by photo-interpretation of high-resolution satellite/aerial orthoimages [38] that are newly acquired every three years. Differently, the new CAP philosophy for controls is expected to be based on EO data and aims at automatically detecting evident inconsistencies between farmers’ declarations and actual crops in the field, thus excluding significant time latency. No standard methodology has been still proposed at the national and European levels, and the task remains a true challenge in the context of services based on EO data. It has to be considered that these tools should be thought of as highly adaptive and customized with respect to the local agricultural landscape with a low probability of being successful as they come for all the situations. Consequently, it is expected that a crucial role will be played by ground data that will be continuously needed to train classification algorithms growing season after growing season. It is the authors’ opinion that no successful tool can ever come if not supported by a proper network of reliable providers of ground data that constantly feed the system with updated training and validation set for crops of interest. With these premises, the present work was aimed at developing and testing a possible approach for classification of the main crops in Piemonte in accordance with the Title III of Reg. (EU) 1307/2013) of CAP 2020. ARPEA indicated the crops of interest (CoI), namely soya, corn, wheat, rice, and meadow. Local agronomic calendars were taken into consideration to support the recognition of the phenological development of CoI. Crop development can greatly differ around the world. In particular, its phenology can vary in terms of length, strength, and period of the year. Consequently, crop detection is a real challenge without any local information. Agronomic calendars play, therefore, a key role in classification processes and cannot be neglected. In Figure 2, calendars of the abovementioned CoI are reported.

2.3. Copernicus Satellite Data

According to local agronomic calendars, a total of 137 S2 Level 2A images were collected covering the period 17 September 2019–12 November 2020. They were obtained from the Copernicus Open Access Hub geoportal (scihub.copernicus.eu). Level 2A products are supplied already calibrated “at-the-bottom of the atmosphere” (BOA) reflectance, permitting an immediate use for land applications. S2 images are supplied as 100 km × 100 km tiles or those projected in the WGS84 UTM reference frame [39]. NIR (band 8) and RED (band 4) bands were used to compute the correspondent maps of the Normalized Difference Vegetation Index (NDVI), having a geometric resolution of 10 m and a nominal temporal resolution of 5 days. S2 L2A data set also contains the scene classification layer (SCL), which is useful to automatically detect and remove bad observations (mainly clouds and shadows) from images while composing time series. Technical features of the S2 multi-spectral sensor (multi-spectral instrument, MSI) and SCL encoding are reported in

Table 1. Sentinel-2 MSI technical features and SCL code pixel assignment.

MSI Technical Features			SCL Codes
Geometric Resolution (m)	Bands	Wavelength (nm)	Code	Description
10	b2	458–523	0	No data
	b3	543–578	1	Saturated or defective
	b4	650–680	2	Dark area pixels
	b8	785–900	3	Cloud shadows
20	b5	698–713	4	Vegetation
	b6	733–748	5	Not vegetated
	b7	773–793	6	Water
	b8a	855–875	7	Unclassified
	b11	1565–1655	8	Cloud medium probability
	b12	2100–2280	9	Cloud high probability
60	b1	433–453	10	Thin cirrus
	b9	935–955	11	Snow
	b10	1360–1390	-	-

.

2.4. Farmers’ GSAA

GSAA contains all information about fields that CAP contributions are requested for (e.g., location, size, crop types, etc.). For this work, about 196,500 GSAA, sizing a total of about 98,000 ha, were made available by ARPEA in vector format for the 2020 agronomic season. These data are for PAs reserved use and, therefore, not currently accessible for all users (not open data).

2.5. Ground Data

During summer 2020, ARPEA conducted several GCs in order to calibrate and validate the proposed methodology. Crop type was detected by ground inspections, as well as several auxiliary information derived by interviews with local farmers such as sowing and harvesting date, type and date of operated agronomic practices, and eventual known and detected anomalies along crop growing season. GCs information was georeferenced by Topcon GRS-1 (Topcon Positioning Italy Srl, Ancona, Italy) GNSS (global navigation satellite system) receiver coupled with Mercury© (Mercury Systems, Inc., Andover, MN, USA) post-processing software [40]. A total of 1026 GSAA, covering about 3193 ha, were surveyed to collect data for training and validating classification. GC data set was randomly split into training (T) and validation (V) sets, corresponding to the 60% (615 fields) and 40% (323 fields), respectively. This was achieved at the class level (n. of fields) in order to ensure equal representativeness of samples for all CoI. Figure 3 shows the spatial distribution of data sets;

Table 2. Number and size of the surveyed plots carried out by ARPEA per crop type and relative split of GCs involved in TP and VP.

Crops	Total Number of Surveyed Fields	Total Area of Surveyed Fields (ha)	Training Set (n. Fields)	Training Set (ha)	Validation Set (n. Fields)	Validation Set (ha)
Soya	220	705.2	132 (60%)	393.2 (56%)	88 (40%)	312 (44%)
Corn	244	493.2	146 (60%)	287 (58%)	98 (40%)	206.2 (42%)
Wheat	182	246	109 (60%)	167.9 (68%)	73 (40%)	78.1 (32%)
Rice	233	1554.1	140 (60%)	921.5 (59%)	93 (40%)	632.6 (41%)
Meadows	147	194.3	-	-	147 (100%)	194.3 (100%)

reports the number and size of surveys for each crop.

2.6. Data Processing

2.6.1. Compliance of GSAA with S2 Data

The benefits and limitations of remote sensing in agriculture are well known in the literature [41,42]. One of the most important issues that have to be considered is the compliancy of field geometry and shape with the technical features (mainly geometric) of remote data. With a special focus on the agricultural Italian context, it can be easily found that it appears as highly fragmented, thus determining, in many cases, unfavorable size and shape of fields. Depending on the shape and size of agricultural fields, deductions generated with remotely sensed data can significantly vary [43,44]. A preliminary selection of “proper” fields (or GSAA-related geometries) to be monitored is therefore mandatory when designing a new control service based on a specific remotely sensed data set [45]. This immediately highlights that properness of detection/controls can be a priori associated only with a subset of the GSAA that must be controlled. In other words, controllers must be informed that the service will provide information only about larger and well-shaped fields. In fact, in small or badly shaped (highly anisotropic) fields, remote spectral measures can be unreliable since a great number of “mixed” pixels (made of different cover types) can generate spectral responses, significantly shifted from the reference ones. In this work, the geometric resolution of S2 data (10 m) was, therefore, crucial to a priori mask out those GSAA polygons not suitable for remote control. Consequently, to take care of this important issue, supplied GSAA polygons were analyzed and the correspondent area and shape index (SI, Equation (1)) were computed by ordinary GIS tools available in SAGA GIS 7.9 [46].

$$SI=\frac{P}{2\sqrt{\pi A}}$$

(1)

where A and P are the polygon area and perimeter, respectively [47].

2.6.2. NDVI Image Time Series

NDVI is one of the most popular spectral indices [48] able to derive information concerning vegetation with special concerns about its biomass and phenology. In this work, NDVI was computed from all the available scenes within the explored period (growing season 2020) and the correspondent time series generated from band 8 (NIR) and band 4 (Red) stacks. This was achieved by taking care of the available SCL layer that was used to filter out, at the pixel level, all bad acquisitions while composing the NDVI time series. TS was generated from the native band 8 and band 4 by a self-developed routine implemented in IDL v 8.0.1 [49] in charge of removing bad observations depending on the local SCL code, smoothing the remaining observations by a symmetric Savitzky–Golay filter (kernel = 3, derivative = 0, degree = 1) and regularizing the profile by spline interpolation (tensor value = 10) to obtain a time series with a time frequency of 5 days [50,51]. Finally, a TS of 85 NDVI (×10,000) maps were obtained for the investigation period. Local NDVI temporal profile from TS is assumed to describe crop phenology and time behavior (including effects of management practices) along its growing season [52,53].

2.6.3. Minimum Distance and Random Forest Classification of Crops

Minimum distance (MD, [54]) and random forest (RF, [55]) algorithms are largely used in agriculture for supervised crop classification [56,57,58,59]. In particular, MD is well known to be one of the simplest algorithms for supervised classification that, exactly for this reason, made possible a complete control of results and a more reasonable tuning of the few parameters that it requires. In general, it can be said to be compliant with a user profile characterized by a high level of skill concerning the application domain he is managing (e.g., agriculture) and a low level of technological/digital consciousness. According to a crop classification approach based on NDVI temporal profile, MD is in charge of comparing, in the hyperdimensional space (time domain) of TS, the Euclidean distance between the generic pixel and the class centroids as derived from the training set [60]. MD classification (hereinafter called MDC in order to distinct this classification from the rule-based phases) makes possible to define a threshold able to test if the winning (shorter) distance relating the generic pixel to the centroid of the closest class is reasonable or not. This determines the eventual migration of pixels into the unclassified class hosting pixels that are assumed to not belong to any of the classes of interest.

Differently, RF is a more nested approach to classification, basing its deductions on an arbitrary number of decision trees. RF has been extensively used in the remote sensing context, showing very suitable performances in crop classification [55,61,62]. Conversely, the variety of the parameters the user has to set is larger, and results (mainly failures) are, in general, trickier to be interpreted and eventually corrected by tuning the parameters more properly.

2.7. Rule-Based Hierarchical Classification

Literature reports many classification approaches for land use mapping based on satellite data [63]. Complex algorithms able to manage large quantities of data during the training step have often been used to improve classification results. Some of the most popular and newest techniques are based on the deep learning approach; results appear to confirm very suitable performances of these algorithms [64,65]. Nevertheless, these often require users with a high level of technical skills in information technology that, generally, do not belong to the knowledge domain the classification is required for. Moreover, they make it difficult to interpret criticalities and, consequently, to adjust algorithm parameters that, in most cases, cannot be related to tangible factors close to the knowledge domain the user belongs to. This criticality appears to be especially important in agriculture when the agronomic interpretation of both training sets and results is highly desirable. With these premises, a rule-based hierarchical classification approach (HI) built around easy-to-use and controllable algorithms (i.e., MD and thresholding) can be preferable [66,67]. HI is, in fact, intended to approach classification by applying simple rules or tasks, to disaggregate a complex problem into a succession of simple and more controllable ones, where domain knowledge can play a crucial role in improving classification results.

In this framework, the authors decided to approach CoI classification in AOI by a self-developed HI procedure and compare results with those obtainable through MDC and RF. Proposed HI relies on auxiliary agronomic data and knowledge that are used to define agronomically based rules useful to identify CoI. The correspondent workflow is shown in Figure 4. Details are reported, with reference to the single CoI, in the following sections.

2.8. Meadow Detection

Agricultural management of meadows significantly affects NDVI temporal profile [68,69]. In fact, meadows show an NDVI level averagely high and stable if compared with other crops; moreover, its management is ordinarily characterized by several cuts along the growing season that do not completely remove biomass [70]. Consequently, changes of NDVI values are limited, thus determining, along the whole season, low values of NDVI standard deviation (${\sigma }_{NDVI}$). Differently, others CoI (e.g., corn, wheat, soya) are characterized by a greater NDVI variability along the year; they present the typical phenological behavior where NDVI starts to grow after plow, reaches the maturity plateau and finally decrease after the harvest. This highly moved behavior generates higher values of ${\sigma }_{NDVI}$ along the year thus suggesting that ${\sigma }_{NDVI}$ can be used as discriminant to detect meadows pixels. A proper threshold value has therefore to be found. GCs can provide this type of information. To achieve this task, the 5th, 25th, 50th, 75th, and 95th percentiles of ${\sigma }_{NDVI}$ values were computed at class level for all CoI and compared by boxplot. The threshold was expected to fall in the upper (i.e., >75th) and lower (i.e., <25th) percentile for meadows and other classes, respectively. A proper threshold was finally found and meadow fields mapped. Meadow pixels were therefore removed (masked) from TS and no more considered in the next classification steps aimed at recognizing other CoI.

2.9. Wheat Detection

In order to facilitate classification, TS was twice subset to generate a winter (WTS) and a summer (STS) time series useful to classify winter and summer crops, respectively. According to the local agronomic calendar of winter crops, WTS was assumed to cover the period between 15 September 2019 and 15 July 2020, resulting in a stack of 64 NDVI images. According to the local agronomic calendar of summer crops, STS was assumed to cover the period between 15 March 2020 and 15 October 2020, resulting in a stack of 51 NDVI images. Since winter crops are expected to be characterized by a medium-high NDVI value in Spring (typically in April, [31], only pixels showing an NDVI value > 0.62 on 24 April were selected and considered for classification as supported by De Petris [71].

Candidate wheat pixels were detected by MD with reference to the previously thresholded WTS. Since two classes were expected-wheat and other winter crops, and MD training was achieved with reference to a single class (wheat), other winter crops were placed in the unclassified class. Labeling of unclassified pixels strictly depends on the distance threshold that is used during MD running. To make the threshold selection objective, the Euclidean distance layer (DL), generated as auxiliary data during MD classification as implemented in SAGA GIS, was used. DL image histogram was analyzed by Otsu’s method (OM), searching for an automatic and objective threshold able to separate the wheat from other winter crops. OM is a nonparametric and unsupervised thresholding approach that locates the threshold where the separability of image pixels is maximum [72]. An optimization is operated iteratively, looking for a threshold that contemporarily minimizes within-class and maximizes between-class variances. The method is data-dependent and, consequently, adaptive; this makes it possible to automatically detect a new threshold value depending on the operational situation (area, time, band/index, etc.) that time to time one has to be analyzed.

With these premises, once a proper threshold value was found, all pixels showing a DL value greater than the threshold were labeled as “other winter crops”.

2.10. Corn Detection

NDVI profile from STS was used to separate the corn from other summer crops (soya and rice). Previous classification experiences in the same area based on TS profile analysis proved that corn was well separable from other summer crops [10]. The classification was achieved with reference to all the pixels that were not previously classified as meadows or wheat.

STS was initially analyzed to select only vegetated pixels [73] by NDVI thresholding (NDVI > 0.62 @ 8 July 2020), admitting that summer crops reach their maximum NDVI values in this period of the year.

Again, MD was used, and OM was applied with reference to the DL image histogram to separate corn fields from other summer crops. Detected corn pixels were removed from STS and archived as representatives of the corn class in AOI.

2.11. Soya and Rice Detection

Previous studies from Sarvia [10] in the same area found that rice and soya are difficult to be separated if classification is achieved with reference to the yearly NDVI time series. A different but more traditional approach based on the classification of a single multi-spectral image acquired at the right time along the year appeared to be more successful in separating these two classes. The most critical issue at this point was the selection of the proper summer image that maximizes spectral differences between these two classes. A subset of 3 summer native L2A S2 images (hereinafter called SMSI, summer multi-spectral image) was used for this task. They were selected according to the agronomical criteria that are reported in

Table 3. S2 images and corresponding phenological stage of rice and soya crops.

S2 L2A Image	Crop	Phenological Stage
15 June 2020	Soya	Leaf and node development
	Rice	Tillering
18 July 2020	Soya	End node development-bloom
	Rice	Maximum tiller number-panicle formation
14 August 2020	Soya	End bloom-beans develop
	Rice	Flowering-dough

, mainly relying on crops’ local phenological behaviors.

A separability analysis, based on statistical concerns, was therefore performed in order to detect the most suitable moment and bands useful to separate rice and soya. Soya and rice class mean (${\mu }_i^{\lambda }$) and standard deviation (${\sigma }_i^{\lambda }$) values were computed for all the bands and the selected images (June, July, and August) according to GCs. A separability index was computed at band and image level according to Equation (2):

$$S_{i,j}^k\left(t\right)=\left|\frac{{\mu }_i^{\lambda }\left(t\right)-{\mu }_{j,t}^{\lambda }\left(t\right)}{\sqrt{{\sigma }_{i,t}^{\lambda }{}^2\left(t\right)+{\sigma }_{j,t}^{\lambda }{}^2\left(t\right)}}\right|$$

(2)

where ${\mu }_i^{\lambda }\left(t\right)$ and ${\mu }_j^{\lambda }\left(t\right)$ are the mean values of λ-th S2 band for i-th and j-th class (i.e., soya and rice) at the date t; ${\sigma }_i^{\lambda }{}^2\left(t\right)$ and ${\sigma }_j^{\lambda }{}^2\left(t\right)$ are the variances of λ-th spectral band for i-th and j-th class at the date t. This index quantifies the ratio between the average distance between two classes in the spectral domain of the λ-th spectral band (numerator) and the uncertainty affecting the distance measure (denominator). Separability is statistically possible when $S_{i,j}^{\lambda }\left(t\right)$> 1, consequently, only bands and dates satisfying this condition can be considered. $S_{i,j}^{\lambda }\left(t\right)$ was therefore plotted against the wavelength and the most promising combination selected. This corresponds to a subset of bands of the best performing acquisition in terms of $S_{i,j}^{\lambda }\left(t\right)$ values. Once the most promising date and bands were selected, a new MD was carried out. Again, OM applied on the DL image histogram was used to locate a proper threshold to separate soya and rice from other summer crops not considered in this work (e.g., tomato, courgette, apples, sunflower, etc.).

Finally, all previous classifications were merged into a single map, hereinafter called crop types map (CTM). All crop types not considered in previous steps were masked out from CTM.

2.12. Comparing HI with MDC and RF

A comparison of CTMs from the different approaches was performed at the parcel level (GSAA polygons). Consequently, pixels belonging to the same parcel were merged by a majority operator to assign a unique class to the same parcel. A total of 323 GCs were used as the validation set, and the corresponding confusion matrices were generated. Overall accuracy (OA), User’s accuracy (UA), Producer’s accuracy (PA), and K coefficient were calculated [74] and compared. To highlight improvements/limits of the compared approaches, accuracies were compared as percentage differences ($\mathsf{\Delta }{\%}_{MDC}$, $\mathsf{\Delta }{\%}_{RF}$, Equation (3)) and some discussions given.

$$\mathsf{\Delta }{\%}_{MDC}=\frac{\left(P_{HI}-P_{MDC}\right)}{P_{MDC}}·100;\mathsf{\Delta }{\%}_{RF}=\frac{\left(P_{HI}-P_{RF}\right)}{P_{RF}}·100$$

(3)

where $P_{HI}$is the generic accuracy parameters for HI classification, $P_{MDC}$ and $P_{RF}$ the correspondent ones for MDC and RF, respectively.

3. Results and Discussion

3.1. Compliance of GSAA Geometry with S2 Data

All GSAA polygons showing a SI < 3 (long and tight shape) and an area < 0.1 ha were excluded from the analysis. These values were suggested by Sarvia [10] for the same area. Results are reported in

Table 4. Number and surface of monitorable GSAA.

N° of fields before filtering	196,573
N° of fields after surface filtering	66,095
N° of fields after geometrical filtering	57,230
Area of field before filtering (ha)	97,978
Area of fields after surface filtering (ha)	95,306
Area of fields after geometrical filtering (ha)	93,230
Monitorable fields (%)	29.11%
Monitorable area (%)	95.15%

.

Table 4 shows that only 29% (57,230 out of 196,573) of the plots within the GSAA satisfy the geometric criteria required for proper detection with satellite S2 data. The high drop in terms of the number of plots was mostly due to the peculiar fragmentation of the Italian agricultural context, characterized by many small-sized plots with anisotropic geometries [75]. Although this result was quite critical, the reduction in terms of the controllable area does not vary significantly. Specifically, the area that can be monitored with S2 data was about 95% of AOI (from 97,978 ha to 93,230 ha). This outcome highlights the great advantage of S2 data to support controls for the entire area under CAP payments. Moreover, it can also be affirmed that the high number of plots removed after the geometric survey has a very low impact on CAP payments as their area is very limited.

3.2. MDC and RF Classification

MDC classification was achieved as proposed in [10] by SAGA GIS 7.9.1 [46]. Accordingly, a single value (class independent) of 15,000 (NDVI points) was adopted for DL image histogram thresholding to map all CoI.

Similarly, RF classification was achieved by SNAP v.8.0.0 [76] as proposed by Sarvia [10] using a number of trees equal to 10 and a number of training samples equal to 5000 pixels.

Classification maps from MDC and RF classification are reported in Figure 5. Figure 5c confirms that rice was the main crop within AOI. With regard to soya, significant differences were found between the two classifications.

The percentage of soya parcels was found to be 25% and 9% from MDC and RF, respectively, in AOI. The percentage of rice fields was found to be 40% and 54% for MDC and RF, respectively, thus highlighting a significant misclassification problem affecting rice and soya classes. No validation is given here; it is demanded in the section “Validation and Comparison of CTMs”.

Figure 5c shows that the number of plots was not proportional to their areal size. This is probably due to rice fields that were generally characterized by medium to large size, while other crops, e.g., meadow, generally show smaller areas.

3.3. Rule-Based Hierarchical Classification

3.3.1. Meadow Detection

Meadows were identified by per-pixel thresholding of NDVI standard deviation in the time domain (σ_NDVI, Figure 6a). σ_NDVI threshold was set equal to 1695 falls between the 75th percentile of meadow and the 5th percentile of wheat, as shown in Figure 6b, providing suitable separability of the meadow from other crops. It is worth highlighting that this technique does not require any training set. Figure 6b demonstrates that meadows present very low σ_NDVI values if compared with other CoI. This is probably due to their development cycle, which is very long, being a permanent culture. Figure 6c reports the obtained map of meadows.

3.3.2. Wheat Detection

Wheat was identified by MD classification of WTS trained with 109 fields from GCs; OM was used to automatically find an objective threshold with respect to the DL image histogram (Figure 7a,b) able to properly separate wheat from other winter crops. DL histogram shows two maxima. One is centered around low values (about 10,000) and is probably related to the wheat class. The other one is located around high values (about 26,000) and is probably related to other winter crops. OM placed the threshold at 15,000, i.e., between the two histogram peaks. Figure 7c shows wheat fields as mapped by the proposed approach.

3.3.3. Corn Detection

Corn detection was achieved by MD classification of STS trained with 146 fields from GCs. OM was used to locate a proper threshold (4500) along the DL image histogram (Figure 8b) that, again, showed two peaks. The first, centered around DL low values (about 2000), is probably related to corn fields. The second, centered around DL high values (about 12,000), is probably related to other summer crops.

It is worth noticing that the DL map (Figure 8a) alone is already able to provide a preliminary overview of the spatial distribution of corn (red) and other summer crops (green). Figure 8c shows the map of corn fields as classified by the proposed method.

3.3.4. Soya and Rice Detection

Soya and rice were classified using the three selected SMSI in place of TS. Separability was computed for all SMSI, and its spectral profile (Figure 9) was analyzed by visual interpretation. It highlights that for all the acquisitions: (i) visible bands show low separability; (ii) NIR and MIR bands show high separability values; (iii) NIR and MIR related separability is generally lower in June (S ≈ 1) than in July and August (S > 2). From an agronomical point of view, this can be interpreted as probably due to the similar earlier development stage that the two analyzed crops present in June (Table 3). Consequently, June’s spectral signature cannot be assumed as a strong discriminant to separate soya and rice in AOI.

Conversely, in July spectral response of the two crops appears to maximize their separability. Agronomically speaking, in this period of the year, soya and rice are showing two different phenological stages (see Table 3) in AOI. In fact, in July, soya is experiencing its node development and bloom; rice is approaching panicle formation while exiting its tillering phase.

With reference to the selected band subset (b5-b8a, b11, and b12) from the July acquisition, MD was run trained with 132 soya and 140 rice fields from GCs. OM was used to locate a proper threshold in the correspondent DL image histogram (Figure 10b) to separate these two classes from the remaining ones. DL image histogram shows a drastic fall just after the OM threshold (2500). Figure 10c shows the final map of soya and rice fields.

CTM was finally generated by merging all the abovementioned partial maps in order to provide a comprehensive representation of CoI in AOI (Figure 11a).

Some statistics were therefore computed to synthesize the agricultural landscape of the area. In Figure 11b, a plot is reported showing that: (i) rice is the main crop in AOI (about 60% of total plots and almost 75% in terms of area); (ii) soya is poorly present in the area covering about 2% both in terms of parcel number and area. This value is significantly different from the ones estimated by MDC and RF (25% and 9%, respectively). Further differences can be found in the next section.

3.4. Comparing HI with MDC and RF

To evaluate the performance of HI, a comparison was made with respect to MDC and RF. After computing the main performance parameters (see Materials and Methods section) of each classifier with reference to the validation set reported in Table 2, they were compared and discussed. Results are reported in Figure 12. In particular, Figure 12a,c,e report the absolute value of the computed accuracy parameters. Figure 12b,d,f report the percentage differences as defined in Equation (3).

Concerning PA, Figure 12a shows that MDC was the best performing one for wheat and meadow: in particular, MDC PA was 94% and 92% for wheat and meadow, respectively, against 86% and 73% from HI and 77% and 68% from RF. For all other CoI, HI PA was higher than the ones from RF and MDC. MDC proved to be the one generating the highest variability in PA accuracies. Differently, RF showed a fairly low variability and fairly high PA; HI was the best performing one showing low variability and high PA.

As far as the analysis of PA performance differences is concerned, Figure 12b shows that: (i) HI performed better than RF for all CoI (Δ% > 0) except wheat, where it was slightly negative; (ii) HI performed better than MDC for soya, corn, and rice, while it was slightly worse for meadow (−20%) and wheat (−8%); (iii) the major improvement given by HI was found for soya (+220% and +38% with respect to MDC and RF, respectively).

Concerning UA, Figure 12c shows that, again, MDC was better for detecting wheat and meadow MDC (89% and 92%, respectively). Correspondent UA were 78% and 86% for HI and 88% and 66% for RF. For all other CoI, HI UA was higher than those from both RF and MDC. Moreover, it can be noted that both MDC and RF showed fairly high variability in class UA if compared with HI.

As far as the analysis of UA performance differences is concerned, Figure 12d shows that: (i) HI performed better than RF for all CoI (Δ% > 0) except wheat, where it was slightly negative (−12%); (ii) HI performed better than MDC for soya, corn, and rice, while it was slightly worse for meadow (−7%) and wheat (−13%); (iii) with respect to MDC Δ% was higher for soya (+43%) and rice (+68%). Concerning RF, Δ% was higher for soya (+29%) and meadow (+30%).

It is worth stressing that the high-class commission affecting both MDC and RF classification appeared to be solved by the proposed HI approach, which is driven by agricultural knowledge.

Figure 12e shows OA and K values. OA values were found to be 63%, 80%, and 89%, respectively, for MDC, RF, and HI. K values were found to be 44% (MDC), 70% (RF), and 84% (HI), proving that, in the local context, HI is preferable.

Figure 12f compares the improvements that HI generated with respect to both OA and K values. Specifically, Δ%OA was found to be +42% and +11% with respect to MDC and RF approaches, respectively. Δ%K was found to be +87% and +18%, respectively. It can be, therefore, assumed that the HI approach can provide better results from this point of view, as well.

To preliminarily test the generalization of our conclusions, a comparison with similar works was performed. It was found that Foerster [77] obtained an OA close to the one from this work (86%) using a hierarchical winter-summer and grass/permanent crops classification in NE Germany. Li also obtained their best results in classifying crops in California through a hierarchical approach based on deep learning approaches [78]. Specifically, they report a Δ% OA equal to +10%, proving how hierarchical approaches can effectively improve crop classification.

These few references provide a first benchmark useful to affirm that our results can be somehow assumed as general, even if depending on the degree of agricultural knowledge one can provide during rules definition in HI.

Nevertheless, this approach suffers from some limitations and constraints. The most important one is the need to possess proper agronomic knowledge about local crops and some ground reference data for training classifiers (in this work, ARPEA GCs). Furthermore, the quality and truthfulness of GCs used to train classifiers and validate results are basic in determining the final quality of CTM. Since GCs campaigns require a significant effort in terms of time and cost from public administrations, they have to carefully evaluate the sustainability of such an approach within their ordinary operative/administrative framework. With these premises, the proposed approach can be defined as a reliable tool for CAP controls.

4. Conclusions

In this work, a hierarchical classification of crops was proposed in order to support CAP controls in agriculture. This work was supported by ARPEA, which provided several GCs for five crops: soya, corn, wheat, rice, and meadow. The proposed hierarchical approach was compared with other more traditional approaches, namely MDC and RF, trying to overcome some classification problems that previous studies highlighted, especially for detecting rice and soya. The analysis was focused on those fields that can actually be monitored by S2 data (about 57,230 parcels) according to their geometric features. Consequently, a preliminary step was achieved to select them out of the whole.

The proposed approach was structured in the following several steps: (a) TS standard deviation analysis in the time domain for meadows mapping; (b) MD classification of winter part of TS in the time domain for wheat detection; (c) MD classification of summer part of TS in the time domain for corn classification; (d) selection of a proper summer multi-spectral image (SMSI) useful for separating rice from soya with MD operated in the spectral domain. To separate crops of interest from other classes, consequent MD classifications defining HI were thresholded by Otsu’s method. Overall accuracy for MD, RD, and HI were found to be 63%, 80%, and 89%, respectively. It was found that especially the SMSI-based approach within HI generated a significant improvement in soya and rice detection. Based on this work, it is advised to not proceed with crop classification using a single algorithm but rather, if possible, to structure a hierarchical approach based on agronomical and local calendars knowledge in order to maximize the accuracy of the classification. Moreover, it is worth reminding that satellite data allow monitoring of all GSAAs in a cost and effective way; however, the minimum size of the plot and the geometric parameters of the satellite sensor must always be considered.

It is worth reminding that, presently, the ordinary procedure for controls is based on manual photo-interpretation and ground surveys. Only 5% of the total number of fields declared by farmers are verified based on risk and randomness criteria. Consequently, 95% of applications cannot be verified, and the possibility of supplying illegitimate CAP grants is very high. The new EU CAP controls will be supported by remote sensed data, making public administrations able to focus their checks on those fields where satellite-based classification founds some inconsistencies with farmers’ declarations. Precisely for this reason, crop classification and mapping from satellite data (S2 mission) is getting more and more important [79,80], and the HI approach well fits this point.