Abstract
Satellite crop detection technologies are focused on the detection of different types of crops in fields. The information of crop-type area is more useful for food security than the earlier phenology stage is. Currently, data obtained from remote sensing (RS) are used to solve tasks related to the identification of the type of agricultural crops; additionally, modern technologies using AI methods are desired in the postprocessing stage. In this paper, we develop a methodology for the supervised classification of time series of Sentinel-2 and Sentinel-1 data, compare the accuracies based on different input datasets and find how the accuracy of classification develops during the season. In the EU, a unified Land Parcel Identification System (LPIS) is available to provide essential field borders. To increase usability, we also provide a classification of the entire field. This field classification also improves overall accuracy.
1. Introduction
Satellite crop detection technology is focused on the detection of different types of crops in the field in the early stage before harvesting. There exists a large area of domains where such technologies can be used [1,2,3,4,5]. As examples, we can mention:
- The public sector and organizations dealing with food security, e.g., the Common Agriculture Policy [6] in Europe, GEOGLAM/GEO monitoring [7] and FAO agriculture production monitoring [8];
- The food industry, investors and business owners for their strategic decisions, investment making and sustainability forecasts [9];
- Insurance brokers (risk assessment, data collection, client claim verification, etc.) [10];
- Agriculture machinery producers (information about crops are important for combinations with other information and management) [11].
Multispectral satellite images are used in remote-sensing crop detection. Remote sensing has the advantage of providing information over a large area in a relatively short time. After processing, the images can be used to produce thematic maps. The act of processing the data into maps is called image classification [12]. Two types of classification exist: supervised and unsupervised classification [13].
1.1. Classification
In supervised classification, the analyst selects pixels from the input image based on knowledge of the land cover, also called “training sites” [14]. Each training site is placed in the spectral space based on its values of input layers. The analyst then chooses a statistical rule (an algorithm) that assigns every pixel of the input layers to one of the predefined classes.
Unsupervised classification does not require any prior information about the area of interest [12]. A large number of pixels are analyzed and then classified into several classes based on statistical groupings of pixels in this type of categorization [14].
The first technique, supervised classification, is the most commonly used for quantitative analysis of remote-sensing image data [13]. Generally, supervised classification includes the following practical steps [13]:
- Output classes—the supervisor determines the number and meaning of the desired classes.
- Training sites—the supervisor chooses known pixels which represent each of the output classes defined in step 1. Training data can be acquired using site visits, additional measurements, maps, imagery of different origins or photo interpretation of the input layers. When a set of training pixels lies in a region enclosed by a border, we call it a training field;
- Spectral signature—irrespective of the chosen algorithm, the positions of the training pixels in the spectral space are calculated by the classification software.
- Classification—each pixel in the image is assigned to one of the classes defined in step 1, whereas in step 2 the supervisor labels only a small portion of the pixels, and all pixels are assigned to a predefined class based on the chosen statistical rule (algorithm);
- Thematic map production—classification is visualized, the number of pixels in each class may be summarized and from that class, the area can be derived.
- Accuracy—an important step is to assess the accuracy of the final product.
Supervised classification has the potential to be more accurate than unsupervised classification. However, it is highly dependent on the training sites as well as the skill of the image analyst and the spectral distinction of the classes [14]. If several classes are very similar in terms of spectral reflectance (e.g., annual versus perennial grasslands), classification errors will tend to be high. Supervised classification requires more care in processing the training data. If the training data is poor or unrepresentative, the classification results will also be poor. In general, supervised classification requires more time and money than unsupervised classification, so both methods have advantages and disadvantages.
Most land-cover types are usually classified using a single-date image. This is because land cover does not change rapidly. Crops cultivated on fields, however, undergo fast changes in several months. During the vegetation season, the crop is planted, grows and is harvested. The difference among phenology stages of individual crop types can be used to distinguish crop types from each other. The substitution of single-image classification by time-series classification has a great positive impact on classification accuracy [15,16,17,18].
1.2. Overview of Relations of Phenology and Earth Observation
Phenology is a branch of science that studies the periodic events of biological life cycles that depend on many external environmental influences, such as weather and climate changes and other ecological factors. Over time, species have evolved in response to their environment and adapted specifically to biotic and abiotic factors. Because of these interconnections, the study of phenology is useful in many ways. For example, the study of a plant can provide information about the environment in which it evolves, and conversely, the study of biotic and abiotic factors can help to understand how a plant responds to environmental factors [19]. Moreover, phenological events are easy to observe. Therefore, this science is used in many disciplines such as ecology, climatology, forestry and agriculture.
In agriculture and horticulture, phenology has been used for a very long time. These observations are essential for many practical purposes. They allow, among other things, the careful selection of crops and varieties adapted to the environment and the organization of rotations. They also play an important role in the choice of irrigation, fertilization and protection against pests and diseases. These observations can also be useful in preventing the risk of frost damage and in predicting harvest dates. By studying the phenophases of different crops and taking the right measures at the right time, it is therefore possible to improve management, increase yields, achieve greater stability in production and have better-quality food [20].
Today, climate change impacts all ecosystems and threatens the balance of global food production. In addition, the world population continues to grow (9.7 billion people estimated in 2050 according to the United Nations) [21]. The scientific community must analyze the impacts of climate change and anticipate their consequences in order to propose concrete solutions in terms of the management of living resources. Phenological traits are key characteristics of climate adaptation and are of particular interest to the scientific community [19,22].
Efforts have been made worldwide to enlarge the phenology databases. Data collection and observations have been facilitated by technological advances, progress in computing and satellite remote sensing, which has allowed the development of research methods and models on phenology [19].
Nevertheless, in situ phenological data are only available over limited areas [19]. Since 1970, technical advances in ground-based satellite observations have made it possible to observe phenology on a larger scale. Several satellites can be used for such observations, such as AVHRR (since 1980), MODIS (since 2000) and more recently VIIRS (since 2012) [19,23,24]. The phenology observed at the landscape scale by earth-observation satellites is called land-surface phenology (LSP) [25]. To study phenology at this scale, vegetation indices (VI) are created from land-surface reflectance acquired by satellite optical sensors. The phenology observations obtained by LSP are different from in situ phenology observations. Because they are based on a regional and global scale, these observations can be compared with regional climate information. This makes LSP remote sensing an important biological indicator for detecting the response of terrestrial ecosystems to climate variation.
It is now possible to use the time series of vegetation index response curves to track crops over a growing season. Based on results presented in [26], the development of a vegetation index during the vegetation season was visualized (Figure 1).
Figure 1.
Time-series of Enhanced Vegetation Index profiles of field averages for chosen crops in Rostenice farm in 2018.
1.3. Vegetation Indices
Vegetation indices have been widely used in agriculture for decades. They serve as a tool for monitoring crop health and supporting farmers’ decisions. Vegetation indices are, along with soil and water indices, part of spectral indices which come from satellite multispectral sensors. Mathematically, spectral indices are combinations of surface reflectivity of two or more wavelengths, converting values of multiple spectral bands into a single value [27]. The normalized difference vegetation index (NDVI) and enhanced vegetation index (EVI) are among the most widely used indices [19]. The spectral response of green leaves on which these two indices are based is characterized by strong chlorophyll absorption in the red band and strong reflectance of the leaf structure in the near-infrared band of optical sensors [19]. Nevertheless, in dense vegetation NDVI suffers from saturation [28] which makes the index less suitable for time-series monitoring. Other widely used VI are the (modified) soil-adjusted vegetation indices (SAVI and MSAVI) [29]. They are used most commonly in medium- and low-resolution imagery and low-density vegetation cover due to their ability to minimize soil brightness influences [30]. An appropriate well-established index not suffering from mentioned limits is the enhanced vegetation index (EVI) [31].
This index was originally developed over NDVI by optimizing the vegetation signal in areas of high leaf area index (LAI). It is most useful in high LAI regions where NDVI may saturate. It uses the blue reflectance region to correct for soil background signals and to reduce atmospheric influences, including aerosol scattering [32]. Interpretation of the EVI values is not important in this study because classification takes into account only the index values.
The radar vegetation index RVI (RVI4S1 for using Sentinel-1 radar data) has been proposed as a method for monitoring the level of vegetation growth, particularly when time series of data are available. RVI is a measure of the randomness of the scattering and is sensitive to biomass; vegetation water content also has low sensitivity to environmental condition effects.
The RVI is the measure of randomness of scattering and can be written as
where and are polarized backscattering coefficients and is the cross-polarization coefficient in power units. RVI generally ranges from 0 to 1 mostly, similar to how NDVI may exceed 1 in some cases when double-bounce scattering is encountered. The RVI is near zero for a smooth bare surface and increases as the crop grows (up to a point in the growth cycle) [33].
1.4. Land Parcel Information System
The Common Agricultural Policy (CAP) finances, via the European Agricultural Guarantee Fund, direct payments to farmers and measures to face environmental challenges. To guarantee payments are well-distributed, the CAP depends on the Integrated Administration and Control System (IACS). This system comprises solid administrative and on-the-spot checks of subsidy applications and is managed by the member states. The Land Parcel Information System (LPIS) is a key component of the IACS. It contains imagery (aerial or satellite photographs) of all agricultural parcels in the EU. LPIS aims to locate all eligible agricultural land and calculate their maximum eligible area. The area is key information for the calculation of the subsidy amount. Contrariwise, LPIS serves as a basis for cross checking during the administrative control procedures and on-the-spot checks by the paying agency. Member states usually use their LPIS for other environmental rules and restriction applications [34].
1.5. Objectives of Research
The main objectives of our research are:
- To determine which input dataset gives the best crop classification accuracy;
- To answer the questions, “What is the development of crop classification accuracy during the vegetation season?” and “When is the classification sufficient for serious yield prediction?”;
- To determine how to ensure data availability for crop classification—what is the accuracy of classification based on Sentinel-1 data?
2. Materials and Methods
2.1. Pilot Areas
The first experiments with supervised classification were carried out on the Rostenice farm, South Moravia, Czech Republic (Figure 2). In the study year of 2020, the temperature was significantly higher (+2.5 °C) and the year was extraordinarily rich in precipitation (+144 mm) compared to the long-term averages (10.8 °C and 517 mm, respectively). It was the second-rainiest year since the beginning of the weather records (1961). Despite the cloudy and rainy weather, it was possible to find useful satellite images for crop analyses. The terrain is flat, sometimes slightly undulating. The altitude ranges from 194 to 376 m above sea level.
Figure 2.
Overview of the experiment site.
2.2. Used data
2.2.1. Field data
LPIS is regularly used by the farmers, as subsidies need to be updated yearly. It is therefore a valuable source of field data. The availability of LPIS data is dependent on the member state which manages it. In the Czech Republic, field borders from LPIS are publicly available through public export. This dataset does not contain crop data. Each farmer, however, can export their data through private export which also contains crop-type attributes.
This study uses data from Rostenice farm. The farm granted the crop data together with field borders. The Rostenice farm company manages 870 fields, with a total area of 10,094 ha. In this study, we used crops that occupy more than 1.5% of the overall area listed in Table 1.
Table 1.
Crops whose area is greater than 1.5% of the total.
2.2.2. Satellite Data
In the study, satellite data from Sentinel-1 and Sentinel-2 satellites were used. These data have an optimal combination of spatial resolution (10 m), time resolution (2–3 days) and cost (free). The Sentinel satellites are part of ESA’s space program, Copernicus. The data from all Sentinel satellites are freely available on the Copernicus Open Access Hub maintained by ESA [35].
Sentinel-2 Level 1C images are downloaded from ESA Open Access Hub [35]. Atmospheric corrections are being calculated utilizing Sen2Cor software resulting in Level 2A images that are used for further image processing and vegetation index calculation (NDVI, EVI and others).
Sentinel-1 data request quite extensive preprocessing in several steps, using Sentinel-1 toolbox functions placed in the SNAP Graph Builder process. These include calibration, speckle filtering, terrain correction and others (see Figure 3).
Figure 3.
Sentinel-1 processing steps.
After finalizing the preprocessing steps, the Radar Vegetation Index for Sentinel-1 (RVI4S1) was calculated.
All the satellite-data processing was carried out on our own built cloud environment based on open-source software (OpenStack, CentOS Linux, etc.). Image- and spatial-data-processing software is also open-source (SNAP, QGIS, JupyterHub, GDAL, OrfeoToolbox, etc.)
Sentinel-2 is a state-of-the-art satellite delivering optical data of the studied locality every two or three days with band resolutions 10, 20 and 60 m. Based on the phenology of the crops planted on the area of interest, the period of interest was determined and named “vegetation season”. For the studied crops (Table 1), it runs from the start of March to the end of August. Following the survey of data availability, processing and infrastructure preparation, the methodology was prepared in 2019. The main research was carried out in the vegetation season of 2020. In 2021 it was impossible to use a meaningful amount of Sentinel-2 images due to lingering cloud cover over the studied area.
There were 74 Sentinel-2 images of the studied locality in the vegetation season 2020. However, it was impossible to use 44 of them due to clouds covering most of the image (Figure 4). The remaining 30 images could be hypothetically used for the analysis of part of the farm or individual fields; however, to carry out crop classification, it was essential to use cloud-free images as input layers. Satellite images downloaded from the ESA hub contained metadata about the approximate cloud cover of the scene. There were 19 images that had cloud cover under 10% of the scene area and 9 images with less than 1% of the area under the cloud cover. To have the images equally distributed, one image per month (from March to August) was selected to enter the classification calculation. The dates were 18 March 2020, 22 April 2020, 22 May 2020, 4 June 2020, 1 July 2020 and 28 August 2020, making six EVI layers in total.
Figure 4.
Cloud coverage of the images available across the whole vegetation season 2020.
Filtering Sentinel-1 images was significantly easier. Sentinel-1 satellites make images of Rostenice farm every 1 or 2 days. To make the calculations less computationally expensive and retain classification quality at the same time, the number of input images was reduced to one image every 6 days starting 16 March 2020 and ending 31 August 2020. There were 26 RVI4S1 layers in total.
2.3. Used Tools and Algorithms
The preprocessed data were loaded into QGIS version 3.16 and classified using Semi-Automatic Classification Plugin version 7 (SCP). The plugin is intensively developed by Luca Congedo [36] and enables many earth-observation operations, including the downloading of satellite images of various sources, preprocessing, clustering, classification and accuracy calculation.
Supervised classification is a common tool to determine land cover. The quality of the output product is strongly dependent on the quality of the defined training areas [37]. The process of definition of output classes is work-intensive, as the supervisor must select a representation of all the desired classes. The training samples for each class should be distributed throughout the layer.
The SCP plugin offers three classification algorithms:
- Minimum Distance firstly calculates the mean vector for each output class [14]. Each pixel is then assigned to a class based on the shortest distance in the multidimensional space. The minimal distance means maximum similarity [12]. The Minimum Distance algorithm is widely used for classification using remote-sensing data [38]. Theoretically, the algorithm can use one of the following distances: (a) Euclidean Distance, (b) Normalized Euclidean Distance and (c) Mahalanobis Distance. The SCP Minimum Distance algorithm uses Euclidean Distance [36]. This algorithm was chosen to perform the supervised classification because it gives good results in good time.
- Maximum Likelihood is based on the probability of each belonging to a predefined class [39]. It assumes normal distribution for each class in each band. Each pixel is put in the class that has the highest probability [14]. Maximum likelihood is a computationally expensive and accurate classifier [40]. It is an optimal classifier when the probability distribution functions of classes are Gaussian [12].
- Spectral Angle Mapping uses the n-dimension angle to match unclassified pixels to training data. The spectral similarity between unclassified pixels and training data is calculated as the spectral angle. Mathematically it is a vector with dimensionality equal to the number of bands [14]. Spectral angle goes from 0, when signatures are identical, to 90 when signatures are completely different. Therefore, a pixel belongs to the class with the lowest angle [36]. It is usually used with data of high spectral dimensionality—a high number of bands or spectrometers recorded [13].
2.4. Experiments Provided
The SCP plugin enables the user to create band sets which are then used as an input to further analyses. The bands in the band sets can be of different origins. The innovative approach of this experiment was to put index layers from different dates and satellites into one band set. By making a vegetation index from a multispectral image, we preserve the important information while making a single-band raster from a multiband image. Thanks to this, we can put more layers from different dates in one band set. Index layers come from both Sentinel-1 and Sentinel-2, radar vegetation index for Sentinel-1 (RVI4S1) and enhanced vegetation index (EVI), respectively.
2.4.1. Supervised Classification for Agricultural Land
As mentioned above, there are two steps when performing supervised classification of a satellite image(s). First is the learning step, in which the supervisor (human) manually identifies the desired categories in the image. The database of the polygons which contains attribute information about the output class is called the “seed sample” or an SCP “training input”. In our experiment, there were altogether 43 training inputs (polygons) for seven categories of the crop (Table 2). For these vector layers, their place in the spectral space—the so-called signature—was calculated. Second is the prediction step, where the algorithm predicts the class for all the pixels of the input layers based on the signature calculated in the first step. In pixel-based classification, the algorithm takes each pixel individually, and using specific decision rules, puts the pixel in one of the predefined classes. In our study, the minimum distance algorithm was applied. It assigned each pixel to one of the seven predefined classes. This process was repeated for all the band sets. As there are many isolated pixels, the performance of the classification was improved by a sieve filter, making the result more compact. The described process is illustrated in Figure 5.
Table 2.
Training inputs.
Figure 5.
Flowchart of methodological process.
2.4.2. Majority Classification for Field
Although the number of isolated pixels was reduced by the sieve filter, there were usually more classes in one field. When the desired output information should be clear and not a probability of the present categories, the whole polygon must be assigned to the dominant class. Each field was assigned to the most frequent class using the Zonal Statistics tool. The accuracy of this product differs from pixel-by-pixel accuracy and can also be measured. With label rules, the results can be visualized to be suitable for visual interpretation.
2.4.3. Individual Crop Classification
For further improvements and a more detailed understanding of crop types, the classification and accuracy development of individual crops was desired. The accuracy was assessed for all seven classified crops and months of interest. The calculations were realized only for a combination of Sentinel-1 and Sentinel-2 data. The accuracies of individual crop classification are parts of the overall accuracy calculation for a combination of EVI and RVI4S1.
3. Results
3.1. Statistical Evaluation of Results
3.1.1. Supervised Classification of Agricultural Land
Several layers of supervised classification were created in order to compare the results from three aspects (see also Figure 6):
Figure 6.
Comparison of all classification products.
- The input band sets—Sentinel-1 vs. Sentinel-2. Three types of input were compared. The input layers come from the indices EVI, RVI4S1 or a combination of both. RVI4S1 layers had the lowest accuracy. When the classification was made based on 26 RVI4S1 layers from March till August and was improved by the sieve filter, the overall accuracy came to 67%. When the input was made of seven EVI layers, the overall accuracy reached up to 91% without sieve improvement. The third set of inputs consisted of the combination of EVI and RVI4S1 layers. The overall accuracy of the classification coming from the band set reached 89% without the sieve filter.
- Number of layers. As expected, the overall accuracy of the supervised classification roses when more data came into the input band set. If we want to identify the crop in March, there is 51% overall accuracy with March satellite images from both Sentinel-1 and Sentinel-2. When we add April data, the accuracy jumps to 68%. By further adding index layers from later months, we can obtain up to 93% overall accuracy.
- Sieve filtering. The resulting layers, which were postprocessed by the sieve filter, show higher accuracy than without it in all classifications. The effect of the sieve filter is stronger on classification with RVI4S1 because these classifications contain more isolated pixels and small islands of pixels than classifications coming from EVI. The sieve filter applied on EVI+RVI4S1 classifications has a higher effect, with fewer data (early months) and lower accuracy than with high-accuracy classifications.
From the example of the confusion matrix of individual classes (Table 3) it is possible to explore how many pixels of a specific crop were misclassified as other crop types. This shows how similar the crops are during the vegetation season. The rest of the confusion matrices are appended in Appendix A.
Table 3.
Pixel-count confusion matrix of RVI4S1+EVI May classification after sieve filtering. Classified rows contain classified classes and the number of pixels that were assigned to a reference class. In the reference columns, there are true crops from the farmer’s data.
3.1.2. Majority Classification for Field
In order to make the results easy to understand and practical to use, every field was assigned one prevalent class based on the highest representation of a class within the field. The real and classified attributes were visualized with multilabels (Figure 7).
Figure 7.
Visualization of the majority classification of Sentinel-1’s whole vegetation season classification (26 layers) [41].
The accuracy of the majority classification is summarized in Table 4. The accuracy values are mostly slightly higher compared to the pixel classification. The highest value is 96.3%. When processing Sentinel-1 data, the area of the field had an influence over the accuracy. When small fields (less than 10 ha) were filtered out, the accuracy of the majority classification increased in all months. On average, the accuracy improved by 9.6%. On the contrary, if only small fields were kept in the dataset, the classification accuracy decreased significantly; the mean fall was 10.7%.
Table 4.
Overall accuracy for majority field classification.
3.1.3. Individual Crop Accuracy
Individual crop classification shows more details than the overall accuracy of the whole classified layer. The accuracy development is not equal for all the crop types. While Poppy starts with 0% accuracy in March and ends with 97.5% accuracy in August, the accuracy of Winter wheat rises from 62.0% to 80.1%, which makes only an 18.1% increase. We see that Spring barley already has 91.4% classification accuracy in May and Corn is recognized with 96.2% accuracy in June (Table 5).
Table 5.
Classification accuracy development for individual crop types of combination of RVI4S1 + EVI layers after sieve filtering.
3.2. Visual Interpretation
Chosen classification outputs were published using the HSLayers-NG web mapping framework [41] and are available on the Agrihub web portal (Figure 8).
Figure 8.
Comparison of crop classification development in the season. Sentinel-1 and Sentinel-2 data-based pixel classification with input data from March + April on the left side and input data from March to August on the right side [41].
This application allows to visualize time series of data during the season by using date selection control, applying data transparency and combining data with field data and data from ground measurement. The map window can also be split and multiple layers can be compared at one moment using the swipe control. Any other relevant data can be also added to the map from other resources (WMS, files, etc.) to find possible correlations.
4. Discussion
The agriculture of developed countries is strongly dependent on subsidies and there is no sign that this should change in the future. More likely, subsidies will become even higher and more important for landscape management. A potential utilization of this method is checking agriculture subsidies.
Public availability of field borders (LPIS) supports necessary innovation in the agricultural sector, especially in combination with open and quality satellite data of sufficient space and time resolution, such as the Sentinel mission.
Sentinel-2 satellites produce excellent data but also have a serious limitation that cannot be ignored. Due to the physical principle of the limit, Sentinel-2′s optical sensor cannot be improved to overcome clouds. The ambition of satellite earth observation is to be an essential part of the food industry in many ways. To make this possible, the data need to be reliable and regularly available even if it is cloudy for a long period. In this study, we investigated possible Sentinel-1 alternatives in case of scarcity of Sentinel-2 data. The radar data are not degraded by clouds, so a usable Sentinel-1 image is ready on a regular basis. Further preprocessing steps are needed to obtain the correct radar signal values. Even after these adjustments, the radar vegetation index values for Sentinel-1 vary significantly within a field.
In this study, EVI was identified as the most suitable vegetation index to be used as input for the classification. However, not only one single index needs to be used. Similarly, as EVI was combined with RVI4S1, it can be combined with any other optical vegetation index. Ninety-one vegetation indices coming from the Sentinel-2 image were used as an input for crop classification in [42]. Single-image classification used in that study was upgraded to multi-image classification in our study. The available computational sources together with an increasing number of Earth-observing satellites have led to increased attention to the potential of multi-source satellite imagery, as is reviewed in [43]. There are plenty of other satellites that could be added to crop-classification development efforts, especially Landsat missions. Potential improvements of classification accuracy development during the vegetation season can be achieved by comparison of the classification algorithms, development of improved classification algorithms or application of machine learning algorithms [44]. Further investigation in this field of study with emphasis on development during the season would be beneficial as most of the studies care only about the highest accuracies at the end of the season [17,18,42,43,44].
Another modern alternative coming into question is Unmanned Aerial Vehicles (UAVs) [45]. They are ready to collect data whenever the user needs it and can go high enough for even cloud cover to not be an obstacle. On the other hand, wind can make it impossible to fly a UAV. Moreover, the time consumption of a UAV flight is disproportionately high compared with the satellite data download. The key decision element of choosing the data acquisition platform is spatial resolution. If the purpose of the application is crop detection on a regional or national scale, there is no point in thinking about UAVs. A resolution of 10 m which Sentinel satellites offer is just enough for most applications concerning most common crops such as cereals, corn and oilseed rape (in Europe). It is more profitable to grow these crops on large fields, as big machines are used to handle them. It is not an exception when the fields are tens of hectares. On the other hand, there are a lot of niche crops that are typically grown on a few acres. UAV’s spatial resolution of several centimeters per pixel is fine for these high-value crops (grapevines, vegetables, various kinds of berries, herbs, etc.). A promising idea worthy of further research is the combination of satellite and drone data. The drone data can be used to increase the spatial resolution and to calibrate the satellite data.
Crop-phenology dynamics inspired us to use time series of vegetation indices for crop detection during different stages of vegetation growth. If the experiment will be reproduced at another site, it is important to determine the specific period of interest based on the phenology of the crops to be detected. Our work shows that the use of time series can significantly improve the accuracy of the classification of individual crops compared to single-image classification. Surprisingly, it is possible to have good estimates already in April (72%) and very accurate results in May (84%). This could be important for many purposes, especially for food security strategies but also for the food market. The earlier reliable predictions are available, the more effective the reaction can be. Remote-sensing crop detection is the key to global yield estimates. If we know how big an area is sown by which crop, we only need an average yield per hectare to predict the yield and we do not have to be surprised with the yields after harvest.
This study was intended as an experiment to learn how reliable satellite-data-based crop classification is. We classified only fields where classification could be verified, i.e., where we have crop data from the farmer. This did not bring any added value to the farmer, but we confirmed that it is possible to use this method in unknown fields and expect similar accuracy. Supervised classification always needs some training data so the area cannot be unknown. Nevertheless, when the field borders are available (for example LPIS), it is possible to classify all the fields in the Sentinel tile using data from only a farmer or a few farms. It is important to train all the classes which are desired as the outcome.
5. Conclusions
The main contribution of this study is the accuracy development of crop detection during the vegetation season.
The results reveal a higher classification accuracy based on Sentinel-2 data than the Sentinel-1-based classification. However, this does not mean that Sentinel-1 data are useless. When there are enough Sentinel-2 data, it is better to use them. Nevertheless, a lack of Sentinel-2 data can occur, and in this case, we are ready to use the Sentinel-1 alternative.
Based on the experiments, further research will continue. We assume that there is still some space for improvement of crop-classification accuracy. Individual crop-detection accuracy assessments showed that there are significant differences among crops. From the confusion matrices, appended in Appendix A, we can learn which crop types were hard for the classifier to distinguish between. These crop types can be better trained by adding some training data. We will also deal with the utilization of unsupervised classification. It is much less time-demanding, as there is no training stage and we do not need any other data besides satellite data. Extensive cloud cover for most of the 2021 vegetation season demonstrated that further research of a Sentinel-2 alternative is needed, as there is no guarantee of the availability of these data.
Author Contributions
Conceptualization, K.C. and H.S.; methodology, H.S.; software, J.K.; validation, J.K. and F.Z.; formal analysis, H.S.; investigation, H.S., V.O.; resources, H.K.; data curation, J.K.; writing—original draft preparation, H.S., V.O., J.S. and I.B.; writing—review and editing, K.C.; visualization, F.Z.; supervision, K.C.; project administration, H.K.; funding acquisition, H.K. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by EU Research and Innovation Programme H2020, EO4Agri, Grant number 821940; and SmartAgriHubs, Grant number 818182.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
Appendix A. Confusion matrices of the supervised classification
Explanatory notes:
Reference—from the farmer’s dataset, true
Classified—classification layer, prediction
| 1—Spring barley |
| 2—Winter oilseed rape |
| 3—Corn for grain and silage |
| 4—Winter wheat |
| 5—Spring field peas |
| 6—Poppy |
| 7—Two-row winter barley |
RVI4S1+EVI-based classification before sieve filtering
Table A1.
Error matrix for RVI4S1 + EVI-based classification before sieve filtering in March.
Table A2.
Error matrix for RVI4S1 + EVI-based classification before sieve filtering in March + April.
Table A3.
Error matrix for RVI4S1 + EVI-based classification before sieve filtering in March + April + May.
Table A4.
Error matrix for RVI4S1 + EVI-based classification before sieve filtering in March + April + May + June.
Table A5.
Error matrix for RVI4S1 + EVI-based classification before sieve filtering in March + April + May + June + July.
Table A6.
Error matrix for RVI4S1 + EVI-based classification before sieve filtering in March + April + May + June + July + August.
EVI-based classification before sieve filtering
Table A7.
Error matrix for EVI-based classification before sieve filtering in March.
Table A8.
Error matrix for EVI-based classification before sieve filtering in March + April.
Table A9.
Error matrix for EVI-based classification before sieve filtering in March + April + May.
Table A10.
Error matrix for EVI-based classification before sieve filtering in March + April + May + June.
Table A11.
Error matrix for EVI-based classification before sieve filtering in March + April + May + June + July.
Table A12.
Error matrix for EVI-based classification before sieve filtering in March + April + May + June + July + August.
RVI4S1-based classification after sieve filtering
Table A13.
Error matrix for RVI4S1-based classification after sieve filtering in March.
Table A14.
Error matrix for RVI4S1-based classification after sieve filtering in March + April.
Table A15.
Error matrix for RVI4S1-based classification after sieve filtering in March + April + May.
Table A16.
Error matrix for RVI4S1-based classification after sieve filtering in March + April + May + June.
Table A17.
Error matrix for RVI4S1-based classification after sieve filtering in March + April + May + June + July.
Table A18.
Error matrix for RVI4S1-based classification after sieve filtering in March + April + May + June + July + August.
RVI4S1 + EVI-based classification after sieve filtering
Table A19.
Error matrix for RVI4S1 + EVI-based classification after sieve filtering in March.
Table A20.
Error matrix for RVI4S1 + EVI-based classification after sieve filtering in March + April.
Table A21.
Error matrix for RVI4S1 + EVI-based classification after sieve filtering in March + April + May.
Table A22.
Error matrix for RVI4S1 + EVI-based classification after sieve filtering in March + April + May + June.
Table A23.
Error matrix for RVI4S1 + EVI-based classification after sieve filtering in March + April + May + June + July.
Table A24.
Error matrix for RVI4S1 + EVI-based classification after sieve filtering in March + April + May + June + July + August.
EVI-based classification after sieve filtering
Table A25.
Error matrix for EVI-based classification after sieve filtering in March.
Table A26.
Error matrix for EVI-based classification after sieve filtering in March + April.
Table A27.
Error matrix for EVI-based classification after sieve filtering in March + April + May.
Table A28.
Error matrix for EVI-based classification after sieve filtering in March + April + May + June.
Table A29.
Error matrix for EVI-based classification after sieve filtering in March + April + May + June + July.
Table A30.
Error matrix for EVI-based classification after sieve filtering in March + April + May + June + July + August.
References
- Šafář, V.; Charvát, K.; Horáková, Š.; Orlickas, T.; Rimgaila, M.; Kolitzus, D.; Bye, B.L. D2.2 Initial Workshop—User Requirements and Gap Analysis in Different Sectors Report; EO4AGRI Consortium: Madrid, Spain, 2019. [Google Scholar] [CrossRef]
- Šafář, V.; Charvát, K.; Horáková, Š; Orlickas, T. D2. 3 Mid-Term Workshop-User Requirements and Gap Analysis in Different Sectors. EO4AGRI Consortium. 2020. Available online: https://zenodo.org/record/4247303/files/EO4AGRI_D2.3-Mid-Term-Workshop---User-Requirements-and-Gap-Analysis-in-Different-Sectors-Report_v2.0.pdf (accessed on 21 December 2021).
- Kubíčková, H.; Šafář, V.; Kozel, J.; Král, M.; Křivánek, K.; Řezník, T.; Šmejkal, J.; Vrobel, J.; Zadražil, F.; Charvát, K.; et al. D2.4 Final Workshop—User Requirements and Gap Analysis in Different Sectors Report, EO4AGRI Consortium. Available online: https://ec.europa.eu/research/participants/documents/downloadPublic?documentIds=080166e5d0f69ec0&appId=PPGMS (accessed on 21 December 2021).
- Budde, M.E.; Rowland, J.; Funk, C.C. Agriculture and food availability—Remote sensing of agriculture for food security monitoring in the developing world. In Earthzine; IEEE: Washington, DC, USA, 2010. [Google Scholar]
- Young, O.R.; Onoda, M. Satellite Earth Observations in Environmental Problem-Solving. In Satellite Earth Observations and Their Impact on Society and Policy; Springer: Singapore, 2017; pp. 3–27. [Google Scholar]
- Schmedtmann, J.; Campagnolo, M.L. Reliable Crop Identification with Satellite Imagery in the Context of Common Agriculture Policy Subsidy Control. Remote Sens. 2015, 7, 9325–9346. [Google Scholar] [CrossRef] [Scilit]
- Whitcraft, A.K.; Becker-Reshef, I.; Justice, C.O. A Framework for Defining Spatially Explicit Earth Observation Requirements for a Global Agricultural Monitoring Initiative (GEOGLAM). Remote Sens. 2015, 7, 1461–1481. [Google Scholar] [CrossRef] [Scilit]
- Reynolds, C.A.; Yitayew, M.; Slack, D.C.; Hutchinson, C.F.; Huete, A.; Petersen, M.S. Estimating crop yields and production by integrating the FAO Crop Specific Water Balance model with real-time satellite data and ground-based ancillary data. Int. J. Remote Sens. 2000, 21, 3487–3508. [Google Scholar] [CrossRef] [Scilit]
- Rembold, F.; Atzberger, C.; Savin, I.; Rojas, O. Using Low Resolution Satellite Imagery for Yield Prediction and Yield Anomaly Detection. Remote Sens. 2013, 5, 1704–1733. [Google Scholar] [CrossRef] [Scilit]
- Benami, E.; Jin, Z.; Carter, M.R.; Ghosh, A.; Hijmans, R.J.; Hobbs, A.; Kenduiywo, B.; Lobell, D.B. Uniting remote sensing, crop modelling and economics for agricultural risk management. Nat. Rev. Earth Environ. 2021, 2, 140–159. [Google Scholar] [CrossRef] [Scilit]
- Vlasova, A. Digitalization of agriculture. In Digital Agriculture-Development Strategy, Proceedings of International Scientific and Practical Conference (ISPC 2019), 21-22 March 2019, Ekaterinburg, Russia; Atlantis Press: Paris, France, 2019; pp. 405–409. [Google Scholar]
- Jog, S.; Dixit, M. Supervised classification of satellite images. In Proceedings of the 2016 Conference on Advances in Signal Processing (CASP) IEEE, Pune, India, 9–11 June 2016; pp. 93–98. [Google Scholar]
- Richards, J.A. Supervised Classification Techniques. In Remote Sensing Digital Image Analysis: An Introduction; Richards, J.A., Ed.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 247–318. [Google Scholar]
- Boori, M.; Paringer, R.; Choudhary, K.; Kupriyanov, A.; Ras, M. Comparison of hyperspectral and multi-spectral imagery to building a spectral library and land cover classification performance. Comput. Opt. 2018, 42, 1035–1045. [Google Scholar] [CrossRef] [Scilit]
- Bargiel, D. A new method for crop classification combining time series of radar images and crop phenology information. Remote Sens. Environ. 2017, 198, 369–383. [Google Scholar] [CrossRef] [Scilit]
- Simonneaux, V.; Duchemin, B.; Helson, D.; Er-Raki, S.; Olioso, A.; Chehbouni, A.G. The use of high--resolution image time series for crop classification and evapotranspiration estimate over an irrigated area in central Morocco. Int. J. Remote Sens. 2008, 29, 95–116. [Google Scholar] [CrossRef] [Scilit]
- Zhong, L.; Hu, L.; Zhou, H. Deep learning based multi-temporal crop classification. Remote Sens. Environ. 2018, 221, 430–443. [Google Scholar] [CrossRef] [Scilit]
- Xu, L.; Zhang, H.; Wang, C.; Zhang, B.; Liu, M. Crop Classification Based on Temporal Information Using Sentinel-1 SAR Time-Series Data. Remote Sens. 2018, 11, 53. [Google Scholar] [CrossRef] [Scilit]
- Liang, L. Phenology. In Reference Module in Earth Systems and Environmental Sciences; Elsevier: Amsterdam, The Netherlands, 2019; ISBN 9780124095489. [Google Scholar] [CrossRef] [Scilit]
- Schwartz, M.D. (Ed.) Phenology: An Integrative Environmental Science; Kluwer Academic Publishers: Dordrecht, The Netherlands, 2003; p. 564. [Google Scholar]
- United Nations. World Population Prospects 2019 Highlights, Department of Economic and Social Affairs. Available online: https://www.ined.fr/fichier/s_rubrique/29368/wpp2019.highlights_embargoed.version_07june2019_vf.fr.pdf (accessed on 21 December 2021).
- Donnelly, A.; Yu, R. The rise of phenology with climate change: An evaluation of IJB publications. Int. J. Biometeorol. 2017, 61, 29–50. [Google Scholar] [CrossRef] [Scilit]
- Moon, M.; Zhang, X.; Henebry, G.M.; Liu, L.; Gray, J.; Melaas, E.K.; Friedl, M.A. Long-term continuity in land surface phenology measurements: A comparative assessment of the MODIS land cover dynamics and VIIRS land surface phenology products. Remote Sens. Environ. 2019, 226, 74–92. [Google Scholar] [CrossRef] [Scilit]
- Zhang, X.; Liu, L.; Yan, D. Comparisons of global land surface seasonality and phenology derived from AVHRR, MODIS, and VIIRS data. J. Geophys. Res. Biogeosciences 2017, 122, 1506–1525. [Google Scholar] [CrossRef] [Scilit]
- Wu, C.; Gonsamo, A.; Chen, J.M.; Kurz, W.; Price, D.T.; Lafleur, P.M.; Jassal, R.S.; Dragoni, D.; Bohrer, G.; Gough, C.M.; et al. Interannual and spatial impacts of phenological transitions, growing season length, and spring and autumn temperatures on carbon sequestration: A North America flux data synthesis. Glob. Planet. Chang. 2012, 92-93, 179–190. [Google Scholar] [CrossRef] [Scilit]
- Masialeti, I.; Egbert, S.; Wardlow, B. A Comparative Analysis of Phenological Curves for Major Crops in Kansas. GIScience Remote Sens. 2010, 47, 241–259. [Google Scholar] [CrossRef] [Scilit]
- Gu, Z.; Shi, X.; Li, L.; Yu, D.; Liu, L.; Zhang, W. Using multiple radiometric correction images to estimate leaf area index. Int. J. Remote Sens. 2011, 32, 9441–9454. [Google Scholar] [CrossRef] [Scilit]
- Garonna, I.; Fazey, I.; Brown, M.E.; Pettorelli, N. Rapid primary productivity changes in one of the last coastal rainforests: The case of Kahua, Solomon Islands. Environ. Conserv. 2009, 36, 253–260. [Google Scholar] [CrossRef] [Scilit]
- Giovos, R.; Tassopoulos, D.; Kalivas, D.; Lougkos, N.; Priovolou, A. Remote Sensing Vegetation Indices in Viticulture: A Critical Review. Agriculture 2021, 11, 457. [Google Scholar] [CrossRef] [Scilit]
- Huete, A.R. A soil-adjusted vegetation index (SAVI). Remote Sens. Environ. 1988, 25, 295–309. [Google Scholar] [CrossRef] [Scilit]
- Huete, A.R.; Liu, H.Q.; Batchily, K.V.; Van Leeuwen, W.J.D.A. A comparison of vegetation indices over a global set of TM images for EOS-MODIS. Remote Sens. Environ. 1997, 59, 440–451. [Google Scholar] [CrossRef] [Scilit]
- Huete, A.; Didan, K.; Miura, T.; Rodriguez, E.P.; Gao, X.; Ferreira, L.G. Overview of the radiometric and biophysical performance of the MODIS vegetation indices. Remote Sens. Environ. 2002, 83, 195–213. [Google Scholar] [CrossRef] [Scilit]
- Kim, Y.; Jackson, T.; Bindlish, R.; Lee, H.; Hong, S. Radar Vegetation Index for Estimating the Vegetation Water Content of Rice and Soybean. IEEE Geosci. Remote Sens. Lett. 2011, 9, 564–568. [Google Scholar]
- European Union. The Land Parcel Identification System: A Useful Tool to Determine the Eligibility of Agricultural Land—But Its Management Could Be Further Improved. European Court of Auditors. Available online: https://www.eca.europa.eu/Lists/News/NEWS1610_25/SR_LPIS_EN.pdf (accessed on 21 December 2021).
- Copernicus Open Access Hub. Available online: https://scihub.copernicus.eu/dhus/#/home (accessed on 21 December 2021).
- Ongedo, L. Semi-Automatic Classification Plugin: A Python tool for the download and processing of remote sensing images in QGIS. J. Open Source Softw. 2021, 6, 3172. [Google Scholar] [CrossRef] [Scilit]
- Chuvieco, E.; Congalton, R.G. Using cluster analysis to improve the selection of training statistics in classifying remotely sensed data. Photogrammetric. Eng. Remote Sens. 1988, 54, 1275–1281. [Google Scholar]
- Kar, S.A.; Kelkar, V.V. Classification of multispectral satellite images. In Proceedings of the 2013 International Conference on Advances in Technology and Engineering (ICATE) IEEE, Mumbai, India, 23–25 January 2013; pp. 1–6. [Google Scholar]
- Morgan, R.S.; Rahim, I.S.; Abd El-Hady, M.A. comparison of classification techniques for the land use/land cover classification. Glob. Adv. Res. J. Agric. Sci. 2015, 4, 810–818. [Google Scholar]
- Mahmon, N.A.; Ya’acob, N.; Yusof, A.L. Differences of image classification techniques for land use and land cover classification. In Proceedings of the 2015 IEEE 11th International Colloquium on Signal Processing & Its Applications (CSPA), IEEE. Kuala Lumpur, Malaysia, 6–8 March 2015; pp. 90–94. [Google Scholar]
- HSLayers-NG. Available online: https://ng.hslayers.org (accessed on 24 November 2021).
- Kobayashi, N.; Tani, H.; Wang, X.; Sonobe, R. Crop classification using spectral indices derived from Sentinel-2A imagery. J. Inf. Telecommun. 2020, 4, 67–90. [Google Scholar] [CrossRef] [Scilit]
- Orynbaikyzy, A.; Gessner, U.; Conrad, C. Crop type classification using a combination of optical and radar remote sensing data: A review. Int. J. Remote Sens. 2019, 40, 6553–6595. [Google Scholar] [CrossRef] [Scilit]
- Maxwell, A.E.; Warner, T.A.; Fang, F. Implementation of machine-learning classification in remote sensing: An applied review. Int. J. Remote Sens. 2018, 39, 2784–2817. [Google Scholar] [CrossRef] [Scilit]
- Messina, G.; Praticò, S.; Badagliacca, G.; Di Fazio, S.; Monti, M.; Modica, G. Monitoring Onion Crop “Cipolla Rossa di Tropea Calabria IGP” Growth and Yield Response to Varying Nitrogen Fertilizer Application Rates Using UAV Imagery. Drones 2021, 5, 61. [Google Scholar] [CrossRef] [Scilit]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).







