Similarity Metrics Enforcement in Seasonal Agriculture Areas Classiﬁcation

: Accurate identiﬁcation of agriculture areas is a key piece in the building blocks strategy of environment and economics resources management. The challenge requires one to deal with landscape complexity, sensors and data acquisition limitations through a proper computational approach to timely deliver accurate information. In this paper, a Machine Learning (ML) based method to enhance the classiﬁcation process of areas dedicated to seasonal crops (row crops) is proposed. To this objective, a broad exploration of data from Moderate Resolution Imaging Spectro-radiometer sensors (MODIS) was made using pixel time-series combined with time-series similarity metrics. The experiment was performed in Brazil, covered 61% of the total agriculture areas, ﬁve different states speciﬁcally selected to demonstrate biome differences and the country’s diversity. The validation was made against independent data from EMBRAPA (Brazilian Agriculture Research Corporation), RapidEye Sensor Scene Maps. For the eight tested algorithms, the results were enhanced and demonstrate that the method can rate the classiﬁcation accuracy up to 98.5%, average value for the tested algorithms. The process can be used to timely monitor large areas dedicated to row crops and enables the application of state of art classiﬁcation techniques, two levels classiﬁcation process, to identify crops according to each speciﬁc need within the areas.


Introduction
The identification of agriculture areas is valuable information for the scientific community, government agencies, farmers, and other members of the society. Agriculture main commodity areas are monitored on a global scale to predict production, yield, demand, prices, climate risks, and so forth [1][2][3][4][5]. Related statistics are produced and published to reduce the economic externalities impact and to balance market information asymmetry [6,7]. Ongoing, accurate, and timely crop information remains a challenge, as data becomes available long after the harvesting time, and the machine learning classification is dependent on reliable data sources and processing needs enhancements, despite the available capacity [8][9][10]. The accuracy of the latest results in distiguishing crops of agriculture areas range from 84% to 95% [11][12][13][14]. The state of the art publications in agriculture remote sensing explore the importance of machine learning applied in combined data sources and types in order to enhance the process and accuracy for multiple crops as classes.
When it comes to process improvements and machine learning, Convolutional Neural Networks (CNN) was applied to achieved significant results in image recognition tasks by automatically dynamics can be detected and properly managed [8,9], (e.g., crop phenology stage, chemicals administration, mechanization, and irrigation management, among others). The critical components for remote sensing are: accurate and current information for training the classifiers; an affordable source of data that qualifies for the specific objective; a storage and processing capacity [10,27,28]. By using MODerate-resolution Imaging Spectrometer (MODIS) products that include the Enhanced Vegetation Index (EVI) [29], among others, we have access to a complete record of data from each of the Terra and Aqua MODIS sensors, at varying spatial (250 m, 0.05 degree) and temporal (8-day) resolutions validated with accuracies depicted by a pixel reliability flag and with globally averaged uncertainties of 0.015 units. Further, the MODIS/EVI combination is a robust set for exploring seasonal crops (soybeans) [22]. The spatial resolution adopted are consistent in expressing accurate cropland information in fields that are larger than 20 ha [13], and appropriate to this task [30]. This experiment relies on the achievements and specifications above to explore the spectral dynamics of extensive areas, in accordance with the specifics purpose stablished for this research.

Brazilian Agriculture in Numbers
In Brazil, agriculture represents almost a quarter of the country GDP's, 24.1% in 2017 [31]. According to the Brazilian Institute of Geography and Statistics (IBGE) rural census [32], the harvested area during the 2016/2017 season represented 7.9% of the total area with 73,797,057 hectare (ha) dedicated for temporary and semi-temporary crops and another 5,184,813 ha for permanent crops, with the final results published in 2019. For the purpose of this study, detailed statistics of the studied areas are summarized on the Table 1. The column "Total" shows the total area (country/State). The column "Agriculture" presents the harvested area (country/State). It is important to highlight that, due to geographical and environmental conditions, which include climate and/or technologies (e.g., irrigation), it is common to have multiple crops per year in tropical areas. Therefore, the conclusion is that the harvests are larger than the total area dedicated to agriculture, a total of 67,547,537 ha according to EMBRAPA [33,34] and MAPA [35].
The main crops cultivated locally are presented below, in Table 2, and the data is organized as a percentage of total area occupied to provided a land use perspective view, and put in evidence of a larger area per crop cultivated.

Studied Area
The study was performed over Brazil at the state level, see Figure 1. The locations were selected to bring the diversity (biome, crop, climate, etc.) and its complexity into the analysis. The states studied were: Goiás (GO), Minas Gerais (MG), Mato Grosso (MT), Parana (PR) and São Paulo (SP), Table 1. In this study the growing season (phonological) cycle is the key aspect taken in consideration, crop recognition requires specifics and may be performed in accordance to them, combining scenes and the growing season maps.

IBGE Census
As a reference in this study, IBGE Agriculture census data was used to contrast results, that is, quantify the total area and row crop areas for each studied state. The original numbers were consolidated and published by municipalities and by state, consequently. For this research objective the productive clusters are represented in maps and at the state level, without municipal geopolitical boundaries. The preliminary census results were published in 2018 and final results late in 2019.

Ontology
The use of expert knowledge management based on ontology process to support the collective understanding of a single event (Data driven approach) was applied to proper address crop growing season specifics to expand the comprehension possibilities in a multidisciplinary context [36]. The same author defined ontology scope as being the "specialist of specifics", meaning the understanding of a domain group. In this environment the essential is the potential for information and knowledge sharing [37]. The proposed Ontology is presented in the concept map below, Figure 2, where the target class are highligted and the profiles are demonstrated. The data was segmented in accordance to the Global Food Security (GFS) [38] and MODIS ontology into two large classes, Agriculture and Other. Agriculture, as the target class, included: Irrigated and dry seasonal crops areas potentially featuring corn, soybean, cotton, barley, potato, alfalfa, sorghum, rye, canola, peanut, manioc, and beet.
The Others class included: perennial and semi-perennial/semi-temporary crops, natural vegetation (forest, cerrado, amazon forest, tropical forest), pasture, urban areas, and water surface, among others. The agriculture class areas are those where we expected to find a high level of EVI values dynamics, which is the purpose of this study, and is also compatible with the objective of the sensor selected. As a reference, the profile images that are representing the target group are from SatVeg [39]. Only the crops that can be found within the region and are described as part of sensor ontology are here in evidence.

Datasets Processing
The criteria for the selection of the sensor was based on the most recent publications and goal, as cited above. The .tiff (Tagged Image Format) dataset files were downloaded from the MODIS repository as .hdf (Hierarchical Data Format) files according to provided guidelines, and then framed in accordance to each studied state and a country view. The experiment was performed using EVI from MOD13Q1 with 250 m of spatial resolution.
Accord to the methodology outlined as a flowchart in Figure 3, a large amount of random Pixels Time Series (PTS) were extracted from each studied area to achieve 99% +1 −1 of confidence level, with 17,000 points for each dataset, and 5 datasets per state. PTS values were extracted as vectors and stored as .csv (Comma Separated Values) files using a expert system created for this purpose. Each vector contained the pixel position, EVI raw data as time series values, and calculated computational distances, with similarity metrics proposed. The data were then pre-classified into two large groups, Agriculture and Others. Agriculture was the target group, which expected areas with high spectral dynamics level, and the Others group contained lower dynamics level areas. Each experiment used 20% of data set for training.
Given the inherent complexity of vegetation and environments which are all reflected within the region selected, for each state, a group of similarity metrics were selected and tested as a way to put in evidence the expected characteristics and work as a hybrid index in the classification process, as in Reference [40]. The computational distances used in this experiment are: Manhattan distance, Minkowski distance, Sum, Mean, Median, Standard deviation, Coefficient of Variation, Variance, and Difference [41,42]. All the similarity metrics that used the distances were calculated for each pixel time-series. In particular, the Minkowski distance used has been explored by References [43,44] to cluster time series that have different temporal resolutions, as the case for the Dynamic Time Warp (DTW), and in this case it was used to measure the length of the time-series. The possible values for the c are useful to accommodate the time difference into the time series. The equation is demonstrated below, Equation (1): where: the distance D, p and q are the data points (x i+1 and x i , respectively) and used c = 2.

Time-Frame
For this research, 36 raster layers from the MODIS sensor were used, with an 18 month time frame coverage, from July 2016 to December of 2017 to match IBGE census data also used as a reference in this study. The time frame selected meets the objective of measuring annual dynamics for temporary crops, thus excluding semi-temporary crops (e.g., sugarcane) from the analysis.

Validation
Tree levels of validation were considered-the classification accuracy, the concordance between algorithms, and the computational metrics relevance. EMBRAPA scene maps were used as an external reference data source to contrast and validate accuracy. The spatial resolution of the the validation maps was 5 m to provide quality data for the training set used, to preserve the characteristics that are relevant to the experiment [8], and to match the field size.
The classification accuracy validation was made according to the proposed ontology, a confusion matrix used to contrast results as True-Positive (TP), False-Negative (FN), False-Positive (FP) and True-Negative (TN). Accuracy stands for all that were correctly classified, as in the Equation (2), and recall stands for positives that were correctly classified, as in the Equation (3), below: The concordance between algorithms assessment were made using the Cohen's Kappa coefficient or simply (k), which has been largely used to evaluated classification algorithms performance as described in Reference [45]. Kappa values ranges from −1 to 1, where −1 represents "complete disagreement", 0 is a "random classification", and 1 is a "perfect agreement".
The computational metrics used are the sensitivity and specificity, Equations (4) and (5) below, where we expect to identify the impact of the selected similarity metrics composition on the classification process of the target group.
where "true positive" is the number of correctly predicted areas, "positive" is the number of agriculture, "true negative" is the number of correctly predicted as others, and "negative" is the number of others shown in the classification. The impact of each similarity metric as an attribute over the target class was evaluated using Shannon entropy index to reveal in a [0:1] scale the cluster behavior [46,47]. The calculation of the distances was made according to the Equation below, Equation (6): where E is the information strengh of the attibute n over the class i, and p is the probability of the class in n.

Computation Tolls
All data manipulation and tests were performed using R R language, Matlab R , RapidMiner R , and QGIS R . Using these tools to evaluate the effectiveness of the proposal, we explored the data with eight Machine Learning algorithms: Naive Bayes, Logistic Regression, Decision Tree, Gradient Boosted Tree, Generalized Linear Model, Deep Learning, Random Forest. This collection represents most of the algorithms used in the cited publications.
The data used were min-max normalized to comply with the classification tolls specs. The values were linearly reduced to a scale between [0:1], where 0 and 1 are the minimum and maximum values, respectively. The z is the normalized value accord to Equation (7) below:

Results
By performing the study as proposed, we observed that the addition improved the specificity and sensitivity of the algorithms. The sensitivity of the algorithms can be verified by contrasting ROC comparison curve (detection probability in machine learning). The Figure 4 shows the classification process for data without similarity metrics and Figure 5 shows the sme process for data with similarity metrics. Specificity raised from 98.6% to 99.2% with a direct effect on accuracy, sensitivity enhancement from 94.62% to 97.4% which improved processing performance despite the data dimensional increase. As we can verify, data with similarity metrics increased sensitivity for all tested algorithms.
The enhancement for each method is demonstrated below, Table 3, where the results for each state are presented. The caption MT, GO, MG, SP and PR are the abbreviations of the states mentioned on Section 2.1. The columns RD and DSM represent the accuracy with the different type od data, Raw Data (RD) and Data with Similarity Metrics (DSM). It is important to point the difference in results between the algorithms, these differences reflect the way each of them internally organizes the data toward the best solution.
The concordance of the algorithms index (Kappa) and the enhancement range achieved for each method with Raw Data and Data with Similarity Metrics (RD-DSM) is demonstrated in the Table 4.
The highest impact was achieved with Deep Learning, increased the accuracy from 83.9% to 98.6% with Kappa index of 0.95. However, the highest accuracy was achieved with Gradient Boosted Tree, results were enhanced from 95.2% to 99.6% and with Kappa index of 0.96.  The hierarchal clustering analysis below, Figure 6, put in evidence the hierarchal importance of each metric for the process. The relevance of each attribute is calculated accord to Shannon information and the normalized score is represented hierarchically. The Minkowski distance played a key part in the classification process and demonstrated to be very representative of the target class, highest score. Canberra distance, Standard deviation, Mean, Coefficient of Variation, and Variance were also very relevant in the high score process, but not as decisive. The removal of the Manhattan distance, Sum, Median, and Difference had no impact on the process accuracy.   The correlation between a group of indexes found in the bibliography were also evaluated as alternatives to the EVI. Results are presented in Table 5, where the correlations are contrasted. From this perspective, none of the indexes were found as strong enough to be used as alternative or to be replaced within the same method. Finally, from the process it was possible to generate a scene map layer identifying row crops areas, as presented on Figure 7, below. The figure shows where the row crops clusters are located.

Discussion
As presented in the Results section, the proposed approach enhanced the classification accuracy for all tested methods. The similarity metrics in combination with time series improved the average accuracy from 88.7% to 98.5%, as presented on Table 4. Therefore, the proper selection of similarity metrics showed potential to enhance the classification efficiency, increase classification accuracy without extra processing time.
The results achieved are higher than all previous work that addressed classification of agriculture row crop areas in Brazil and, in particular, the results achieved by References [13,14,[20][21][22]24] as presented in Table 6. In regard to Reference [14], which achieved the highest result prior to this work, despite the large area, they only study one state (Mato Grosso) and tree crops. The second highest result achieved [13], compared results for 6 algorithms and results ranged from 84% to 96%, covered 5 countries and noticed the lowest accuracy for Brazil (84%). The same author appointed as a key finding that "the site effect dominates the method effect". Table 6. Results contrasted.

Author Area (ha) Accuracy Kappa Comments
Publications that share at least one common aspect: Brazil as studied area [14] 5,617,250 95.0% 0.98 -3 tree seasonal crops in Mato grosso/ [13] 507,728 84.0% -used the same algorithms, achieved 84% accuracy for Brazil/ [22] 724,293 84.0% 0.56 -differentiated Soybean and non-soybean/ [24] 1600 83.0% 0.78 -used high spatial resolution images/ [21] 9658 74.6% 0.57 -seasonal crops group as part of the results/ International Publications that share at least one similarity [48] 258,500 97.0% -seasonal crops group as part of the results/ [12] 2,800,000 94.6% -seasonal crops classification/ It is important to highlight that in Brazil we have summer crop season which is in general rainy and cloudy, more than 90% of the production is rainfed. According to References [49,50], less than 40% of temporal images are useful for analysis. In this context, in this research, we bring the need for the combination of high temporal resolution with high spatial resolution for validation, MODIS and RapidEye, respectively.
According to Reference [48] the combination of temporal and spectral information can improve classification accuracy than only using spectral information, 10-15% higher. In this work, we demonstrated that the proper similarity metric can enhance time series classification accuracy, 3-14%, 10% on average for the tested algorithms. In fact, the combination created condition to increase specificity (accuracy) and sensibility (convergence). Both aspects are demonstrated on Figures 4 and 5 above. The sensibility, in this case the positive impact on the classification process, for each of the algorithms were different. Below we present the sensibility for each one of the algorithms: Two level classification strategy, as proposed by Reference [18], or multi-level as used by [14] are a possibility from the achieved results. The layer identifying temporary crop clusters, as presented on Figure 7, meets the demand presented by References [12,20,22,48], among others.
Another important aspect is the potential of the process to remotely identify agricultural areas as a dynamic census process. Timely spatial information about productive areas with a high confidence level (99% +1 −1 ) is relevant for policy makers and for the private sector. This information can support IBGE statics and contribute to their monitoring process. In regard to private sector, precision agriculture services can be provided in large scale when geospatial data organized and available.
The process also demonstrated some limits that are related to the objective of each search and complexity of the environment. An accurate time-frame collection of maps is required to enhance the uniqueness of the target. It is also important to remember that on top of spectral resolution, temporal and spatial resolutions are key to put in evidence specifics of cultivars. Therefore, sugarcane and cultivated pasture have specifics in growing season, and requires specifics in spectral and temporal resolution to enhance the classification process [51].

Conclusions
This study explored the classification of agriculture areas dedicated to temporary crops using well-known ML algorithms to examine time-series from a collection of vegetative indexes in combination with similarity metrics. As a conclusion we have that:

•
As the primary objective, the results demonstrated that the approach enhanced the classification accuracy for all tested algorithms. This is the highest accuracy for the classification of agriculture areas in Brazil, 99.6% with kappa index of 0.96 using Gradient Boosted Tree algoriothm; • The similarity metrics worked to increase the accuracy within the context proposed, EVI data reflecting the growing season dynamics of temporary crops. The similarity metric added 3-14% points to the accuracy; • The process increased accuracy and without extra computational cost.

•
The results are robust to support policy maker and precision farming, 99% +1 −1 of confidence level.
The field knowledge (scene map generated) allows crop level classification improvements, that is, crop differentiation using the available techniques accordingly to each specific need. This information is useful for further research and also to support the private sector and public sectors on monitoring and spatial planning of annual crops in Brazil.
To enhance results and explore the approach, modifications in the Minkowski equation, c value, should be tested to address specifics in phenology time frame. The size of the data set for training should be quantified, it is expected that the a smaller data set size would maintain accuracy. Moreover, DTW can be used to explore the differences in growing season for the two levels crop classification process, among others. Specifics in computational costs and cloud computing benefits should also be explored and demonstrated.