Application of Google Earth Engine Cloud Computing Platform, Sentinel Imagery, and Neural Networks for Crop Mapping in Canada

Abstract

The ability of the Canadian agriculture sector to make better decisions and manage its operations more competitively in the long term is only as good as the information available to inform decision-making. At all levels of Government, a reliable flow of information between scientists, practitioners, policy-makers, and commodity groups is critical for developing and supporting agricultural policies and programs. Given the vastness and complexity of Canada’s agricultural regions, space-based remote sensing is one of the most reliable approaches to get detailed information describing the evolving state of the country’s environment. Agriculture and Agri-Food Canada (AAFC)—the Canadian federal department responsible for agriculture—produces the Annual Space-Based Crop Inventory (ACI) maps for Canada. These maps are valuable operational space-based remote sensing products which cover the agricultural land use and non-agricultural land cover found within Canada’s agricultural extent. Developing and implementing novel methods for improving these products are an ongoing priority of AAFC. Consequently, it is beneficial to implement advanced machine learning and big data processing methods along with open-access satellite imagery to effectively produce accurate ACI maps. In this study, for the first time, the Google Earth Engine (GEE) cloud computing platform was used along with an Artificial Neural Networks (ANN) algorithm and Sentinel-1, -2 images to produce an object-based ACI map for 2018. Furthermore, different limitations of the proposed method were discussed, and several suggestions were provided for future studies. The Overall Accuracy (OA) and Kappa Coefficient (KC) of the final 2018 ACI map using the proposed GEE cloud method were 77% and 0.74, respectively. Moreover, the average Producer Accuracy (PA) and User Accuracy (UA) for the 17 cropland classes were 79% and 77%, respectively. Although these levels of accuracies were slightly lower than those of the AAFC’s ACI map, this study demonstrated that the proposed cloud computing method should be investigated further because it was more efficient in terms of cost, time, computation, and automation.

1. Introduction

Knowledge of the location, extent, and type of croplands are important for food security systems, poverty reduction, and water resource management [1]. Therefore, it is crucial for stakeholders to have accurate cropland maps. This will help in coordinating managerial plans and policies for croplands. In this regard, remote sensing provides valuable opportunity for classifying and monitoring agricultural areas in a cost- and time-efficient manner especially over large areas [2].

Numerous studies have investigated the potential of remote sensing for cropland mapping worldwide, most of which have been conducted over relatively small geographical areas. For instance, the study sites of the Group on Earth Observation Global Agricultural Monitoring (GEOGLAM) Joint Experiments for Crop Assessment and Monitoring (JECAM) are no larger than 25 × 25 km². Although such studies have large value for demonstrating the capability of new methodologies and algorithms for crop mapping purposes, their ability to be scaled for operational application over much larger regions often remains unclear. In this regard, one of the important issues to be considered is processing big geo data (e.g., thousands of satellite images, often comprising tens of terabytes of data) over a large area in the classification procedure [3]. This task is impossible by commonly used classification software packages. Thus, several platforms, such as Google Earth Engine (GEE), have been developed to effectively process big geo data and produce large-scale maps.

The GEE cloud computing platform contains numerous open-access remote sensing datasets and various algorithms (e.g., cloud masking functions and classifiers) which can be easily accessed for different applications, such as producing country-wide cropland inventories [4,5,6]. So far, several studies have utilized GEE to produce large-scale cropland classifications. For example, Dong et al. [7] mapped paddy rice regions in northeast Asia using Landsat-8 images within GEE. Their results showed that GEE was significantly helpful in annual paddy rice mapping over large areas by reaching the Producer Accuracy (PA) and User Accuracy (UA) of 73% and 92%, respectively. Moreover, Xiong et al. [8] developed an automatic algorithm within GEE to classify croplands over the entire Africa using various in situ and remote sensing datasets. Massey et al. [9] also used Landsat imagery available in GEE to map agricultural areas in North America in 2010. For this purpose, they fused pixel-based Random Forest (RF) and an object-based segmentation algorithm, called Recursive Hierarchical Segmentation (RHSeg). The Overall Accuracy (OA) of the final map was more than 90%, indicating the high potential of their proposed method for continental cropland mapping tasks. Finally, Xie et al. [10] classified irrigated croplands over the conterminous United States with 30 m optical Landsat imagery in GEE. They reported that they were able to obtain the OA and KC of 94% and 0.88, respectively, at the aquifer level in 2012.

Canada contains numerous agricultural areas, making the country one of the largest food producers and exporters in the world [7]. Given the vastness and complexity of Canada’s agricultural regions in terms of crop types, remote sensing is one of the most reliable methods to produce detailed information describing the evolving state of the country’s environment. Sector-wide, there is a need for relevant, timely, and accurate spatially specific information on the state and trends in agricultural production, bio-physical resource utilization and mitigations, domestic and international market conditions, and how these conditions relate to global conditions. These needs coincide with a rapid expansion of data availability brought about by the increasing development of technology that has made geospatial data less expensive, more reliable, more available, and easier to integrate and disseminate. Over the past decade, various research studies have explored the utility of remote sensing methods for agricultural mapping across Canada’s diverse and often complex agricultural landscape. For instance, Deschamps et al. [11] compared Decision Tree (DT) and RF algorithms to map Canadian agricultural areas using satellite imagery. They reported that RF outperformed DT by up to 5% and 7% for OA and class accuracies, respectively. They employed multi-temporal remote sensing images acquired between May and September and reported that the best time to distinguish different crops was between August and early September at the peak biomass and after wheat harvest. Jiao et al. [12] also applied multi-temporal RADARSAT-2 images to an object-based classification algorithm to classify croplands in an agricultural area in northeastern Ontario, Canada. They classified five different crop types with an OA of 95%. Finally, Liao et al. [13] used the RF classifier along with multi-temporal RADARSAT-2 datasets to map croplands in southwestern Ontario, Canada. Their results showed that the elements of the coherency matrix and the imagery acquired between June to November provided the highest classification accuracy (OA = 90%).

As mentioned above, most studies on cropland classification in Canada were conducted over relatively small areas and using relatively traditional methods where a small number of crop types were mapped. The only near nation-wide cropland classification is the Annual Space-Based Crop Inventory (ACI) produced by the Agriculture and Agri-Food Canada with 30 m spatial resolution (AAFC, Figure 1). ACI is updated annually and is available free of charge to the public via the Government of Canada‘s Open Data Portal. ACI contains a map of the agricultural land use and non-agricultural land cover found within Canada’s agricultural extent [14]. The DT methodologies along with optical (e.g., Landsat-8) and SAR (e.g., SGX dual-polarization RADARSAT-2 in the Wide mode) imagery are applied to produce ACI. Although these maps achieve an OA of 85% at the national scale, its accuracy can vary from crop-to-crop, from region-to-region, and from year-to-year, depending on the satellite data availability and training site distribution. Generally, the highest mapping accuracies (>90.0%) are found where crops display significantly different spectral characteristics at the time of the remote sensing data acquisition. Conversely, lower mapping accuracies (70% to 80%) occur where crops are spectrally similar [14].

Developing and implementing novel methods for improving the accuracy of ACI is an ongoing priority of AAFC remote sensing science. Recent advancements in the fields of machine learning and big data processing methods suggest that alternative approaches may exist for more quickly generating agricultural maps of Canada with high accuracy. Consequently, it is required to investigate the most recent machine learning algorithms and cloud computing methods to produce accurate ACI map in a more operational and automatic approach. Thus, this study aims to classify croplands over 10 provinces of the country using GEE and Artificial Neural Network (ANN) for the first time. For this purpose, all the Ground Range Detected (GRD) products of Sentinel-1 and Sentinel-2 acquired in 2018 were applied to produce an object-based ACI map at the spatial resolution of 10 m.

2. Study Area and Datasets

2.1. Study Area

Figure 2 illustrates the study region in the red boundary with an area of 2.803 million km² which comprises ~28% of Canada’s terrestrial landmass. This area includes important agricultural areas over 10 provinces of Canada, and it is the region with which AAFC generates its ACI maps. The study area contains a wide range of ecoregions and is characterized by various ecological factors, including climate, vegetation, and water because of its large extent and geography. Although land cover varies across the study area, it is mostly dominated by forests, wetlands, grassland, and croplands [3,15].

2.2. Field Data

Field data for 2018 were provided by AAFC. AAFC collects tens of thousands of crop type observations from crop insurance information and car-based “windshield surveys” every year. In the latter approach, some primary sampling units were first randomly delineated from the existing databases (i.e., administrative regions) according to their cropland size [14]. Subsequently, the Global Positioning System (GPS)-enabled tablets were used to determine the crop type along the roads from the moving vehicle. This approach allowed a simple and rapid collection of numerous field samples from diverse crop types. In this study, the AAFC database for 2018 that comprises the boundaries of the croplands mapped during the 2018-windshield surveys across the country was utilized. The quality of field boundary delineation was validated using a cross-check with existing high-resolution optical imagery from ArcGIS basemap and Google Earth. Figure 2 illustrates the distribution of field samples across Canada and

Table 1. The list of cropland classes along with their number of field samples (polygons) which were used in this study.

Class	NL	PE	NS	NB	QC	ON	BC	Sum
Barley/Oats/Spring Wheat	11	424	167	711	32,772	1210	79	35,374
Rye	0	16	21	6	829	285	4	1161
Winter Wheat	1	38	128	3	2587	2269	17	5043
Corn	28	143	852	507	61,562	11,065	574	74,731
Tobacco	0	0	0	0	0	221	0	221
Ginseng	0	0	0	0	0	190	3	193
Canola/Rapeseed	0	22	4	48	1789	263	6	2132
Soybeans	0	247	316	262	54,586	12,099	15	67,525
Peas	5	58	0	48	593	58	8	770
Beans	0	0	0	1	1044	239	7	1291
Tomatoes	0	0	0	1	39	110	2	152
Potatoes	34	532	39	1008	1802	231	98	3744
Blueberry	4	66	265	155	486	7	516	1499
Cranberry	12	4	13	28	484	0	93	634
Orchards	0	4	186	55	246	462	728	1681
Vineyards	0	4	63	9	19	346	361	802
Buckwheat	0	12	1	1	658	9	0	681
Sum	91	1570	2055	2848	159,496	29,064	2511	197,634
NL: Newfoundland and Labrador, PE: Price Edward Island, NB: New Brunswick, NS: Nova Scotia, QC: Quebec, ON: Ontario, BC: British Columbia.

provides the information about the field samples in each province. It is worth noting that although AAFC considers about 59 different cropland classes (see Figure 1), only 17 cropland classes, which are the main classes found in Canada, were mapped in this study. The main reason for this was that the number of field samples for some of the classes was insufficient for training and validating the classification algorithm. Moreover, three classes of Barley, Oats, and Spring Wheat were combined. This was because they had similar spectral and backscattering characteristics in the satellite images and contained a high confusion in the classification.

2.3. Satellite Data

A combination of multi-temporal SAR (Sentinel-1) and optical (Sentinel-2) imagery, acquired in 2018, was used in this study. Each of these satellite dataset can detect different spectral and physical characteristics of crops and, thus, a combination could potentially compensate the limitation of using one type of imagery [16,17,18]. For instance, Davidson et al. [14] reported that experiments undertaken at Canadian research sites indicated that a combination of Landsat and RADARSAT-2 data can improve the accuracy of cropland classification by up to 5–8% compared to when only Landsat imagery was used. Additionally, many studies (e.g., [15,17,19]) have discussed that it is essential to use multi-date satellite data to produce highly accurate cropland classification. This is because various croplands have different phenology and temporal patterns, which can be potentially detected by multi-temporal remote sensing images.

Sentinel-1 acquires C-band data in various swath modes (i.e., Interferometric Wide swath (IW), Extra Wide swath (EW), and Strip Map (SM)), and both single (HH: Horizontal transmit and Horizontal receive polarization, or VV: Vertical transmit and Vertical receive polarization) and dual polarizations (HH/HV: Horizontal transmit and Vertical receive polarization, or VV/VH: Vertical transmit and Horizontal receive polarization). The primary conflict-free modes are IW with VV+VH polarization over lands (excluding polar regions) [20]. In this study, the GRD data collected in IW mode and in the ascending orbit with 10 m spatial resolution were used. Furthermore, since the HH and HH/HV polarizations are designed only for polar regions [20], they were not available over the entire Canada and, thus, only the VV/VH dual polarization was used in this study.

Sentinel-2 is a European multi-spectral satellite, which acquires imagery in 13 spectral bands, such as visible (band 2–4), Red Edge (RE, band 5–7), Near Infrared (NIR, band 8), and Shortwave Infrared (SWIR, band 11–12) bands with 10 m, 20 m, 10 m, and 20 m spatial resolutions, respectively. In this study, the Normalized Difference Vegetation Index ($NDVI=\frac{NIR - Red}{NIR + Red}$) and the Normalized Difference Water Index ($NDWI=\frac{NIR - SWIR}{NIR + SWIR}$) with 10 m spatial resolution were only used to reduce the computational cost of processing.

3. Methodology

The flowchart of the method to produce the ACI map using the GEE platform and ANN algorithm is illustrated in Figure 3. The details of each step are also discussed in the following five subsections.

3.1. Satellite Data Pre-Processing

The available Sentinel-1 Ground Range Detected (GRD) products in the ascending mode within GEE (Image Collection ID: COPERNICUS/S1_GRD) were used in this study. This product was already calibrated to backscattering coefficient (σ°, dB), ortho-rectified, and converted to the backscattering coefficient using the Sentinel-1 Toolbox [21]. In fact, the five following pre-processing steps were used by the GEE developers to derive the backscattering coefficient for each pixel of Sentinel-1 image, the details of which are discussed in [22]: (1) applying orbit file, (2) GRD border noise removal, (3) thermal noise removal, (4) radiometric calibration, and (5) terrain correction. In addition to these pre-processing steps, a foreshortening mask correction was applied to reduce the geometric distortions due to SAR data imaging [23].

The level-1C Sentinel-2 images available within GEE (Image Collection ID: COPERNICUS/S2) were used in this study. These products were converted to Top of Atmosphere (TOA) reflectance values scaled by 10,000 through radiometric calibration [24]. Because of the necessity of cloud masking in optical images, all scenes with higher than 10% cloud cover were also removed from further steps. Moreover, the quality band was applied to remove invalid observations as well as to produce the cloud-free mosaic optical image over the study area.

Finally, the pre-processed Sentinel-1 and Sentinel-2 datasets were clipped over the cropland areas using the union of the AAFC’s cropland inventory map (Figure 1) and Moderate Resolution Imaging Spectroradiometer (MODIS) yearly land cover product with 500 m spatial resolution (Image Collection ID: MODIS/006/MCD12Q1) [25]. By doing this, the final study area was the cropland areas identified by either the AAFC or MODIS land cover product in 2018.

3.2. Feature Extraction

The performance of classification algorithms depends on the extraction of suitable features [26]. Therefore, in this section, the extracted features from optical and SAR datasets are discussed.

Regarding SAR features, the mean function was initially applied to preprocessed Sentinel-1 images to produce bi-monthly VV and VH images (see

Table 2. Number of the Sentinel-1 and Sentinel-2 images used for each time interval.

Months	Sentinel-1		Sentinel-2
	Features	# Images	Features	# Images
January–February	VV-1, VH-1	537	NDVI-1, NDWI-1	9552
March–April	VV-2, VH-2	545
May–June	VV-3, VH-3	620	NDVI-2, NDWI-2	14,532
July–August	VV-4, VH-4	705
September–October	VV-5, VH-5	660	NDVI-3, NDWI-3	11,861
November–December	VV-6, VH-6	761
Total	6 VV + 6 VH	3828	3 NDVI + 3 NDWI	35,945

). The mean function enabled the production of 12 mosaic SAR images (6 VV + 6 VH) over the study area for the whole year of 2018. Moreover, the mean function reduced the effects of speckle noise in the feature layer [27].

Spectral indices derived from remote sensing images have been widely employed for crop classification [28]. In this regard, the Normalized Difference Vegetation Index (NDVI) proved to be competent in crop classification [29,30]. Moreover, the Normalized Difference Water Index (NDWI), which contains the amount of plant water content, has been extensively utilized for crop mapping [31,32,33]. Finally, it was reported that using a combination of these two indices provides complementary information about vegetation canopies and, thus, could improve the accuracy of crop classification [34]. Therefore, in this study, the NDVI and NDWI images were calculated from preprocessed Sentinel-2 images. A median function was applied to all NDVI and NDWI images within every four months, resulting in three NDVI and three NDWI images for the year of 2018 (see Table 2). This function allowed producing six single homogenous and cloud-free NDVI/NDWI images over the study area. Moreover, this function removed the wrong values that were calculated because of the existence of very bright/dark or noisy pixels [3,35]. It should be noted that the reason to produce the four-month mosaic images was the fact that four month was the minimum amount of time to produce cloud-free NDVI/NDWI mosaic images over the study area because of the frequent cloud cover in Canada.

Finally, three NDVI, three NDWI, and 12 SAR images were stacked, producing a single mosaic image with 18 layers over the study area. For this purpose, 3,828 Sentinel-1 and 35,945 Sentinel-2 images were processed within the GEE (see Table 2). This final mosaic image was applied to the classification algorithm to map croplands. It is worth noting that utilizing these two features at different time intervals reduced the artifact lines between images acquired at different orbits.

3.3. Segmentation

Many studies have reported that object-based image analysis could improve crop type classification compared to pixel-based methods [36,37]. Therefore, the Simple Non-Iterative Clustering (SNIC) algorithm, which is the best segmentation algorithm in GEE, was applied to segment the final layer-stacked mosaic image. SNIC is an improved version of the Simple Linear Iterative Clustering (SLIC) segmentation algorithm that benefits from a non-iterative procedure and enforces the connectivity rule from the initial stage [38]. The SNIC is initialized by a user-defined number of seed points spread on a regular grid in the image space. Afterward, a distance measure along with the 4/8 connectivity rule are applied to grow each seed point, resulting in final segments. In this study, the SNIC algorithm was applied to the single mosaic image with 18 layers and, then, the values of each segment were averaged to decrease the noises and increase the reliability of the samples that were used in further steps. Finally, these segments were employed to perform an object-based classification to produce the ACI map. The implemented SNIC algorithm within GEE comprises five input parameters [39]: Size, Compactness, Connectivity, Neighborhood Size, and Seeds (an optional parameter). In this study, these parameters were set to 50, 1, 8, 100, and null, respectively based on multiple trials and errors to find the optimum values for these parameters.

Figure 4 illustrates the results of SNIC segmentation over a sample region in the study area.

3.4. Classification

Deep learning algorithms, such as ANN, have been extensively employed for crop type classifications [33,40,41,42]. ANN was inspired by the biological neural networks that simulates the human brain nervous recognition system with high potential for non-linear classification capacity [43].

In this study, the single mosaic image with 18 layers generated within GEE were used to produce the ACI map using the ANN algorithm. To this end, the reference samples were first randomly divided into three sets of training (70%), validation (15%), and test (15%) samples. According to sampling theory, random sampling shall provide independent information and avoid spatial biases [14]. The training, validation, and test samples were applied to train the ANN algorithm, adjust the hyperparameters of the ANN architecture, and accuracy assessment of the final ACI map, respectively. Despite the high computational performance of GEE, this cloud platform has computational limitations for the cases when the numbers of reference samples or input features for classification are too large [3,6,35]. Therefore, the mean values of training and validation samples were first extracted (for each segment) from the single mosaic image with 18 layers in GEE and, then, were transferred into a local computer to train a feedforward ANN algorithm. It should be noted that the training phase was conducted on a local computer with an Intel Core i7-5820K, 3.3 to 3.6 GHz.

Generally, four parameters of the number of layers, number of neurons, type of activation function, and learning algorithm are used to determine the architecture of an ANN model. In this study, an ANN architecture with two hidden layers with 40 and 30 neurons was used, respectively. The numbers of neurons were selected by implementing multiple trials and errors to find the most optimum values. The numbers of neurons for the input and output layers were 18 and 17 that were equal to the number of input features and ACI classes, respectively. The activations function of all neurons was the tangent sigmoid. Furthermore, the minimum performance gradient was used as the stopping criterion, and the model satisfied this criterion after 4,226 iterations. The Scaled Conjugate Gradient (SCG) method was also employed as the back-propagation learning algorithm. The SCG employs second-order techniques in the second derivatives and conjugate direction to find a better local minimum in each step [44]. In contrast to the algorithms that employ gradient direction as the search direction, SCG avoids zig-zag solutions [45]. Furthermore, the SCG algorithm benefits from relatively small storage requirement and fast convergence speed, which can be useful for large-scale studies [45,46]

After training the ANN model, it was transferred to GEE and was applied to the entire study area to produce the ACI map. It is worth noting that all the weights and biases of the trained ANN model on the local computer were manually transferred into GEE API.

3.5. Accuracy Assessment

It is necessary to evaluate the classification result of the final ACI map to ensure the reliability of the proposed method. Therefore, the accuracy of the final classified ACI map was assessed both visually and statistically. For visual assessment, the available high-resolution satellite imagery within ArcGIS and Google Earth were employed. Regarding statistical accuracy assessment, the confusion matrix of the classification was calculated using independent test samples (15% of all samples) within GEE. Subsequently, various metrics such as OA, KC, PA, UA, Commission Error (CE), and Omission Error (OE) were derived from the confusion matrix.

4. Results

Figure 5 illustrates the object-based 2018 ACI map produced using the ANN algorithm and a combination of Sentinel-1 and -2 imagery within the GEE cloud computing platform. The resulting ACI map is visually clear and noise-free because of the incorporation of the SNIC algorithm for noise (salt-and-pepper) reduction, as mentioned in Section 3.3. Through the visual interpretation, it was observed that the proposed method was reasonably able to detect and classify the cropland classes. In this regard, three zoomed regions were explored and illustrated along with the corresponding high-resolution images available in ArcGIS to present the visual accuracy of the produced ACI map (see Figure 6).

The first zoomed region (Figure 6a) is located in the southern part of Alberta in which circular (center-pivot irrigation) and rectangular croplands exist. The proposed approach was able to successfully identify the type of croplands and removed the non-cropland areas. Moreover, the proposed method had high potential in delineating the boundary of croplands (circular patterns) over this region. The dominant cropland classes were Barley/Oats/Spring Wheat over this zoomed region. Likewise, the proposed method efficiently classified the croplands located in the southern part of Manitoba (Figure 6b). Potatoes and Canola/Rapeseed classes were assigned the largest cultivation area in this region. Furthermore, Figure 6c shows the effectiveness of the proposed GEE workflow to classify the croplands in the south-east of Ontario. This region contains several croplands dominated by Corn and Winter Wheat.

Table 3. Area of different cropland classes over the study area calculated from the produced ACI map in this study.

ID	Class	Area (Hectare)
1	Barley/Oats/Spring Wheat	34,773,308
2	Rye	3,859,182
3	Winter Wheat	4,607,312
4	Corn	2,007,945
5	Tobacco	124,157
6	Ginseng	129,507
7	Canola/Rapeseed	12,561,075
8	Soybeans	2,132,346
9	Peas	4,887,516
10	Beans	1,841,635
11	Tomatoes	118,822
12	Potatoes	4,053,364
13	Blueberry	4,932,649
14	Cranberry	1,028,732
15	Orchards	1,583,179
16	Vineyards	425,621
17	Buckwheat	2,091,076
-	Total	81,157,426

provides the area of each cropland type, which were calculated from the produced ACI map. In total, 81,157,426 hectares of the study area were identified and classified as croplands. Generally, three classes of Barley/Oats/Spring Wheat, Canola/Rapeseed, and Blueberry had the highest coverage over the study area with areas of about 34, 12, and 5 million hectares, respectively. The lowest coverages were for Tomatoes, Tobacco, and Ginseng classes with areas of approximately 118, 124, 129 thousand hectares, respectively.

Table 4. Confusion matrix of the produced Annual Space-Based Crop Inventory (ACI) map in terms of the number of pixels.

	Reference Data (ID)
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	Total
Classified Data (ID)
1	2230420	75114	69312	41847	324	427	12583	20015	31924	16198	261	13174	6758	1004	5279	1613	44910	2571163
2	154119	422211	65392	13932	2830	3382	1807	7130	10396	6057	222	5067	3769	266	5334	2229	2336	706479
3	179766	63932	672192	16167	168	537	958	9048	8590	5072	276	1527	3565	508	7783	3805	3651	977545
4	48904	11934	5394	428800	840	702	1131	36133	21025	14373	1340	8291	1409	201	3737	3146	18614	605974
5	111	3230	214	1290	138560	4295	26	3167	505	4206	578	1378	92	7	456	505	9	158629
6	198	3518	925	1021	5108	69627	7	2732	186	2973	80	901	211	54	769	427	1	88738
7	24834	3583	389	2955	90	4	405153	3560	7219	2350	452	9876	478	20	309	324	15018	476614
8	16752	5932	2758	37286	4866	2061	2966	417017	13700	82097	4027	38583	1217	342	3526	2155	9161	644446
9	55878	8840	3878	28620	544	335	8286	14923	558750	36085	116	9680	1952	50	769	162	6531	735399
10	37563	8436	5856	21258	4020	1279	4823	77461	42492	812496	261	23708	810	165	2367	703	10640	1054338
11	582	1240	249	5986	3189	1041	40	6341	772	1427	58848	1203	17	1	286	137	356	81715
12	26778	7867	902	9490	4449	811	8587	49022	10973	39950	1905	423930	1355	81	958	630	19407	607095
13	11913	1982	415	1069	56	146	343	651	686	811	2	302	264369	2301	5829	1614	1242	293731
14	896	602	532	12	0	36	0	7	301	132	0	21	4474	146766	239	558	31	154607
15	7169	2818	4520	2306	336	307	15	2936	581	2621	140	358	5521	545	222276	21522	247	274218
16	1806	1431	4346	1428	56	525	3	1592	86	605	47	43	4148	243	24791	138215	146	179511
17	65946	5049	2627	18185	0	16	10297	6909	6087	5852	72	10213	2509	171	300	297	384835	519365
Total	2863635	627719	839901	631652	165436	85531	457025	658644	714273	1033305	68627	548255	302654	152725	285008	178042	517135	10129567
PA (%)	77.89	67.26	80.03	67.89	83.75	81.41	88.65	63.31	78.23	78.63	85.75	77.32	87.35	96.10	77.99	77.63	74.42	OA = 76.95 %
UA (%)	86.75	59.76	68.76	70.76	87.35	78.46	85.01	64.71	75.98	77.06	72.02	69.83	90.00	94.93	81.06	77.00	74.10
CE (%)	22.11	32.74	19.97	32.11	16.25	18.59	11.35	36.69	21.77	21.37	14.25	22.68	12.65	3.90	22.01	22.37	25.58	KC = 0.738
OE (%)	13.25	40.24	31.24	29.24	12.65	21.54	14.99	35.29	24.02	22.94	27.98	30.17	10.00	5.07	18.94	23.00	25.90
OA: Overall Accuracy, KC: Kappa Coefficient, PA: Producer Accuracy, UA: User Accuracy, CE: Commission Error, OE: Omission Error 1: Barley/Oats/Spring Wheat, 2: Rye, 3: Winter Wheat, 4: Corn, 5: Tobacco, 6: Ginseng, 7: Canola/Rapeseed, 8: Soybeans, 9: Peas, 10: Beans, 11: Tomatoes, 12: Potatoes, 13: Blueberry, 14: Cranberry, 15: Orchards, 16: Vineyards, 17: Buckwheat

provides the confusion matrix of the classified ACI map using over 10 million independent test samples. The OA and KC were respectively 77% and 0.74, indicating the high potential of the proposed cloud computing method for the ACI production. The classes had medium to high PA values, varying between 63% and 96%. The classes of Cranberry and Canola/Rapeseed had the highest PAs, respectively. These classes have distinct spectral (NDVI + NDWI) and backscattering (VV + VH) characteristics compared to other cropland classes. Moreover, the ACI map achieved a satisfactory average UA of 77%. The Cranberry and Blueberry classes obtained the highest UAs, respectively, while the classes of Rye and Soybeans had the lowest UA values. It was also observed that the classes of Barley/Oats/Spring Wheat had the highest confusion with other cropland classes (6 other classes), causing 34% loss of PAs in total. Furthermore, the largest amount of reciprocal confusion was observed between the classes of Soybeans and Beans, resulting in a reduction in PAs by approximately 28% in total.

5. Discussion

In this section, the limitations of the study regarding the filed data, similarity of croplands, and discriminating croplands from non-cropland are discussed. Several suggestions, such as adding more satellite data, producing the Canada-wide cropland map, producing ACI map, and change analysis are also provided.

5.1. Field Data

Similar to most of the remote sensing studies, the quality and quantity of field data play an important role in the accuracy of the classification. This problem is exacerbated in this study area due to the small number of field data collected relative to the very large geographical area mapped. In fact, the number of field data for several classes were insufficient for training and validating the classification algorithm and, thus, were initially removed. Significant variation in the number of reference samples of different classes results in an imbalance references sets, affecting the classifier performance [47,48,49]. Thus, the 59 cropland classes used by AAFC were reduced to 17 cropland classes in this study. It should be noted that these 17 classes are the major croplands in Canada by seeded area. Another characteristic of the current study that made it more sensitive to the number of field data was utilizing ANN as the deep learning classification method. This was another reason that the classes with a small number of field samples were initially removed.

Another issue regarding field data that affected the accuracy of classification was the fact that field data were available over only seven Canadian provinces. Clearly, different provinces have various climates and different land covers/land uses. Thus, lacking field data over some provinces reduced the accuracy and reliability of the final ACI map over several regions.

5.2. Spectral Similarity of Croplands

In this study, 17 classes were distinguished, some of which contained similar spectral and physical characteristics. Therefore, they had similar pixel values in both optical and SAR imagery. Figure 7 demonstrates the box plots of the 17 cropland classes using different spectral and SAR features at different time ranges in 2018. As is clear, there is a high overlap between the values of various croplands, which made the discrimination of these classes more challenging. This fact was also well reflected in the confusion matrix (Table 4) and the final ACI map (Figure 5) where a high confusion between some of the classes was observed and several misclassifications occurred. Furthermore, it was observed that using a specific feature at different time ranges was beneficial in discriminating various croplands. For example, the mosaic SAR images with the VH polarization (i.e., Figure 7g–i) had different patterns of box plots, indicating an improvement in cropland discrimination using the VH features generated at different seasons. It is finally worth noting that the confusion between multiple cropland classes would have been much worse if the numbers of the classes were not decreased to 17 classes.

5.3. Discrimination of Cropland and Non-Cropland Areas

Generally, cropland classification has two main steps: (1) discriminating cropland from non-cropland areas, and (2) classifying various crop types within the cropland area. As discussed in Section 3, the AAFC’s ACI map along with the MODIS yearly land cover product were initially applied to distinguish croplands from non-croplands. However, the accuracies of these products, especially the MODIS product, in identifying the cropland areas are questionable at multiple regions. Thus, a considerable amount of error could have incurred at this step. It is therefore recommended to develop a specific algorithm to initially distinguish these two general categories (i.e., cropland and non-cropland) to produce a more accurate and reliable ACI map in the future.

5.4. Including More Satellite Data

One of the approaches to improve the classification accuracy is utilizing different remote sensing datasets with various characteristics to facilitate discriminating different croplands by generating more information [50]. For instance, AAFC has used the datasets collected by three satellites of Landsat-8, Sentinel-2, and RADARSAT-2 for producing the 2018 ACI map with the spatial resolution of 30 m (Figure 1). However, in this study, the ACI map was produced using Sentinel-1 and Sentinel-2 datsets with 10 m spatial resolution. There are several recently launched and former satellites which could be included in the ACI production to improve the accuracy. For example, Canada has launched RADARSAT Constellation Mission (RCM) on June 12, 2019. This satellite provides circular polarization datasets with high spatial resolution and, thus, can considerably improve the accuracy of future ACI maps.

5.5. Canada-Wide Cropland Inventory Map

As discussed, the area specified by AAFC which is approximately 28% of the country (Figure 2) was considered in this study to produce the ACI map. Although this area contains most croplands in Canada, there are additional croplands outside of this boundary, such as Yukon. Consequently, the future studies should aim to produce ACI map for the entire country or extend the boundaries of classification until all the cropland areas are included.

5.6. ACI Maps at Diffeent Years and Change Analysis

For managing the amount of food being produced and distributed in a country, it is important to track crop production in different years. Moreover, for protecting agricultural areas and other land covers, like wetlands, it is imperial to detect what land covers are changed to agricultural fields or vice versa. Therefore, ACI maps at different years can be produced to answer these questions. The proposed GEE cloud computing method can be effectively applied to produce ACI maps and assess the changes in a more cost-efficient approach compared to the methods that are currently utilized by AAFC.

6. Conclusions

Although a vast portion of Canada is covered by different types of croplands, advanced machine learning and big geo data processing methods have not been investigated for producing ACI maps. Therefore, a cloud computing method was presented in this study to produce object-based ACI maps using a combination of multi-date Sentinel-1 and Sentinel-2 images acquired in 2018. The method was implemented within GEE, and an ANN was applied to delineate 17 cropland classes over 10 provinces of Canada. The overall classification accuracy was 77%, and the average PA and UA for cropland classes were 79% and 77%, respectively. This level of accuracy was reasonable for the first effort to use GEE for ACI production. The proposed method was highly efficient in terms of cost and computation. In fact, using GEE cloud platform and open-access satellite images (i.e., Sentiel-1/-2) instead of local computers and costly images (e.g., RADARSAT-2) used by AAFC made the proposed method more efficient in terms of time and cost and would facilitate automated production of the ACI maps in future. It is believed that the proposed approach could be effectively applied to create Canada-wide cropland inventories and facilitate change detection and monitoring efforts. Future studies should consider the proposed method along with the following factors to produce more accurate and reliable ACI maps: (1) More field data should be utilized to be able to discriminate more classes and improve the class accuracies; (2) an efficient algorithm should be developed to discriminate cropland from non-cropland regions. This will improve the accuracy and reliability of the map and will also facilitate the production of Canada-wide ACI maps; (3) ACI maps should be produced for different years and a change detection algorithm should be developed to assess the amount of changes; and (4) additional satellite data, such as RCM and X-band SAR images, should be incorporated into the classification.