Hierarchical Classification of Soybean in the Brazilian Savanna Based on Harmonized Landsat Sentinel Data

Abstract

The Brazilian Savanna presents a complex agricultural dynamic and cloud cover issues; therefore, there is a need for new strategies for more detailed agricultural monitoring. Using a hierarchical classification system, we explored the Harmonized Landsat Sentinel-2 (HLS) dataset to detect soybean in western Bahia, Brazil. Multispectral bands (MS) and vegetation indices (VIs) from October 2021 to March 2022 were used as variables to feed Random Forest models, and the performances of the complete HLS time-series, HLSS30 (harmonized Sentinel), HLSL30 (harmonized Landsat), and Landsat 8 OLI (L8) were compared. At Level 1 (agricultural areas × native vegetation), HLS, HLSS30, and L8 produced identical models using MS + VIs, with 0.959 overall accuracies (OA) and Kappa of 0.917. At Level 2 (annual crops × perennial crops × pasturelands), HLS and L8 achieved an OA of 0.935 and Kappa > 0.89 using only VIs. At Level 3 (soybean × other annual crops), the HLS MS + VIs model achieved the best performance, with OA of 0.913 and Kappa of 0.808. Our results demonstrated the potential of the new HLS dataset for medium-resolution mapping initiatives at the crop level, which can impact decision-making processes involving large-scale soybean production and agricultural sustainability.

1. Introduction

Brazil is one of the world’s most important food producers and exporters, contributing significantly to meet the global growing demand [1]. For example, the national harvested area of soybeans (Glycine max L.) in 2020 was more than 37 million ha, higher than that from the United States, with approximately 33 million ha [2]. In the 2029/2030 crop growing season, we expect Brazil to produce 318 million tons of grains in addition to soybeans, especially maize (Zea mays L.), wheat (Triticum aestivum L.), beans (Phaseolus vulgaris L.), rice (Oryza sativa L.), coffee (Coffea sp.), cotton (Gossypium sp.), oatmeal (Avena sativa), sorghum (Sorghum bicolor L.), peanut (Arachis hypogaea L.), sunflower (Helianthus annuus L.), and canola (Brassica sp.), an increase of about 27% in relation to the 2019/2020 crop season [3].

The Brazilian tropical savanna region (Cerrado biome), occupying more than 200 million ha in the central part of the country, is the major producer of food and energy in the country [4]. It contributes to 52% of the national soybean production, 54% of maize, 96% of cotton, and 51% of sugarcane (Saccharum officinarum L.) [5]. Most of the grain production in the Cerrado is found in extensive flat terrains (plateaus) [6]. These are the cases, for example, of the western region of the Bahia State (Western Bahia), in the Cerrado/semi-arid transition [7,8]; of the municipalities of Lucas do Rio Verde, Sinop, and Sorriso, in the Cerrado-Amazon ecotone of the Mato Grosso State [9]; and of the MATOPIBA region, an acronym relating the Cerrado portion of four states (Maranhão, Tocantins, Piauí, and Bahia) located in the northern part of the biome [10].

Monitoring the rainfed agricultural production, which accounts for approximately 75% of global food staple [11], using optical satellite remote sensing data is difficult because of the persistent cloud cover conditions during the wet season [12,13], the spectral similarity between some crops (for example, soybean and bean), and varying planting dates. A multisensor approach for increasing the periodicity of satellite overpasses and for taking advantage of complementary information from multiple sources of satellite data can optimize the capability of rainfed crop-type identification [14]. Perhaps one of the most well-known satellite fusion methods is the Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM) that was developed by Gao et al. [15]. STARFM products are made by fusing Landsat and Moderate Resolution Imaging Spectroradiometer (MODIS) data sets and have been widely used for crop monitoring [16,17,18]. According to Zhu et al. [19], other highly adopted fusion models are the Enhanced Spatial and Temporal Adaptive Reflectance Fusion Model (ESTARFM) [20] and the Spatial Temporal Adaptive Algorithm for mapping Reflectance Change (STAARCH) [21], also involving fusion of Landsat and MODIS data sets.

The drawback of these models is the need for the availability of cloud-free Landsat and MODIS images on matching dates, which is not always achievable in practice [14]. Another issue is the necessity of careful image pre-processing to harmonize sets of images acquired at different spatial, spectral, and radiometric resolutions, view angles, and signal-to-noise ratios. Recently, NASA proposed the Harmonized Landsat and Sentinel-2 (HLS) initiative to produce a virtual constellation of analysis-ready, two to four day’s temporal frequency, surface reflectance images acquired by the Landsat Operational Land Imager (OLI) and Sentinel-2 Multispectral Instrument (MSI) sensors [22]. This initiative relies on a set of algorithms for atmospheric correction, cloud and cloud shadow masking, spatial co-registration, common gridding, bidirectional reflectance distribution function normalization, and spectral bandpass adjustments [23,24,25]. Previous studies have demonstrated the potential of the HLS time series for improving crop type mapping [26], crop intensity [27] and biomass [28], crop phenological metrics estimation [29], irrigated area detection [30], livestock intensification indication [31,32], and surface phenology characterization [33,34].

Since 2020, the entire Brazilian territory has been covered by the HLS v2.0 collection. Despite its potential for providing valuable capability to monitor the spatio-temporal dynamics of tropical agriculture, to our best knowledge, detailed studies evaluating the potential and accuracy of HLS data sets to identify croplands in the Cerrado biome are still in initial stages. Within this context, our study aims to analyze the potential of HLS images to identify soybean plantations from other representative Land-Use Land-Cover (LULC) classes in the Cerrado biome using a hierarchical classification system. Ultimately, attempting to separate soybeans, the most important agricultural commodity in Brazil, from other annual crops found in the Cerrado is the main motivation of this study.

2. Materials and Methods

2.1. Study Area

The selected study area corresponds to the region located in Western Bahia, Brazil (south latitude: 11°45′; west longitude: 45°50′), surrounded by the BR-242, BA-459, and BA-460 highways and covering an area of 366 thousand ha (Figure 1). The area partially covers the municipalities of Barreiras, Luís Eduardo Magalhães, and Riachão das Neves and is known as Soybean Ring because of the intensive production of soybean in this region [35].

The region is characterized by the tropical continental climate, i.e., Aw in the Köppen climate classification system [36], with strong climate seasonality: six months of dry season (April to September) and six months of wet season (October to March). According to the rainfall data provided by an automatic station located in the municipality of Luís Eduardo Magalhães and recorded by the National Institute of Meteorology [37] (station code: A404; latitude = −12.15°; longitude = −45.83°; elevation = 760 m; operation date = 17 April 2002), the average annual rainfall is 885 mm. This rainfall gradient increases from east to west. According to the data gathered by the Shuttle Radar Topography Mission (SRTM), the topography of the study area is flat, with an average elevation of 788 m, while highly weathered Oxisols, with low natural fertility and high Al toxicity, are the dominant soil type [38].

Since 1985, this region has faced a rapid process of conversion of native vegetation into large-scale mechanized agriculture for exportation [7]. The main rainfed crop found in the study area is soybean, followed by cotton and maize (

Table 1. Estimated crops harvested area (ha) from the municipalities of Barreiras, Luís Eduardo Magalhães, and Riachão das Neves in the 2020–2021 crop growing season.

Municipality	Cropping Pattern	Crop Type	Harvested Area (ha)
Barreiras	Annual	Soybean	195,500
		Maize	18,598
		Cotton	23,855
		Others (beans, sorghum, sugarcane)	16,435
	Perennial	Coffee, banana (Musa spp.), orange (Citrus sinensis L.), papaya (Carica papaya L.)	6364
Luís Eduardo Magalhães	Annual	Soybean	162,200
		Maize	14,600
		Cotton	16,513
		Others (beans, sorghum, wheat)	22,268
	Perennial	Coffee, banana, orange, papaya	1451
Riachão das Neves	Annual	Soybean	116,500
		Maize	12,200
		Cotton	32,895
		Others (beans, sorghum, cassava)	7973
	Perennial	Coffee, banana, orange, papaya	1175

Source: Brazilian Institute of Geography and Statistics (IBGE) [35].

). Coffee and bean are the most important irrigated crops by center-pivot irrigation systems. In the Landsat 8 OLI image from October 2021, we counted 349 center-pivots in the study area.

2.2. Methods

Our study can be characterized as an applied research with a qualitative–quantitative approach, using technical procedures based on a study case, and Figure 2 summarizes the proposed methodology. The Landsat 8 (L8) and the Harmonized Landsat Sentinel-2 (HLS) images acquired during the 2021–2022 crop growing season were classified by the non-parametric Random Forest (RF) classifier using training samples collected during the field campaign conducted in December 2021. The main steps of the proposed method are detailed in the following subsections.

2.3. Data Sets

In this study, we selected a set of seven Landsat 8 OLI images converted into surface reflectance (path/row = 220/68) from 1 October 2021 to 31 March 2022, which corresponds to the crop growing season in the study area. The images were obtained from the United States Geological Survey (USGS) Earth Explorer platform (https://earthexplorer.usgs.gov/ accessed on 28 April 2022). We preprocessed these images by applying a multiplicative scale factor (0.0000275) and an additive offset (−0.2) [39]. The quality assurance (QA) band was used to mask cloud, cloud shadows, water, and cirrus by using 21,824 as the integer value.

We also selected the Sentinel-2 MSI harmonized surface reflectance data resampled to 30 m over the Sentinel-2 tilling system (HLSS30; 28 images) as well as the Landsat 8 OLI harmonized surface reflectance data resampled to 30 m over the Sentinel-2 tiling system (HLSL30; 6 images), version 2.0 from the same period (Figure 3). The HLSS30 and HLSL30 images are available with geometric, radiometric, and atmospheric corrections [22]. To cover the entire study area, we selected the images from four tiles: 23LLG, 23LLH, 23LMG, and 23LMH. Whenever the Landsat 8 and Sentinel-2 overpasses were coincident, we opted to select the HLSS30 data. All available images were downloaded from the EarthData website (https://search.earthdata.nasa.gov/ accessed on 1 April 2022).

Both Landsat 8 OLI and HLS (HLSS30 and HLSL30) images were preprocessed by the following steps: (1) mosaicking the images per band and per date; (2) reprojection of the mosaics for the Southern Hemisphere (Universal Transverse of Mercator—UTM projection system and WGS84 datum); and (3) cropping the mosaics to the boundary of the study area. We also used the HLS F-mask to remove pixels contaminated by clouds, cloud shadows, water, and high aerosol level, using the digital numbers 64 and 128 (

Table 2. Integer values selected from the F-mask, representing pixels without clouds, cloud shadows, water, and high aerosol level. The value 0 at the indicated bit means absence, while 1 indicates presence. Bit 7 and Bit 6 are the aerosol level (01 represents low level and 10 represents medium level); Bit 5 is water; Bit 4 is snow or ice; Bit 3 is cloud shadow; Bit 2 is adjacent to cloud shadow; Bit 1 is cloud; and Bit 0 is cirrus.

Integer Value	Bit 7	Bit 6	Bit 5	Bit 4	Bit 3	Bit 2	Bit 1	Bit 0
64	0	1	0	0	0	0	0	0
128	1	0	0	0	0	0	0	0

), following the indications of the HLS Users’ Guide v2.0 [40]. For further image processing, as detailed in the next subsection, we selected the following spectral bands common to both Landsat 8 OLI and Sentinel-2 MSI satellites: blue; green; red; near-infrared (NIR); shortwave infrared 1 (SWIR 1); and shortwave infrared 2 (SWIR 2).

Spectral Vegetation Indices (VIs)

The Landsat 8 OLI, HLSS30, and HLSL30 images were converted into the following spectral vegetation indices (VIs): Normalized Difference Vegetation Index (NDVI) [41]; Green Normalized Difference Vegetation Index (GNDVI) [42]; Normalized Water Difference Index (NDWI) [43]; and Soil-Adjusted Vegetation Index (SAVI) [44] (

Table 3. Spectral vegetation indices (VIs) used in the study.

VIs	Name	Equation	Reference
NDVI	Normalized Difference Vegetation Index	$\frac{R\lambda nir-R\lambda red}{R\lambda nir+R\lambda red}$	[41]
GNDVI	Normalized Difference NIR/Green NDVI	$\frac{R\lambda nir-R\lambda green}{R\lambda nir+R\lambda green}$	[42]
NDWI	Normalized Difference Water Index	$\frac{R\lambda nir-R\lambda swir1}{R\lambda nir+R\lambda swir1}$	[43]
SAVI	Soil-Adjusted Vegetation Index *	$\frac{\left(R\lambda nir-R\lambda red\right)}{R\lambda nir+R\lambda red+L}\ast \left(1+L\right)$	[44]

* Factor for soil brightness correction (L) = 0.5.

). The NDVI is the most traditional vegetation index and presents a high correlation with photosynthetic-activity-related parameters such as the leaf area index and leaf chlorophyll [41]. However, some plant phenological phases related to changes in leaf pigments, water content, and crop residues may not be entirely analyzed using only NDVI, especially in regions with complex crop cultivation patterns [45]. GNDVI provides more sensitivity to chlorophyll-a content than NDVI by replacing the red band with the green band, which is more sensitive to chlorophyll [42], improving the potential to detect stressed and senescent vegetation and estimate green crops [42]. NDWI accentuates differences in the leaf water contents of several vegetation types [43]. SAVI minimizes soil background effects in the NDVI by using a correction factor in the NDVI formula [44]. The combined use of these VIs has been valuable for crop mapping purposes [8].

2.4. Hierarchical Classification

The image classification was performed considering three hierarchical levels of LULC classes present in this study. The first level is composed of two groups of LULC classes: the agricultural areas and the natural vegetation. The water bodies and urban areas found in the study area were masked. The water bodies were extracted from the LULC map produced by the MapBiomas project, a multi-institutional initiative for reconstructing historical LULC maps from the Brazilian biomes since 1987, based on the cloud computing in the GEE platform [46]. The urban areas were obtained from IBGE [47]. Since the urban areas corresponded to the year 2015, we manually complemented the expanded areas larger than 1 ha. In the second level, the agricultural areas were divided into annual crops, perennial crops, and cultivated pasturelands. At this level, due to the low occurrence of coffee plantations and silviculture, both were combined into the class of perennial crops because of their similar spectral response. Finally, in the third level, the annual crops were split into soybean and other annual crops. The image classification was also generated considering other datasets: only multispectral bands (MS); only spectral vegetation indices (VI); and a combination of multispectral and VIs subsets (MS + VIs).

To perform the image classification, a set of 192 sampling points was surveyed during a field campaign conducted on 29–30 November 2021, with the support of the AgroTag software application [48] (Figure 4). More specifically, we surveyed 86 points of harvested crops in 2021–2022, 48 of soybean, 38 of other annual crops (maize, cotton, and bean), and 20 of pasturelands and Cerrados’ natural vegetation. AgroTag is a software developed by the Embrapa Environment located in Campinas, São Paulo State to gather and share field information. The software enables the acquisition and storage of Global Positioning System (GPS) coordinates, field photos, and metadata of different sampling points. Seventy percent of ground truth data were used for training the classifier while 30% were used for validation.

The ground truth data were complemented by samples extracted from a high-resolution RGB color composite of 8-m spatial resolution red, green, and blue bands of the China–Brazil Earth Resources Satellite (CBERS-4A) Wide Scan Multispectral and Panchromatic Camera (WPM) fused with the 2–meter spatial resolution panchromatic band acquired on 27 October 2021. These data were used only for the Levels 1 and 2 classification procedures since the remote sensing-based reference data are not quite feasible for complex targets [49]. At Level 1, we collected 167 samples for each class, while at Level 2, we gathered 30 samples for pasturelands, 30 samples for perennial crops, and 50 samples for annual crops.

Image Classification and Parameterization

In this study, we used the Random Forest (RF) classifier [50], which is one of the most used and successful non-parametric algorithms for image classification. It operates by combining multiple random decision trees that group decisions by average, where the most voted class among all the trees in the forest is considered the final response. It can deal with large volumes of data and imbalanced classes, supplying a ranking of the importance of variables for constructed models [51,52]. We have considered other machine learning methods at the initial stage of the study. However, some tests have shown no significant difference. Moreover, RF is a well-established method known to handle data with high dimensionality, robustness against overfitting, and high predictive capacity [53], characteristics that can be considered relevant for our purpose in this study. This situation endorsed the use of RF.

To improve the performance of the RF classification, we employed 10-fold cross-validation to adjust a set of parameters, the number of variables randomly sampled as candidates at each split (mTry), the maximum number of terminal nodes trees (maxnode), and the number of trees to be grown (nTree) for each model, using the caret and randomForest packages available in the R environment (

Table 4. Results of the parametrization considering the number of variables at each split (mTry), the number of trees in the forest (nTree), and the maximum number of terminal nodes (maxnode) for each model, after 10-fold cross-validation.

Level	Parameter	HLS			L8			HLSS30			HLSL30
		MS	VIs	MS + VIs	MS	VIs	MS + VIs	MS	VIs	MS + VIs	MS	VIs	MS + VIs
1	mTry	10	6	14	18	16	19	13	11	12	6	5	6
	maxnode	9	14	8	11	9	7	12	9	9	14	8	6
	nTree	50	250	50	150	50	500	50	100	300	50	150	300
2	mTry	13	6	14	7	4	13	15	15	11	7	14	5
	maxnode	10	12	6	14	12	13	12	9	9	12	8	12
	nTree	50	200	500	250	350	50	100	100	150	450	250	300
3	mTry	9	6	18	9	5	10	14	11	17	8	5	5
	maxnode	6	10	8	5	5	7	6	9	10	13	10	13
	nTree	100	50	400	100	50	50	150	500	200	50	100	200

). The final maps were generated from the best dataset at each level and filtered with the modal function in 3 × 3 and 5 × 5 moving windows, depending on the level, using tools of the raster package.

2.5. Accuracy Assessment and Statistical Analysis

We assessed the accuracies of the RF classifications through the overall accuracy (OA), Kappa index, producers’ accuracy (PA), and users’ accuracy (UA). The performances of each dataset (HLS, HLSS30, HLSL30, and Landsat 8 OLI) and each subset (MS, VIs, and MS + VIs) were evaluated to determine the best model. Therefore, along with the exploratory statistical analysis of our results, we also employed the two-sample Student’s t-test, using the Basic Statistics and Data Analysis (BDSA) package in R [54], to assess if the HLS outcomes (OA and Kappa) were statistically different from the HLSS30, HLSL30, and L8 in a confidence interval of 0.95. This was done after we confirmed the normal distribution of the models’ results for each dataset at each level using the Shapiro-Wilk test.

One default output from RF models is the variable importance ranking, where the predictive power of each variable is measured throughout a series of models’ reruns with selected predictors, being a vital tool for feature selection, dimensionality reduction, and understanding variable interaction [50]. Therefore, we considered the metric of Mean Decrease in Accuracy (MDA), which measures the variable importance using out-of-bag samples and recording the variation in prediction error at each variable permutation, to build ranks with the top 10 most important variables for each dataset (HLS, HLSS30, HLSL30, and L8) at each level.

3. Results

3.1. Influence of Cloud Cover on Satellite Data Availability

Table 5. Cloud cover and Landsat 8 OLI data loss caused by using the Quality Assessment (QA) band in the study area (Path/Row: 220/68).

Overpass	% Cloud Cover over the Entire Tile	% Data Loss over the Study Area
1 November 2021	69	97
17 November 2021	29	29
20 January 2022	6	35
5 February 2022	15	17
21 February 2022	51	93
9 March 2022	4	12
25 March 2022	28	45

shows the influence of cloud cover on L8 data availability over the study area during the crop growing season. Five L8 images from a total of 11 overpasses presented 100% of cloud coverage, while another seven images presented cloud coverage ranging from 4% to 70%. Considering only the area occupied by the Soybean Ring, the percent of data loss due to the cloud cover interference varied from 12% to 97%, with an average of 47%.

Table 6. Total of overpasses, average cloud cover, and percent data loss in the Harmonized Landsat Sentinel-2 images caused using the F-mask in the study area.

Month	Total of Overpasses	% Cloud Cover *	Data Loss (%)
October	4	36	69
November	6	68	61
December	4	77	87
January	7	67	87
February	7	57	71
March	6	44	64

* Average cloud cover among the four tiles (T23LLG, T23LLH, T23LMG, and T23LMH) necessary to cover the study area.

shows the influence of cloud cover on HLSS30 and HLSL30 data availability over the study area. The number of available scenes increased from 7 (L8) to 34 (28 HLSS30 overpasses and 6 HLSL30 overpasses). The data loss due to bad-quality pixels varied from 61% in November to 87% in December and January, with an average value of 74% during the crop growing season. Only 12 scenes presented less than 60% of data loss, while 17 scenes presented cloud coverage and cloud shadow higher than 90%. Some expected images were unavailable for download, possibly because of the high percentage of cloud coverage, which can affect the pre-processing steps. We noted that some of these missing images in the EarthData platform were available in the Landsat 8 L1GS database in the Earth Explorer platform, indicating that these images presented insufficient locational information for terrain correction.

3.2. Classification Results

Figure 5 depicts the classification results obtained via RF classifier based on both spectral bands and VIs of HLS, HLSS30, HLSL30, and L8 images for three hierarchical levels. The HLS Level 1 map shows that most of the study area is occupied by agricultural areas, while the remaining native vegetation mostly corresponds to riparian forests. The proportion of agricultural areas identified by our best models were 73% (HLS and HLSS30), 74% (HLSL30), and 75% (L8) of the Soybean Ring, respectively. The annual crops were also similarly assessed as being 86% (HLS), 83% (HLSS30), 84% (HLSL30), and 88% (L8) of the agricultural areas (approximately 60–65% of the entire study area). The L8 and HLSL30 final maps of Level 2 estimated 200% and 138%, respectively, more perennial crops (coffee under irrigation system and silviculture) than the HLS dataset, which means a difference of 80 and 55 km². Indeed, some of the coffee areas were missed by HLS at Level 1, classified as natural vegetation. HLS found 102 and 75 km² more pastures than the HLSL30 and L8 maps, respectively. The datasets differ in Level 3 classifications, as it will be approached in the next session, reflected in the soybean estimations. While the L8 and HLSL30 best models calculated that 68% and 65% of the annual crops at the first growing season were soybean, respectively, HLS estimated a rate of 77% (45% and 49% of the total area), with a difference up to 145 km² (Figure 6).

3.3. Accuracy Assessment and Statistical Analysis

In general, the models’ performances decrease as the hierarchical levels increase. At Level 1, except for the HLSL30, all models were able to separate agricultural areas from natural vegetation with accuracies over 0.90. The average OA/Kappa obtained by each dataset was 0.938/0.876 (HLS), 0.945/0.889 (HLSS30), 0.866/0.730 (HLSL30), and 0.948/0.897 (L8). The Student´s t-test showed that HLS is a better dataset than HLSL30 with statistical significance. The Cerrado is marked mainly by its heterogeneity, where different environmental conditions build a complex landscape, with several phytophysiognomies that, at the same time, reveal the high biodiversity of this biome and make mapping initiatives more challenging [6]. However, within the Soybean Ring, most of the remaining native vegetation is composed of gallery forests, which have a constant response over time, given the natural composition of their closed tree canopies and the absence of harvests or clear cuts during the harvest period [8].

The performances were also similar at Level 2, with average OA/Kappa of 0.903/0.837 (HLS), 0.860/0.778 (HLSS30), 0.763/0.604 (HLSL30), and 0.892/0.823 (L8). The t-test pointed out that the HLS models were only statistically superior from the HLSL30. At level 3, HLS had a mean OA/Kappa of 0.882/0.744, in opposition to a mean OA/Kappa of 0.840/0.633 (HLSS30), 0.725/0.441 (HLSL30), and 0.754/0.489 (L8). According to the Student´s t-test, the HLS has a statistically significant superiority of OA and Kappa over all datasets. The table containing the results from the hypothesis test can be seen in the Supplementary Material. Neither the HLSL30 nor the L8 data produced models with statistical significance in separating soybean from other crops at this level.

Regarding the performances of the multispectral bands (MS) and spectral vegetation indices (VIs), individually or combined, at Level 1, the combination (MS + VIs) produced higher performances in comparison with the MS and VIs subsets separately, except for the HLSL30 dataset. The combination of bands and indices improved the OA and Kappa of HLS up to 10% and 9.8%, respectively. At Level 2, the L8 VIs subset over-performed the others of this sensor, while the combined subset also generated the best HLS model. The MS subsets of L8 and HLS presented the lower scores, while the VIs and MS + VIs improved the results for these datasets. For the HLSS30 dataset, bands alone produced a model nearly identical to MS + VIs, but only with 3% higher Kappa. At Level 3, the VIs models had the weakest performances for HLS, HLSS30, and L8, while the best overall model was obtained with the combination of bands and indices of HLS data. The overall accuracies and Kappa indices of all models are presented in

Table 7. Overall Accuracy (OA) and Kappa Index for Level 1, 2, and 3 classifications (L8, HLS, HLSS30, and HLSL30) in datasets: multispectral bands (MS), spectral vegetation indices (VIs), and combination of MS and VI (MS + VIs) bands.

Sensor/Data	Classifications	Datasets	OA	Kappa
Landsat-8 Operational Land Imager OLI (L8)	Level 1	L8 MS	0.938 ***	0.877
		L8 VIs	0.948 ***	0.896
		L8 MS + VIs	0.959 ***	0.918
	Level 2	L8 MS	0.839 ***	0.734
		L8 VIs	0.935 ***	0.895
		L8 MS + VIs	0.903 ***	0.840
	Level 3	L8 MS	0.782 ns	0.559
		L8 VIs	0.696 ns	0.349
		L8 MS + VIs	0.783 ns	0.559
Harmonized Landsat Sentinel-2 (HLS)	Level 1	HLS MS	0.917 ***	0.835
		HLS VIs	0.938 ***	0.876
		HLS MS + VIs	0.959 ***	0.917
	Level 2	HLS MS	0.839 ***	0.726
		HLS VIs	0.935 ***	0.892
		HLS MS + VIs	0.935 ***	0.892
	Level 3	HLS MS	0.867 **	0.721
		HLS VIs	0.867 **	0.704
		HLS MS + VIs	0.913 **	0.808
Sentinel-2 Multi-spectral Instrument Surface Reflectance (HLSS30)	Level 1	HLSS30 MS	0.928 ***	0.855
		HLSS30 VIs	0.948 ***	0.897
		HLSS30 MS + VIs	0.959 ***	0.917
	Level 2	HLSS30 MS	0.871 ***	0.788
		HLSS30 VIs	0.839 ***	0.746
		S30 MS + VIs	0.871 ***	0.800
	Level 3	HLSS30 MS	0.869 **	0.721
		S30 VIs	0.783 ns	0.475
		S30 MS + VIs	0.869 ***	0.704
Landsat-8 Land Imager Surface Reflectance and TOA Brightness (HLSL30)	Level 1	HLSL30 MS	0.897 ***	0.794
		HLSL30 VIs	0.845 ***	0.689
		HLSL30 MS + VIs	0.856 ***	0.708
	Level 2	HLSL30 MS	0.742 *	0.568
		HLSL30 VIs	0.774 **	0.622
		HLSL30 MS + VIs	0.774 **	0.622
	Level 3	HLSL30 MS	0.696 ns	0.414
		HLSL30 VIs	0.739 ns	0.425
		HLSL30 MS + VIs	0.739 ns	0.485

Level of significance: p > 0.05 *; >0.005 **; >0.0005 ***; non-significant ^ns.

, and the confusion matrices are shown in Supplementary Material along with the error matrices and classes accuracies.

3.4. Variable Importance

Based on the Mean Decrease Accuracy (MDA) scores for the HLS datasets considered in the RF classification, VIs showed to be more important than spectral bands for Levels 1 and 2 of the hierarchical classifications in all datasets. However, for the Level 3 classification, spectral bands overperformed VIs. Regarding the time of the crop growing season, the beginning of the season showed to be more relevant for the Levels 1 and 2 classifying procedure, while, to discriminate soybeans from other annual crops, the end of the growing season is more prominent. For the L8 and HLSL30 models, there was an evident relevance of the clearer overpass, i.e., 9 March 2022, when cloud cover was lower than 5% and only 12% of the data were lost. Figure 7 displays the top 10 more important variables for the HLS datasets in the three levels of classification, while those from HLSS30, HLSL30, and L8 are shown in the Supplementary Material.

Considering which HLS variables were poor predictors, we observed that, in Level 1, SWIR 1(from 15 December 2021, 5 February and 20 March 2022) and SWIR 2 (from 20 November 2022, 20 and 25 March 2022) were the poorest spectral bands, and NDVI from 10 March was the poorest VI. In Level 2, SWIR 1 from 18 February, Blue from 17 November 2021, and Red from 30 November 2021 were the poorest predictors among the spectral bands. Among VIs, GNDVI from 10 Mach 2022, NDWI from 9 January 2022, and NDVI from 18 February 2022 were the poorest. In Level 3, Red, SWIR 1, and NIR from 26 October 2021 and SWIR1 from 15 March 2022 were the poorest spectral band predictors. Among VIs, GNDVI (from 15 March and 14 January 2022) and NDVI (from 15 March and 3 February 2022) were the poorest. Specifically, at the end of the rainy season, we observed the prevalence of the observations from 10 and 15 March 2022 on those affecting the capability of different VIs and spectral bands. The average data loss among these dates was 73%, with a rate of flagged pixels between 31% and 99%, which reaffirms that having clearer observations along the crop season was determinant to agricultural and soybean mapping.

4. Discussion

4.1. Cloud Cover Interference on Satellite Image Acquisition

The rainfed crop plantation in Western Bahia includes soybean, maize, cotton, and bean. It starts in early October until late December, depending on crop type and crop variety [55]. The harvesting time usually starts from February until May. Except for the peak of the dry season (July to August), the Western Bahia is a region severely affected by persistent cloud coverage, particularly between December and February, when it usually surpasses 60–70% [13]. This study showed that we have seven L8 images with less than 70% of cloud coverage (only two with less than 10%) for the 2021–2022 crop season monitoring in the study area. This number increases to 21 when HLS images are considered (less than 70%). Therefore, regardless of the increased potential of crop production monitoring by HLS data sets related to their better spatial and spectral resolutions in relation to the L8 data sets, increasing the number of available images throughout the crop growing season is of great advantage.

As previously mentioned, we used the F-mask to remove poor-quality pixels, and we observed an average data loss of 74%. Such loss was, on average, 15% higher than the cloud cover at the acquisition moment. Such issue was also observed in South Dakota, U.S., where 38 HLS overpasses presented more than 90% of flagged pixels in less than one year [26]. In previous versions, HLS quality bands had known issues regarding bright targets [32] as well as commission and omission errors [56]. Some alternative approaches have been evaluated, such as the application of RF or decision tree classifiers over the QA bands and spectral thresholds using the NDWI [33]. Taking regional tuning and mask refinement into consideration can be a strategy to increase the number of valid HLS observations over cloud-persistent regions and therefore improve mapping accuracies; however, cloud masking still is an open matter to be continually addressed in future studies with HLS data [26].

This high cloud cover interference is a trade-off that has been part of the context of rainfed crop type identification and monitoring over the Brazilian agricultural frontiers using optical remote sensing data, especially in the Cerrado biome [57]. The development of initiatives for expanding satellite revisit frequency such as the HLS has the potential to overcome the limitation of cloud-free satellite images [22]. In our study, HLS higher data frequency proved to be fundamental for the detection of soybean production compared to HLSS30, HLSL30, and L8 datasets, providing a more detailed and precise mapping of Brazil’s most important crop. They were also efficient for mapping LULC over the rainy period, even with a high rate of data loss.

Therefore, big LULC mapping projects in Brazil such as MapBiomas and TerraClass based on Landsat 30 m and MODIS 250 m data can benefit from HLS, since the temporal resolution is still a limitation, even outside of rainy periods. However, part of the trade-off is the increased amount of data; therefore, NASA HLS products should become available through cloud processing platforms such as the GEE so that regional, national, and global mapping initiatives worldwide can rely on these virtual constellations. Some authors have been developing routines to harmonize Landsat and Sentinel data using resources of the GEE [58]. However, although such processes can improve harmonization coefficients at the regional level, the amount of work and expertise in programming, along with computational limitations, mainly in developing countries, can be limiting factors for many users.

4.2. Impact of Parametrization on the RF Classification Performance

Parameter setting on classification with machine learning algorithms has a significant impact on models’ performances [51], and though it is widely used for agricultural monitoring and mapping, the parameterization usually is less explored [52]. In this study, after a series of tests, we chose to tune three RF parameters: the number of variables at each split (mTry) from 5 to 20, the number of trees in the forest (nTree) from 50 to 500, and the maximum number of terminal nodes (maxnode) from 5 to 15. To assess the impact of parametrization on models’ performances, we used the same seeds to rebuild the best models using the worst values for the three parameters tuned. At Level 1, parameterization increased HLS MS + VIs OA by 2.2% and Kappa by 4.6%, HLSS30 MS + VIs OA by 4.5% and Kappa by 9.7%, but displayed no difference for the L8 MS + VIs model (+0.02 and 0.03% of OA and Kappa). At Level 2, the combination of the worst parameters did not impact HLS and HLSS30 MS + VIs models (<1% variation) but led to an increase of 10.3% of OA and 18% of Kappa in the L8 VIs model. Level 3 was impacted by parameter setting, presenting an increase of 4.8% of OA and 12.9% of Kappa in the HLS MS + VIs model and +5.6% of OA and +13.2% of Kappa in the L8 MS + VIs model. Therefore, although the impact of parameterization in our study was discrete, it became more significant as the level increased, and led to important improvements regarding mainly Kappa coefficients.

4.3. LULC Mapping Challenges and Variables Importance

In general, one of the main challenges to accurately mapping the main crops on regional and national scales in Brazil is the lack of ground truth samples, as was the case in this study. Even in a landscape mainly dominated by soybean in the first growing season (possibly followed by maize and other annual crops), the number of ground samples was smaller than intended due to the bad weather conditions, leading to poor road conditions, along with some inaccessible farms. To overcome these limitations, we adopted a hybrid sampling strategy, using high-resolution remote sensing data for the less complex levels 1 and 2, which is acceptable [49].

However, Brazil is a continental country where highly dynamic agriculture is the main economic activity. Therefore, building a representative database is not always possible, and the development of new strategies to monitor land use and main crop plantation to minimize the above-mentioned limitations is fundamental. In this context, remote sensing tools such high-resolution images and spectral indices temporal signatures are complementary and worked well in our study. The feasibility of spatial and time transference of crop samples from the U.S. Cropland Data Layer (CDL) to Brazil was evaluated by Ajadi et al. [59], using MODIS and Synthetic Aperture Radar (SAR) data. These authors produced a large-scale mapping of soybean production in Brazil in the first growing season with 88% accuracy. Unsupervised techniques can lead to up to 80% accuracy in distinguishing between soybean and maize when ground data is insufficient [60]. However, sharing and integrating databases based on the Findable, Accessible, Interoperable, and Reusable (FAIR) principles, generated by research groups, can also be an interesting way to improve agricultural monitoring where ground data acquisition is challenging [61,62].

The continuous an integrated development of Sentinel-2 and Landsat 8/9 scientific activities that enable the sensors’ interoperability and the virtual constellation of HLS [63] is a successful model that could be trespassed to initiatives of collection, storage, and sharing ground data from different users, especially in developing countries, where there is a lack of cropland data and the monitoring of food production is a sensitive issue to decision-makers. The sharing of methods, data, and findings is essential in the current trend of big data on remote sensing [64]. Although acquired two years ago, the database built and shared by Oldoni et al. [65] with some coincident areas was useful for better understanding of our study area, since it was not possible to go through it entirely.

The AgroTag application presents a high potential to overcome the limitations of ground sample acquisition in Brazil if data from different users could be accessed. Finally, it is also valid to reassure the relevant role of high-resolution (2-m) pan-sharpened images from the CBERS-4A satellite and of ready-to-use MODIS NDVI temporal patterns made available by Embrapa through the SATVeg [66] in the collection of samples in less complex classes, making the process possible and more reliable.

At Level 2, areas with straw were a source of misclassification and confusion. Two situations happened: harvesting in the previous season followed by fallow until later February or March, and forest plantation areas in the harvesting process during this assessed period. Similar circumstances were found in previous studies on the agricultural activity in the Brazilian savanna [67,68], and are related to its temporal dynamics, which often propitiate regions in transition, which vary in time, space, concept, and categorization. In both situations, when using the HLS data set, these cases were classified as pasturelands, and when using the L8, they were classified as annual crops. Figure 6 illustrates such issues, showing that the classification based on the HLS and HLSS30 overestimated pastures when compared to Landsat datasets (L8 and HLSL30), while those detected up to three times more perennial crops than HLS. Unfortunately, it was not possible to add more field inspections on the Soybean Ring, and there are no official reference data to obtain a better understanding of some particularities present in this area. During the field work, it was noted that, apart from the annual crop predominance, there is a complex agricultural dynamic, as in the Cerrado itself. An example is the large variety of crops (soybean, bean, maize, cotton, and coffee or pastures for seeds) and the irrigation pivots occupied with more than one crop, increasing the challenge of mapping and monitoring crop types in this region.

Moreover, we observed several fields in the conversion process from natural vegetation to possible annual cropping or pasture, as well as silviculture areas being converted into pasturelands, or even abandoned after harvesting. Considering the dynamics of LULC conversion in this modern agricultural frontier, where LULC change decisions are mainly driven by the agribusiness sector [10], this situation is quite common [7].

These challenges are present in the natural and anthropic Cerrado dynamics. Although some strategies that were not used in this study can be further employed with HLS datasets to assess how they can help reduce confusion and produce even better results, such as the estimation of phenometrics and the use of larger time series [69], it is important to state that the 2021–2022 crop season was unusual for the region in terms of precipitation, since the accumulated rainfall between October and March was 1068 mm, while the average for the same period was 790 mm in the last two decades [37]. This caused even more cloud cover, and therefore, harmed the total of useful remote sensing optical observations.

Regarding the spectral data, earlier HLS images (mainly October) and the VIs showed to be more appropriate to discriminate between agricultural lands and natural vegetation (Level 1), as well as to discriminate between annual crops, perennial crops, and pasturelands (Level 2). This situation was somewhat expected, since at the beginning of the wet season, most of the agricultural lands are dominated by bare soils and straws [60,67,68], highlighting the spectral contrast with the evergreen gallery forests.

The discrimination between soybean and other annual crops (Level 3) in HLS models was more effective at the end of the wet season with multispectral bands alone or combined with vegetation indices (MS + VIs). The HLS, HLSS30, and L8 VIs model also presented the weakest performance among the three subsets at this level. Adding the MS subset increased the results up to 12% in OA and 60% in Kappa, and seven out of the 10 more important variables were from March. Using L8 data to analyze the spectral-temporal response of different types of crops, such as maize, soybean, and sugarcane, Montibeller et al. [70] also found spectral bands more efficient to retrieve such responses than the spectral vegetation indices. These patterns can be explained by the differences in the growing season between soybean and other annual crops, especially cotton, the second most-cultivated crop in this harvest period. In March, most of the soybean is entering harvesting time, while cotton is still in the growing season. This difference in crop calendars is a proxy for separating both crop types, as each spectral profile pattern varies in time and amplitude, as previously reported [71,72,73,74,75].

Overall, MS + VIs subsets outperformed the others, enhancing the importance of VIs for LULC mapping purposes. Capable of expressing differences in plant responses under different soil, weather, environment, and management conditions [76], VIs were essential to detect soybean production throughout our hierarchical classification system. HLS-based results showed their relevance for Levels 1 and 2. In Level 1, NDVI, SAVI, and GNDVI from 6 October 2021 were the three most relevant features to the RF algorithm, while NDWI from 25 November 2021 and 5 February 2022 also were significant. In Level 2, SAVI, NDWI, and NDVI from the same date (6 October 2021) were the three most relevant features, with GNDVI being important on different dates (25 November 2021, 6 October 2021, 5 February 2022, 26 October 2021, and 28 February 2022). L8-based results showed the prevalence of spectral bands for Levels 1 and 3. In both cases, Green, SWIR1, and Red from the observation acquired on 9 March 2022 were the three most relevant features, with Blue, SWIR2, and NIR presenting complementary relevance in different observations of January and March 2022, highlighting the period of maximum vegetative vigor. Level 2 was the only L8-based result where VIs were the most relevant features: GNDVI, NDVI, SAVI, and NDWI from 9 March 2022. As it was mentioned, the top 10 most important variables of the HLSS30, HLSL30, and L30 datasets are show in the Supplementary Material.

4.4. HLS Applications in Agricultural Monitoring

The impact of the higher satellite revisit provided by HLS, although it was not evident in all three levels of models, was noticeable in the mapping results (Figure 5). At Level 3, our main subject, the HLS datasets outperformed HLSS30, HLSL30, and L8 with statistical significance. This difference was evident in the final maps, where the HLS MS + VIs model provided a more confident representation of crops’ spatial distributions over the study area. In contrast, for example, the L8 MS + VIs final map was visibly harmed by the lack of data, presenting a higher occurrence of misclassifications and granular effects (Figure 8). These circumstances may have been due to the spectral similarity between soybean, maize, and cotton. Regionally, all are rainfed and dryland crops with relatively close growth times. Confusion involving soybean and maize can derive from the intercropping between them commonly adopted by local farmers [77] and the similarity of crop type phenology and their spectral responses [69].

In addition to causing a greater accumulation of clouds, the uncommon rainfall volume in this harvest period may have approximated sowing and phenological calendars for different crops, as reported in previous studies [78,79]. In this scenario, the HLS’ higher frequency of valid observations may have improved the detection of relevant dissimilarities between these crops.

The increase in temporal resolution provided by the HLS is a significant differential in the monitoring of surface dynamics in a spatial resolution that is more suitable for management on a regional scale, contributing to the improvement of the characterization of surface phenology under different stimuli [32,34,80,81] and the detection of phenological metrics that are key to agricultural management such as emergence [29,82], which in turn can improve the separability between crop types under different environmental conditions [26,58,83,84,85,86], as was the case in this study.

The HLS datasets from NASA or independent harmonization methods were also tested and improved results in many tasks that were related to agricultural monitoring and management such as tracking agricultural extension and intensification [27,87,88], provided denser VIs time series to support yield estimates [23], were included as input data in crop growth models [28], and can be used to assist in managing the use of water resources in agriculture, recovering and modeling evapotranspiration [89] and surface temperature [90], and detecting the distribution of irrigation areas [30].

Despite presenting a high potential for agricultural management and monitoring on different fronts, the balance between temporal and spatial resolution of HLS data also proved to be important in monitoring diverse surface events, such as the detection of pasture cutting [31,91] and the occurrence of fires [92], flooding [93], and disturbances [94,95], among others. However, this is a new approach that remains underexplored, with few studies employing such data. Therefore, we expect growth in its exploration since many issues involving the combined use of Landsat and Sentinel-2 data [96] still need to be addressed to enhance the capacity to obtain dense time series useful for land surface monitoring purposes.

5. Conclusions

Our results show that the time series of HLS images acquired over the crop growing season can map rainfed soybean plantations more effectively than HLSS30, HLSL30, and L8. The Landsat images (HLSL30 and L8) were not capable of accurately distinguishing soybean from other annual crops at Level 3. The RF classifier was able to deal with inherent limitations related to the reduced number of cloud-free images over the crop growing season in the region as well as with spectral similarities between some annual crops (e.g., soybean and bean) and varying crop planting dates. This study demonstrated that the RF classifier applied to HLS images can address the challenges of identifying different crop types, which can be useful for public policies of monitoring and forecasting agricultural commodities over tropical regions.

There are still other important issues to be addressed in future work. For example, the use of different classification algorithms and variables such as phenometrics, the inclusion of all 13 spectral bands of the Sentinel-2 images, and the comparison between Sentinel-2 and HLS30 images need to be evaluated. In the Cerrado region, the crop–livestock integration management system has been increasingly adopted by farmers, especially the integration between soybeans and maize, both with different Brachiaria varieties used for pasture. Ground truth involving crop–livestock integration from representative sites, which is the case of Cerrado/Amazon ecotone in the Mato Grosso State, should be also considered.