CNN-Random Forest Hybrid Method for Phenology-Based Paddy Rice Mapping Using Sentinel-2 and Landsat-8 Satellite Images

Abstract

The agricultural sector plays a vital role in achieving the second Sustainable Development Goal: “Zero Hunger”. To ensure food security, agriculture must remain resilient and productive. In Indonesia, a major rice-producing country, the conversion of agricultural land for non-agricultural uses poses a serious threat to food availability. Accurate and timely mapping of paddy rice is therefore crucial. This study proposes a phenology-based mapping approach using a Convolutional Neural Network-Random Forest (CNN-RF) Hybrid model with multi-temporal Sentinel-2 and Landsat-8 imagery. Image processing and analysis were conducted using the Google Earth Engine platform. Raw spectral bands and four vegetation indices—NDVI, EVI, LSWI, and RGVI—were extracted as input features for classification. The CNN-RF Hybrid classifier demonstrated strong performance, achieving an overall accuracy of 0.950 and a Cohen’s Kappa coefficient of 0.893. These results confirm the effectiveness of the proposed method for mapping paddy rice in Indramayu Regency, West Java, using medium-resolution optical remote sensing data. The integration of phenological characteristics and deep learning significantly enhances classification accuracy. This research supports efforts to monitor and preserve paddy rice cultivation areas amid increasing land use pressures, contributing to national food security and sustainable agricultural practices.

1. Introduction

According to data from The Sustainable Development Goals (SDG) Report 2024 in 2023, 733 million people faced hunger globally, including 148 million children under the age of five who suffered from stunted growth, which represents approximately 22.3% of children in this age group [1]. One of the efforts to overcome poverty and hunger is to sustain agricultural production and ensure that all people suffering from hunger and malnutrition have access to nutritious food. The agricultural sector contributes to achieving the second goal in the SDG, namely “zero hunger”, significantly [2]. To achieve this goal by 2030, solutions and policies are needed to transform the food system by investing in sustainable agricultural practices to reduce and mitigate the impact of conflict and pandemics on global nutrition and food security [3].

Over half of the world’s population consumes rice as a staple food [4]. Indonesia is a country that is in third place as the highest rice-producing country in the world [5]. However, the Agricultural Sector in Indonesia is facing the problem of land conversion to other uses, such as industry and warehousing, which impacts food security threats. Therefore, it is necessary to map agricultural land, especially paddy rice fields. Traditional mapping of paddy rice by surveys requires a lot of time, money, and energy. To form a better mapping system, remote sensing satellite imagery has been developed to accurately produce the paddy rice map.

According to a study conducted by Zhao [6], the evolution of the paddy rice mapping method in the period 2010–2020 began with applying a phenology algorithm, which was then developed by utilizing SAR and optical remote sensing satellite image data. Paddy rice mapping using the phenology method is based on the cyclic and seasonal growth shown during their planting cycle [7]. The rice plant has unique phenological characteristics where during the flooding and transplanting phases, the surface is covered by a mixture of water and soil. Meanwhile, other crops are dry and not flooded with water [8]. Phenological characteristics are represented through time-series vegetation indices derived from multi-temporal remote sensing images [9]. According to a study by Zhang [7,10], paddy rice mapping carried out using a phenology and data fusion method of Landsat and MODIS increased accuracy by 6.7%. Based on this study, rice temporal vegetation indices (NDVI, EVI, LSWI) are extracted from fused time-series data to observe the rice planting cycle [5]. However, the use of Landsat-8 data is less effective in extracting phenological information such as NDVI or EVI values due to scarcity, considering its temporal resolution of 16 days.

Classifying paddy rice with phenological characteristics is a simple method, but it is less effective when applied to complex areas. To solve those issues, a spectral learning method was developed based on rice spectra, divided into three methods: classical statistics, machine learning, and deep learning [6]. In 2020, Ramadhani conducted a study mapping rice growth phases in the provinces of West Java and East Java, Indonesia, using multi-temporal Sentinel-2 satellite image data as well as a combination of Sentinel-1 and MOD13Q1 satellite images, which produced the highest accuracy of 90.6% with Sentinel-2 and the SVM classifier [11]. The results of this study show that the higher the spatial resolution of the satellite imagery used, the higher the accuracy of the resulting classification model.

According to a study by Arjasakusuma [12], multi-sensor and multi-temporal analysis can effectively identify paddy rice cover classes based on their temporal characteristics. The study was carried out in paddy rice fields in the northern part of Central Java Province, Indonesia, using remote sensing satellite data Sentinel-1 and Landsat-8, applying the Random Forest classifier, which produced an accuracy value of 93.48%. Based on studies that have been conducted previously [12,13,14,15], Random Forest has better capabilities in classifying paddy rice and non-paddy rice fields than other machine learning classifiers. A study by Choudhary [16] proves that the classification of paddy rice and non-paddy rice fields using Sentinel-2 data and various secondary data reached an accuracy value above 85%. However, Random Forest and other classical machine learning classifiers require preliminary feature engineering to capture complex patterns or relationships in the data. Therefore, a deep learning method is now being developed for mapping paddy fields using remote sensing satellite imagery.

The deep learning algorithm commonly used in plant classification using remote sensing satellite image data is the Convolutional Neural Network (CNN). Based on the results of a study conducted by Thorp and Drajat [17], deep learning classifiers can specifically model the temporal dimensions of satellite data so that they are suitable for use in temporal analysis in paddy rice field mapping. Zhu et al. proposed mapping paddy fields by integrating phenology and deep learning algorithms without field data, which produced an accuracy value of 88.8% using the LSTM model with Sentinel-2 data [8]. Based on a study conducted by Rawat, a multi-source method based on deep learning by integrating Sentinel-2 and Landsat-8 data produces a good accuracy of 93.75% with the 1D-CNN model [18]. This study used optical satellite data because it has higher spatial and temporal resolution than radar (SAR) satellites.

Based on the results of a study conducted by Zhu [8] and Rawat [18], the 1D-CNN method provides an effective solution for handling multi-source remote sensing data and can represent the spectral-temporal domain as a 1D time-series signal. The Hybrid CNN-RF model is widely applied to crop classification using multi-temporal remote sensing satellite image data, which produces better accuracy than the CNN model [19,20]. A study conducted by Yang et al. proved that crop classification using the CNN-RF method increased accuracy by 1.68% compared to the CNN method with Sentinel-2 data [20]. Based on these studies, it can be said that the CNN-RF method has great potential in the representation of temporal features and produces higher accuracy compared to the CNN in crop classification using remote sensing satellite imagery.

To confirm this assumption, we studied the implementation of multi-temporal Sentinel-2 and Landsat-8 to increase paddy rice classification using the CNN-RF Hybrid model. In this study, we used optical satellite image data because it has higher spatial and temporal resolution than SAR data and is easier for visualization [18]. In classifying paddy rice, we need to pay attention to the dynamics of land conditions. To tackle this issue, we used multi-temporal data from Sentinel-2 and Landsat-8 satellite images so that the rice growth phases can be observed in the process of classifying paddy rice.

Therefore, in this paper, we aim to develop a model for paddy rice mapping based on the phenological method with Sentinel-2 and Landsat-8 images. The contributions are as follows:

• Develop a paddy rice mapping method with CNN-RF Hybrid, which would increase the accuracy of the model;
• Interpret the rice growth by the phenological method using four vegetation indices (NDVI, EVI, LSWI, and RGVI);
• Propose a paddy rice mapping scheme including data pre-processing, dataset preparation, model training, and analysis of multi-temporal data.

This paper is organized as follows: Section 2 introduces the study materials and methods; Section 3 presents the mapping results and relevant analysis; Section 4 discusses the limitations and some explorations of our work; Section 5 is the concluding chapter.

2. Materials and Methods

2.1. Study Area

In this study, we chose the southern part of Indramayu Regency, West Java Province, Indonesia, as a study area for mapping paddy rice fields, as shown by the red rectangle in Figure 1. It is located at coordinates 108°18′–108°21′ east longitude and 6°21′–6°24′ south latitude. According to data from the Indonesian Central Statistics Agency in 2023, Indramayu Regency is the regency with the highest rice production in Indonesia, with an average rice production of 4151 t per year. The selection of the southern area of Indramayu Regency is considered to have a large paddy rice field area because it is a national rice barn, so it can avoid the occurrence of mixed pixels in the paddy rice field classification process. Based on its location, Indramayu Regency has a monsoon climate with two seasons: rainy and dry. The rainy season lasts from October to March, while the dry season lasts from April to September [6].

2.2. Dataset

The satellite images used in this study are multi-source optical satellite images, Sentinel-2 and Landsat-8, available on the Google Earth Engine (GEE) platform. GEE is a cloud-based geospatial analysis platform with JavaScript. We used Sentinel-2 MSI: Multispectral Instrument, Level 2A, which has been available since 28 March 2017, with a spatial resolution of 10 m and a temporal resolution of five days. As for the Landsat-8 images, we used the USGS Landsat-8 Collection 2 Tier 1 top-of-atmosphere (TOA) Reflectance, which is a collection of Landsat-8 satellite images with the highest scene quality and is suitable for use in time-series analysis. It has a resolution of 30 m. As stated in

Table 1. Date of acquisition and band used of multi-temporal Sentinel-2 and Landsat-8.

Sensor	Date of Acquisition	Band Used
Sentinel-2	11 April, 6 May, 10 June, 30 June, 9 August, 14 August, 19 August, 28 September	Red, Blue, Green, VRE1, VRE2, VRE3, VRE4, SWIR1, SWIR2
Landsat-8	20 March, 10 July, 26 July, 11 August, 12 September	Red, Blue, Green, NIR, SWIR1, SWIR2

, we selected the optical remote sensing satellite imagery with cloud cover effects of less than 20%, resulting in scarce data. Therefore, in this study, we combined optical images from Sentinel-2 and Landsat-8 to fill the temporal gaps caused by cloud cover and differences in revisit cycles.

To ensure accurate pixel alignment, the two satellite images need to share the same spatial resolution. Several studies [21,22,23,24,25] have found that classification accuracy, particularly using Random Forest, increases with higher spatial resolution. Higher resolution enables better feature discrimination and reduces mixed-pixel errors, especially in complex landscapes. Because Landsat-8 images have a lower spatial resolution, which is 30 m, resampling is carried out at the pre-processing stage so that they have a spatial resolution of 10 m for better spatial capturing.

The data acquisition date chosen in this study is based on the rice planting season in Indonesia, especially in West Java Province. The total rice planting time ranges from 120 to 150 days depending on the variety [7]. Indonesia’s first rice planting season generally starts in November, and the second planting season starts in March. Meanwhile, for downstream areas, the first rice planting season starts in February due to the prolonged rainy season. Meanwhile, the second planting season starts in July [11,26]. As shown in Table 1, in this study, multi-temporal Landsat-8 and Sentinel-2 data were acquired from 11 April to 28 September 2022. The dates were chosen to represent 1–2 rice planting cycles in Indramayu Regency, West Java Province, Indonesia.

The acquisition dates for Sentinel-2 and Landsat-8 were dynamically selected based on cloud-free area availability during the paddy cultivation season. Instead of fixed intervals, the selected images represent the key growth stages, and the selection took into account the exclusion of areas with cloud cover exceeding 20%. To fill the temporal gap, Sentinel-2 and Landsat-8 images were integrated, taking advantage of the complementary revisit cycles to ensure adequate temporal coverage without requiring interpolation.

2.3. Workflow

The methodology of this study is illustrated in Figure 2, which generally consists of four main processes, including image pre-processing, dataset preparation, model construction, and mapping the predicted class on satellite images. The pre-processing stage to obtain multi-temporal vegetation indices data was carried out using the Google Earth Engine (GEE) platform with JavaScript programming until an image was obtained with dimensions of 543 × 550 pixels in TIFF format. Dataset preparation was carried out by utilizing the QGIS platform by creating polygons for each paddy and non-paddy class in vector form by referring to Google Satellite high-resolution image reference data. These vectors are then converted into raster files, which are used as input for the classification model.

The CNN-RF model development was carried out using Python 3.11.13 programming with the Google Colaboratory platform. The CNN was placed at the front of the model to extract features, which were then fed to the RF classifier as input. Before being passed to the RF, the flatten layer in the CNN transformed the CNN output into a one-dimensional feature vector suitable for the RF input. During the training phase, hyperparameters such as the number of decision trees and data splitting ratio were tuned to achieve optimal accuracy. Training only updated the RF model, while the CNN was not trained end-to-end with the RF, meaning there was no backpropagation from the RF output to the CNN.

After the model is trained, an evaluation is carried out to analyze the performance of the model built using the confusion matrix method and evaluation parameters consisting of Overall Accuracy (OA), Cohen’s Kappa, Precision, Recall, and F1-Score. Validation of mapping results using the CNN-RF method was carried out using reference data consisting of high-resolution images and field surveys conducted on 2 June 2023.

2.4. Image Pre-Processing and Vegetation Indices

In this study, all stages of pre-processing of multi-temporal optical remote sensing satellite images were carried out using the GEE platform. First, resampling is carried out so that all Sentinel-2 and Landsat-8 multi-temporal images have the exact spatial resolution of 10 m, as shown in Figure 3. The resampling method applied is the nearest neighbor method, where each “corrected” pixel from the nearest “uncorrected” pixel is assigned a value. The multi-temporal image data that has been resampled is then cloud-masked to produce an image with a cloud cover of 5%.

Then, the selection of spectral bands and calculation of vegetation indices consisting of NDVI, EVI, LSWI, and RGVI are carried out. Mapping paddy fields using the phenological method is based on paddy growth patterns throughout their life cycle [7], which consists of four phases: transplanting and flooding, vegetative growth, reproductive, and ripening. Paddy plants have unique phenological characteristics where, during the flooding and transplanting phases, the surface is covered by a mixture of water and soil [27]. Therefore, the phenology method is applied to differentiate rice plants from other crops. Phenological characteristics are represented by the time-series vegetation indices, which are derived from multi-temporal remote sensing images [9,16]. Based on a study by Zhou, vegetation indices with the highest correlation values to rice phenology are NDVI, EVI, LSWI, and RGVI [28].

The four vegetation indices are calculated using the equations listed in

Table 2. The equation of spectral indices used.

Vegetation Indices	Equation
Normalized Difference Vegetation Index (NDVI)	$NDVI={\displaystyle \frac{NIR-RED}{NIR+RED}}$
Enhanced Vegetation Index (EVI)	$EVI2=2.5+{\displaystyle \frac{NIR-RED}{NIR+2.4\times RED+1}}$
Land Surface Water Index (LSWI)	$LSWI={\displaystyle \frac{NIR-SWIR}{NIR+SWIR}}$
Rice Growth Vegetation Index (RGVI)	$RGVI2=1.05{\displaystyle \frac{BLUE+RED}{NIR+SWIR1+0.5}}$

, which are derived from the spectral bands of optical remote sensing satellite images. NDVI is an index that quantifies the difference between reflected NIR and RED light from the crop. The chlorophyll content in the crop can reflect more NIR and green light and absorb more red and blue light [29]. As shown by the information provided in Table 2, NDVI is calculated by considering the NIR and RED channels. Therefore, in this study, NDVI is used to differentiate rice plants from other plants based on the level of greenness or chlorophyll content. By implementing the phenology method, we need to pay attention to changes in water on the surface of the paddy fields [30,31]. Therefore, in this study, we used LSWI and EVI to consider the characteristics of paddy plants during the growth period other than the paddy transplantation period. Based on a study conducted by Choudhary et al., 2021 [16], RGVI is an index that is more appropriate to use to improve and obtain information on paddy rice phenology.

Several studies [8,9,16] have reported different threshold values for some indices that indicate phenological phase transition. For example, reproductive phase was indicated by NDVI with slightly different peaks, which are around 0.8 [8], around 0.7 [9], and between 0.7 and 0.8 [16]. Meanwhile, the thresholds of LSWI for transplanting phase were LSWI > NDVI or LSWI > EVI [8], LSWI > 0 [9], and LSWI > 0.05 [16]. The differing values across studies could be caused by regional variability, seasonal difference in observation times, rice variety, and water management conditions. Therefore, classifying paddy fields across phenological phases requires a model that can accommodate the complexity of these temporal patterns, for which machine learning and deep learning are well-suited approaches.

2.5. Dataset Preparation

The dataset is divided into training and validation data which consists of two classes, consisting of paddy and non-paddy. We employed a binary classification approach that categorizes pixels into either paddy or non-paddy. The paddy category includes rice plants at various phenological stages, while the non-paddy category consists of all pixels that do not belong to the paddy class. Accordingly, all subcategories under non-paddy are classified as non-paddy.

In this study, we used optical remote sensing image data that has color hue information. Therefore, in the data preparation stage, we created the polygons for each class based on different colors. The non-paddy class is divided into several sub-classes consisting of tree, pond, river, built-up red, and built-up white. The built-up red and built-up white subclasses categorize man-made structures based on roofing materials; built-up red refers to buildings topped with tile roofs, while built-up white includes those constructed with asbestos roofing. These polygons were created based on Google Satellite high-resolution remote sensing images with QGIS version 3.24 software. The results of these polygons were then converted into raster form. The total number of pixels from the data was calculated.

2.6. Model Construction

The CNN-RF Hybrid model is a combination of a Convolutional Neural Network (CNN) to extract high-dimensional features and a Random Forest (RF) classifier as a replacement for the Fully Connected (FC) Layer in CNN to make the final classification decision [19,20]. As Figure 4 illustrates, in traditional CNN networks, the FC layer is used to make the final classification decision and overfitting usually occurs in the case of insufficient data samples. In the Hybrid CNN-RF model, the RF classifier is used instead of the FC layer to make the final decision and prevent overfitting [11]. The use of an RF classifier can also produce high accuracy even though the number of samples used is low. The main problem in plant classification using remote sensing data is finding useful information from massive data to balance classification accuracy and training time. Therefore, in this study, we offer a paddy rice mapping method with a Hybrid CNN-RF model where the CNN automatically extracts features from the data that will be used in the RF classifier to obtain final modeling results.

The Hybrid CNN-RF model architecture that we built consists of three convolution layers followed by a Max Pooling layer and a fully connected layer. At each convolution layer, a kernel of size 2 is used with an ReLU activation function. The number of filters in each convolution layer is 128, 64, and 64, respectively. The Softmax function is used in the output layer to produce output in the form of probabilities for each class in the multiclass classification. In the Hybrid CNN-RF model that we built, hyperparameters were also set using the Adam optimizer with a learning rate of 0.001, batch size of 10, and number of epochs of 150. These parameters were set empirically, based on the optimal results obtained. The results of the classification and mapping of paddy rice using the Hybrid CNN-RF model were also compared with the results of mapping using the RF and 1D-CNN with the same architecture and parameters. The RF architecture used has a total of 100 estimators, like the RF architecture in the Hybrid CNN-RF model we offered in this study.

The mathematical formulations of the CNN-RF model are represented by Equations (1)–(5). The prediction result ($\widehat{y})$ of the CNN-RF is described as follows:

$$\widehat{y}=mode \left\{t_1\left(z\right), t_2\left(z\right), \dots , t_n\left(z\right)\right\}$$

(1)

showing that RF takes the mode of decision tree results, following the majority-voting mechanism, where $t_i\left(z\right), i=1, 2, \dots , n$ denotes the prediction result of the $i$-th decision tree, $n$ is the number of decision trees, and $z$ is the input of RF, which is also the output of CNN.

The output of CNN is the flattened feature, as follows:

$$z=Flatten(MaxPool(Dropout(h^{\left(3\right)})$$

(2)

where

$$h^{\left(3\right)}=\mathit{max}(0, f(W^{\left(3\right)}\ast h^{\left(2\right)}+b^{\left(3\right)})), h^{\left(3\right)}\in {\mathbb{R}}^{L_3\times d_3})$$

(3)

$$h^{\left(2\right)}=\mathrm{m}\mathrm{a}\mathrm{x}(0, f(W^{\left(2\right)}\ast h^{\left(1\right)}+b^{\left(2\right)})), h^{\left(2\right)}\in {\mathbb{R}}^{L_2\times d_2})$$

(4)

$$h^{\left(1\right)}=\mathrm{m}\mathrm{a}\mathrm{x}(0,f(W^{\left(1\right)}\ast x +b^{\left(1\right)})), h^{\left(1\right)}\in {\mathbb{R}}^{L_1\times d_1})$$

(5)

where $h^{\left(k\right)}, W^{\left(k\right)}, b^{\left(k\right)}$ denote the output, weight, and bias of the $k$-th convolution layer. The dimension of the $k$-th layer output is $L_k\times d_k$, where $L$ is the vector length, and $d$ is the number of filters in the layer, and $x \in {\mathbb{R}}^{L_0\times 1}$ is the input vector that corresponds to each pixel, consisting of spectral bands and vegetation indices, with the length $L_0$.

2.7. Classification Scheme

In this study, paddy rice mapping using the CNN-RF Hybrid method was carried out using 11 schemes, as listed in

Table 3. Input features for each classification scheme.

Scheme	Input Features
1	Raw Spectral Bands
2	Raw Spectral Bands+All VI (NDVI, EVI, LSWI, RGVI)
3	All VI (NDVI, EVI, LSWI, RGVI)
4	Raw Spectral Bands+NDVI
5	Raw Spectral Bands+EVI
6	Raw Spectral Bands+LSWI
7	Raw Spectral Bands+RGVI
8	NDVI
9	EVI
10	LSWI
11	RGVI

. The classification scheme was determined based on input features from multi-temporal data from Sentinel-2 and Landsat-8. The aim of creating a classification scheme in this study is to obtain optimal features as inputs for the CNN-RF Hybrid model to map paddy rice. Schemes #1–#3 were developed to analyze the effect of four vegetation indices (NDVI, EVI, LSWI, and RGVI) on the representation of phenological characteristics. Schemes #4–#7 were designed to examine the impact of integrating each of the four vegetation indices (NDVI, EVI, LSWI, and RGVI) with raw spectral bands as inputs for paddy rice mapping features. Meanwhile, schemes #8–#11 were formulated to determine whether each vegetation index can effectively represent the phenological characteristics of paddy rice. The classification results for each scheme with the CNN-RF Hybrid method are compared with the CNN-1D and Random Forest methods to analyze the performance of the proposed mapping method.

2.8. Accuracy Assessment and Evaluation Parameter

To evaluate the performance of the proposed Hybrid CNN-RF model, we used several evaluation parameters, including overall accuracy (OA), precision, recall, F1-Score, and Cohen’s kappa. Overall accuracy represents the probability that data can be correctly classified by the CNN-RF model. OA is calculated as the sum of the number of data points correctly identified as paddy (true positives) and the number of data points correctly identified as non-paddy areas (true negatives), divided by the total number of test data points. Meanwhile, precision represents the proportion of pixels classified as paddy rice out of the total pixel predictions. Recall indicates the number of paddy rice areas that were correctly classified. Then, F1-Score is a parameter that represents the average of recall and precision. Cohen’s Kappa is a statistical measure that assesses the performance of the proposed CNN-RF Hybrid method based on the agreement between the model’s predictions and the actual data.

The value of Cohen’s Kappa ranges from −1 to 1. A Cohen’s Kappa value of 0 indicates that the classification is no better than random classification, while an increasingly negative value suggests that the classification is significantly worse than random classification. In contrast, if Cohen’s Kappa is close to 1, it indicates that the classification is substantially better than random classification.

$$OA={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(6)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(7)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(8)

$$F1 Score=2\times {\displaystyle \frac{(Precision\times Recall)}{(Precision+Recall)}}$$

(9)

$$Cohe{n}^{’}s Kappa={\displaystyle \frac{OA-p_e}{1-p_e}}$$

(10)

where $p_e$ denotes the percentage of the difference in measurements between predicted results and reference data.

3. Results

3.1. Implementation Details

The binary classification of paddy and non-paddy areas was carried out using the CNN-RF Hybrid model with various input features in schemes #1–#11, which are listed in Table 3. Then, to assess the performance of the classification model, an evaluation was carried out with several parameters consisting of overall accuracy (OA), Cohen’s Kappa, recall, precision, and F1-Score. The highest OA value obtained from scheme #2 is 0.950, with a Cohen’s Kappa value of 0.841, a recall and precision of 0.950, and an F1-Score of 0.949. For scheme #1, an OA of 0.937 was obtained, with a Cohen’s Kappa of 0.841, recall and precision of 0.937, and an F1-Score of 0.936.

Based on these results, it can be said that the paddy rice mapping using the CNN-RF Hybrid method, using features consisting of all spectral bands and the four vegetation indices as listed in Table 3, increases the OA value by 0.013 compared to using only the spectral band features alone. Based on the recall and precision results, whose values are above 0.9 in schemes #1 and #2, it proves that the validation data can be predicted by the model correctly. In addition, the Cohen’s Kappa value obtained by schemes #1 and #2 is more than 0.8, proving that there is strong agreement between the predicted results and the actual data.

The analysis presented in Schemes #4 to #7 investigates the influence of phenological characteristics captured by various vegetation indices, including NDVI, EVI, LSWI, and RGVI. These indices are evaluated alongside spectral band features extracted from multi-temporal Sentinel-2 and Landsat-8 imagery. Scheme #7 demonstrates the highest overall accuracy (OA) among these schemes. These findings indicate that incorporating all spectral bands together with the RGVI index as input features in the CNN-RF Hybrid model yields an increase in OA of 0.005 compared to using only spectral bands as input features.

In addition, Schemes #8 to #11 were formulated to assess the outcomes of mapping paddy and non-paddy areas utilizing each vegetation index, consisting of NDVI, EVI, LSWI, and RGVI as input features. The highest OA value is produced by scheme #10, where the LSWI index is used as feature input for paddy and non-paddy mapping with multi-temporal data from Sentinel-2 and Landsat-8. These results prove that the unique characteristics of rice plant growth phenology can be effectively represented by the LSWI index, which is an index that is sensitive to changes in the volume of water covering the surface of rice plants during the flooding and transplanting phases [17,18].

3.2. Mapping Results

The aim of this study is to obtain a map of paddy and non-paddy areas in the specified area with accurate results from the binary classification. As explained in the previous section, the best performance is produced by scheme #2, where all spectral bands and four vegetation indices consisting of NDVI, EVI, LSWI, and RGVI from Sentinel-2 and Landsat-8 multi-temporal data are used as the input features. Figure 5a shows the results of mapping paddy and non-paddy areas using the CNN-RF Hybrid method for scheme #2. As can be seen in the figure, the mapping results produced by scheme #2 with the CNN-RF Hybrid method produce quite good maps of paddy and non-paddy areas. The results obtained prove that raw spectral bands and four vegetation indices consisting of NDVI, EVI, LSWI, and RGVI from multi-temporal data from medium-resolution optical remote sensing satellite images can effectively map paddy and non-paddy areas during their growth phase using the CNN-RF Hybrid method.

Figure 5b shows the results of paddy and non-paddy mapping in scheme #3, which only uses four vegetation indices consisting of NDVI, EVI, LSWI, and RGVI as input features. The mapping results prove that involving only vegetation indices without raw band spectral data as input results in the loss of some information regarding the phenological characteristics of rice during its life cycle, considering that the data used in this study is multi-temporal data acquired based on the rice planting period in West Java.

Figure 5c shows the paddy and non-paddy mapping results obtained in scheme #7. The results obtained show that there is no significant difference between using raw spectral bands and combining them with the RGVI index as input features. These results indicate that the use of the RGVI vegetation index does not significantly improve the results of paddy and non-paddy mapping with multi-temporal data. In this study, the RGVI index was used in multi-temporal analysis of rice growth phases.

3.3. Performance Comparison

The performance evaluation results for paddy and non-paddy mapping for schemes #1–#11 using the CNN-RF Hybrid method proposed in this study were compared with the performance evaluation results using the CNN-1D and Random Forest methods. As shown in

Table 4. Evaluation of paddy and non-paddy mapping performance for schemes #1–#11 using RF, CNN-1D and CNN-RF Hybrid methods.

Scheme	RF Classifier
	OA	Cohen’s Kappa	Recall	Precision	F1
1	0.895	0.745	0.895	0.888	0.886
2	0.899	0.756	0.899	0.890	0.891
3	0.589	0.076	0.589	0.596	0.592
4	0.802	0.785	0.802	0.801	0.800
5	0.768	0.703	0.760	0.755	0.768
6	0.816	0.835	0.816	0.816	0.818
7	0.816	0.835	0.816	0.816	0.816
8	0.598	0.098	0.598	0.605	0.601
9	0.641	0.183	0.641	0.642	0.641
10	0.679	0.273	0.679	0.681	0.680
11	0.589	0.076	0.589	0.596	0.592
Scheme	CNN(1D)
	OA	Cohen’s Kappa	Recall	Precision	F1
1	0.915	0.789	0.915	0.915	0.915
2	0.928	0.818	0.928	0.927	0.927
3	0.720	0.028	0.720	0.667	0.618
4	0.841	0.548	0.841	0.847	0.826
5	0.784	0.335	0.784	0.790	0.745
6	0.843	0.564	0.843	0.843	0.831
7	0.843	0.562	0.843	0.845	0.830
8	0.756	0.243	0.756	0.743	0.709
9	0.794	0.381	0.794	0.797	0.762
10	0.843	0.561	0.843	0.845	0.830
11	0.720	0.038	0.720	0.669	0.623
Scheme	CNN-RF
	OA	Cohen’s Kappa	Recall	Precision	F1
1	0.937	0.841	0.937	0.937	0.936
2	0.950	0.873	0.950	0.950	0.949
3	0.631	0.083	0.631	0.630	0.631
4	0.939	0.847	0.939	0.939	0.939
5	0.775	0.347	0.775	0.761	0.749
6	0.940	0.849	0.940	0.940	0.940
7	0.942	0.854	0.942	0.942	0.941
8	0.665	0.160	0.665	0.661	0.663
9	0.712	0.280	0.712	0.709	0.711
10	0.765	0.419	0.765	0.765	0.765
11	0.632	0.086	0.632	0.631	0.631

, of the three methods tested in mapping paddy and non-paddy areas with multi-temporal data, Sentinel-2 and Landsat-8 produced the best performance in scheme #2, which uses a combination of all spectral bands with four indices of vegetation consisting of NDVI, EVI, LSWI, and RGVI as input features. The best performance produced by scheme #2 produces OA values of 0.950, 0.928, and 0.899 using the CNN-RF Hybrid, CNN-1D, and RF methods. These results prove that mapping paddy and non-paddy areas using multi-temporal data from Sentinel-2 and Landsat-8 images by combining all spectral bands and four vegetation indices (NDVI, EVI, LSWI, and RGVI) consistently increases accuracy values compared to using only spectral bands and four vegetation indices (NDVI, EVI, LSWI, and RGVI) only for all classification methods applied. Table 4 is an evaluation of paddy rice mapping performance for schemes #1–#11 using Random Forest, CNN-1D, and the CNN-RF Hybrid method.

As stated in Table 4, between schemes #4 and #7, the highest OA produced by scheme #7 for each of the CNN-RF Hybrid, CNN-1D, and RF methods is 0.942, 0.843, and 0.816. These results prove that using all spectral bands combined with the RGVI vegetation index effectively increases the OA value compared to just using all spectral bands from multi-temporal image data as input features. Between schemes #8 and #11, the highest OA value was obtained in scheme #10 for each classification method applied in mapping where the values were 0.765, 0.843, and 0.679 using the CNN-RF Hybrid, CNN-1D, and CNN-RF methods, respectively. These results show that of the four vegetation indices consisting of NDVI, EVI, LSWI, and RGVI, the use of LSWI produces the highest accuracy in paddy and non-paddy mapping with multi-temporal Sentinel-2 and Landsat-8 data.

Based on training time data from the three methods tested and listed in Table 4, the largest training time was generated in the learning process using the CNN-1D method. Between schemes #1 and #11, the largest training time was produced by scheme #2 for the three classification methods applied, namely 4.754 s, 743.325 s, and 2.309 s using the CNN-RF Hybrid, CNN-1D, and RF methods, respectively. These results prove that the use of all spectral bands and four vegetation indices consisting of NDVI, EVI, LSWI, and RGVI from Landsat-8 and Sentinel-2 multi-temporal data has an impact on processing memory requirements because it requires a larger number of neurons in the deep learning model. Apart from that, it also affects the length of the learning process so that the classification time becomes long. Apart from that, these results also prove that the application of the CNN-RF method proposed in this research in mapping paddy and non-paddy areas with multi-temporal optical remote sensing image data can effectively lighten the computational burden, considering the CNN-RF architecture. The use of the CNN-1D model is used to extract features which then become input for the RF classifier in making the final decision or a replacement for the fully connected layer in CNN-1D.

The models reported in Table 4 are built with the optimum hyperparameter setting. To show that CNN-RF results were statistically meaningful, we report the mean, standard deviation, and variance values of each model with scheme #2 in

Table 5. Statistics of the overall accuracy of the compared models with varying hyperparameter settings.

Model	OA (µ ± σ)	OA Variance (σ2)
RF classifier	0.8952 ± 0.0043	0.0000189
CNN-1D	0.9050 ± 0.0076	0.0000588
CNN-SVM	0.9067 ± 0.0048	0.0000226
CNN-RF	0.9411 ± 0.0042	0.0000199

. We also compare them with CNN-SVM to highlight the superiority of the proposed CNN-RF. The tuned hyperparameters were the data splitting ratio, the number of decision trees, and the dropout ratio. The results indicate that the statistical performance of CNN-RF was significantly superior to that of the other methods.

4. Discussion

4.1. Phenological Analysis

In this study, we also carried out quantitative and qualitative analysis of rice growth phenology by plotting the median value of each vegetation index, namely NDVI, EVI, LSWI, and RGVI, in a time series obtained via the GEE platform. Rice plant growth generally consists of four stages, namely bare-land, vegetative, reproductive, and ripening [32]. Bare land is the phase of preparing the land to be planted with rice. Then, the vegetative stage is the stage where seeding is carried out until the tillering phase, which occurs for 65 days, also known as the flooding and transplanting phase, because the surface of the soil is covered with water [33]. The next stage is the reproductive stage, which occurs for 30 days and ends with ripening, until finally the rice can be harvested [16].

Figure 6 is a graph that represents the median value of NDVI from multi-temporal Sentinel-2 and Landsat-8 data acquired from 20 March to 28 September 2022. As seen in Figure 6, the NDVI value from the end of March to April experienced a significant decline, tending to be low for the paddy rice class and continuing to decline from May to the end of June. These results prove that the rice plants have been harvested and entered a transition phase until the next planting period. Meanwhile, NDVI values for non-paddy classes (tree, river, pond) increased until mid-April. The NDVI value for the paddy class increased again at the end of June to July, which indicates that the rice growth phase is already in the vegetative phase. In mid-August, the NDVI value for the paddy class experienced a significant decline in the paddy class. These results prove that rice has entered the generative phase so that it can be harvested, which is marked by the plant’s color changing to yellowish brown [34]. Meanwhile, the NDVI value for the non-paddy class in this period had a gentler graphic trend. This indicates that there was no significant change in NDVI values during that period.

Figure 7 is a graph that represents the median EVI value from multi-temporal Sentinel-2 and Landsat-8 data acquired from 20 March to 28 September 2022. In general, the EVI pattern for paddy class is not much different from the NDVI pattern in the same period. However, in the EVI pattern, there was a very significant increase compared to NDVI in mid-June to July. This happened because during that time period, there were significant structural changes in rice plants from the vegetative phase to the generative phase.

Figure 8 is a graph that represents the LSWI value of the paddy class, where there was no significant change from June to July when compared to the EVI pattern. This is in accordance with the theory, which states that the LSWI index is an index that is sensitive to changes in water volume on the surface of rice plants. Meanwhile, the EVI index is an index that is sensitive to structural changes in vegetation [16,33,35]. These results are also strengthened by a study conducted by Ramadhani et al. [11], which states that rice plants are at the peak of the vegetative phase in mid-July and then enter the generative phase until they can be harvested, starting at the end of July to mid-August.

Figure 9 is a graph that represents the median RGVI value of the paddy and non-paddy classes acquired from 20 March to 28 September 2022. RGVI is an index used to obtain a high level of contrast between the growth of rice plants and the background and minimize differences between them [16]. As shown in the figure, the median RGVI value for the paddy class has a significant difference from the non-paddy class. The median RGVI value for the paddy class in the period mid-March to mid-April did not differ significantly from the non-paddy class. This is because rice is still in the flooding and transplanting or early vegetative phase during that period. Then, the value increases significantly from mid-June until it reaches its peak in mid-July when rice has entered the generative phase towards maturity. Then, the value decreases until August, when the rice has entered the ripening phase and is ready to be harvested until the land is prepared for the next rice planting period.

4.2. Limitations and Future Work

In this study, the classification model is trained to learn the phenological features derived from multitemporal data, rather than on rigid thresholding. However, under ideal conditions, the transitions between rice growth stages can generally be interpreted using reference thresholds reported in previous studies [8,9,16]. Although the actual values in this study differ slightly from those reported, the temporal pattern of increase and decrease that characterizes rice phenology remains consistent. These indicators help encode phenological understanding into the model’s input features. Such differences may result from the variability in paddy varieties and regional characteristics. In addition, a deeper analysis of the contribution of each index to the classification can be achieved through the use of Explainable AI (XAI), as demonstrated in [36,37,38]. Hence, the future directions include training the model with a larger and more diverse dataset in terms of geographical regions and the paddy varieties, as well as incorporating XAI to analyze the contribution of various indices.

Even though CNN was not explicitly designed for temporal-sequential learning, our method utilizes multitemporal features of the images that represent the phenological development of seasonal rice crops and incorporates them as an input channel. This enables the CNN filters to capture short-term temporal dependencies. By integrating RF as the decision layer, this model leverages the generalization capability of ensemble methods. However, future research comparing this hybrid approach with temporal models such as LSTM or TCN is recommended to assess its scalability and sensitivity to temporal patterns.

A limitation of this study is the lack of an extensive validation to assess the generalizability and robustness of the model across years and regions. Our model was currently trained on multitemporal data that represents the typical planting season, so it may require retraining or domain adaptation if applied to years with unusual climate conditions or to regions with different phenological calendars. The vegetation indices and the model performance may vary in particular conditions such as delayed planting, climate anomalies, or partial harvest failures. These factors can influence phenological signals and the classification robustness. Future research should incorporate a multi-year dataset, simulate planting schedule irregularities due to climate variability, and assess the model in various agroecological zones to ensure robust generalization.

5. Conclusions

This study presents a comprehensive approach to mapping paddy rice fields using a CNN-RF Hybrid model that leverages the phenological characteristics of rice plants. By integrating multi-temporal Sentinel-2 and Landsat-8 satellite imagery, we demonstrated that the hybrid model significantly improves classification accuracy, achieving an overall accuracy of 95% and a Cohen’s Kappa of 0.893. The use of both raw spectral bands and vegetation indices (NDVI, EVI, LSWI, and RGVI) as input features proved to be highly effective in capturing the phenological changes in rice growth cycles. The findings underscore the potential of the CNN-RF Hybrid model in agricultural applications, particularly in regions where land use changes pose a threat to food security. Future work could explore the application of this model to other crop types, climate, and regional variabilities, the use of XAI for indices analysis, as well as integrating additional data sources to further refine and validate the model’s performance.