Optimized Ensemble Learning for Enhanced Crop Recommendations: Leveraging ML for Smarter Agricultural Decision-Making ^†

Abstract

Agriculture is the backbone of a country and plays a vital role in shaping its economic performance. Factors such as natural disasters, extreme weather changes, pests, and soil quality significantly impact productivity, often leading to economic losses. Accurate predictions in agricultural practices, particularly crop recommendations, can substantially boost productivity and resource management. This research aims to develop a robust crop recommendation system using ensemble learning (EL), which integrates multiple machine learning (ML) models for improved performance. This study utilizes two datasets: a real-time dataset available on Kaggle, collected using IoT sensors, and a synthetic dataset generated using CTGAN. These datasets provide crop recommendations for 22 different crops, based on key features like nitrogen, phosphorus, potassium, soil pH, humidity, and rainfall. The performance of various ML models—such as linear regression (LR), support vector machine (SVM), decision tree (DT), naïve Bayes (NB), K-nearest neighbor (KNN), random forest (RF), extra tree classifier, XGBoost, and gradient boost—is compared with that of EL models, including voting, bagging, boosting, and stacking ensemble techniques. The stacking ensemble model achieved the highest accuracy at 99.36% across all ensemble techniques. By further optimizing this model using the Optuna hyper-parameter tuning technique, the accuracy was improved to 99.43%.

1. Introduction

Agriculture is a vital asset for any country, contributing to its strength and independence. A nation thrives when it can successfully meet its agricultural needs, becoming self-sufficient and reducing reliance on other countries for daily necessities. The wealth of a country lies in its farming industry and its farmers. However, modern agriculture faces numerous challenges, including global warming, wars, infectious diseases, and pests [1]. To combat these issues, AI can be used in crop prediction, weather forecasting, soil health analysis, precision farming, yield prediction, pest control, and many more applications [1,2].

In this research paper, we propose a machine learning (ML)-based crop recommendation system. The dataset used in this study was sourced from Kaggle (https://www.kaggle.com/datasets/atharvaingle/crop-recommendation-dataset, accessed on 19 September 2024) and was collected using IoT sensors. These sensors measure critical soil parameters including moisture, pH, temperature, humidity, and essential soil nutrients such as phosphorus (P), nitrogen (N), and potassium (K). Based on these parameters, our system recommends the most suitable crop to achieve improved yield.

2. Literature Review

Dey et al. (2024) [3] divided their dataset into two parts: one containing 11 agricultural plants and the other containing 10 horticultural plants. The authors implemented five distinct machine learning models—SVM, XGBoost, RF, KNN, and DT—on each of the separate datasets rather than on a combined dataset. The XGBoost model achieved an accuracy of 99.09%.

Islam et al. (2023) [4] proposed an ML-enabled IOT device to monitor soil nutrients like N, P, K, pH, temperature, and soil humidity. The author employed ML-based algorithms like catBoost, voting, and bagging to predict the recommended crop based on the parameters of the soil studied. CatBoost obtained the highest accuracy at 97.5%.

Kiruthika et al. (2023) [5] proposed a method based on improved-distribution-based chicken swarm optimization (IDCSO) with weight-based long short-term memory (WLSTM) for crop prediction. The author achieved an accuracy of 95% by employing the IDCSO algorithm for feature selection.

Ramzan et al. (2024) [6] implemented ML and EL models on two types of data: real-time data and hybrid data (real-time data and manual data). The author implemented ML algorithms to predict the recommended crop and compared the performance of ML and EL models.

Nikhin et al. (2024) [7] used ML models to predict crop yield with weather, soil, and crop data. The author found that the extra tree regressor achieved the highest performance among the other ML models, followed by the random forest regressor and LGBM regressor.

Elbasi et al. (2023) [8] used fifteen different ML algorithms with a new feature combination scheme. The author achieved 99.59% accuracy using the Bayes net algorithm and 99.46% using the naïve Bayes classifier and Hoeffding tree algorithm.

S.P. Raja et al. (2022) [9] developed a range of feature selection and classification techniques to predict the yield size of plant cultivations. Their study likely involved identifying key features (such as soil quality, temperature, humidity, and other environmental or agronomic factors) that influence crop yield. By using various machine learning or statistical models, they aimed to classify and predict the expected yield based on these selected features.

Sharma et al. (2024) [10] performed a crop prediction by demonstrating how different ML models, such as K-nearest neighbors (KNN) and deep learning algorithms, can achieve high accuracy in crop selection and disease prediction.

Parween et al. (2021) [11] explored the integration of IoT with ML techniques to create a precise crop prediction system, improving decision-making for farmers through real-time environmental monitoring. The system helped to reduce input costs and boosted productivity by recommending the most appropriate crops based on current soil and weather conditions.

Bakthavatchalam et al. (2022) [12] proposed a machine learning-based system to recommend the best high-yielding crops based on a combination of eight different agricultural attributes. Their aim was to improve precision agriculture using supervised learning algorithms implemented in WEKA. The study evaluated different classification algorithms for crop prediction using a multilayer perceptron and rule-based classifier. The performance of the models was evaluated based on accuracy metrics. The results showed that the selected classifiers achieved a high level of prediction accuracy, with a performance rate of 98.2273%.

3. Data Pre-Processing

The dataset used in this study is an IOT sensor dataset available in Kaggle. The dataset includes soil nutrients measures like nitrogen (N), phosphorous (P), and potassium (K) and other parameters such as the pH of soil, moisture, and rainfall, with the type of recommended crop as the target variable, as shown in Figure 1. The Kaggle dataset has 2200 rows, and the class distribution of the dataset is shown in Figure 2.

Figure 1. Sample dataset.

Figure 2. Class distribution of original dataset.

The Kaggle dataset is relatively small, is well-balanced, and contains no missing values. However, in real-world scenarios, sensor data can have missing values, noisy data, and errors. To account for this and expand our dataset, we generated approximately 1000 synthetic rows using the CTGAN (conditional tabular generative adversarial network) [13], a deep learning model. CTGAN is designed to create synthetic datasets by learning the distribution of the original tabular data, which helps in maintaining the same statistical properties. The CTGAN was trained for 200 epochs with a generator learning rate of 0.0002 and a discriminator learning rate of 0.0001. The quality of the synthetic data and the Kaggle dataset was evaluated, and a graph is given in Figure 3. The synthetic data generated by CTGAN were then concatenated with the original data to create a more robust dataset. This distribution of the combined dataset with 3200 rows is shown in Figure 4. The SMOTE [14] technique is applied to address the issue of imbalanced class data. After implementing SMOTE, each class contains 161 samples each. The details of the dataset are available in Table 1.

Figure 3. Class distribution of original + synthetic dataset.

Figure 4. Evaluation of original and synthetic dataset using CTGAN.

Table 1. Dataset details.

	Kaggle Dataset	Kaggle + Synthetic Dataset
No. of rows	2200	3200
No. of samples per class	100	161
No. of classes for target variable	22	22

4. Methodology

In this research paper, we have developed a classification model to predict the recommended crop based on seven features: N, P, K, soil pH, temperature, humidity, and rainfall. We have implemented various machine learning (ML) models, including linear regression (LR), support vector machine (SVM), decision tree (DT), naïve Bayes (NB), and K-nearest neighbor (KNN). Additionally, we have utilized ensemble learning (EL) models such as random forest (RF), extra tree, XGBoost, bagging, gradient boosting, voting, and stacking. Ensemble learning methods have provided promising results in many fields [15,16]. The details of the EL model implemented in this research work are given in Table 2.

Table 2. Ensemble model parameters.

Model	Base Estimator	No. of Estimator	Meta-Classifier
Bagging	Random Forest	100	Nil
Boosting	Gradient	100	Nil
Voting	ExtraTreesClassifier, RandomForestClassifier, XGBClassifier, Decision Tree Classifier	Voting Method: hard
Stacking	ExtraTreesClassifier, RandomForestClassifier, XGBClassifier, Decision Tree Classifier		LinearRegression

These machine learning (ML) and ensemble learning (EL) models were evaluated on two types of datasets: the original dataset sourced from Kaggle and a concatenated dataset comprising both the Kaggle dataset and a synthetic dataset. The synthetic dataset was generated using the CTGAN generative AI technique. Pre-processing steps were applied to the datasets to normalize the values and convert categorical variables into numerical values, as illustrated in Figure 5. The datasets were then divided into training and testing datasets in the ratio 80% and 20%, respectively. The model was created using the training dataset and was tested using the testing dataset.

Figure 5. Model of proposed approach.

Our findings indicate that the ensemble learning models outperformed the individual machine learning models. However, the synthetic dataset contains more noisy data compared to the Kaggle dataset, resulting in lower accuracy. The accuracy, recall, and precision of the ML and EL models are given in Table 3.

Table 3. Performance evaluation of ML and EL models.

Method	Kaggle—Data Set			Synthetic Dataset
	Accuracy	Precision	Recall	Accuracy	Precision	Recall
LR	90.20	87.5	87.621	55.85	55.54	57.42
SVM	89.74	88.18	87.83	46.26	46.10	45.31
DT	97.44	97.27	97.27	69.95	69.75	70.19
NB	98.70	98.64	98.63	57.68	57.49	58.25
K-Neighbor	96.40	95.91	95.90	70.38	70.29	71.55
RF	99.16	99.09	99.09	75.66	74.33	74.50
Extra Tree	98.36	98.18	98.19	75.41	74.33	74.46
XGBoost	98.71	98.64	98.63	73.64	72.64	72.76
Bagging	98.95	98.86	98.86	73.52	72.92	72.95
Gradient Boosting	98.10	97.73	97.74	72.27	70.38	70.60
Voting	98.68	98.64	98.63	73.94	72.78	72.85
Stacking	99.36	99.32	99.32	74.75	73.77	73.87

5. Model Evaluation

The experiment was conducted in Google Colab Pro with Python 3 Google Compute Engine backend with 15 GB of GPU RAM and 12 GB of system RAM. The dataset was divided into training and testing in a ratio of 80% and 20%, respectively. Seven ML models, namely, LR, SVM, DT, NB, K-neighbor, RF, and extra tree, were tested on the Kaggle and synthetic dataset. The model performance was improved by implementing EL methods such as bagging, boosting, voting, and stacking [17]. The performance of the ML and EL models on the Kaggle and synthetic datasets is given in Table 3.

The performance of the models was evaluated based on the metrics accuracy, precision, and recall, as shown below:

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

The performance of the stacking ensemble model created with four base learners, namely, the extra tree classifier, random forest classifier, XGB classifier, and decision tree classifier, and with one meta-learner logistic regression obtained an accuracy of 99.36%, and it is the highest of all the ML and EL models, as shown in Figure 6. The proposed stacked ensemble model was compared with the existing models, as shown in Table 4. The proposed stacked ensemble model has the highest accuracy of 99.36% when compared to all the other existing models in [3,5,6,9,12]. However, the accuracy of the Bayes net algorithm discussed in [13] is greater as the author selected only specific features of the dataset for classification. In our proposed method, we utilize the full dataset for prediction. Further, the stacking ensemble model is optimized using Optuna, an automatic hyper-parameter tuning framework [18]. Optuna finds the optimal combination of hyper-parameters for the ML models. After tuning the stacked ensemble model, its accuracy increased to 99.43%. However, the accuracy of the synthetic dataset, as given in Table 4, is lower compared to the Kaggle dataset. In this synthetic dataset, the random forest (RF) model achieved the highest accuracy of 75.66%, outperforming the other ML and EL models.

Figure 6. Performance of ML and EL models.

Table 4. Comparison of proposed and existing models.

Reference	Model	Dataset	Accuracy
Elbasi et al. (2023) [13]	Bayes net algorithm	Kaggle dataset with feature selection	99.59%
S.P. Raja et al. (2022) [9]	Bagging	Kaggle with MRFE feature selection	97.29%
Biplob et al. (2024) [3]	XGBoost	Kaggle dataset	99.09%
Ramzan et al. (2023) [6]	KNN	Kaggle dataset	97.81%
Kiruthika et al. (2023) [5]	IDCSO-WLSTM	Kaggle dataset	92.68%
Bakthavatchalam et al. (2022) [12]	MLP	Kaggle dataset	98%
Proposed	Stacking ensemble with Optuna optimization	Kaggle dataset	99.43%

6. Conclusions

In this research, the performance of ML models and EL models are compared on both Kaggle and synthetic datasets. We found that EL models perform well on the Kaggle dataset; in particular, the stacked ensemble model created with four base learners—the extra tree classifier, random forest classifier, XGB classifier, and decision tree classifier—and with one meta-learner logistic regression outperformed the other ML and EL models. The performance of the stacked EL model is further improved using the Optuna optimizer. However, on the synthetic dataset generated using CTGAN, random forest achieved the highest accuracy, outperforming both the ML and EL models. This study highlights that EL models may not perform well if the dataset contains noisy data, as demonstrated by the lower accuracy in the synthetic dataset.