Enhancing Soil Fertility Prediction Through Federated Learning on IoT-Generated Datasets with a Feature Selection Perspective ^†

Abstract

Introduction: Fertile soil has a balanced pH and nutrient profile (potassium, phosphorus, and nitrogen), water retention capability, and organic substances. Fertile soil allows for better plant growth, leading to better production. The soil fertility requirements vary from crop to crop. So, it is essential to identify the soil fertility level according to the crop type. Objective: The objective of this paper is to develop a robust model that is capable of predicting the soil fertility. The model is integrated with IoT-generated data and federated learning-based feature selection techniques to improve the accuracy of the dataset. Materials/Methods: Different feature selection techniques were applied to the dataset. Then, we applied machine learning algorithms such as logistic regression, decision tree, and naïve Bayes, as well as their combinations to analyze and improve the performance. The federated learning approach was implemented to train the local models using the individual partitioned datasets. Each local model of the client shared the cryptic output weight and bias without sharing the raw data. There was a centralized model at the server end that collected these weights and biases, preserving data privacy. These collected data were aggregated and applied to find the least square error (LSE). Then, a gradient descent curve (GDC) was applied to identify the optimized weight and bias, which were fed back again to improve the accuracy of the predictions. Result: From our experimental observations, we analyzed the performance metrics of different ML classifiers, and it was revealed that the ensemble of logistic regression and decision tree had a better performance than the other models. One of our client models generates weight and bias with a precision of 87%, an accuracy of 87%, a recall of 87%, and an F1-score of 86%. Further, we collected two of our client system model outcomes from a server model and applied the LSE to identify the optimal W and B. In future work, we wll improve the performance of our model with a recursive approach by verifying the W and B at the client model in a feedback process.

1. Introduction

Crop production is mainly based on soil and weather conditions, in which soil health plays a very important role in crop cultivation. Soil nutrients need to vary from crop to crop. Hence, it is important to obtain updated knowledge of the soil nutrients so that suitable crops can be selected, and nutrient deficiencies can be addressed.

In this approach, our main goal is to identify whether the soil is fertile or not. This is because fertile soil will have a balanced nutrient profile. When the soil is fertile, crops can grow better, leading to better production, along with an optimized temperature and the proper availability of water and organic matter.

To identify the fertility status of a soil sample, we prepared a prediction model. Before training a model, feature selection is very important in the preprocessing phase. So, different selection techniques such as variance threshold, chi-square, recursive feature elimination (RFE), PCA loadings, random forest (RF) importance, mutual information, and lasso coefficients were applied. Once the feature selection and preprocessing were conducted, the dataset was used to train a model. In reality, many clients may not be interested in sharing their data with others. So, for the better security of the data of each client and also to provide a better prediction model, a novel approach was implemented.

Federated learning allows us to maintain data security at the client end and also prepare a model at the server end with high accuracy. So, the individual models were trained at the client end, and then the results were encrypted and shared with the server. The server contained the cumulative details of all the predictions of the local models and were used for further analysis. Here, some aggregation and filtering techniques were applied to the dataset collection so that the model at the server end improved its performance in predictions.

In this regard, different machine learning techniques and an ensemble of their combinations were used for predictions. So, a balanced integration of IoT sensors and federated learning approaches allows for the maintenance of data security for each client and also improves the performance of the centralized server model. The local clients’ training models at their end generate weight and bias. There is a centralized model at the server end that collects these weights and biases, aggregates them, and supplies them back to improve their performance. This approach maintains data privacy and also improves accuracy.

Then, soil mineral profile data, regarding moisture levels, pH, nutrient levels, etc., were collected through the IoT sensors from different agricultural fields. These data were supplied as input to the centralized model, which predicts whether the soil is fertile or not.

2. Literature Review

Soil fertility analysis is essential for identifying suitable crops for cultivation. Fertile soil is useful and can produce better crops, and by identifying when the soil fertility rate has decreased, we can take the necessary steps to make it fertile and ready for crop cultivation. Here, we review the relevant research papers to identify the work conducted so far on soil fertility analysis and crop predictions. In [1], the authors applied an explainable AI technique based on random forest to predict the soil fertility. An analysis was conducted on a dataset of the European Union with an accuracy of 97% [2], in which the authors introduced a weighted K-mean algorithm to evaluate the soil fertility and integrated the analytic hierarchy process to obtain the weights and attributes of soil nutrient. It was observed that this approach had a better accuracy of 96.91% compared to traditional K-mean clustering. In [3], the authors analyzed an Indian agriculture dataset for soil fertility using classification techniques. It was found that support vector machine obtained a higher accuracy of 80% compared to other machine learning classifiers. In [4], the authors found that prediction was achieved with an accuracy of 78% when analyzing soil fertility using the multiple linear regression (MLR) method. A model was proposed that demonstrated effective analysis and prediction using the MLR technique [5]. The authors reviewed 1328 articles, out of which 20 articles were chosen based on their efficiency in making predictions. Soil was analyzed for its minerals, water, air, and organic matter using different machine learning techniques such as linear regression, support vector machine, K-nearest neighbors, decision tree, artificial neural networks, naïve Bayes, etc. Random forest and deep learning methods had better performance than the others [6]. The authors proposed a framework using an ANN for soil nutrient quality analysis. It was observed that the ANN with Relu and Tanh gave an average accuracy of 90% in fertility classification. The authors analyzed the area of Uttarakhand state, India, to identify the soil’s fertility needs to avoid the unnecessary use of fertilizers [7]. The authors studied the nutrient levels of mulberry gardens in Tamil Nadu. They applied an extreme learning machine (ELM) with various activation functions and observed that the soil had normal electric conductivity, abundant in nitrogen, potassium, and sulfur, but there was a deficiency in magnesium, copper, zinc, etc. [8]. The authors proposed a model for testing whether a soil is fertile or not in terms of 1 or 0. Then, the output is compared with actual values. Then, the model predicts the crops based on the available soil features. Various machine learning techniques were applied, of which random forest obtained 100% accuracy in prediction [9]. The authors applied machine learning techniques such as decision tree, K-nearest neighbor, support vector machine, etc., to identify the soil fertility and in turn, improve the crop productivity. The performance was measured in terms of cross validation, absolute error, and accuracy. The accuracy rate was 99% for the decision tree technique [10]. The authors applied an extreme learning machine (ELM) along with many activation functions to classify the soil features. Here, the Gaussian radial basis function was applied and obtained a higher accuracy of 90%. So, it optimized the use of nutrients and also reduced the fertilizer purchasing cost.

Overall, in this review, we observe that the soil analysis to determine fertility has been conducted using different learning techniques and methodologies to measure the fertility levels to identify the suitable crops, as well as the fertility needs, so there can be less use of fertilizers and maintenance of the soil health.

3. Materials and Methods

The methods at the client and server were different. At the client, the model was applied uniformly to its existing dataset. We used feature selection techniques on a dataset in each client to improve the model performance, reduce the complexity, and enhance the data quality. Then, we applied a model consisting of techniques such as logistic regression, decision tree, naïve Bayes, and their combinations to identify the best model fits for an optimized analysis. Then, the encrypted results were sent to the server-end model.

3.1. Feature Section Techniques

We applied different features and compared their performance. The variance threshold was used to select the features whose variance exceeded the threshold value based on the given formula in Equation (1):

$$Var\left(X_j\right)>threshold.$$

(1)

The chi-square was calculated to identify the dependencies between a feature and the target variables shown in Equation (2):

$${\mathsf{\chi }}^2=\sum {{(O}_i-E_i)}^2/E_i,$$

(2)

where O_i is the observed frequency and $E_i$ is the expected frequency. The recursive feature elimination was used to eliminate the least important features based on the model coefficient. In addition, the PCA loadings were applied to identify the important features in the dataset. Random forest was applied to identify the importance of certain features based on entropy, as shown in Equation (3):

$$Importance\left(X_j\right)={\sum }_{t\in T}{∆I}_t\left(X_j\right),$$

(3)

where ${∆I}_t\left(X_j\right)$ is the decrease in the impurity for feature X_j in tree t. Mutual information was applied to measure the amount of information contained by our target variables. The lasso coefficients were obtained by reducing the coefficients to zero using regularization. Recursive feature elimination with cross-validation support was used to identify the best number of features. Boruta support was applied along with random forest to identify the relevant features.

3.2. Machine Learning Techniques

Logistic regression was applied for binary classification to predict the probability of the target variable soil fertility status, as shown in Equation (4):

$$P\left(Y=1|X\right)={\displaystyle \frac{1}{{1+e}^{-({\beta }_0+{\beta }_1{X}_1+\dots +{\beta }_p{X}_p)}}}.$$

(4)

Decision tree was used to split the whole dataset into subsets based on the features for prediction. The formula is shown in Equation (5):

$$y={\sum }_{m=1}^M\begin{array}{c}{c}_{m}1\left(x\in R_m\right)\\ \end{array},$$

(5)

where $c_m$ is the prediction for the region, and R_m is defined by the tree.

Naive Bayes was applied based on Bayes Theorem to identify the independence between the features, as shown in Equation (6):

$$P\left(Y|X\right)={\displaystyle \frac{P\left(X|Y\right)P\left(Y\right)}{P\left(X\right)}},$$

(6)

where $P\left(X|Y\right)$ is modeled as the product of individual feature probabilities.

Ensemble methods were applied by combining multiple models such as LR + DT, DT + NB, and LR + DT + NB to improve the overall accuracy. Here, the formula is shown in Equation (7):

$$\widehat{y}={\sum }_{m=1}^M{w}_m{\widehat{y}}_m\left(x\right),$$

(7)

where $w_m$ is the weight assigned to the prediction of the mth model in the ensemble.

3.3. Sensors with Arduino Board

We used IoT sensors along with an Arduino board for the collection of soil nutrient values, which were used as test samples for analysis.

The Arduino board was used to connect with sensors collecting the soil nutrient data. We used the NPK sensor to collect nitrogen, phosphorus, and potassium. The pH sensor was used to identify whether the soil was alkaline or acidic, and the EC sensor was used to identify the ability to conduct electricity. The NIR sensor was used to identify the amount of organic carbon in the soil, the ion-selective electrode for sulfur, and the X-ray fluorescence (XRF) sensor and the colorimetric RGB sensor for zinc, iron, copper, manganese, and boron. These data were fed to the model at the server end for testing and generating the result.

3.4. Research Questions

RQ1: How do the different feature selection techniques impact the performance of the machine learning model?

RQ2: Can a hybrid model of different classifiers improve overall the predictive accuracy and robustness for soil fertility prediction?

RQ3: How can we generate and share client model results with the central model while maintaining privacy?

RQ4: How can the central model improve its performance in prediction using optimization techniques?

4. Proposed Model

The proposed model shown in Figure 1 has three phases, of which Phase I comprises the local model and its training using its dataset at individual client systems. Phase II is applies a set of techniques to improve the accuracy of the server-end model. Further, Phase III focuses on verifying the centralized model with sample data collection using a sensory system. Figure 1 shows a detailed explanation of the working process of each local model. It represents the training of a machine learning model using a nutrient dataset to identify whether the soil is fertile or not.

4.1. Phase I

At the local client machine, a standard dataset for soil nutrients was used, which consists of 12 different features such as N, P, K, pH, EC, OC, S, Zn, Fe, Cu, Mn, B, and output. The dataset consists of 11,440 records, which represents whether the binary classification status of the soil is fertile or not in the feature output. The dataset was cleaned to remove duplicate and missing values, outliers, and noise, and some features were normalized to make it suitable for binary classification. Then, we applied feature selection techniques to reduce the overfitting and enhance the accuracy. We determined the variance threshold, chi-square, RFE ranking, PCA loadings, RF importance, mutual information, lasso coefficients, RFECV support, and Boruta support. Further, we applied logistic regression, decision tree, naïve Bayes, and their ensemble approaches. Then, we estimated the ranking of the techniques based on the accuracy, precision, recall, and F1-score. Then, based on the ranking, the best technique was selected to predict the soil fertility status with higher accuracy, as shown in Figure 1.

Figure 2 Phase I shows that there are multiple local models of individual clients, which generate the local results of only the weight, bias, and accuracy to supply to the server-end model without supplying the actual results. So, the results are kept securely at individual local machines without sharing with others.

4.2. Phase II

This phase consisted of a centralized model at a server that received the weight, bias, and accuracy from each client and stored them by combining them. The least square error method was applied to these collected details to identify the error data of each sample. Based on a minimum error and with an aggregation approach to the suitable weight, bias values were generated, which was further verified using a gradient decent curve. Then, it fed back to the client models to test the dataset repeatedly.

4.3. Phase III

This phase contained mainly soil nutrient collection from fields with the help of sensory devices. A set of sensors connected with an Arduino board collected nutrients such as nitrogen, phosphorus, potassium, pH, electrical conductivity, organic carbon, sulfur, zinc, iron, copper, manganese, and boron. The Arduino was connected to an NPK sensor, a pH sensor, an EC sensor, an NIR sensor, an ion-selective electrode sensor, and a colorimetric RGB Sensor with a supply of 5 volts. We used these sensors and collected the soil nutrients in a systematic way, which we used for testing the soil sample. There is a central model that holds the optimized weight and bias with the least error. The central model tested and analyzed the soil nutrients collected and identified its fertile state with a higher accuracy.

5. Results and Discussion

We initially applied feature selection techniques to verify their role in improving the performance of the fertility analysis. The research questions and the solution during implementation are discussed in a table of ranking, while analyzing different attributes.

5.1. Research Question #1

Is it possible to use the feature selection technique for soil fertility analysis? If so, how do the different feature selection techniques impact the performance of the machine learning model?

Solution: Yes, it is possible to use the various feature selection techniques for soil fertility analysis. Our experimental result revealed that the RFECV (recursive feature elimination with cross validation) was the best feature because it is one of the iterative methods. This FS algorithm removed the least important features from our dataset. We repeated this process to obtain the best features from our dataset. Features such N, P, pH, and Cu were included, which suggests these features are robust across multiple folds. It is the best because it accomplishes feature elimination as well as model validation.

Table 1. An analysis table representing the rankings for different features of the dataset.

	Variance Threshold	Chi-Square	RFE Ranking	PCA Loadings	RF Importance	Mutual Information	Lasso Coefficients	RFECV Support	Boruta Support
N	5982.230	11,447.547	9	0.107	0.550	0.494	0.005	TRUE	TRUE
P	482.034	1347.885	7	0.108	0.127	0.144	0.006	TRUE	TRUE
K	15,413.778	101.166	11	0.107	0.029	0.012	0.000	TRUE	FALSE
pH	0.216	0.223	2	0.143	0.048	0.077	0.000	TRUE	TRUE
EC	0.020	0.025	3	0.004	0.026	0.026	0.000	TRUE	FALSE
OC	0.710	1.420	10	0.151	0.024	0.000	0.000	TRUE	FALSE
S	19.551	10.896	4	0.166	0.026	0.000	0.001	TRUE	FALSE
Zn	3.584	14.134	5	0.167	0.029	0.005	0.000	TRUE	FALSE
Fe	9.661	7.822	6	0.177	0.038	0.028	0.003	TRUE	TRUE
Cu	0.217	3.891	1	0.113	0.039	0.049	0.000	TRUE	TRUE
Mn	18.459	5.506	8	0.179	0.034	0.000	0.000	TRUE	TRUE
B	0.325	1.350	12	0.120	0.030	0.032	0.073	FALSE	FALSE

shows the outcome for seven different techniques applied on 12 features, where we can identify their rankings.

In addition, we tried to represent the importance scores in the chart in Figure 3, which describes the importance scores of different techniques. It shows how each feature ranks according to the different feature selection techniques.

Further, the heat map generated and shown in Figure 4 implemented a multiple feature selection method to visualize the ranking of different features of our dataset. It compared all the results of the feature selection techniques and identified the most important features for our predictive model. Here, it showed that the soil nutrient properties were very important. Potassium is identified as a key factor for analysis, and based on cross validation, the ranking of nitrogen is high. Overall, by focusing on the soil properties, the prediction model can be optimized.

In the context of machine learning, the FS plays a vital role in model building and recognizing the important features of the dataset. In this paper, we used various techniques (such as variance threshold, chi-square, recursive feature elimination (RFE), PCA loadings, random forest (RF) importance, mutual information, and lasso coefficients) to suggest the best model. We estimated the rank of all the features as well as how they performed on the model accuracy, precision, recall, and F1-score. The chi-square as well as RF had high scores for some of the features as compared to other ones. The RFECV and Boruta were represented as 1 and 0 for true or false. With these, we obtain more consistent features for model development.

5.2. Research Question #2

Is it possible to make a hybridization model by considering the individual models for soil fertility? Can a hybrid model combining all these classifiers improve the overall predictive accuracy and robustness in a soil fertility prediction system?

Solution: In this paper, we took the baseline classifier, and the experimental result revealed that there was a significant difference in performance metrics when dealing with the variability of the dataset. We also considered the hybridization of models like logistic regression + decision tree, decision tree + naive Bayes, and logistic regression + decision tree + naive Bayes). The experimental result showed that the combination of logistic regression + decision tree performed well in comparison to other combinations. The performance metrics’ accuracy as well as precision increased. The ensemble model suggested one hybridization i.e., (LR + DT) to improve overall performance in soil fertility prediction.

Table 2. The performance analysis details of different algorithms and their ensemble approaches.

Techniques	Logistic Regression	Decision Tree	Naive Bayes	LR + DT	DT + NB	LR + DT + NB
Metrics
Accuracy	0.83	0.82	0.49	0.87	0.85	0.85
Precision	0.78	0.81	0.59	0.87	0.84	0.83
Recall	0.83	0.82	0.49	0.87	0.85	0.85
F1-Score	0.81	0.81	0.41	0.86	0.83	0.83

shows the techniques of logistic regression, DT, and NB along with their ensemble approaches applied for data analysis. These methods were verified for four measured parameters, including the accuracy, precision, recall, and F1-score, to identify the most suitable technique.

Figure 5 shows a comparative analysis using a bar chart based on precision, recall accuracy, and F1-score. It was observed that the naïve Bayes had low performance in each metric, whereas the ensemble of logistic regression with the decision tree had higher performance.

In addition, the performance comparison was conducted on the hybridization of naïve Bayes, decision tree, and logistic regression using a heat map and box plot shown in Figure 6.

The boxplot provided the accuracy of the best hybridization model. It determines which model is the best according to the accuracy of the performance metrics. The red identifies the highest accuracy (LR + DT).

Figure 7 represents the ROC curve and confusion matrices of individual methods along with the hybrid approach.

6. Conclusions and Future Work

It is observed that a necessary approach is applied at every stage to improve the performance of the model. At the client-end model, for the selection of appropriate features, seven different selection techniques were applied, and based on calculated Boruta support the feature selection was conducted. It allows us to have better generalization, reduces overfitting, removes highly correlated features, leads to a more reliable model, and improves the accuracy.

Then, the implementation of algorithms such as naïve Bayes, decision tree, logistic regression, and ensemble approaches allows us to analyze and improve the performance of the predictions. It is observed that the accuracy of the decision tree is 82%, and logistic regression is 83%, whereas the ensemble of LR + DT led to an accuracy of 87% in predicting the soil fertility.

Again, an optimized approach is incorporated at the server-end centralized model with the least square error and gradient decent curve to offer recursive feedback to clients to improve the weight and bias values. The implementation of the recursive feedback approach is in progress, which shall be the future work of contributions. Further, a sensory system was introduced at the server end to test a new sample nutrient value collected from fields.