Using a One-Dimensional Convolutional Neural Network on Visible and Near-Infrared Spectroscopy to Improve Soil Phosphorus Prediction in Madagascar

Abstract

As a proximal soil sensing technique, laboratory visible and near-infrared (Vis-NIR) spectroscopy is a promising tool for the quantitative estimation of soil properties. However, there remain challenges for predicting soil phosphorus (P) content and availability, which requires a reliable model applicable for different land-use systems to upscale. Recently, a one-dimensional convolutional neural network (1D-CNN) corresponding to the spectral information of soil was developed to considerably improve the accuracy of soil property predictions. The present study investigated the predictive ability of a 1D-CNN model to estimate soil available P (oxalate-extractable P; Pox) content in soils by comparing it with partial least squares (PLS) and random forest (RF) regressions using soil samples (n = 318) collected from natural (forest and non-forest) and cultivated (upland and flooded rice fields) systems in Madagascar. Overall, the 1D-CNN model showed the best predictive accuracy (R² = 0.878) with a highly accurate prediction ability (ratio of performance to the interquartile range = 2.492). Compared to the PLS model, the RF and 1D-CNN models indicated 4.37% and 23.77% relative improvement in root mean squared error values, respectively. Based on a sensitivity analysis, the important wavebands for predicting soil Pox were associated with iron (Fe) oxide, organic matter (OM), and water absorption, which were previously known wavelength regions for estimating P in soil. These results suggest that 1D-CNN corresponding spectral signatures can be expected to significantly improve the predictive ability for estimating soil available P (Pox) from Vis-NIR spectral data. Rapid and accurate estimation of available P content in soils using our results can be expected to contribute to effective fertilizer management in agriculture and the sustainable management of ecosystems. However, the 1D-CNN model will require a large dataset to extend its applicability to other regions of Madagascar. Thus, further updates should be tested in future studies using larger datasets from a wide range of ecosystems in the tropics.

1. Introduction

Phosphorus (P) deficiency is a major constraint for crop production in low-input agricultural systems in the tropics [1], and stems from the predominance of strongly weathered soils in which the availability of P is lowered by strong sorption to aluminum (Al) and iron (Fe) (hydr)oxides [2,3]. Even in natural ecosystems, limited soil available P can lead to a decline in the climax ecosystem by reducing biomass productivity [4]. Hence, rapid and quantitative information on soil fertility status is essential for improving biomass production in agricultural and natural ecosystems and developing sustainable land management. Among many extraction methods for evaluating available P, the method using acid ammonium oxalate solution is known to be suited to tropical weathered soils because it can solubilize the active reductant-soluble P, which is the dominant P pool for P cycling in tropical ecosystems [5,6].

Among the proximal soil-sensing techniques, laboratory visible and near-infrared (Vis-NIR) spectroscopy has long been adopted as a rapid, cost-effective, and quantitative analytical method for predicting soil properties [7]. Efforts have been made to characterize the chemical, physical, and mineralogical composition of soil using Vis-NIR spectra (400–2500 nm) [8]. However, making reliable predictions at larger scales requires a large spectral and reference dataset collected from various soil conditions, and this requires the development of a spectral library [9,10]. With the increasing number of data sources, a standard approach based on a spectral library is required [11]. Currently, soil spectral libraries are available at the country [11,12], continental [13,14], and global [15] scales. These spectral libraries allow the development of calibration models for predicting soil properties.

For the calibration method, the partial least squares (PLS) regression [16] has long been used as a standard approach in Vis-NIR spectroscopy because it can extract information on target soil properties from a spectral matrix with hundreds or even thousands of wavebands [17,18]. The predictive accuracy is improved by waveband selection in PLS analysis by removing redundant wavebands [19,20]. Our previous papers showed that the application of waveband selection improves the performance of PLS analysis for estimating the total carbon (TC) content of paddy soils in Madagascar [21,22,23]. However, as a linear multivariate calibration, the accuracy of PLS analysis tends to decrease because of the nonlinear nature of the relationship between spectral data and the dependent variable [24]. To overcome this issue, machine learning techniques have been increasingly adopted because they can account for the nonlinearity associated with soil spectral responses. The major machine learning approaches are artificial neural networks (ANNs) [25], support-vector machines (SVM) [26], and random forest (RF) [27,28,29].

More recently, deep learning approaches have rapidly evolved in machine learning techniques with promising results for data analysis in nature [30] and have also been tested for soil spectral calibration [31,32]. Among deep learning approaches, convolutional neural networks (CNNs) [33] are one of the most popular learning architectures. In the remote sensing of agriculture, CNN-based approaches have been applied to various images with different research objectives, such as land cover classification [34], weed mapping [35], and crop yield prediction [36]. CNN-based deep learning was originally suitable for 2D image data, but its applicability has been extended to one-dimensional (1D) spectral data in recent years. For example, Padarian et al. [32] demonstrated that the CNN model, by converting the 1D soil spectra into a 2D spectrogram as input, can predict the soil TC, cation exchange capacity (CEC), clay, sand, and pH with better accuracy than conventional methods (PLS and Cubist). Ng et al. [31] developed a 1D-CNN model and compared it with a 2D-CNN model to estimate the major soil properties (TC, organic carbon (OC), CEC, clay, sand, and pH)) based on the Kellogg Soil Survey Laboratory (KSSL) database. The results indicated that the 1D-CNN model was more effective than the 2D-CNN model. For areas not included in the soil spectral libraries, a local model can be developed or improved by transfer learning from the global model [37]. However, the prediction accuracy for P is lower than that for other soil components, and thus, improvements in predictive accuracy are required. Tsakiridis et al. [38] reported that the R² for soluble P in sodium hydrogen carbonate was 0.42, while the R² for clay content, soil organic carbon, and total nitrogen was 0.86, 0.86, and 0.83, respectively, using the visible, near-infrared and shortwave-infrared (VNIR-SWIR) spectra of the Land Use and Coverage Area Frame Survey (LUCAS) soil spectral library in combination with a localized multi-channel 1-D CNN model.

As mentioned above, state-of-the-art deep learning approaches with soil spectral libraries are currently available to construct models for predicting soil properties at local, regional, and global scales. However, soil P predictions remain challenging despite their importance to the sustainable management of agricultural systems aimed at addressing both soil degradation in the tropics and the environmental impacts of its excess use [39]. This is because soil P has no specific absorption in the Vis-NIR wavelength region [40]. Our previous studies in a range of rice cultivation soils in Madagascar identified the important wavelength for estimation and developed a reliable prediction model for oxalate-extractable P in soils, which is a suitable indicator of P availability for rice in the region [41], using PLS analysis with waveband selection [23,42]. Furthermore, different P cycling between natural and cultivated systems resulted in different soil P forms, which consequently decreased the accuracy of a common model in both land-use systems (cultivated vs. natural) [42]. The estimation accuracy should be improved by applying a comprehensive model to soil diagnosis in farm fields and other ecosystems for data compatibility and usability.

Therefore, the purpose of the present study was to develop a model based on the 1D-CNN architecture for estimating the soil available P content (Pox) in Madagascar. We used the dataset from our previous reports [23,42] because in Madagascar, only a small number of qualified datasets have been recorded in the soil spectral libraries: continent scale (n = 82) [13] and global scale (n = 18) [15]. Furthermore, because most soil spectral libraries use total P (TP) as reference data, the transfer learning approach cannot be applied to localize the model to Madagascar. Therefore, we compared the predictive ability of the 1D-CNN model to the previously used regression approaches, PLS and RF. We also performed a sensitivity analysis to identify the important wavebands used by the CNN model to predict soil Pox, and then evaluated the importance of the wavebands showing high sensitivity compared to PLS and RF.

2. Materials and Methods

2.1. Study Site and Dataset

The combined dataset included soil Pox content and spectral measurement data from 318 sampling points from the central highlands and eastern forest regions in Madagascar (Figure 1). Soil sampling was carried out in cultivated (n = 244) and natural (n = 74) systems. The climate in the area is sub-humid to humid tropics with an annual rainfall of 1381–2500 mm and a mean annual temperature of 16.9–24.0°C, depending on the elevation (110–1667 m above sea level). The area is dominated by Geric Ferralsols, which are generally acidic with low available P [43,44].

In the central highlands, soil sampling was performed in farmers’ fields under irrigated (n = 173) and upland (n = 71) rice systems in 2015–2016 and 2018–2019. Surface soil samples were collected at a depth of 0–15 cm as composites of three to four cores in each field. In the eastern forest region, 74 soil samples were collected in 2014 and 2015 from forest (n = 16) and non-forest (n = 58) sites. The non-forest sites included fallow and degraded land. The soil samples were air-dried for 14 days and sieved to <2 mm. Soil P was extracted using the acid ammonium oxalate method [45], and the concentration of P in the oxalate extraction was analyzed using the malachite green colorimetric method [46].

2.2. Spectral Measurements and Preprocessing

Spectral measurements of soil samples were taken in dark rooms at the Japan International Research Center for Agricultural Science (JIRCAS) and the Laboratoire des Radioisotopes, Antananarivo University, Madagascar, using ASD FieldSpec 4 Hi-Res (ASD Inc., Longmont, CO, USA) and an ASD contact probe. Compared to multispectral sensors, the hyperspectral soil reflectance data in laboratory Vis-NIR spectroscopy has some advantages such as waveband richness, sharpness of wavebands, and spectral continuity [47]. The ASD FieldSpec measures the spectral reflectance in the 350–2500 nm wavelength region. The spectral resolution (full width at half maximum; FWHM) was 3 nm in the 350–2500 nm range and 6 nm in the 1000–2500 nm range, which were resampled to 1 nm resolution wavelengths for output data using the cubic spline interpolation function in ASD software (RS3 for Windows; ASD Inc.). A Spectralon (Labsphere Inc., Sutton, NH, USA) reference panel (white reference) was used to optimize the ASD instrument prior to taking Vis-NIR reflectance measurements for each sample.

Bulk soil samples (<2 mm) were filled into an optical-glass Petri dish with a diameter of 85 mm and pressed to form a layer about 19 mm tick. The soil surface reflectance measurements were done on the surface for 25 times with five replications for each soil sample, and the spectral readings were averaged.

During preprocessing, spectral data were initially converted from reflectance (R) to absorbance (A = log(1/R)). Then, the absorbance spectra were converted to the first derivative absorbance (FDA) spectra using a Savitzky-Golay smoothing filter [48] with a third-order, 15-band moving polynomial. In addition, FDA spectra were standardized using a standard normal variate transform (SNV) to reduce the particle size effect [49]. Finally, the FDA data of both edge wavelength regions (350–419 nm and 2401–2500 nm) were removed because of the low signal-to-noise ratios in the instruments. Thus, the remaining 1981 spectral bands between 420 and 2400 nm were used in the analyses.

2.3. Model Development

2.3.1. Partial Least Squares (PLS) Regression

PLS is one of the most widely used linear regression approaches to analyze high-dimensional datasets, such as hyperspectral and Vis-NIR data, because it combines all available waveband information without multicollinearity issues, unlike standard multiple linear regression analysis. The PLS regression treats each waveband as an independent explanatory variable for estimating the response variable of the target component (soil Pox in the present study). The regression coefficient of PLS is computed by weighting the optimal number of PLS factors; it is also called a weighted coefficient (β_w). Here, a leave-one-out cross-validation (CV) method was used to select the optimal number of PLS factors to be included in the regression model. The important wavebands can be determined by the high absolute value of β_w.

2.3.2. Random Forest (RF) Regression

RF is a machine-learning approach that uses a non-linear statistical ensemble method that builds a large number of decision trees (ntree) for classification or regression and then averages them [50]. This approach evaluates the relationship between explanatory and response variables using a set of decision rules constructed by recursively dividing the input space into smaller regions. In the procedure, the model is developed with a set of trees by selecting a subset based on a bagging approach. Each tree grows until it reaches a certain minimum number of nodes (node size). Then, the remaining subset (called the out-of-bag (OOB) sample) is used for internal cross-validation to assess the average accuracy and error rate over all predictions [51]. In addition, the OOB is used to estimate the variable (feature) importance. Finally, the output class is calculated using the maximum votes from the ntree in the forest. In this study, we set ntree = 5000 and node size = 5 to construct the RF model. The importance of spectral wavebands in the model was evaluated by the Gini impurity, which was calculated by summing all decreases in Gini impurity at each tree node split, normalized by the number of trees.

2.3.3. One-Dimensional Convolutional Neural Network (1D-CNN)

CNNs are the most widely used architectures in deep learning approaches. CNN architectures are generally composed of an input layer, several hidden layers (convolution layers, pooling, and fully connected), and an output layer. Among the CNNs, 1D-CNN has an input layer and 1D filters on the convolution layers suitable for one-dimensional spectral data [52]. In the present study, we used the 1D-CNN architecture proposed by Ng et al. [31] (Figure 2) because the structure is simple and well described, and there are existing reports on other soil properties. This is good to compare the accuracy of our newly performed soil oxalate-P estimation with the results of other soil properties. The architecture included 10 hidden layers (

Table 1. Architecture of one-dimensional convolutional neural network (CNN).

Type	Filter Size	No. of Filters	Activation
Convolutional	20	32	ReLU
Max-pooling	2	-	-
Convolutional	20	64	ReLU
Max-pooling	5	-	-
Convolutional	20	128	ReLU
Max-pooling	5	-	-
Convolutional	20	256	ReLU
Max-pooling	5	-	-
Dropout (0.4)	-	-	-
Flatten	-	-	-
Fully-connected	-	100	ReLU
Dropout (0.2)	-	-	-
Fully-connected	-	1	Linear

ReLU, rectified linear unit.

): four convolutional layers, four max-pooling layers, and two fully connected layers. The activation function used a rectified linear unit (ReLU) for all hidden layers [53]. Two dropout rates of 0.4 and 0.2 were used to avoid overfitting [31].

2.4. Data Handling and Implementation

Our whole dataset (n = 318) was split into training (n = 238) and test (n = 80) subsets. The training data were used to develop the PLS, RF, and 1D-CNN models. Then, the models were applied to the test dataset to assess the predictive abilities of the models for soil Pox prediction.

Table 2. Summary statistics of soil oxalate-extractable P data.

Dataset	System	n	Min	Max	Median	Mean	SD
Training	All	238	21.9	1172.0	67.7	214.7	278.0
	Cultivated	183	23.7	1172.0	106.0	268.7	296.5
	Natural	55	21.9	53.9	34.8	35.1	7.2
Test	All	80	22.3	1225.2	68.5	220.9	290.0
	Cultivated	62	22.3	1225.2	106.2	274.8	309.6
	Natural	18	22.9	57.9	33.8	35.5	9.5

n, number of samples; SD, standard deviation

summarizes the minimum, maximum, median, mean, and standard deviation (SD) values of Pox (mg P kg^–1) for the training (n = 238) and test (n = 80) datasets, including data collected from different land-use systems (natural and cultivated). Soils in cultivated systems showed higher Pox values and wider ranges compared with soils in natural systems, probably due to the effect of fertilization on cultivated land [42].

The data distributions of soil Pox for all systems in the training and test datasets are shown in Figure 3. The data range of soil Pox content was similar for the training (21.9–1172.0 mg P kg^–1) and test (22.3–1225.2 mg P kg^–1) datasets. Moreover, the SD (278.0 and 290.0 mg P kg^–1) showed similar values. This is important for correct validation by applying the model created with the training data to the test data. The data distribution was left-skewed because of the predominance of strongly weathered soils in the region [43,44], with higher mean (214.7 and 220.9 mg P kg^–1 for training and test datasets, respectively) than median (67.7 and 68.5 mg P kg^–1) values. Theoretically, the SD, data range, and distribution pattern affect the accuracy of the regression analysis. Compared to previous studies (121–991 mg P kg^–1) [54], our data set included a larger variance of Pox values, which can be expected to improve the predictive ability of a model.

In the present study, PLS and RF models were implemented using R ver. 3.6.3 [55] with the packages ‘pls’ ver. 2.7-2 [56] and ‘randomForest’ ver. 4.6-14 [57]. The CNN was implemented using Python ver. 3.6.9 [58] with the Keras library ver. 2.2.4 [59], and TensorFlow ver. 1.14.0 [60] backend. All computations were performed on a desktop PC with Intel^® CoreTM i9-9900X 3.5 GHz processor, 64 GB RAM, and the NVIDIA GeForce RTX 2080 Ti.

2.5. Predictive Accuracy Evaluation

To assess the predictive accuracy of the PLS, RF, and 1D-CNN models, a 5-fold cross-validation was performed in the training dataset (n = 238). In the procedure, the training data were split randomly into 5-folds. Each model was built on 4 (=5 − 1) folds, and then the error of the remaining 1-fold was recorded as validation data. The process was repeated until each of the 5-folds served as a validation data set. The mean values of the coefficient of determination (R²), root mean squared error (RMSE), and bias were used to assess model accuracy. The RMSE and bias were defined as

$$\mathrm{RMSE}=\sqrt{\frac{{\sum }_{i=1}^n{\left(Y_i-{\widehat{Y}}_i\right)}^2}{n}},$$

(1)

$$\mathrm{bias}=\frac{{\sum }_{i=1}^n\left({\widehat{Y}}_i-Y_i\right)}{n},$$

(2)

where Y and Ŷ are the observed and predicted soil Pox values, respectively, and n is the total number of observations i.

Moreover, the models were applied to the test dataset (n = 80), and then the predictive ability was evaluated from the R², RMSE, bias, and the ratio of performance to the interquartile range (RPIQ) [61]. The RPIQ is an auxiliary indicator of the model, and the performance of predictive ability and future reliability can be judged as follows: (i) >2.5: excellent model, (ii) 2.0–2.5: very good model with predictive ability, (iii) 1.7–2.0: good model, (iv) 1.4–1.7: fair model in need of some improvement, and (v) <1.4: model with very poor predictive ability [62].

2.6. Sensitivity Analysis of 1D-CNN Model for Evaluating Important Wavebands

In the PLS and RF models, the importance of wavebands for estimating soil Pox can be assessed using PLS regression coefficients and RF importance (Gini purity index). However, it is difficult to directly evaluate the important wavebands in the 1D-CNN model. To assess the importance of wavebands in the 1D-CNN model, a sensitivity analysis was performed using the method proposed by Ng et al. [31], in which the sensitivity is calculated as a function of the variance of the model for each window of spectra [63]:

$$S_i=\frac{V\left(f\left(X_1,\dots ,X_2,\dots ,X_n\right)-f\left(\overline{X}\right)\right)}{V\left(Y\right)},$$

(3)

where V is the variation calculation, f(X₁,…X₂,…, X_n) is the prediction of spectra due to variation in waveband i with the other wavebands held constant on average, $f\left(\overline{X}\right)$ is the prediction value using the mean values of the spectra, and Y is the observed value of the soil Pox. In essence, this approach compares the sensitivity of the wavebands and calculates how the model changes [31].

3. Results

3.1. Comparison of Predictive Abilities in PLS, RF, and 1D-CNN Models

To compare the predictive accuracy in the PLS, RF, and 1D-CNN models using FDA spectral datasets, the mean values of R², RMSE, bias, and the relative improvement (%RMSE) from 5-fold cross-validation were compared in the training dataset (n = 238) (

Table 3. Mean values of R², RMSE, bias, and relative improvement (%RMSE) from 5-fold cross validation in the training dataset for developing the partial least squares (PLS), random forest (RF), and one dimensional convolutional neural network (1D-CNN) models.

Model	R2	RMSE	Bias	%RMSE 1
PLS	0.827	114.854	16.577	-
RF	0.842	108.820	13.517	5.254
1D-CNN	0.989	35.636	-2.202	68.973

¹ Relative improvement (%RMSE) of the RF and CNN models in comparison to PLS model. RMSE, root mean squared error.

). Computational times for developing the PLS, RF, and 1D-CNN models were 3.9, 264.1, and 5908.45 s (CPU time), respectively. Clearly, the 1D-CNN required a lot of learning time to develop the model, but it showed much better predictive accuracy (R² = 0.989, RMSE = 35.636) than the RF (R² = 0.842, RMSE = 108.820) and PLS (R² = 0.827, RMSE = 114.854) models. Compared with the RMSE values of the PLS model, the RF model showed a slight improvement in accuracy (%RMSE = 5.254%), while the 1D-CNN model showed a significant improvement (68.973%).

In addition, to confirm the reliability of the models, predictive ability was assessed from the relationship between the observed and predicted soil Pox content from the PLS, RF, and 1D-CNN models in the individual test dataset (n = 80) (Figure 4). The R², RMSE, bias, RPIQ, and the relative improvement in RMSE (%RMSE) are summarized in

Table 4. Comparison of predictive abilities in the partial least squares (PLS), random forest (RF), and one dimensional convolutional neural network (1D-CNN) models.

Model	R2	RMSE	Bias	RPIQ	%RMSE 1
PLS	0.792	132.694	15.606	1.900	-
RF	0.808	126.894	11.884	1.986	4.371
1D-CNN	0.878	101.154	−4.035	2.492	23.769

¹ Relative improvement (%RMSE) of the RF and CNN models in comparison to PLS model. RMSE, root mean squared error; RPIQ, ratio of performance to the interquartile range.

. Similarly to the cross-validation results in the training dataset, the 1D-CNN showed higher predictive accuracy (R² = 0.878, RMSE = 101.154) than the RF (R² = 0.808, RMSE = 126.894) and PLS (R² = 0.792, RMSE = 132.694) models. Compared with the PLS model, the RF and 1D-CNN models indicated a 4.37% and 23.77% relative improvement in %RMSE, respectively. Based on the RPIQ values, the PLS and RF models (RPIQ < 2.0) could be judged as good models, but they required further work to improve quality and increase future applicability, while the 1D-CNN model (RPIQ = 2.492) was considered to be a very good model with high predictive ability.

3.2. Important Wavelengths

The importance of each waveband in the models for soil Pox prediction was assessed. Figure 5 shows the FDA spectra in logarithmic form to easily refer to the wavelength characteristics depending on the soil P values, and the absolute values of PLS coefficients, RF importance (Gini purity index), and 1D-CNN sensitivity (s) as indicators of the importance of each waveband in the soil Pox estimation models.

The important wavebands for the PLS model were revealed in the visible (433, 446, 509, 550, and 590 nm) and NIR (1001, 1412, 1904, and 2219 nm) wavelength regions. The RF model showed constant values over the entire wavelength range, but the 446, 1336, 1366, 2365, and 2397 nm were particularly important. In the 1D-CNN model, high sensitivity was observed in the narrow and limited wavelength regions, with peaks at 432, 590, 1433, and 2274 nm. The three peaks (432, 590, and 1433 nm) did not exactly match but also showed a high value in PLS coefficients, and thus were considered to be important for estimating soil Pox.

4. Discussion

As a deep learning approach suitable for 1D spectral data, 1D-CNN with Vis-NIR spectral data was applied to predict soil Pox content in Madagascar, and its predictive ability was compared with that of PLS and RF models. The 5-fold cross-validation in the training dataset demonstrated that the 1D-CNN model can estimate soil Pox from Vis-NIR spectral data with improved accuracy compared to the other common methods of PLS and RF. Even though our soil samples were collected from different land- use systems, the best predictive ability in the 1D-CNN model was also confirmed in the individual test dataset. These results indicated that a comprehensive model to predict Pox in soils with high accuracy could be developed irrespective of land use systems using a deep learning approach with a 1D-CNN model rather than PLS (conventional method) and RF (another machine learning approach). Similarly, the lowest bias was observed in the 1D-CNN model. Based on the RPIQ value, our 1D-CNN model could be considered to have a very good predictive ability, and therefore could provide high-quality results in future applications. These results confirm that the CNN-based model can outperform PLS and other machine learning approaches for estimating soil properties, as suggested by previous studies [31,32]. Another update from our previous report using PLS regression analysis [42] was that the 1D-CNN model enabled the prediction of soil P as a single model, even based on the dataset collected from different land-use systems in Madagascar. These improvements are considered to be advances in the holistic understanding of soil P dynamics and their rational management in agriculture and natural ecosystems in Madagascar. Furthermore, the timely and accurate prediction of available P content in soils using our 1D-CNN model can be expected to contribute to effective fertilizer management in agriculture and the sustainable management of ecosystems.

We also assessed the important wavebands of the 1D-CNN model using a sensitivity analysis method developed by Ng et al. [63]. Assessing the important specific spectral features and their regions may help clarify the relationship between spectral signatures and underlying molecular activity within the spectral wavelength regions used in the calibration [64]. In the Vis-NIR wavelength region, theoretically, there is no specific absorption by Pox; thus, differences in spectral shape due to P content cannot be clearly determined [65,66,67]. However, indirect correlations between P forms and spectrally active soil properties can be studied. For example, Pätzold et al. [68] reported that the most relevant variables for the estimation of soil P by PLS were selected based on the variable importance in projection (VIP) method, and that PLS regression coefficients were located around 500 nm and 2200–2400 nm. Our previous study in Madagascar identified important Pox regions at 454–660 nm and 1732–2312 nm based on five runs of the variable selection method genetic algorithm followed by PLS regression [23].

In the present study, our 1D-CNN model showed high sensitivity in the narrow and limited wavelength regions with peaks at 432, 590, 1433, and 2274 nm. Since these wavebands correspond to the regions reported previously [23,68], it can be suggested that soil Pox tends to be primarily associated with the visible (400–660 nm) and longer NIR (1700–2400 nm) wavelength regions. The visible part of the spectral region was dominated by absorption due to organic matter (OM) and electronic transitions of Fe, and the selected wavebands at 432 and 590 nm were attributed to absorption by ferric oxide and OM, respectively [40,69,70,71]. Our previous study also reported that oxalate-extractable Al and Fe were positively correlated with soil TC and organic P content, respectively [5]. The longer NIR wavelength region (1700–2400 nm) was dominated by a combination of bands from intense fundamental vibrations of O-H, C-H bonds, and Al metal-OH groups [7,72]. The waveband at 1433 nm was attributed to sorptive water, and that at 2274 nm was associated with gibbsite (Al-oxide mineral) [73]. The significance of the sorptive water in air-dried soils for predicting Pox was corroborated by our previous finding that active Al was clearly correlated with air-dried soil moisture content (Nishigaki et al., under review).

Overall, our findings were in agreement with those of earlier studies that reported that machine learning outperformed PLS regression. However, such approaches may not be suitable for all situations because the deep learning approach requires large, qualified datasets to develop good prediction models [31]. Therefore, many CNN-based deep learning approaches have been performed using a large topsoil spectral dataset, such as the KSSL database (n = 14,594) [31] or LUCAS soil spectral libraries (n = 19,036) [32,37,38,74]. Our dataset collected from the central and eastern regions of Madagascar was relatively small (n = 318). This is a limitation of our study for developing a robust model on a larger spatial scale. However, the Pox targeted in our study was more highly correlated with other soil properties (e.g., oxalate-extractable Al and Fe, and soil pH) than total P or easily soluble P [5]. This possibly resulted in a higher prediction accuracy using spectral data for Pox than that achieved in previous reports. Further evaluation by updating the model with a larger dataset will be required in future studies to map and assess the spatial distribution of soil Pox status on a larger spatial scale in Madagascar.

5. Conclusions

The present study investigated the performance of a 1D-CNN model through comparison with two conventional methods, PLS and RF, for estimating soil Pox content with Vis-NIR spectral data in soil samples collected from cultivated (upland and flooded rice fields) and natural (forest and non-forest) systems in Madagascar. The main conclusions are as follows:

• With the potential to provide high predictive ability and performance in deep learning approaches, Vis-NIR spectroscopy with 1D-CNN is a promising method for predicting soil Pox content.
• Our 1D-CNN model provided the best predictive ability to estimate soil Pox content compared with the PLS and RF models.
• The RPIQ value from the 1D-CNN is suggested to be a very good model with high predictive ability for future applicability.
• The important wavebands from the sensitivity analysis of the 1D-CNN model were revealed in the visible region (432 and 590 nm) associated with Fe-oxides and diverse functional groups in soil OM; at 1433 nm, associated with water absorption; and at around 2270 nm with gibbsite (Al oxide mineral). These wavelength regions are known to be of high importance in the PLS model, and are in line with previous studies.
• The 1D-CNN model we developed allowed soil P prediction based on a single model, even using data from different land-use systems.

Our findings illustrate the potential of deep learning approaches for predicting soil P availability using a proximal sensing technique. The main contribution from our study is the demonstration of a 1D-CNN model that can be applied to soil Pox prediction in different ecosystems from the central to eastern regions of Madagascar. Moreover, rapid and accurate prediction of soil Pox using our results can be expected to contribute to effective fertilizer management in agriculture and the sustainable management of ecosystems. Nevertheless, the 1D-CNN approach employed in this study should be updated with a larger dataset and further evaluated on a larger spatial scale in Madagascar.