Deep Learning-Enabled Dynamic Model for Nutrient Status Detection of Aquaponically Grown Plants

Abstract

Developing models to assess the nutrient status of plants at various growth stages is challenging due to the dynamic nature of plant development. Hence, this study encoded spatiotemporal information of plants within a single time-series model to precisely assess the nutrient status of aquaponically cultivated lettuce. In particular, the long short-term memory (LSTM) and deep autoencoder (DAE) approaches were combined to classify aquaponically grown lettuce plants according to their nutrient status. The proposed approach was validated using extensive sequential hyperspectral reflectance measurements acquired from lettuce leaves at different growth stages across the growing season. A DAE was used to extract distinct features from each sequential spectral dataset time step. These features were used as input to an LSTM model to classify lettuce grown across a gradient of nutrient levels. The results demonstrated that the LSTM outperformed the convolutional neural network (CNN) and multi-class support vector machine (MCSVM) approaches. Also, features selected by the DAE showed better performance compared to features extracted using both genetic algorithms (GAs) and sequential forward selection (SFS). The hybridization of deep autoencoder and long short-term memory (DAE-LSTM) obtained the highest overall classification accuracy of 94%. The suggested methodology presents a pathway to automating the process of nutrient status diagnosis throughout the entire plant life cycle, with the LSTM technique poised to assume a pivotal role in forthcoming time-series analyses for precision agriculture.

1. Introduction

Lettuce (Lactuca sativa L. var. longifolia) is a globally cultivated and consumed plant species. This common leafy vegetable is abundant in essential nutrients such as fiber, vitamins (A, B, C, and K), chlorophyll, and carotenoids, which are crucial in promoting human health [1]. Under the circumstances of growing lettuce in aquaponics systems, the bio-available nutrients from fish metabolism may be insufficient for the plant’s optimal growth, and the lettuce may experience nutritional deficiency disorders [2]. This can impact the quality and productivity of lettuce and its morphological characteristics [3,4]. For instance, Yang and Kim compared hydroponics and aquaponics for growing tomatoes, basil, and lettuce. Their findings demonstrated that the mean concentrations of all nutrients were markedly reduced in aquaponics compared to hydroponics [3]. Thus, it could be concluded that the plants in aquaponics systems need nutritional supplements for optimal quality and growth.

Traditionally, identifying nutrient deficiency depended on visual assessments or chemical techniques, which are known to be time-consuming, labor-intensive, and expensive. The rapid development of monitoring and machine learning technologies has recently provided innovative opportunities for real-time monitoring of plant nutritional status [5]. Spectroscopic monitoring methods have proven effective in identifying plants’ health status [6]. Spectral reflectance in the range of 350–2500 nm is mainly reliable in applications related to plant status detection, such as determining the contents of water, chlorophyll, and nutrients [7]. The spectral characteristics of plants serve as a dependable indicator of leaf surface qualities, internal structure, and biochemical characteristics (macro- and micro-nutrient) [8,9]. In addition to leveraging hyperspectral sensors to deepen insights into plant behaviors, notable progress in big data analysis and advancements in computational capabilities have ushered in novel opportunities for crafting cutting-edge techniques to extract plant information [10].

Recently, the integration of machine learning techniques, informed by spectral and image data inputs, has constructed diagnostic models for various crop traits. More recently, deep learning has emerged as a subclass of machine learning that addresses some of the problems of traditional machine learning, such as the problem of handcrafted features [11]. Deep learning has shown great success in various disciplines of the agricultural field thanks to the emergence of deep convolutional neural networks (DCNNs) [12]. CNNs learn automatically and select high-level features closely related to the target during the transfer learning process [13]. While convolutional neural networks (CNNs) have demonstrated impressive capabilities in diagnosing plant nutrient deficiencies, they may not be suitable for modeling dynamic systems, particularly time-series spectral datasets spanning the entire growth cycle. Long short-term memory (LSTM) networks have shown promise in analyzing dynamic systems across various applications [14,15], including the monitoring of quality features in lettuce using spectral time-series data [16] and the quantitative characterization of tobacco components using time-series spectral data [17]. Additionally, Dandıl et al. used the LSTM model with spectral data to diagnose pseudo-brain tumors, achieving high classification accuracy for brain tumors and normal brain tissue [18]. Also, Wang et al. combined Raman spectroscopic data with the LSTM network to classify kinds of blood species; the model achieved a high classification accuracy of more than 93% [19,20]. Irrespective of the model employed, the feature extraction method significantly influences the accuracy and robustness of the model. This involves extracting features that best represent the target group. Additionally, various dependable feature extraction techniques exist, including both traditional methods like GA and SFS, as well as more novel deep approaches like DAE [21]. Extracting features with DAE before applying classification approaches like LSTM or CNN substantially enhances the classification results and better modeling of spatial and spatiotemporal behavior [22].

Despite LSTM’s potential for modeling dynamic systems, a significant gap exists among various agricultural systems, particularly evident in the distinct characteristics of aquaponically grown plants compared to their counterparts. Consequently, developing models explicitly tailored for aquaponics plant datasets becomes vital. This study introduces a hybrid deep learning model that merges the strengths of long short-term memory (LSTM) and DAE to enable timely and precise diagnosis of nutrient status in aquaponically cultivated lettuce. In addition to LSTM, we propose a novel CNN model and utilize a support vector machine (SVM) model as a performance evaluation benchmark. To the best of our knowledge, this paper presents the first attempt to hybridize an autoencoder and LSTM model to detect the nutritional status of aquaponically grown plants using time-series spectral data. The outcomes of this study are expected to bolster the sustainability of aquaponics systems and promote their adoption as precision agriculture (PA) technology, thereby enhancing the intelligence of aquaponic systems and facilitating their deployment on a commercial scale.

2. Materials and Methods

2.1. Experimental Design and Set-Up

The proposed protocol to detect the nutrient status of aquaponically grown lettuce is depicted in Figure 1. The proposed methodology has two main stages: data collection and processing, followed by the modeling stage in which the data are divided, feature extraction, classification, and, finally, model performance evaluation. An aquaponics system was developed and built following the standard construction criteria. Given that lettuce is the predominant and widely favored crop in aquaponics systems, it was selected as the subject of investigation in this study’s research tests [23]. Lettuce seedlings were cultivated in a culture tub with four different levels of nitrogen (N), phosphorous (P), and potassium (K) using an aquaponics solution and nutritional supplements. The seedlings were transplanted into the tub cultures when they were two weeks old. The optimal growth of lettuce requires a full-strength nutrient solution containing 28, 36, and 2.4 g/L of N, P, and K, respectively [24]. However, the aquaponics system provides only 60% of N (17 g/L), 50% of P (18 g/L), and 5% of K (0.1) [3,25,26]. Accordingly, as shown in

Table 1. N, P, and K concentrations for each treatment.

Nutrient	Nutrient Concentration at Each Level, g/L
	Low	Medium	Optimal
N	20	23	28
P	23	28	36
K	0.6	1.1	2.4

, 9 nutritional levels were prepared (low N, medium N, optimal N, low P, medium P, optimal P, low K, medium K, optimal K). Figure 2 shows examples of the image dataset. The nutrient concentration was maintained by measuring pH (6.9) and electrical conductivity (0.1 dSm⁻¹) at the experiment’s beginning; if there was any daily change, the nutrient solutions were replenished.

2.2. Spectral Dataset Collection

Ten lettuce seedlings were assigned to each nutrient level, with a total of 90 plants. Leaf reflectance was measured starting from the third or fourth leaf during the growing season. A total of 8100 hyperspectral reflectance measurements were obtained from lettuce plants cultivated under various nutrient levels using a full-range hyperspectral ASD FieldSpec 4 Hi-Res (Analytical Spectral Devices Inc., Boulder, CO, USA) spectroradiometer. The total number of hyperspectral reflectance measurements and acquisition dates are summarized in

Table 2. Hyperspectral reflectance measurements and acquisition dates.

No.	Acquisition Date	Number of Measurements	Total
1	15 November 2023	900	8100
2	22 November 2023	900
3	3 December 2023	900
4	10 December 2023	900
5	16 December 2023	900
6	23 December 2023	900
7	28 December 2023	900
8	4 January 2024	900
9	8 January 2024	900

. The spectral range of the ASD detector was 350 to 2500 nm, with a 1 nm sampling interval between contiguous bands comprising 2151 points for each spectrum. The spectral dataset was partitioned into training (60%), validation (20%), and test (20%) sets. Model construction and data analysis were performed on the Kaggle platform, which offers free access to NVidia K80 GPUs in kernels. An Intel Core i7-3630QM processor operating at 2.4 GHz and 8 GB of RAM was used to test this platform on the computer. The CNN and LSTM modules from the TensorFlow library version 2.6.2 were employed for the classification task. Figure 3 shows the average spectra of lettuce plants grown in low (20, 23, 0.6 g/L), medium (23, 28, 1.1 g/L) and optimal (28, 36 and 2.4 g/L) nutritional levels of N, P and K, respectively. Interestingly, as the content of macronutrients in plants (such as nitrogen, phosphorus, and potassium) increases, reflectance in the blue bands (475 nm), green bands (530 nm), red bands (668 nm), and the red edge (717 nm) significantly decreases [2]. This phenomenon occurs due to higher nutrient levels, which lead to an increase in chlorophyll content and photosynthesis, resulting in greater absorption of visible light and reduced reflectance [2]. Reflectivity transitions from a negative correlation to a positive correlation from the beginning of the red edge (720 nm) to the near-infrared range (1300 nm), indicating that a higher presence of nutrients corresponds to increased reflectivity. This effect is largely attributed to elevated nutrient levels, which enhance canopy structure, leaf area index, water content, and biomass, thereby improving the absorption of chlorophyll and dry matter. Additionally, increased multi-photon backscattering in the near-infrared region leads to heightened near-infrared reflectance [2].

2.3. Deep Autoencoder (DAE)

The DAE is a deep-based methodology for feature extraction [22]. Autoencoders are unsupervised neural network approaches to building models based on unlabeled data. These approaches learn the representation of input datasets by training the neural network to discard non-target data (noise). This restructures the input data and reduces the loss between inputs and outputs. A typical autoencoder consists of an input layer, an output layer, and hidden layers. Autoencoding processes are separated into encoding, decoding, and reconstruction loss, as shown in Figure 4. Autoencoder-based feature extraction has been proven reliable in many scientific contributions. Hinton et al. were the first to demonstrate the effectiveness of autoencoders in dimensionality reduction (DR) on large datasets [27]. Han et al. developed FlowNet, a framework that uses autoencoders to extract features such as streamlines and stream surfaces [28]. Guo et al. introduced a sophisticated convolutional autoencoder that focuses on minimizing reconstruction and clustering losses to acquire embedded information for clustering [29].

In the encoding phase, the inputs ($x_1$, $x_2$, $x_3$, …… $x_n$) are high-dimensional vectors that are mapped to represent the low-dimensional bottleneck layer (h) after removing unimportant features, as shown in Equation (1).

$$h=f_1\left(w_{i}x+b_i\right)$$

(1)

where $w_i$ is the weight matrix, $b_i$ is a bias, and $f_1$ is an activation function.

During the decoding step, the bottleneck layer representation of ($h$) was used to generate the output $\widehat{x}$ that maps back to the reconstruction of $x$, as shown in Equation (2):

$$\widehat{x}=f_2(w_{j}h+b_j)$$

(2)

The activation function for the decoder is denoted as $f_2$. The weight matrix is denoted as $w_j$, the bias is represented by $b_j$, and $\widehat{x}$ represents the reconstructed input sample.

The reconstruction loss ($L$) is computed to minimize the discrepancy between the output and the input, as depicted in Equation (3):

$$L\left(x-\widehat{x}\right)={\displaystyle \frac{1}{n}}{\sum }_{n=1}^n\left|\widehat{x}-x\right|$$

(3)

where $x$ represents the input data, $\widehat{x}$ indicates the output data, and $n$ is the number of samples in the training dataset.

2.4. Genetic Algorithm (GA) and Sequential Forward Selection (SFS)-Based Feature Extraction

Two traditional methods, namely GA and SFS, were used as comparison benchmarks to evaluate the performance of the DAE. GA is one of the most popular data dimension reduction methods. It has shown remarkable performance when used to determine the chemical composition of plants compared to many techniques for selecting optimal wavelengths [30,31]. To obtain the optimal solution to the targeted problem, the genetic parameters of the GA, such as crossover rate, mutation rate, and population, must be adjusted. This work used a crossover rate of 0.8 and a mutation rate of 0.15, and the number of generations was assigned as 10. The sequential forward selection (SFS) method selects the optimal wavelengths without data loss or distortion, which gives it an additional advantage over its counterparts [32].

2.5. Long Short-Term Memory (LSTM)

Figure 5 depicts the primary constituents of a long short-term memory (LSTM) unit. LSTM is a recurrent neural network (RNN) type that can acquire knowledge about immediate and long-lasting relationships among time steps in sequential input. LSTM was initially proposed by Hochreiter and Schmidhuber (1997), who made fundamental changes to the architecture of standard RNNs by incorporating memory cells and a gate mechanism in hidden layers [33]. The LSTM model has proven its worth in addressing sequential problems such as time series, speech recognition, and signal analysis. Notably, LSTM is the foundation for most contemporary classification models. It has also been demonstrated that LSTM is relatively successful at classifying spectral dataset [18,19].

The LSTM module architecture consists of a cell state (c) and hidden state (output state; h). Four gates regulate these states: 1. forget (sigmoid activation function, f), 2. filter (g), 3. input (sigmoid activation function, i), 4. output (sigmoid or tanh function, o). Figure 4 shows the main components of the LSTM model. The cell state retains information acquired from earlier time steps, and at each time step, gates regulate the flow of information to and from the cell state. First, the forget gate remembers or forgets the data flowing from the prior cell state, $c_{t-1}$, where the decision to forget is formed by processing the input information, $x_t$, and the previous hidden state, $h_{t-1}$, through a sigmoid activation function, whose output, $f_t$, is between [0, 1], as in Equation (4). The input gate, $g_t$, then generates a new memory state by passing the input information $x_t$ and the hidden state of the previous time step ($h_{t-1}$) to the cell activation function as in Equation (5). In the meantime, the input gate takes any parts of $g_t$ that will be forgotten by creating an input state $i_t$, as in (6), and subsequently generates the new memory cell state, $c_t$, as in (7). The output gate generates the updated hidden state, $h_t$, using the recent memory cell state, $h_t$, and the output state, $o_t$, as in (8) and (9).

$$f_t={\sigma }_g(W_f{x}_t+R_f{h}_{t-1}+b_f)$$

(4)

$$g_t={\sigma }_c(W_g{x}_t+R_g{h}_{t-1}+b_g)$$

(5)

$$i_t={\sigma }_g(W_i{x}_t+R_i{h}_{t-1}+b_i)$$

(6)

$$c_t=f_t\odot c_{t-1}+i_t{\odot g}_t$$

(7)

$$h_t=o_t\odot {{\sigma }_c(c}_t)$$

(8)

$$o_t={\sigma }_g(W_o{x}_t+R_o{h}_{t-1}+b_o)$$

(9)

where W, R, b, σ_g, and σ_c denote the input weights, recurrent weights, offset, gate activation function, and cell activation function, respectively.

2.6. Deep Autoencoder–LSTM (DAE-LSTM)

First, an LSTM-based nutrient status detection using hyperspectral measurements was built. A dropout layer was incorporated into this model to mitigate overfitting in deep learning models [34]. The dense layer, which is a fully connected neural network layer, ensures that each neuron in the dense layer receives input from every neuron in the preceding layer. This layer is utilized to adjust the dimensions of a vector through matrix-vector multiplication, with backpropagation assisting in training and updating the matrix values. Dense layers are primarily employed for the output layers, utilizing the softmax activation function to provide a probability for each class. The model then predicts the class with the highest probability. Then, the deep autoencoder and LSTM unit were combined. The autoencoder extracts the deep features and feeds them directly to the LSTM unit. Figure 6 presents the suggested DAE-LSTM model for diagnosing nutrient deficiencies in aquaponically grown lettuce. The time-series spectral dataset was fed to the autoencoder to extract the most representative features. The extracted features were fed to the LSTM model as a j × 1 cell array, where j is the number of samples at each time step explained in Table 2, which comprises 900 spectra (samples) for the four classes (low, medium, optimal) in each run. Each cell comprises an m × n matrix, where m represents the number of feature values and n is the number of time steps (the number of time steps for each dataset is described in Table 2). A backpropagation approach was used to train the end-to-end LSTM model. After completing the training process, the final hidden state h_t−S encodes significant information about the sequential spectral data. h_t−S was used as a representation vector. The fully connected (FC) layer was employed to convert this vector into a vector of the same length as the number of classes (aquaponics, low, medium, optimal). Finally, the softmax layer classifies the datasets into target categories. In the softmax layer, the number of neurons is adjusted to match the number of classes. In training the LSTM model, the extracted features were randomly split into 60% for training, 20% for validation, and 20% for testing. The sigmoid activation function was utilized for the input gate and forget gate. Both sigmoid and tanh were employed for the output gate, while tanh was used as the hidden layer function. The Adam optimization function was applied, with cross-entropy serving as the loss function.

2.7. Convolutional Neural Network (CNN)

A CNN architecture was developed as a comparison benchmark for the performance of the DAE-LSTM model, as depicted in Figure 7. The standard CNN architecture comprises three main types of layers: the input layer, responsible for providing the network with a collection of spectral data; the hidden layers, commonly consisting of convolutional, pooling, and flatten layers; and lastly, the output layer, which is a fully connected layer that flattens the outputs from the preceding layers and transforms them into the desired target categories. We added some other layers to our framework, such as dense layers, leaky layers, and leaky rectified linear units (LeakyReLU). All properties of the schematic CNN model layers are exhibited in

Table 3. Components of the proposed deep CNN structure.

Layer	Type	Input Size	Layer	Type	Input Size
0	Input data	1 × 120 × 1	8	Droput	1 × 30 × 64
1	Conv2D	1 × 120 × 1	9	Conv2D	1 × 30 × 64
2	LeakyReLU	1 × 120 × 32	10	LeakyReLU	1 × 30 × 128
3	Maxpooling2D	1 × 120 × 32	11	Maxpooling2D	1 × 30 × 128
4	Droput	1 × 60 × 32	12	Droput	1 × 15 × 128
5	Conv2D	1 × 60 × 32	13	Dense	1 × 15 × 128
6	LeakyReLU	1 × 60 × 64	14	Flatten	1 × 15 × 128
7	Maxpooling2D	1 × 60 × 64	15	Dense	1 × 15 × 128

. The first layers in the CNN architecture are the convolutional layers, which are the most important in the CNN. Convolutional layers apply a series of filters (2, 2) to the input volume in convolutions. These layers act as feature extractors from the input dataset. The LeakyReLU layers alter input values that are below zero by multiplying them with a fixed scalar via a specific thresholding procedure. The pooling layers perform a spatial downsampling operation on the input volume by computing the maximum value (max pooling) or the mean value (average pooling) [34]. To reduce the chance of overfitting problems occurring during the training process, a max-pooling layer of size 2 × 2 was applied [34]. Mathematically, max-pooling is described by Equations (10)–(12):

$${\mathrm{m}}_1^{\left(\mathrm{L}\right)}={\mathrm{m}}_1^{(\mathrm{L}-1)}$$

(10)

$${\mathrm{m}}_2^{\left(\mathrm{L}\right)}={\displaystyle \frac{{\mathrm{m}}_2^{(\mathrm{L}-1)}-\mathrm{F}\left(\mathrm{L}\right)}{{\mathrm{S}}^{\mathrm{L}}}}+1$$

(11)

$${\mathrm{m}}_3^{\left(\mathrm{L}\right)}={\displaystyle \frac{{\mathrm{m}}_3^{(\mathrm{L}-1)}-\mathrm{F}\left(\mathrm{L}\right)}{{\mathrm{S}}^{\mathrm{L}}}}+1$$

(12)

where ${\mathrm{S}}^{\mathrm{L}}$ denotes the stride and ${\mathrm{m}}_1^{\left(\mathrm{L}\right)}$, ${\mathrm{m}}_2^{\left(\mathrm{L}\right)}$, and ${\mathrm{m}}_3^{\left(\mathrm{L}\right)}$ are filters for feature maps such as 2 × 2 or 3 × 3.

Dropout layers were added to reduce overfitting problems [34]. The dense layer is a type of fully connected neural network layer in which each neuron is connected to every neuron in the previous layer. The last layer is fully connected, and it acts as a classifier through the softmax function.

2.8. Traditional Machine Learning

A multi-class support vector machine (MCSVM) is one of the most common classification methods; therefore, it was utilized in this study to assess the performance of the LSTM model. In this case, the spectral data were treated as a single growth phase (as a single time step).

2.9. Performance Evaluation of the Proposed Framework

To evaluate the performance of competing approaches, several performance metrics were used, including accuracy (Acc), precision (Pr), recall (Re), and F-measure (Fm) (Equations (13)–(16)). Acc indicates the number of correctly classified samples relative to the test dataset’s total samples. Pr is the ratio of correctly predicted observations to the total number of predicted observations. Re is the ratio of correctly predicted observations to total observations. Fm is the harmonic mean of Pr and Re and it is a better measure of incorrect observation cases.

$$\mathrm{A}\mathrm{c}\mathrm{c}={\displaystyle \frac{\sum \mathrm{T}\mathrm{P}+\sum \mathrm{T}\mathrm{N}}{\sum \mathrm{T}\mathrm{P}+\sum \mathrm{T}\mathrm{N}+\sum \mathrm{F}\mathrm{P}+\sum \mathrm{F}\mathrm{N}}}\times 100$$

(13)

$$\mathrm{P}\mathrm{r}={\displaystyle \frac{\sum \mathrm{T}\mathrm{P}}{\sum \mathrm{T}\mathrm{P}+\sum \mathrm{F}\mathrm{P}}}\times 100$$

(14)

$$\mathrm{R}\mathrm{e}={\displaystyle \frac{\sum \mathrm{T}\mathrm{P}}{\sum \mathrm{T}\mathrm{P}+\sum \mathrm{F}\mathrm{N}}}\times 100$$

(15)

$$\mathrm{F}\mathrm{m}=2\times \left({\displaystyle \frac{\mathrm{P}\mathrm{r}\times \mathrm{R}\mathrm{e}}{\mathrm{P}\mathrm{r}+\mathrm{R}\mathrm{e}\mathrm{l}}}\right)\times 100$$

(16)

3. Results and Discussion

3.1. Evaluation of the Training Process

Both intra- and inter-class heterogeneity influence the accuracy of model training. Figure 8 illustrates the training accuracy, validation accuracy, training loss, and validation loss for each period of the training process for DAE-LSTM and DAE-CNN. Both models’ training and validation accuracies improved gradually, even after 400 iterations, after which they were reasonably established. Similarly, the decrease in loss for models was gradual until it reached the halting threshold at 600 iterations (Figure 8). The DAE-LSTM model exhibited superior performance by achieving the highest training accuracy, at 98.5%, and validation accuracy, at 94.7%. The DAE-CNN model achieved a training accuracy of 92.7% and a validation accuracy of 83.3%. We also observed a slight difference between the training and validation accuracy of the DAE-LSTM model, indicating that the model fits the data better than the DAE-CNN. To avoid overfitting, the network was trained by the back-propagation algorithm with early stopping [34]; therefore, no overfitting issues were observed during the training process.

3.2. DAE vs. GA and SFS

This study utilized three distinct feature extraction methods on spectral data: two conventional techniques (GA and SFS) and a deep learning approach (DAE). These extracted features were employed as inputs for a range of classification models. DAE extracts high-level representative features for each nutrient without knowing the nature of these features, which directly feed into the deep classification model. The GA and SFS selected the most representative features of the three nutrients, each selecting 900 spectral features for nitrogen in the spectral range of 450–1728 nm, 900 spectral features for potassium in the spectral range of 512–1899 nm, and 900 spectral features for phosphorus in the spectral range of 503–1881 nm. These results align with the study of Sun et al., who identified the spectral range of 544.1–1229.00 nm for determining NPK in tomatoes [35]. In addition, Pacumbaba and Beyl found that nutrient shortage significantly affected the spectral reflectance of lettuce leaves in the visible range of 401–780 nm [36]. Moreover, Herrmann et al. reported that the shortwave infrared (SWIR) region (1200–2500 nm) is highly sensitive to NPK [37]. A comparative study and quantitative analysis were undertaken to delve deeper into the impact of deep features on the diagnostic capabilities for plant nutritional status, focusing on key metrics such as Acc, Pr, Re, Fm, and IoU. The findings revealed that deep feature-based models extracted through DAE outperformed traditional feature-based approaches. This may be due to the ability of DAE to extract deep features and wavelengths that are more discriminative between different classes than those extracted by traditional methods. Specifically, within the same classification model (LSTM), substituting GA with DAE to substantially improve Acc, Pr, Re, and Fm by 29.1%, 27.9%, 29.1%, and 31.3%, respectively. Similarly, in the case of the CNN model, replacing GA with DAE resulted in very significant enhancements in Acc, Pr, Re, and Fm by 49.2%, 31.7%, 49.01%, and 48.6%, respectively, as illustrated in

Table 4. Comparison of the classification accuracy of all proposed methods.

Model	Method	Classification Performance Metrics, %
		Acc	Pr	Re	Fm
LSTM	DAE	94.7	94.8	94.7	94.7
	GA	73.4	74.1	73.4	72.1
	SFS	80.4	81.5	81.4	80.5
CNN	DAE	83.3	83.2	83.3	83.3
	GA	55.8	56.3	55.9	56.09
	SFS	78.9	80.9	78.9	78.9
MCSVM	GA	72.2	72.1	72.2	72.2
	SFS	70.7	70.3	70.7	70.7

.

3.3. LSTM vs. CNN and MCSVM

This study demonstrates the suitability of the LSTM model for monitoring plant nutritional status using time-series spectral data. Table 4 shows the performance evaluation of LSTM, CNN, and MCSVM using deep feature extraction methods and choosing the optimal wavelengths. Regardless of the input method, the LSTM model performed impressively in the classification task, outperforming both CNN and MCSVM. The LSTM model obtained the highest values for all performance criteria, demonstrating its effectiveness for monitoring the nutritional status of aquaponically growing plants. The DAE-LSTM model achieved the highest performance scores for accuracy (Acc), precision (Pr), recall (Re), and F-measure (Fm). Specifically, the LSTM model achieved scores of 94.7, 94.8, 94.7, and 94.7 for Acc, Pr, Re, and Fm, respectively. These results are consistent with Petkovski and Shehu, who used DAE-LSTM to detect anomalies in aquaculture time-series sensor data, achieving a classification accuracy of 95.1, 93.4, and 95.1 for the pH, dissolved oxygen, and water temperature datasets, respectively [38]. The DAE-CNN model demonstrated good performance, achieving good values for Acc, Pr, Re, and Fm of 83.3, 83.2, 83.3, and 83.3. On the contrary, the MCSVM-SFS model exhibited the lowest performance across all performance criteria, at 70.7, 70.3, 70.7 and 70.7. These results are consistent with the study of Abdalla et al., who compared LSTM and SVM to classify oilseed rape based on its nutritional status, demonstrating the significant superiority of LSTM (Acc = 95) over SVM (Acc = 29) [14]. The findings presented here showcase the effectiveness of the LSTM model in accurately identifying and understanding the temporal dynamics in feature changes. Several contributions compared the performance of LSTM, CNN, and traditional models (e.g., SVM) for classifying spectral data, proving the superiority of deep models, in particular LSTM [39,40,41]. The hybridization of DAE and LSTM performed better than other deep learning methods, as demonstrated in this study and some previous contributions. Yu et al. developed a model that combined stacked auto-encoders (SAEs) and a fully connected neural network (FNN) to detect nitrogen in oilseed rape leaves using hyperspectral images, achieving an accuracy of 90.3%. SAEs were used to extract deep spectral features from hyperspectral images [42]. Zhou et al. presented a deep method involving wavelet transform (WT) and stack convolution auto encoder (SCAE) to extract deep features from visible–near-infrared (400.68–1001.61 nm) hyperspectral image of lettuce for monitoring the content of cadmium (Cd) and lead (Pb) in lettuce, achieving an accuracy of 93.1 and 94.1% [43]. To detect nitrogen and potassium in rice, Fubing Liao et al. combined CNN and LSTM using sequential images collected by an unmanned aerial vehicle (UAV) achieved an accuracy of 88.38% [44], which also demonstrates the superiority of DAE-LSTM.

Confusion matrices were generated to assess the predictive efficacy of the proposed classification models in discerning various nutrient levels. A confusion matrix is a detailed tabular representation of classification outcomes, illustrating the correspondence between actual and predicted classes. The confusion matrix’s vertical axis corresponds to the actual classes, while the horizontal axis represents the predicted classes. The first row of Figure 9 displays the confusion matrices of the DAE-LSTM model, followed by the second row showing those of the DAE-CNN model and the third row depicting MCSVM with SFS.

The study revealed that DAE-LSTM exhibited superior performance in distinguishing between different levels of nutrient symptoms (low, medium, optimal) with minimal classification errors, as depicted in Figure 9a. Despite incorporating CNN with deep features extracted by DAE, it demonstrated comparatively inferior performance when contrasted with LSTM. Figure 9b illustrates an escalation in classification errors for CNN. The amalgamation of features selected by SFS, GA, and both LSTM and CNN led to a notable rise in classification errors compared to models relying on deep features extracted with DAE. Moreover, the combination of MCSVM and SFS exhibited notably poor performance, indicating a substantial increase in classification errors and the inability of the traditional machine learning classifier to differentiate between various nutrient levels. Increasing the model size of the DAE-LSTM can significantly increase the accuracy of nutritional status detection systems. However, the continual expansion of DAE-LSTM’s size may pose challenges for deploying deep learning in embedded systems for real-time applications [14]. Leveraging pre-trained deep learning networks and reducing their size by model pruning or explicitly searching for more straightforward and compact network topologies is a recent focus of study. Pruning pre-trained networks involves removing redundant weights and connections that have minimal impact on improving the objective function. By implementing pruning techniques, the computational demands of the network are significantly reduced, enabling training and deployment on resource-constrained devices. As a result, forthcoming research endeavors are centered on computational complexity and its implications. Plants at various growth stages exhibit distinct responses to nutritional stress and other stimuli. Hence, depending solely on one growth stage to assess a plant’s health and nutritional condition may lead to incorrect conclusions. Moreover, modeling each plant growth stage individually is difficult and does not provide a comprehensive view for decision-makers. Recent developments in various sensing technologies, such as imaging and non-imaging spectral sensors, have reached a state of maturity, enabling them to obtain time-series data with high spatiotemporal data accuracy [45]. Hence, to reach a dependable determination regarding the nutritional status of plants, it is imperative to construct models based on spatiotemporal data acquired at various stages of growth. Conventional classification models such as MCSVM cannot capture dependencies from time-series data due to their inability to retain past information. Despite the high classification ability of CNN models, they do not prove worthy when modeling dynamic systems such as a time-series spectral dataset. Long short-term memory (LSTM) has demonstrated its efficacy in modeling dynamic systems due to its ability to learn short- and long-term dependencies between time steps in sequential data [16,18]. Hence, in this investigation, the LSTM model exhibited remarkable efficacy in identifying the nutritional status of aquaponically cultivated plants using spectral data collected at various stages of plant development.

4. Conclusions

This study introduced a robust deep learning time-series approach incorporating deep features to detect the nutritional status of aquaponically cultivated lettuce. The results indicated the superior performance of deep feature extraction methods compared to conventional approaches. Moreover, this research highlighted the significance of integrating multiple temporal data into a cohesive, dynamic model for effective plant status diagnosis. Furthermore, this study demonstrated the superiority of DAE over both SFS and GA. The DAE-LSTM model achieved the highest overall classification accuracy of 94.7%. The presented method offers a pathway to identify the type of nutrient stress and classify the nutrient status in aquaponically grown lettuce plants. Additionally, the adaptive nature of this approach with further investigation, study, and re-training on appropriate dataset may allow it to be easily expanded to other applications, such as disease diagnosis and drought stress, with minor modifications.