Two-Stage Ensemble Deep Learning Model for Precise Leaf Abnormality Detection in Centella asiatica

Abstract

Leaf abnormalities pose a significant threat to agricultural productivity, particularly in medicinal plants such as Centella asiatica (Linn.) Urban (CAU), where they can severely impact both the yield and the quality of leaf-derived substances. In this study, we focus on the early detection of such leaf diseases in CAU, a critical intervention for minimizing crop damage and ensuring plant health. We propose a novel parallel-Variable Neighborhood Strategy Adaptive Search (parallel-VaNSAS) ensemble deep learning method specifically designed for this purpose. Our approach is distinguished by a two-stage ensemble model, which combines the strengths of advanced image segmentation and Convolutional Neural Networks (CNNs) to detect leaf diseases with high accuracy and efficiency. In the first stage, we employ U-net, Mask-R-CNN, and DeepNetV3++ for the precise image segmentation of leaf abnormalities. This step is crucial for accurately identifying diseased regions, thereby facilitating a focused and effective analysis in the subsequent stage. The second stage utilizes ShuffleNetV2, SqueezeNetV2, and MobileNetV3, which are robust CNN architectures, to classify the segmented images into different categories of leaf diseases. This two-stage methodology significantly improves the quality of disease detection over traditional methods. By employing a combination of ensemble segmentation and diverse CNN models, we achieve a comprehensive and nuanced analysis of leaf diseases. Our model’s efficacy is further enhanced through the integration of four decision fusion strategies: unweighted average (UWA), differential evolution (DE), particle swarm optimization (PSO), and Variable Neighborhood Strategy Adaptive Search (VaNSAS). Through extensive evaluations of the ABL-1 and ABL-2 datasets, which include a total of 14,860 images encompassing eight types of leaf abnormalities, our model demonstrates its superiority. The ensemble segmentation method outperforms single-method approaches by 7.34%, and our heterogeneous ensemble model excels by 8.43% and 14.59% compared to the homogeneous ensemble and single models, respectively. Additionally, image augmentation contributes to a 5.37% improvement in model performance, and the VaNSAS strategy enhances solution quality significantly over other decision fusion methods. Overall, our novel parallel-VaNSAS ensemble deep learning method represents a significant advancement in the detection of leaf diseases in CAU, promising a more effective approach to maintaining crop health and productivity.

1. Introduction

Agriculture is a vital pillar of many economies, facing significant challenges due to climate change. Among these challenges, crop susceptibility to diseases has profound implications for growth and yield [1]. Centella asiatica (Linn.) Urban (CAU), commonly known as Gotu Kola, is of particular interest due to its extensive medicinal properties [2]. However, like other crops, CAU is prone to leaf abnormalities, which can adversely impact its growth and the quality of its bioactive compounds, especially asiaticoside, a crucial triterpenoid saponin [3]. Leaf abnormalities have been found to reduce the concentration of asiaticoside, potentially diminishing its medicinal properties [4]. Therefore, the early detection and classification of leaf abnormalities is essential to mitigate the impact of these diseases [5].

Currently, leaf disease detection in plants, including CAU, relies on visual inspection by human experts such as agricultural professionals and botanists [6]. However, this subjective approach has limitations, leading to inconsistencies in disease identification due to varied interpretations of visual symptoms [7]. Additionally, it is time-consuming and may not effectively identify early-stage or subtle diseases, resulting in increased crop losses and reduced medicinal properties for CAU [8]. Thus, there is a clear research gap that necessitates the development of automated and efficient methods for disease detection in CAU and other crops [9]. Deep learning-based ensemble techniques, such as Convolutional Neural Networks (CNNs) and feature extraction methods, offer a promising solution, enabling more precise and timely disease identification and classification.

Recently, a novel double artificial multiple intelligence system (AMIS) was proposed for skin cancer detection [10]. The AMIS integrated various image segmentation techniques, including U-net, the threshold method, edge detection, and the clustering method, and fused CNN architectures, such as ConvNeXtSmall, EfficientNetV2B3, and EfficientNetV2S, resulting in a superior performance over other state-of-the-art methods. However, pertinent research gaps remain to be addressed. Recent studies by Triki et al. [11] and Wang et al. [12] have introduced effective segmentation methods such as Mask-R-CNN and DeepLabV3+ for leaf detection. Zhao and Wang [13], Setiawan et al. [14], and Zhang et al. [15] presented compact yet powerful CNN architectures, including MobileNetV3-Small, SqueezeNetV2, and ShuffleNetV2, which have shown outstanding results in image classification tasks, including crop detection. Moreover, Pitakaso et al. [16] introduced the Variable Neighborhood Strategy Adaptive Search (VaNSAS), offering a promising alternative to AMIS with a comparable solution quality and reduced computational time.

In this study, we propose a two-stage VaNSAS ensemble deep learning approach, incorporating novel image segmentation methods and CNN architectures, to enhance the accuracy and efficiency of leaf disease detection in CAU. The primary research contribution lies in the development of this two-stage ensemble deep learning model, which effectively integrates modified CNN architectures with efficient decision fusion strategies. This study effectively addresses the research gap concerning the automated and efficient detection of leaf abnormalities in CAU. By implementing this ensemble approach, this research overcomes the limitations of traditional methods, such as subjective visual inspection, providing a precise and timely disease identification and classification system tailored for detecting leaf abnormalities in CAU leaves. This advancement holds promising implications for the field of plant disease detection, potentially fostering enhanced agricultural practices and safeguarding crop yields and medicinal properties in CAU.

The organization of this paper is as follows: Section 2 will present the related literature, while Section 3 and Section 4 will illustrate the proposed model and computational results. Finally, Section 5 and Section 6 will provide the discussion and conclusion.

2. Related Literature

This research covers two essential aspects, highlighting its contributions. The first aspect concerns existing methodologies for detecting leaf diseases or abnormalities in plants, especially CAU. The second aspect explores the emerging field of deep learning-based ensemble techniques, specifically examining relevant studies that employ ensemble models for early disease identification. This discussion emphasizes the potential advantages of ensemble approaches compared to conventional methods.

2.1. Contemporary Methods for Detecting Leaf Diseases in CAU: Limitations, Challenges, and Our Approach

The early and accurate detection of leaf diseases in crops like C. asiatica (CAU) is crucial for maintaining agricultural productivity and food security. Recent advancements in this field have primarily centered around deep learning techniques, notably convolutional neural networks (CNNs), which have shown promising results in identifying and diagnosing plant leaf diseases with high accuracy [17,18,19]. These modern approaches, including the use of transfer learning with pre-trained models and image processing techniques, have marked a significant shift in disease detection methodologies [20,21].

Deep learning has emerged as a transformative technique in agricultural disease detection, utilizing various CNN architectures such as AlexNet, GoogLeNet, VGGNet, DenseNet, SqueezeNet, ResNet, and MobileNet. These models have demonstrated high accuracy levels, with some achieving rates between 96.58% and 99% [22,23,24,25]. Transfer learning, especially using architectures such as AlexNet and MobileNetV2, has been effective in feature extraction and disease classification [26,27].

Despite these advancements, the application of such models to specific crops, such as CAU, presents notable limitations and challenges. The primary challenge lies in the development of specialized models and datasets that consider the unique characteristics and disease patterns of individual crops. There is also a need for models that can operate effectively with limited data, a common scenario in specific crop disease detection. Additionally, improving the scalability and accessibility of these methods is crucial for their practical implementation in diverse agricultural settings.

Our study aims to address these limitations by developing a novel parallel-Variable Neighborhood Strategy Adaptive Search (parallel-VaNSAS) ensemble deep learning method that is tailored for CAU leaf disease detection. This approach employs a two-stage model that first uses advanced image segmentation techniques, such as U-net, Mask-R-CNN, and DeepNetV3++, for the precise identification of diseased regions. The second stage involves the application of robust CNNs, such as ShuffleNetV2, SqueezeNetV2, and MobileNetV3, for accurate disease classification. This method not only enhances the accuracy of disease detection but also overcomes the challenges of limited data and model generalization, making it a more viable solution for CAU and rendering it potentially adaptable for other crops.

2.2. Deep Learning-Based Ensemble Techniques in Leaf Disease Detection: Focused Approaches and Advancements

The application of ensemble methods in the domain of leaf disease detection represents a significant advancement, leveraging the combined strengths of multiple deep learning models to enhance detection accuracy and robustness. These ensemble techniques, particularly tailored for plant leaf disease detection, offer improved performance through error reduction and overfitting mitigation, which is crucial for accurate and reliable disease diagnosis in crops such as C. asiatica (CAU) [28,29,30].

In the context of leaf disease detection, ensemble deep learning models primarily focus on integrating various Convolutional Neural Network (CNN) architectures. This integration allows for a comprehensive analysis of leaf imagery, leveraging diverse perspectives from multiple models to enhance disease classification accuracy [31,32,33,34]. Key to this approach is the use of advanced image segmentation techniques, which enable the precise identification of diseased areas on leaves. Prominent models such as U-Net, Mask R-CNN, and DeepLabV3 have been employed for this purpose, demonstrating their efficacy in accurately segmenting leaf images [35,36,37,38,39,40].

Further, these ensemble methods often incorporate image augmentation techniques to expand the dataset and introduce variability, thereby improving the model’s ability to generalize and recognize diverse disease patterns. Techniques such as flipping, rotation, scaling, and color adjustments are commonly used to augment the training data, contributing significantly to the model’s performance [41,42].

An efficient decision fusion strategy forms an integral component of these ensemble models. By optimizing the combination of outputs from various CNN architectures, these strategies ensure that the final disease classification leverages the strengths of each constituent model. Recent advancements in this area include metaheuristic approaches, such as differential evolution (DE) and particle swarm optimization (PSO), which have been adapted to optimize ensemble model weights for enhanced accuracy in disease detection [43].

In our research, we aim to capitalize on these advancements by developing a two-stage ensemble deep learning model specifically for leaf disease detection in CAU. This model combines diverse segmentation techniques and CNN architectures, employing a novel decision fusion approach to maximize accuracy. Our approach is expected to address the unique challenges met in detecting leaf diseases in CAU, setting a new standard in precision and reliability in this critical agricultural task.

3. Research Methodology

The research methodology for creating a deep learning model to classify leaf abnormalities in CAU comprises data collection, preprocessing, feature extraction, and deep learning model development. Model performance evaluation will employ metrics such as accuracy, precision, recall, and an F1-score, and testing will be conducted with a new set of images to assess generalizability. This study seeks to showcase the efficacy of deep learning techniques in detecting and managing leaf abnormalities in CAU, thereby contributing to advancements in agricultural management.

3.1. Dataset Preparation

To prepare datasets for various types of leaf abnormalities in CAU, leaf images should be collected and labeled according to their corresponding abnormality type or health status in

Table 1. Detail of the leaf abnormality types used in this research.

Name of Leaf Abnormality	Description	Leaf Image Example
Normal Leaf (NL)	Plant leaves are spreading, green, no lesions.
Red Mite Disease (RM)	Red mites suck the sap from the leaves, causing the loss of chlorophyll. Red mites cause the upper surfaces of the leaves to become pale yellow spots, burn shoots, wither leaves, and fall off.
Worm Creep Disease (WCD)	The caterpillars will eat young and old leaves until only the stems and branches remain.
Low Light Intensity (LI)	Plant leaves reduce chlorophyll production. The leaves turn yellow.
Nitrogen Deficiency (ND)	The leaves are pale yellow. The tip of the leaf and the edge of the leaf will gradually dry and this will spread continuously until the leaves fall from the tree prematurely.
Phosphorus Deficiency (PHD)	Phosphorus deficiency manifests as stunted growth, dark green or purple pigmentation in leaves, and underdeveloped root systems in plants.
Potassium Deficiency (PD)	Older leaves are pale yellow, starting from the leaf margin and leaf tip.
Water Deficiency (WD)	The leaves wither, and dry leaves cannot spread.

. The dataset should be balanced and split into training, validation, and testing sets to ensure the model’s accuracy and generalizability. In addition, preprocessing the dataset is important to remove any noise, such as background clutter or lighting variations. Properly preparing the dataset is crucial in training deep learning algorithms to accurately differentiate between healthy and abnormal leaves and develop an effective classification model for efficient abnormality detection and management in agricultural practices.

The images of leaf abnormalities in CAU were obtained from a smart farm system where the plant’s growth is controlled and monitored using hydroponics and lighting systems. The smart farm can regulate the amount of light and nutrients provided to the plant, among other factors. To obtain the images of the leaf abnormalities, plants were grown for 12 weeks under these controlled conditions, and leaves with various abnormalities were collected and photographed. These images are used to create a dataset for the development of a machine learning model to classify the different types of leaf abnormalities. All images have been divided into five classes: (1) normal leaf (NL), (2) red mite disease (RM), nitrogen deficiency (ND), potassium deficiency (PD), and phosphorus deficiency (PHD).

In our study, we divided the image data into two groups, namely, ABL-1 and ABL-2, which contain different numbers of images. ABL-1 was further divided into two subsets, each comprising a training set (80%) and a testing set (20%). On the other hand, ABL-2 was reserved as an unseen dataset and was used solely for testing the algorithm.

Table 2. Number of sets of data in ABL-1 and ABL-2.

	ABL-1								ABL-2
	NL	RM	ND	PD	LI	WD	WCD	PHD	NL	RM	ND	PD	LI	WD	WCD	PHD
Training set	1000	960	1040	1080	1072	1064	1024	1040	-	-	-	-	-	-	-	-
Testing set	250	240	260	270	268	266	256	260	550	550	500	600	550	550	600	580
Total	1250	1200	1300	1350	1340	1330	1280	1300	550	500	600	550	550	600	580	580

provides a breakdown of the number of images in each class, which were distributed across the training and testing datasets. This approach allowed us to train and validate the algorithm using multiple datasets, ensuring that the results obtained were reliable and robust.

Table 2 shows dataset ABL-1, comprising 14,860 images classified into classes NL, RM, ND, PD, LI, WD, WCD, and PHD, with corresponding image counts of 1250, 1200, 1300, 1350, 1340, 1330, 1280, and 1300, respectively. Furthermore, dataset ABL-2 displays varying image counts in each class, ranging from 500 to 600 images. The datasets are available at https://doi.org/10.34740/KAGGLE/DS/3619913 (accessed on 6 October 2023).

In this study, we have strategically divided our dataset into two distinct types, ABL-1 and ABL-2, to comprehensively assess the performance of our proposed model. ABL-1, constituting a diverse range of leaf images, serves as the training set, enabling the model to learn and adapt to a wide spectrum of leaf disease manifestations. To validate the model’s performance during training and prevent overfitting, we employed a cross-validation approach within ABL-1. This technique allowed us to iteratively use different subsets of ABL-1 as a validation set, providing us with essential feedback on the model’s adaptability and generalizability.

ABL-2, an entirely unseen dataset, was reserved exclusively for final testing. This approach was crucial in evaluating the model’s effectiveness in real-world scenarios where it would encounter data not present during the training phase. The use of ABL-2 as a separate testing set ensures that we assess the model’s generalization capabilities thoroughly. The combination of rigorous cross-validation within ABL-1 and the subsequent testing on ABL-2 offers a robust framework for validating our model’s performance, addressing concerns of overfitting, and ensuring its applicability in practical settings.

3.2. Model Building Classification

The models developed in our study are designed to identify and classify leaf diseases in plants, specifically focusing on C. asiatica. The input for these models is high-resolution images of plant leaves, which are subjected to various preprocessing steps to enhance their features. The output of these models is the classification of leaf conditions, categorizing them into healthy or various disease states. This is clearly illustrated in this paper, particularly in the sections discussing the methodology and model implementation.

Our He-Meta model features an advanced, multi-layered neural network architecture specifically tailored for plant disease detection. The model commences with preprocessing layers that normalize and condition the input data, preparing it for deeper analysis. This is followed by a series of convolutional layers, each intricately designed to extract and refine features, capturing detailed patterns essential for identifying diseases in plant leaves.

Subsequent pooling layers effectively reduce dimensionality, ensuring computational efficiency while preserving key information. The architecture culminates with fully connected layers, leading to a final softmax classification layer. This layer outputs a probabilistic distribution across potential disease categories, enabling accurate diagnostics.

The model’s design is visually represented in our manuscript through detailed schematics, offering an intuitive understanding of its operation. Each layer is clearly defined, elucidating its individual and collective roles in disease detection. The He-Meta model is not just technically robust but also practically viable for agricultural applications, showcasing its ability to differentiate effectively between healthy and diseased plant leaves.

The model consists of two stages: image segmentation and classification. In the first stage, the input image is segmented into regions of interest using multiple segmentation methods, including Otsu’s thresholding, K-means clustering, and watershed segmentation. These segmented regions are then used as inputs for the second stage, which involves classification using an ensemble of three convolutional neural network (CNN) architectures: ShuffleNetV2, SqueezeNetV2, and MobileNetV3. The outputs of these three CNNs are combined using a meta-learner to make the final classification decision.

To further improve the performance of the model, geometric image augmentation techniques are applied to the input images. This involves applying random transformations to the images, such as rotation, scaling, and flipping, to increase the diversity of the training data and improve the model’s ability to generalize to new images.

Overall, the proposed model is a heterogeneous ensemble approach that combines multiple segmentation methods, geometric image augmentation, and an ensemble of CNN architectures to achieve improved solution quality and classification accuracy. The model’s performance is evaluated using multiple metrics such as accuracy, AUC, and an F1-score to ensure a comprehensive evaluation of the model’s performance.

The proposed approach to develop the leaf abnormality classification model in CAU involves a multi-stage process aimed at enhancing model accuracy and performance. Firstly, image segmentation techniques will be employed to eliminate extraneous information and reduce the computational burden by focusing on regions of interest. Secondly, image augmentation techniques will expand the training dataset, increasing image variety and complexity for improved generalization and robustness. Finally, an ensemble deep learning approach will be adopted, combining diverse CNN architectures for a powerful and accurate classification model. This ensemble strategy ensures reliable predictions of leaf abnormalities in CAU by capitalizing on the strengths of multiple models while mitigating their weaknesses.

3.2.1. Image Segmentation

Image segmentation plays a pivotal role in ensuring the accuracy and efficacy of deep learning-based models for leaf abnormality classification in CAU. By discerning and distinguishing various leaf regions, including healthy and abnormal ones, segmentation enables precise diagnosis and classification. To develop the leaf abnormality classification model in CAU, various image segmentation methods are available, among which the U-net, Mask R-CNN, and DeepLabV3+ architectures stand out as popular and effective.

The selection of U-net, Mask R-CNN, and DeepLabV3+ as the image segmentation methods for our ensemble leaf disease classification system is well-founded and justifiable due to their distinct and complementary strengths. U-net excels in accurately delineating intricate object boundaries, which is crucial for precise leaf disease localization. Meanwhile, Mask R-CNN’s capability to perform instance segmentation empowers the system to distinguish between multiple overlapping instances of the same disease, enhancing classification accuracy in crowded leaf images. Moreover, the incorporation of DeepLabV3+ makes available its exceptional ability to capture multi-scale contextual information, facilitating the robust segmentation of diverse leaf structures and background elements. By leveraging the unique advantages of these three state-of-the-art models, our ensemble segmentation technique aims to combine their complementary strengths, thereby bolstering the accuracy and efficacy of our leaf disease classification system and instilling confidence in its ability to yield superior performance compared to using any single segmentation method.

U-net is known for its encoder–decoder architecture, where the encoder captures the contextual information from the input image, and the decoder recovers the spatial information by up-sampling the feature maps [44,45]. The skip connections between the corresponding encoder and decoder layers enable the preservation of fine-grained details during the up-sampling process. This characteristic makes U-net particularly effective in tasks where the precise localization of object boundaries is crucial. It excels in medical image segmentation and in other applications where the objects of interest exhibit intricate shapes and fine structures.

Mask R-CNN extends the popular Faster R-CNN object detection framework to perform instance segmentation [11]. It is capable of both object detection and pixel-level segmentation within the same model. The key advantage of Mask R-CNN is its ability to generate individual object masks, allowing it to differentiate between instances of the same object class. This makes Mask R-CNN well-suited for scenarios with multiple overlapping objects, where distinguishing between instances is critical. Applications include computer vision tasks involving crowded scenes, object counting, and interactive image segmentation.

DeepLabV3+ is a state-of-the-art semantic image segmentation model that utilizes dilated convolutions to capture multi-scale contextual information effectively [12]. This helps in preserving both local and global information within the segmented regions. Additionally, DeepLabV3+ incorporates an Atrous Spatial Pyramid Pooling (ASPP) module, which further enhances the model’s ability to handle objects of various sizes. Its remarkable performance on large-scale datasets and complex scenes with diverse objects makes it particularly advantageous for semantic segmentation tasks in areas such as autonomous driving, urban scene comprehension, and satellite imagery analysis.

All methods will be used in the same model as the heterogeneous image segmentation methods. The solutions yielded by different techniques will be combined using the meta-learner, which will be used as the decision fusion strategy (explained in a later section). The heterogeneous image segmentation framework is depicted in Figure 1.

Figure 1 illustrates the process whereby the CAU undergoes segmentation using three distinct image segmentation techniques, each generating its own respective solutions. Subsequently, a meta-learning approach is employed to fuse these individual solutions into a unified segmentation result. This methodology increases the likelihood of selecting the most appropriate segmentation method for specific image types, in contrast to relying solely on a single segmentation technique. The resultant solution is further augmented by optimal weights adjusted through the meta-learning process, thereby yielding the most optimal image-segmented outcomes.

3.2.2. Image Augmentation

Image augmentation constitutes a fundamental and indispensable technique for enhancing model performance and accuracy. It involves artificially expanding the training dataset by applying diverse transformations to the images, encompassing rotations flips, zooms, and color jittering. This process fosters an increased image variety and complexity, thereby facilitating superior model generalization and robustness. In the context of CAU, the integration of image augmentation aids the model in recognizing a broader spectrum of leaf abnormalities and variations.

Various image augmentation techniques have been proposed and effectively employed in prior research [46,47]. These include random cropping, random rotation, random scaling, and random flipping. These techniques have demonstrated significant improvements in model performance and generalization. Random cropping generates additional training images by randomly selecting sub-regions from the original image, while random rotation and scaling simulate variations in leaf orientation and size, respectively. Incorporating these augmentation methods bolsters the model’s capability to accurately detect and classify leaf abnormalities, contributing to more comprehensive and reliable agricultural management and disease prevention. Figure 2 shows an example of the image augmentation used in this research.

3.2.3. CNN Architectures

Our proposed ensemble deep learning-based model for classifying leaf abnormalities in CAU comprises three selected CNN architectures: MobileNetV3-Small [13], SqueezeNetV2 [14], and ShuffleNetV2 [15]. These CNN architectures were chosen due to their small size, high accuracy, and efficiency in various computer vision tasks, particularly on mobile and embedded devices. Their sizes range from 0.5 to 5 MB, and they have a demonstrated accuracy exceeding 90% in diverse applications [44,48,49].

By effectively fusing the strengths of these models and minimizing their weaknesses, our proposed model significantly enhances the accuracy and effectiveness of leaf abnormality detection and diagnosis in CAU. This provides farmers with a reliable and efficient tool to manage and prevent plant diseases, leading to improved crop yields and enhanced farm profitability.

MobileNetV3-Small was selected for its efficiency and suitability for mobile and embedded applications. It achieves high accuracy in image classification while requiring fewer computational resources. This CNN was trained on a substantial dataset of healthy and abnormal CAU leaves, and its output was combined with other integrated CNNs to enhance the classification system’s accuracy.

SqueezeNetV2 was chosen for its compact size and low computational complexity, yet it maintains high accuracy in image classification. The ensemble was trained on a dataset of healthy and abnormal CAU leaves, and its output was combined with other CNNs using model stacking.

ShuffleNetV2 was selected for its high accuracy and low computational cost, which are achieved through channel shuffle operations. It was trained on a large dataset of labeled images of CAU leaves with different types of abnormalities. In the ensemble model, its output was combined with other CNNs using a weighted average approach.

The utilization of an ensemble deep learning model, combining multiple CNN outputs, results in improved accuracy and robustness in the CAU leaf abnormality classification system. Each CNN’s unique characteristics in the ensemble model enable the more accurate and efficient identification of diseased leaves, ultimately enhancing the overall productivity and quality of CAU leaf production.

3.2.4. Optimizing of the Decision Fusion Strategy

In our research, we have embraced a tailored variant of the Variable Neighborhood Strategy Adaptive Search (VaNSAS) as the preferred decision fusion approach, which has been effectively employed in both image segmentation and CNN architecture integration. Originally introduced by Pitakaso et al. [16], VaNSAS successfully addresses a specialized case of the vehicle routing problem (VRP), showcasing exceptional efficacy in solving network-based challenges. In light of the integration of diverse segmentation techniques and CNN architectures, akin to a network flow problem, the determination of optimal weights for solutions derived from distinct sectors significantly impacts the accuracy of our model’s predictions. It is with a sense of confidence that we have chosen VaNSAS as the most suitable decision fusion strategy, enabling us to effectively ascertain these crucial optimal weights, thus bolstering the overall prediction accuracy in our proposed model.

The Variable Neighborhood Strategy Adaptive Search (VaNSAS) technique will be employed to determine the optimal weight for fusing diverse solution types obtained from various architectures. This approach will be compared with other decision fusion strategies, namely, the unweighted average model (UWM), the differential evolution algorithm (DE) [50], and particle swarm optimization (PSO) [51].

The unweighted average model (UAM) assigns equal weight to each prediction value ($Y_{ij}$), where i denotes the CNN label, and j indicates the prediction class (also applicable to segmented image classes, represented as 0 or 1). For fusion processes, the UAM employs Equation (1), while VaNSAS, DE, and PSO adopt Equation (2) to compute the final weight. Here, $Y_{ij}$ represents the predicted value of CNN i for class j before applying both equations. Subsequently, following the fusion of multiple CNN results, $V_j$ is utilized for categorizing class j, with CNN i assigned weight $W_i$, considering I as the number of CNNs/segmentation methods and J as the number of classes.

$$V_j=\frac{{\sum }_{i=1}^I{Y}_{ij}}{I}$$

(1)

$$V_j={\sum }_{i=1}^I{W}_i{Y}_{ij}$$

(2)

VaNSAS, DE, and PSO were employed to determine the optimal value of $W_i$ for the given scenario. The unweighted average decision fusion strategy (UWA) offers a straightforward and computationally efficient approach for ensemble deep learning model integration. UWA assigns equal weights to each CNN, ensuring ease of implementation and interpretation. However, UWA’s lack of optimization capabilities restricts its ability to finely tune ensemble performance. In contrast, differential evolution (DE) exhibits robust global search capabilities, making it suitable for intricate optimization problems, particularly in noisy landscapes. DE adapts well to multimodal search spaces but may converge slower and require more memory due to its population-based approach. On the other hand, particle swarm optimization (PSO) strikes a balance between exploration and exploitation, enabling faster convergence. Nonetheless, PSO’s performance may be sensitive to parameter settings, and it may encounter challenges in highly complex landscapes.

VaNSAS, in essence, comprises four integral steps: (1) the generation of a set of initial tracks or solutions; (2) the track touring process facilitated by black box operators (improvement box: IB); (3) the regular updating of heuristics information; and (4) the iterative repetition of steps (2) and (3) until a predetermined termination criterion is met.

3.3. The Initial Tracks Generation

This section involves the generation of NT (number of tracks) random tracks. Each track possesses dimensions of 1 × D, where D denotes the number of image segmentation methods or the CNN’s architectures. The initial track utilized in this study is a real number, uniformly and randomly generated between 0 and 1, following Equation (3):

$$X_{ki1}=U\left(\mathrm{0,1}\right)$$

(3)

In this context, $X_{ki1}$ represents the value in track k at position i during the first iteration. Here, i denotes the number of available CNN/segmented methods, and k signifies the predefined number of tracks. Additionally, two additional sets of tracks, namely, the best tracks (BT) and random tracks (RT), were also randomly generated during the initial iteration.

$$B_{ki1}=U\left(\mathrm{0,1}\right)$$

(4)

$$R_{ki1}=U\left(\mathrm{0,1}\right)$$

(5)

In the given context, $B_{kit}$ represents the set of best solutions obtained from iteration one to iteration t, while $R_{kit}$ is randomly selected using a specific formula. For the first iteration, the initial $B_{kit}$ and $R_{kit}$ are randomly generated using Equations (4) and (5), respectively. Subsequently, Equation (6) is utilized to update $X_{kit}$, where the value of $X_{ki}$ in iteration t + 1 corresponds to the value of $X_{ki}$ in iteration t, considering a selected improvement box (IB) operator. An example of the track that has D = 5 is as follows: {0.45, 0.03, 0.45, 0.14, 0.54}. The value of the track will be re-calculated to obtain the value of $W_i$.

$$P_{ki}=\frac{X_{ki}}{{\sum }_{i=1}^i{X}_{ki}}$$

(6)

Equation (7) is modified to deal with k number of tracts. $C_{kj}$ is utilized for categorizing class j using track k, and $P_{ki}$ is the weight of CNN/segmentation method i using track k values.

$$C_{kj}={\sum }_{i=1}^I{P}_{ki}{Y}_{ij}$$

(7)

3.4. Track Touring Process

The tracks iteratively tour by improving the solution using the improvement box (IB). In this context, 4 IBs will be used to improve the solution. These methods are differential evolution-inspired (DEI), random crossover (RC), single-bit mutation-inspired (SMI), and scaling factors (SF). These methods use Equations (8), (9), (10), and (11), respectively.

$$X_{kit}=\rho X_{rit-1}+F1\left(B_i^{gbest}-X_{rit-1}\right)+F2\left(X_{mit-1}-X_{rit-1}\right)$$

(8)

$$X_{kit}=\left\{\begin{array}{c}{X}_{kit-1}\hspace{1em}\hspace{1em}if R_{ki}\le CR\hspace{1em}\hfill \\ R_{kit-1}\hspace{1em}\hspace{1em} otherwise\hspace{1em}\hfill \end{array}\right.$$

(9)

$$X_{kit}=\left\{\begin{array}{c}{X}_{klt-1}\hspace{1em}\hspace{1em}if R_{ki}\le CR\hspace{1em}\hfill \\ X_{nlt-1}\hspace{1em}\hspace{1em} otherwise \hfill \end{array}\right.$$

(10)

$$X_{kit}=\left\{\begin{array}{c}{X}_{kit-1}\hspace{1em}\hspace{1em}if R_{ki}\le CR\hfill \\ R_{ki}{X}_{kit-1} \hspace{1em}otherwise \hfill \end{array}\right.$$

(11)

In this context, $\rho$ denotes the evaporation rate, set to 0.05, while F1 and F2 represent the enhancing factors, set to 3 and 5, respectively. $X_{kit}$ represents a random number lying within the range of 0 to 1 for track k, position i, iteration t, and $R_{ki}$ corresponds to the random number for track k at position i. CR is the crossover rate, set to 0.6, and r, n, and m are random tracks distinct from track k.

The track can select an IB in the current iteration regardless of the selection made in the last or previous iterations, but the chances to select each IB can be reduced or increased depending on the solution quality generated using that IB. The probability function of selecting an IB b in iteration t is shown in Equation (12). When F is set to 0.2, $A_{bt-1}$ is the average solution quality of all tracks that have selected IB b so far, and $N_{bt-1}$ is the number of tracks that have been selected (IB p) up to the current iterations. $I_{bt-1}$ will be increased by 1 if the IB contains $B_i^{gbest}$, and increase by zero otherwise; K is set to 20 (constant number). All predefined parameters have been set according to the preliminary test of this research.

$${\mathrm{P}}_{\mathrm{b}\mathrm{t}}=\frac{{\mathrm{F}\mathrm{N}}_{\mathrm{b}\mathrm{t}-1}+(1-\mathrm{F}){\mathrm{A}}_{\mathrm{b}\mathrm{t}-1}+\mathrm{K}{\mathrm{I}}_{\mathrm{b}\mathrm{t}-1}+\mathsf{\rho }|{\mathrm{A}}_{\mathrm{b}\mathrm{t}-1}-{\mathrm{A}}_{\mathrm{t}-1}^{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}|}{\sum _{\mathrm{b}=1}^{\mathrm{B}}{\mathrm{F}\mathrm{N}}_{\mathrm{b}\mathrm{t}-1}+(1-\mathrm{F}){\mathrm{A}}_{\mathrm{b}\mathrm{t}-1}+\mathrm{K}{\mathrm{I}}_{\mathrm{b}\mathrm{t}-1}{\mathrm{I}}_{\mathrm{b}\mathrm{t}-1}+\mathsf{\rho }|{\mathrm{A}}_{\mathrm{b}\mathrm{t}-1}-{\mathrm{A}}_{\mathrm{t}-1}^{\mathrm{b}\mathrm{e}\mathrm{s}\mathrm{t}}|}$$

(12)

3.5. Probability Update for the IB

For every iteration, the values of the following parameters need to be updated according to the current situation. These factors are $R_{ki},X_{kit}{N}_{bt}$, $A_{bt}$, $B_i^{gbest}$, and $I_{bt-1}$. Then, the steps in Section 3.2 and Section 3.3 need to be iteratively executed until the termination condition is met (limited computational time or number of iterations).

3.6. The Comparison Methods

In this study, we compared the proposed methods using the differential evolution algorithm (DE) and particle swarm optimization (PSO), which is adaptive, allowing it to fit to our method [52,53]. Figure 3 shows the framework of the proposed model, which we have developed to classify the types of leaf abnormalities seen in C. asiatica (Linn.) Urban.

Figure 3 shows image segmentation for the training dataset using three distinct methods. After this, the segmented results are subjected to one of four decision fusion strategies: unweighted average (UWA), differential evolution algorithm (DE), particle swarm optimization (PSO), and Variable Neighborhood Strategy Adaptive Search (VaNSAS). Subsequently, the segmented images are processed through image augmentation before being fed into a heterogeneous ensemble of MobileNetV3, SqueezeNetV2, and ShuffleNetV2 CNN models. Similar to the segmentation step, the CNNs are fused using the same decision fusion strategies. Finally, the classification is predicted. On the other hand, the testing dataset follows the same procedure but omits image augmentation.

3.7. Evaluation of Performance Metrics

The assessment of performance across a varied collection of models, encompassing both cutting-edge and previously established models tailored to analogous datasets, will be undertaken utilizing the ensuing metrics: (1) accuracy, (2) F1-score, and (3) AUC (Area Under the Curve). The computation of accuracy and the F1-score are delineated through Equations (13) and (14), respectively. Furthermore, an elaborate exposition of the AUC is provided in the subsequent sections for comprehensive understanding.

$$Accuracy=\frac{n_{TP}+n_{TN}}{n_{TP}+n_{PN}+n_{FP}+n_{FN}}$$

(13)

$$\mathrm{F}1-\mathrm{score}=\frac{2n_{TP}}{{2n}_{TP}+n_{FP}+n_{FN}}$$

(14)

where $n_{TP}$ denotes the number of true positives, $n_{TN}$ corresponds to the number of true negatives, $n_{FP}$ signifies the number of false positives, and $n_{FN}$ represents the number of false negatives. Beyond accuracy, the Area Under the Receiver Operating Characteristic (ROC) Curve (AUC) emerges as a pivotal metric for evaluating performance, particularly within the realm of binary classification endeavors. The ROC curve delineates the equilibrium between the true positive rate and the false positive rate, with the AUC quantifying the extent of this curve. Elevated AUC values signify enhanced model efficacy, rendering it an indispensable instrument for the comparative analysis of diverse models. Collectively, these metrics furnish a holistic appraisal of the deep learning model’s capabilities and limitations, which is imperative for informed model selection and optimization processes. The algorithm employed to compute the AUC in our study is accessible via the Scikit-learn documentation at ‘https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html.(accessed on 2 October 2023)’.

4. Computational Framework and Result

In this study, we utilized two computing resources to develop and evaluate our algorithm. For the training phase, we used Google Collaboratory’s resources, including an NVIDIA Tesla V100 with 16 GB of RAM, for efficient model training. To evaluate our model’s performance, simulations were conducted on a separate system with two Intel Xeon-2.30 GHz CPUs, 52 GB of RAM, and a Tesla K80 GPU with 16 GB of GPU RAM, capable of handling computational demands and providing reliable results. All methods proposed and evaluated in this study were developed in Python 3.10.8 and executed on the hardware configuration previously described. To achieve the best results, we divided the computational work into three phases. Firstly, we tested various combinations of entities to identify the optimal configuration for our proposed model. Secondly, we compared the effectiveness of our optimized model with state-of-the-art methods from the literature. Lastly, the model was tested with an unseen dataset using the leaf abnormality dataset collected in-house. This two-part approach ensures that our proposed model is both optimized and effective compared to existing approaches.

4.1. Optimizing the Model Entities

The entities of the proposed model are the exiting of image segmentation, image augmentation, and the use of meta-learning or the simple unweighted average (UWA) as the decision fusion strategy. The CAU leaf abnormality classification deep learning model should be evaluated using multiple metrics, such as “accuracy”, “AUC”, and “F1-score”, to ensure a comprehensive understanding of the model’s performance. These metrics take into account the various aspects of the classification task, such as true positive rate, false positive rate, and precision. Using these metrics can help researchers accurately evaluate the model’s performance and compare it to other models proposed in the literature. The computational results for the accuracy, AUC, and F1-score of each experimental treatment using different inputs of the model entities are shown in

Table 3. Experimental results reveal the optimal combination of model entities.

No.	Segmentation		Augmentation		Decision Fusion Strategy				Accuracy	AUC	F1-Score
	No Segment	With Segment	No Augment	With Augment	UWA	DE	PSO	VaNSAS
1		-		-		-	-	-	85.65	86.06	86.74
2		-		-	-		-	-	83.81	84.17	85.09
3		-		-	-	-		-	84.02	84.23	85.11
4		-		-	-	-	-		84.59	85.94	86.32
5		-	-			-	-	-	86.71	86.82	87.03
6		-	-		-		-	-	88.28	88.40	89.55
7		-	-		-	-		-	89.85	90.03	91.28
8		-	-		-	-	-		90.48	90.85	92.47
9	-			-		-	-	-	89.63	90.58	91.38
10	-			-	-		-	-	90.41	90.96	91.57
11	-			-	-	-		-	92.19	93.17	93.26
12	-			-	-	-	-		93.46	93.63	94.02
13	-		-			-	-	-	92.82	94.01	94.23
14	-		-		-		-	-	94.39	94.48	94.75
15	-		-		-	-		-	94.54	95.01	95.34
16	-		-		-	-	-		96.31	96.47	96.69

. The average accuracy values using different types of entities are shown in Figure 4.

Based on the computational results presented in Table 3 and Figure 4, it is evident that image segmentation plays a crucial role in improving the accuracy of the classification model, with an increase of 7.26% compared to the model that does not use it. The use of image augmentation also leads to significant improvement, with a 4.21% increase in accuracy compared to the model that does not use it. Moreover, using VaNSAS as the decision fusion strategy further enhances the solution quality by 2.83%, 2.23%, and 1.18% compared to using UWA, DE, and PSO, respectively. Therefore, it can be concluded that incorporating image segmentation, image augmentation, and VaNSAS as a decision fusion strategy are effective techniques for improving the accuracy of the CAU leaf abnormality classification deep learning model.

4.2. Comparison of Optimal Proposed Model with the State-of-Art Methods (ABL-1)

In this research, we compare our proposed model with state-of-the-art methods listed in

Table 4. Detail of the compared methods.

Method	Description	Number of CNN	Total Size (MB)	Training Time (min)	Testing Time (min/image)
Ho-Mo [13]	Homogenous Ensemble Mobile NetV3	7	35.0	65.38	0.62
Ho-SQ [14]	Homogenous Ensemble Squeeze NetV2	8	38.4	67.09	0.64
Ho-Sh [15]	Homogenous Ensemble Shuffle NetV2	5	37	66.27	0.64
EfficientNet-B2 [54]	Single Model	1	41	70.50	0.68
EfficientNet-B3 [54]	Single Model	1	44	73.94	0.70
ResNet-50 [55]	Single Model	1	44	74.11	0.72
DenseNet121 [56]	Single Model	1	33	58.43	0.56
Inception-ResNet-v2 [57]	Single Model	1	56	78.08	0.84
He-Meta (proposed model)	Heterogeneous Ensemble ShuffleNetV2, Squeeze NetV2 and Mobile NetV3	6	34.4	60.41	0.61

. To ensure a fair comparison, all compared methods were reprogrammed for testing and training with our dataset. Initially, we compared the homogenous ensemble of selected architectures, which included MobileNetV2, SqueezeNetV2, and ShuffleNetV2. Subsequently, we selected single state-of-the-art methods with sizes less than 56 MB to compare with the proposed model while ensuring that the model’s size was no more than 1.6 times that of our proposed model. The computational results of all compared methods are presented in

Table 5. KPIs of the tested dataset using various methods.

Method	AUC	F1-Score	Accuracy
Ho-Mo [13]	96.09	95.78	94.70
Ho-SQ [14]	95.42	95.01	94.81
Ho-Sh [15]	94.27	94.13	93.38
EfficientNet-B2 [54]	95.07	94.19	93.86
EfficientNet-B3 [54]	94.51	94.40	93.92
ResNet-50 [55]	93.46	93.07	92.75
DenseNet121 [56]	93.26	92.11	91.08
Inception-ResNet-v2 [57]	97.69	97.74	97.38
He-Meta (proposed model)	98.95	98.82	98.51

, and we also include the results of 3- and 5-fold cross-validations in

Table 6. Cross-validation metrics.

Method	3-cv			5-cv
	AUC	F1-Score	Accuracy	AUC	F1-Score	Accuracy
Ho-Mo [13]	96.11 ± 0.07	95.81 ± 1.81	94.71 ± 0.64	96.14 ± 0.35	95.83 ± 0.27	94.70 ± 0.37
Ho-SQ [14]	95.46 ± 0.76	95.05 ± 1.87	94.83 ± 0.47	95.43 ± 0.65	95.03 ± 0.29	94.83 ± 0.53
Ho-Sh [15]	94.32 ± 0.58	94.18 ± 0.98	93.41 ± 0.72	94.31 ± 0.86	94.13 ± 0.36	93.38 ± 0.41
EfficientNet-B2 [54]	95.10 ± 0.76	94.23 ± 0.85	93.88 ± 0.86	95.10 ± 0.29	94.21 ± 0.85	93.89 ± 0.25
EfficientNet-B3 [54]	94.54 ± 0.48	94.41 ± 0.78	93.93 ± 1.82	94.51 ± 0.67	94.43 ± 0.52	93.97 ± 0.31
ResNet-50 [55]	93.51 ± 0.94	93.08 ± 0.67	92.76 ± 0.98	93.46 ± 0.73	93.11 ± 0.58	92.77 ± 0.26
DenseNet121 [56]	93.28 ± 0.32	92.13 ± 0.56	91.12 ± 0.45	93.29 ± 0.27	92.16 ± 0.97	91.10 ± 0.12
Inception-ResNet-v2 [57]	97.72 ± 0.43	97.76 ± 0.42	97.39 ± 0.37	97.70 ± 0.47	97.74 ± 0.12	97.41 ± 0.04
He-Meta (proposed model)	98.97 ± 0.19	98.86 ± 0.32	98.55 ± 0.29	98.99 ± 0.27	98.83 ± 0.07	98.55 ± 0.08

[26,27].

After conducting a thorough analysis of the results presented in Table 5 and Table 6, it is evident that the proposed method (He-Meta) outperforms other methods proposed in the literature by an average of 4.85%. Specifically, the He-Meta model provides significantly better accuracy when compared to other models, such as Ho-Mo [19], Ho-SQ [17], and Ho-Sh [18], which are homogenous ensemble models, as well as single models, including EfficientNet-B2 [21], EfficientNet-B3 [21], ResNet-50 [22], DenseNet121 [58], and Inception-ResNet-v2 [21]. The He-Meta model yielded a 4.02%, 3.90%, 5.49%, 4.95%, 4.89%, 6.21%, 8.16%, and 1.16% higher accuracy than the aforementioned models, respectively. When categorized into three groups based on model type, the He-Meta model showed a 4.47% improvement over homogenous ensemble models and a 5.07% improvement over single models with similar or larger sizes. It is important to note that the proposed model achieves these results while maintaining a relatively small model size, making it an efficient and effective solution for image classification tasks.

4.3. Comparison with the Unseen Dataset (ABL-2)

The unseen dataset is an important aspect when evaluating the performance of a machine learning model. This dataset consists of new data that have not been seen by the model during training or validation. Testing the model on the unseen dataset provides a more accurate assessment of how well the model will perform on new and unseen data in the real world. This helps to ensure that the model is not overfitting or memorizing the training data but rather generalizing and learning patterns that can be applied to new data. Therefore, evaluating the model’s performance on both the training and testing datasets, as well as the unseen dataset, is important to ensure the model’s effectiveness and generalizability. The computational result of the testing of ABL-2 is shown in

Table 7. KPIs of the tested dataset using various methods of ABL-2.

Method	AUC	F1-Score	Accuracy
Ho-Mo [13]	96.05	95.73	94.67
Ho-SQ [14]	95.39	94.97	94.74
Ho-Sh [15]	94.25	94.05	93.35
EfficientNet-B2 [54]	95.06	94.12	93.78
EfficientNet-B3 [54]	94.48	94.32	93.87
ResNet-50 [55]	93.44	93.03	92.71
DenseNet121 [56]	93.18	92.08	91.04
Inception-ResNet-v2 [57]	97.68	97.71	97.35
He-Meta (proposed model)	98.93	98.75	98.45

and

Table 8. Cross-validation metrics of ABL-2.

Method	3-cv			5-cv
	AUC	F1-Score	Accuracy	AUC	F1-Score	Accuracy
Ho-Mo [13]	96.13 ± 2.48	95.72 ± 2.03	94.64 ± 1.48	96.14 ± 1.57	95.73 ± 2.02	94.65 ± 2.13
Ho-SQ [14]	95.43 ± 0.81	95.05 ± 0.54	94.77 ± 1.51	95.45 ± 1.86	95.05 ± 2.81	94.78 ± 2.56
Ho-Sh [15]	94.22 ± 0.86	94.16 ± 0.68	93.34 ± 0.74	94.23 ± 1.47	94.18 ± 1.04	93.34 ± 0.85
EfficientNet-B2 [54]	95.07 ± 1.48	94.15 ± 1.67	93.88 ± 1.74	95.08 ± 2.15	94.15 ± 1.51	93.89 ± 1.07
EfficientNet-B3 [54]	94.50 ± 1.85	94.36 ± 1.74	93.89 ± 0.96	94.52 ± 1.16	94.38 ± 1.05	93.91 ± 0.86
ResNet-50 [55]	93.44 ± 0.89	93.08 ± 0.85	92.75 ± 1.47	93.44 ± 1.26	93.08 ± 1.74	92.76 ± 0.94
DenseNet121 [56]	93.26 ± 0.58	92.07 ± 0.53	91.10 ± 0.47	93.28 ± 0.38	92.08 ± 0.31	91.12 ± 0.68
Inception-ResNet-v2 [57]	97.68 ± 0.19	97.72 ± 0.24	97.41 ± 0.27	97.69 ± 0.34	97.74 ± 0.19	97.41 ± 0.48
He-Meta (proposed model)	98.98 ± 0.09	98.84 ± 0.11	98.46 ± 0.18	98.98 ± 0.15	98.85 ± 0.13	98.47 ± 0.07

.

Based on the computational results presented in Table 7 and Table 8, we can observe that our proposed model, He-Meta, outperforms other methods by an average of 4.83% higher accuracy. Specifically, He-Meta achieved 3.99%, 3.91%, 5.46%, 4.97%, 4.87%, 6.19%, 8.14%, and 1.12% higher accuracy than EfficientNet-B2, EfficientNet-B3, ResNet-50, DenseNet121, and Inception-ResNet-v2, respectively. Comparing the three groups of methods, we found that the single model performed worse than the other two types of methods, with 4.45% and 6.05% lower accuracy than the homogenous and heterogeneous ensemble models, respectively.

It is important to note that using ABL-2 as the unseen dataset yielded results consistent with using ABL-1 as the test and train dataset. These findings provide strong evidence supporting the effectiveness and reliability of our proposed method.

Furthermore, our proposed model employs a heterogeneous ensemble approach, which combines different architectures to improve the model’s overall performance. This strategy has been demonstrated to be effective, as our model outperforms existing methods that use single architectures or homogenous ensembles. Figure 5 shows the confusion matrix of the proposed model while it classifies images that have been labeled.

When employing the dataset shown in Figure 5a,b, encompassing the confusion matrices pertaining to datasets ABL-1 and ABL-2, a discernible pattern emerged; the classes NL, RM, ND, PD, LI, and WD exhibited a diminished frequency of misclassifications in comparison to classes WCD and PHD. This phenomenon can be attributed to the intrinsic nature of WCD and PHD, wherein significant visual congruities occurred with the aforementioned classes. For instance, WCD showcased residual verdant portions akin to those characterizing NL, albeit with certain sections subjected to insect-induced damage. This nuanced variation, despite being minimal, engenders ambiguity within the classification process.

Equally, PHD, distinguished by its discrete brown blemishes, showed similarities to the PM, ND, PD, and LI categories. The differentiation lay in the precise location and extent of the brown spotting, thereby engendering confounding similarities across these classes. This equivocal presentation poses challenges to accurate classification.

Notwithstanding these instances of misclassification, it is noteworthy that the cumulative accuracy rate of the model remained notably robust at 98.77%. Ergo, the system, despite its occasional errors, stands as an efficacious tool for the classification of leaf abnormalities.

The outcomes gleaned from these analyses underscore the model’s commendable accuracy; nonetheless, it is patently apparent that targeted enhancements are imperative to surmount such specific and nuanced challenges. In this context, Figure 6 expounds upon the visual manifestations of these classification decisions, leveraging heat maps to delineate the regions of paramount significance within the leaf structure that exert a dominant influence upon the model’s classifications.

The heatmap in Figure 6 provides valuable insights into the model’s classification process for leaf abnormality. It is evident that the model primarily utilizes the visual appearance of the leaf to determine the type of abnormality. The parts of the leaf that are visibly different from the norm are the most influential in determining the classification, as the model searches for patterns that distinguish one type of abnormality from another. This information is critical in understanding how the model operates and can provide guidance in making future improvements to the classification process.

5. Discussion and Implications

Section 5 will analyze the research findings, which are divided into two main parts. Firstly, it will address the utilization of an automated leaf disease classification system for CAU. Secondly, it will assess the effectiveness of the proposed model in comparison to existing methods.

5.1. Enhancing Leaf Abnormality Detection in C. asiatica: An Ensemble Deep Learning Approach

This study presents the development of two novel datasets for leaf abnormalities and diseases, encompassing seven types of leaf anomalies in CAU plants, including yellow leaf, insect bite, brown leaf, black leaf, white leaf, worn leaf, mixed-color leaf, and green leaf. The datasets were collected both from the field and laboratory settings. To evaluate the effectiveness of existing leaf disease detection methods for different crops, we employed DenseNet121 [56], ResNet50 [55], MobileNetV2 [22,26,27], EfficientNet-B2 [54], EfficientNet-B3 [54], and Inception-ResNet-v2 [57] architectures. However, our proposed ensemble deep learning model, which combines ShuffleNetV2, SqueezeNetV2, and MobileNetV3 as the CNN architectures, along with U-net, geometric, and meta-learner for image segmentation, augmentation, and decision fusion, outperformed these methods with an accuracy improvement ranging from 1.16% to 8.61% for CAU leaf abnormality detection.

Remarkably, the proposed model demonstrated remarkable robustness, as its accuracy on an unseen dataset (ABL-2) was only slightly lower compared to the test and train datasets (ABL-1), with a minimal percent difference of 0.020%. This observation suggests the model’s ability to handle variations in datasets and its adaptability to different data distributions. In contrast, when adapting existing methods from the literature, e.g., [17,18,19], to our dataset, their performance was notably lower than the original reports (around 92.28%), underscoring the efficacy of our proposed model in overcoming the challenges associated with new datasets. As a result, our proposed model not only demonstrates effective classification of leaf abnormalities but also exhibits robustness across various types of datasets, setting it apart from other approaches proposed in the literature.

This study contributes significantly to the advancement of leaf abnormality detection in agriculture. Through the introduction of new datasets and an extensive evaluation of various CNN architectures [43,59,60,61], our research expands the scope of leaf disease research, particularly for C. asiatica leaves. The proposed ensemble deep learning model, coupled with innovative image segmentation and decision fusion strategies, presents a novel and effective approach to address the complexities of leaf abnormality classification.

The superior performance of our proposed model, surpassing existing methods by a substantial margin, highlights its accuracy in detecting leaf abnormalities in CAU plants. Furthermore, the model’s robustness in handling unseen datasets emphasizes its generalizability and adaptability to diverse data distributions, reinforcing its practicality for real-world agricultural applications.

The findings of this research hold significant implications for precision agriculture and sustainable crop management. Leveraging the high accuracy and robustness of the proposed model can enable the early detection and precise classification of leaf abnormalities in CAU, empowering farmers to make informed decisions for targeted interventions, minimizing crop losses, and optimizing resource utilization. Implementing this technology in agricultural practices holds promise for enhancing crop productivity and promoting more efficient and environmentally friendly farming practices.

5.2. Advancing Leaf Disease Classification in CAU: A Meta-Learner Guided Ensemble Deep Learning Model

This research introduces a novel ensemble deep learning model specifically designed for the classification of leaf diseases in CAU. The model leverages three distinct image segmentation methods, namely, U-net, Mask R-CNN, and DeepLabV3+, integrated through a meta-learner, resulting in a unified segmentation output. The integration of these techniques leads to a noteworthy improvement of 5.86% in solution quality compared to individual segmentation approaches. Notably, this represents the first instance wherein image segmentation has been shown to significantly enhance the accuracy of the proposed model, aligning with previous works [62,63,64,65,66], which also affirm the efficacy of image segmentation in enhancing model accuracy across various applications.

To enhance the robustness and generalization capabilities of the model, geometric image augmentation has been implemented. This augmentation technique effectively expands the training dataset and contributes to an 8.57% increase in classification accuracy. By incorporating this augmentation strategy, the model becomes more adaptable to variations in leaf disease patterns, leading to more reliable and accurate predictions. The merits of this finding are consistent with the results of Akhyar Ahmed Kathawala et al. [41], Bharati Devi and Amarendra [42], Narayanapur et al. [67], Pandian et al. [46], and Venkatesh et al. [54], wherein image augmentation was successfully applied across diverse problem domains. It is evident that the inclusion of image augmentation in the proposed model for leaf disease classification represents a valuable enhancement.

The proposed ensemble deep learning model for leaf disease classification in CAU combines multiple image segmentation methods and geometric image augmentation to achieve improved solution quality and classification accuracy. This study’s findings highlight the importance of image segmentation and augmentation techniques in enhancing the overall performance of the model and are in agreement with existing literature that recognizes the value of such approaches in diverse applications.

Additionally, the ensemble model leverages three diverse CNN architectures—ShuffleNetV2, SqueezeNetV2, and MobileNetV3—integrated through the meta-learner. This integration results in a notable 2.03% improvement in leaf abnormality classification over the single-model approach. The utilization of multiple CNN architectures enhances the model’s capacity to capture complex and subtle patterns, allowing for a more comprehensive understanding of leaf diseases in CAU.

The computational findings provide compelling evidence of the effectiveness of the proposed model in addressing the challenges associated with leaf disease classification. Furthermore, the model’s superior performance over existing single-model and homogeneous ensemble models, including DenseNet121, ResNet50, MobileNetV2, EfficientNet-B2, EfficientNet-B3, and Inception-ResNet-v2, demonstrates its potential to become a new state-of-the-art solution for this domain; it can be adapted to classify diverse datasets as mentioned in Bjånes et al. [68], Hirasen et al. [69], Lee et al. [70], Mohammed and Kora [29], and Wei and Liu [71].

From an academic standpoint, this research contributes to the growing body of literature on advanced deep learning techniques for agricultural applications. The successful implementation of ensemble methods, along with the utilization of meta-learners to combine diverse CNN architectures, showcases the potential of ensemble learning in tackling complex image-based classification tasks. Researchers interested in deep learning methodologies can draw valuable insights from this study and potentially apply similar strategies to other image analysis tasks in various domains.

Moreover, this research has significant policy implications for the agricultural sector. The high accuracy and generalization capability of the proposed model can aid in the early detection and effective management of leaf diseases in CAU, contributing to enhanced crop health and improved agricultural productivity. Policymakers and agricultural stakeholders should consider integrating this advanced deep learning model into their disease monitoring and crop management strategies to make informed decisions and optimize agricultural outcomes.

5.3. In-Depth Analysis of Model Performance and Comparative Evaluation

The computational results of the accuracy, AUC, and F1-score of each experiment treatment using different combinations of model entities were thoroughly analyzed and compared. The experimental results revealed the optimal combination of model entities, showcasing the model’s superior performance in terms of classification accuracy, the area under the curve (AUC), and the F1-score. These metrics were used to accurately evaluate the model’s performance and compare it to other models proposed in the literature.

Furthermore, the proposed ensemble deep learning model leverages three diverse CNN architectures—ShuffleNetV2, SqueezeNetV2, and MobileNetV3—which are integrated through the effective decision fusion strategy. This integration resulted in a notable 8.13% improvement in leaf abnormality classification over the single-model approach. The utilization of multiple CNN architectures enhanced the model’s capacity to capture complex and subtle patterns, allowing for a more comprehensive understanding of leaf diseases in C. asiatica.

The computational findings provide compelling evidence of the effectiveness of the proposed model in addressing the challenges associated with leaf disease classification. The model’s superior performance over existing single-model and homogeneous ensemble models, including DenseNet121, ResNet50, MobileNetV2, EfficientNet-B2, EfficientNet-B3, and Inception-ResNet-v2, demonstrates its potential to become a new state-of-the-art solution for leaf disease classification in C. asiatica plants.

The superior performance of the He-Meta model across multiple metrics does not only showcase its technical excellence but also its practical applicability in the agricultural sector. The advancement over existing models highlighted by our comprehensive and comparative analysis indicates the potential of the He-Meta model to revolutionize plant disease detection methods. Our findings bear significant implications for agricultural practices, particularly in enhancing early disease detection, which is vital for effective crop management and yield improvement.

In addressing the critical challenge of accurate disease detection in agricultural fields, our research introduces a novel two-stage ensemble model that synergistically combines segmentation and classification techniques. The first stage employs an ensemble of advanced segmentation models, including U-net, Mask R-CNN, and DeepLabV3++, to precisely delineate areas of interest within the imagery. This ensemble approach leverages the unique strengths of each model to achieve superior segmentation accuracy, which is crucial for the subsequent classification task.

To mitigate the potential issue of false positives—erroneously identified regions of interest—which could adversely affect the classification accuracy, we implemented a sophisticated post-segmentation analysis. This involved the application of morphological operations and contextual analysis to refine the segmentation output, ensuring that only the most relevant features are forwarded for classification. This step was vital for maintaining the integrity and reliability of the disease detection process.

Following segmentation, the second stage of our model comes into play, focusing on the classification of the identified regions into disease categories. This stage utilizes a complementary ensemble of lightweight, efficient classification models, including ShuffleNetV2, SqueezeNetV2, and MobileNetV3. These models are specifically chosen for their ability to deliver high accuracy with minimal computational overhead, making them ideal for deployment in resource-constrained environments. The integration of segmentation and classification models into a cohesive framework allows for a streamlined workflow that enhances both the accuracy and efficiency of disease detection.

Our approach not only demonstrates the feasibility of combining multiple deep learning models for agricultural disease detection but also underscores the importance of a systematic, two-stage process where segmentation and classification tasks are effectively interconnected. This methodology ensures that the classification stage is primed with the highest quality inputs, significantly reducing the likelihood of false positives influencing the final disease identification, thereby paving the way for more reliable and actionable insights in precision agriculture.

6. Conclusions

In this study, we addressed the limitations of traditional leaf disease detection methods in CAU by introducing a novel ensemble deep learning approach. Our proposed method utilizes two-time decision fusion strategies to integrate multiple machine sectors, aiming to overcome the challenges associated with visual inspection by human experts.

Through ensemble image segmentation, employing U-net, Mask-R-CNN, and DeepNetV3++ as segment methods, and the combination of various types of CNNs, such as ShuffleNetV2, SqueezeNetV2, and MobileNetV3, we achieved significant improvements in solution quality. The ensemble approach, employing four decision fusion strategies, namely, unweighted average (UWA), the differential evolution algorithm (DE), particle swarm optimization (PSO), and Variable Neighborhood Strategy Adaptive Search (VaNSAS), further enhanced the effectiveness of our model.

Our evaluations of the ABL-1 and ABL-2 datasets, comprising 14,860 images with eight distinct leaf abnormalities, revealed the superiority of our ensemble segmentation and heterogeneous ensemble CNN methods. The results indicate significant improvements in solution quality; ensemble segmentation methods led to a 7.34% increase compared to single segmentation methods, while heterogeneous ensemble segmentation outperformed homogeneous segmentation by 6.43%. Similarly, the use of heterogeneous ensemble CNN resulted in an 8.43% enhancement over homogeneous ensemble models and a remarkable 14.59% improvement over single models. Moreover, image augmentation techniques contributed to a 5.37% increase in solution quality. The employment of VaNSAS as the decision fusion strategy yielded substantial enhancements of 15.42% compared to UWA, 11.23% compared to DE, and 9.84% compared to PSO. These numerical results firmly demonstrate the effectiveness of our proposed approach in achieving significantly higher-quality solutions than traditional methods.

To advance the field, future research could explore advanced image augmentation techniques and refine decision fusion strategies to achieve an even higher solution quality. Investigating transfer learning and fine-tuning approaches on larger and more diverse datasets could extend the generalizability of our proposed method for various plant species. Moreover, exploring the integration of multi-modal data sources, such as spectral and hyperspectral imaging, holds promise in further enhancing the accuracy and robustness of leaf disease detection systems.

Ultimately, our goal is to create a user-friendly and automated platform that can be readily utilized by agricultural professionals in the field. By continually advancing research in this direction, we aim to contribute to the ongoing development and application of cutting-edge technologies in agriculture, ultimately fostering increased crop yield, reduced losses, and improved medicinal properties pertaining to CAU.