Real-Time Visual Identification System to Assess Maturity, Size, and Defects in Dragon Fruits ^†

Abstract

In the Philippines, dragon fruit has become an essential, high-value crop and is important to the country’s economy. However, due to inefficient manual inspection methods, farmers need help with quality control and market preparation. Therefore, we developed an automated, real-time visual identification system to detect the maturity, size, and defects of dragon fruits. Advanced deep learning models (EfficientNet and YOLOV8) were trained on a diverse dataset of dragon fruit images collected from online sources and a local farm using DSLR and smartphone cameras. A Raspberry Pi 4B with an HQ camera and wide-angle lens was used as a cost-effective and accessible device for farmers. The developed system showed an accuracy of 93.33% for maturity and size classification, 96.67% for defect detection, and an overall accuracy of 83.33%. Regarding accuracy and reliability, the developed method presents a technological advancement for dragon fruit identification and classification.

1. Introduction

Dragon fruit, known for its striking red skin that resembles dragon scales, is a distinctive tropical fruit. Inside, its flesh is red, yellow, or white, dotted with tiny black seeds. The fruit grows green and ripens to a deep red. Locally, dragon fruit farming is a profitable agricultural business, with the fruit selling for 120 to 150 Philippine pesos (PHP) per kilogram, depending on seasons and locations. Though the industry is still growing in the Philippines, it shows great potential. The quality and safety standards for dragon fruit in the Philippines are guided by the Philippine National Standard (PNS)/Bureau of Agricultural and Fisheries Product Standards (BAFPS) for Fresh Fruit—Dragon Fruit (Hylocereus spp.)—Grading and Classification (PNS/BAFPS 115:2013). These documents outline criteria such as minimum requirements and classifications, with dragon fruit categorized based on size, maturity, defects, and overall quality. Dragon fruit has gained attention locally and internationally, although it is still less recognized than mangoes, bananas, and pineapples.

The potential for advanced technology in improving fruit classification and quality control has been recognized in research. Lozano et al. [1] demonstrated high accuracy in defect classification for vegetables such as ladies’ fingers (90.59%), bitter gourd (90.78%), and cucumbers (89%) using the convolutional neural network (CNN)-based MobileNetV2 model. Hsu et al. [2] achieved 94.1% accuracy for dragon fruit ripeness classification using You Only Look Once (YOLO) v3 and the CNN-based DRC-7 model. Similarly, Parco et al. [3] applied a CNN-based system for avocado defect detection, with an accuracy of 93%.

To maximize profits, farmers must accurately assess the ripeness of dragon fruit before harvest and sale. Consumers tend to avoid unripe fruits, while overripe ones are unappealing due to spoilage. The detection of such defects enables farmers to address crop quality issues, preventing deterioration. Dong et al. [4] explored dragon fruit disease segmentation using the FCM algorithm and the two-dimensional Otsu’s thresholding method (OTSU) algorithm to improve accuracy in defect detection. Although various studies have examined dragon fruit’s maturity, size, and defects, none have been successful. The accuracy depends on the artificial intelligence used, the components integrated, and the correct tuning of these parameters.

We developed and implemented a real-time visual identification system using EfficientNet and YOLOv8, which analyzed dragon fruits’ size, maturity, and defects. The system contained a Raspberry Pi (RPi) 4 Model B, an HQ RPi camera with a 6 mm wide-angle lens, and an LCD for image capture. Image processing techniques and algorithms were employed to accurately assess the features of the dragon fruits. Additionally, a user-friendly graphical user interface (GUI) was designed to generate detailed summary reports about the readiness of each dragon fruit for harvest. The results obtained by the system were compared with those obtained from an experienced dragon fruit farmer to calculate its accuracy.

The results of this study benefit the agricultural industry and those involved in the dragon fruit industry. By delivering real-time data on crop conditions, farmers can predict harvest and sorting times, thereby improving productivity and profitability. Retailers and distributors can benefit from reduced spoilage and consistent product quality monitoring, while consumers can enjoy dragon fruits of better quality.

2. Literature Review

2.1. Dragon Fruit Cultivation

The dragon fruit, Hylocereus spp., is a plant well-known in Central and South America. Also known as “pitaya”, the fruit is cultivated in countries with tropical climates, including the Philippines. Its cultivation and market value are increasing, and its popularity is increasing thanks to its reputation as a profitable crop with high potential [5].

The standards for dragon fruits in the Philippines were formulated by the Philippine National Standard/Bureau of Agricultural and Fisheries Product Standards PNS/BAFPS 115:2013 [6]. The minimum standards of Philippine dragon fruits must be ripe, firm, sound, and clean. Its ripeness is dictated by the five categories listed in the document: jumbo (>700 g), extra-large (531–700 g), large (431–530 g), medium (331–430 g), and small (230–330 g). The minimum weight must be 230 g. Quality is classified into Extra Class, Class I, and Class II. Dragon fruits in the Extra Class must show superior quality with minimal defects. Dragon fruits in Class I have minor defects of no more than 5% of their surface area, while Class II fruits have defects of up to 10%. For the post-harvesting and disease management of the dragon fruit, research is necessary to enhance production efficiency.

2.2. CNN

Significant strides have been made in using deep learning models for computer vision, notably with the advent of the EfficientNet and YOLOv8 architectures. EfficientNet has improved CNNs through its compound scaling method. EfficientNet scales the network’s width, depth, and input resolution uniformly [7]. Such an approach allows EfficientNet to deliver top-notch performance with a reduced number of parameters with conventional models. Its high efficiency is ensured by its proficiency in capturing intricate features and an ideal equilibrium between the model’s size and accuracy across many computer vision endeavors. YOLOv8 is the newest algorithm in YOLO, honing in on real-time object detection, segmentation, and image classification. It advances by adopting an anchor-free method to enhance the detection of smaller objects and utilizing a tweaked cross-stage partial darknet (CSPDarknet) for superior feature extraction [8]. Enhancements in Mosaic augmentation enhance loss functions that are transitioned to the PyTorch framework. YOLOv8 is a robust tool with speed and precision, and is especially useful in identifying fruits and assessing their quality [9]. There are numerous machine-learning models for tackling fruit identification, including CNNs. In Ref. [10], 86.67% accuracy was obtained in identifying dragon fruit ripeness using a backpropagation neural network, while a smaller visual geometry group net (VGGnet) achieved 96.67% accuracy [10].

2.3. Algorithms for Image Processing

Image processing algorithms enable precise and efficient crop monitoring and management. By employing computer vision, machine learning, and remote sensing technologies, the algorithms identify images collected from drones, satellites, or terrestrial sensors. Various aspects of crop welfare, including health, nutrient levels, pest activity, and overall field conditions, are informed by the algorithm.

In this study, we used three algorithms: Otsu’s thresholding, the Watershed algorithm, and RGB masking. Otsu thresholding segments images into foreground and background by determining a threshold to minimize the variance within classes while maximizing the variance between classes [11]. This method is effective in distinguishing objects from their backgrounds, though its efficiency is compromised by overlapping intensity distributions or inconsistent lighting [12]. Based on mathematical morphology, the Watershed algorithm identifies and segments objects that overlap or are closely situated by conceptualizing pixel intensities as elevations in a topographical map. By “flooding” this landscape, it generates watershed boundaries and demarcates object edges. Although it is prone to over-segmentation, its accuracy is improved with marker-controlled methodologies, making it a valuable tool in medical imaging and remote sensing applications. RGB masking isolates objects within an image by filtering colors through the red, green, and blue (RGB) channels. This method is adept at distinguishing objects based on color characteristics, though it is affected by fluctuations in lighting. Therefore, it requires supplementary techniques for enhanced feature extraction.

2.4. Size, Maturity, and Defects

Image-based techniques are critical to detecting the size, maturity, and defects in agricultural products to improve the efficiency of sorting and packaging and ensure product quality. Cameras and sensors are used to determine the size of agricultural products by evaluating visual properties. Mengoli et al. applied the YOLOv5 Neural network algorithm for AI-driven classification of fruit sizes [5].

Digital images are vital for extracting dimensions of length, width, volume, and morphological operations. Wang et al. underscored the significance of 3D shape data in assessing the quality of fruits [13]. To gauge fruit maturity, spectral analysis methods are used. The methods are used to analyze the electromagnetic radiation that fruits emit or reflect. Models are created for predicting maturity, as evidenced by Tushar et al., who developed hyperspectral technology that surpassed traditional RGB imaging in determining the maturity of peanuts [14]. Texture analysis is used for categorizing fruit maturity by examining physical attributes such as firmness and suppleness. Al Riza et al. used this method to evaluate mangoes’ ripeness through reflectance and fluorescence imaging systems [15]. In identifying defects, algorithms for detecting edges are essential to detect surface anomalies, including bruises or cuts [16]. Machine learning algorithms, trained on detailed datasets [14], efficiently pinpoint defects such as fungal infections or physical damage by analyzing characteristics such as shape, color, and texture. By incorporating advanced image processing techniques such as EfficientNet and YOLOV8, agricultural practices can be improved to increase accuracy and efficiency.

3. Methodology

3.1. Research Framework

Figure 1 illustrates the research framework of this study. Initially, images of dragon fruits were captured by a camera. The captured images were preprocessed by specially designed algorithms to determine features such as maturity, size, and any defects. The results were presented on the GUI and the LCD.

3.2. System Components

Figure 2 illustrates the system’s architecture. The Raspberry Pi High Quality (HQ) camera was responsible for image acquisition in a 12 MP resolution using a 6 mm wide-angle lens. To ensure the ideal distance between the camera and the object, an E18-D80NK proximity sensor was used. The Raspberry Pi 4 Model B was employed for image collection and processing. A 7-inch Raspberry Pi touchscreen display displayed the images and facilitated user interaction. The system was powered by a 20,000 mAH power bank, ensuring sustained operation.

3.3. Design of the GUI

The GUI was constructed as shown in Figure 3. Tkinter V 8.6 was used as it is a widely recognized standard GUI toolkit in Python V 3.12.4. The layout of the GUI components was designed with a proximity sensor connected to the GPIO pins to sense the presence of an object in a specific range. The system showed a “shutter button” once the object was detected. The button was used to trigger the camera to capture and display the image and allow the user to retake, save, or generate a report. The GUI concealed unnecessary buttons and revealed the shutter button if necessary. The captured images were numbered and named automatically. A reset button was added to reinstate the initial state of the GUI, and reset the image counter and file names saved in the directories. The report was created based on the analysis results of maturity, defects, and size using TensorFlow Lite models (converted from .h5 files for better compatibility) and PyTorch files (.pt file). A separate background process was linked to the GPIO sensor to control the visibility of the buttons and the live feed from the camera. The “Instructions” button provided all the necessary information the user needed before using the device, while the “Exit” button allowed the camera and the GPIO configurations to shut down correctly.

3.4. Image Processing

The dataset was compiled using online images from Google and locally produced dragon fruits. The datasets included acquired images from online sources, specifically Google. The images showed various maturities, sizes, and defects. Fruit maturity was determined by size and color. The dataset contained 3593 images for maturity analysis (with 88% or 3144 images allocated for training post-augmentation, 8% or 299 images for validation, and 4% or 150 images for testing), 3729 images for defect identification (with 88% or 3264 images for training post-augmentation, 8% or 311 images for validation, and 4% or 154 images for testing), and 864 images for sizing (with 88% or 756 images for training post-augmentation, 8% or 72 images for validation, and 4% or 36 images for testing). The data were split in a 70:20:10 ratio for training, validation, and testing datasets.

The system processed the images to enhance image classification accuracy. The preprocessing involved resizing or cropping images, normalizing, standardizing, augmenting, and transforming. Filters and smoothing techniques were also applied. Feature extraction is a significant technique in image segmentation for identifying maturity, size, and defects. Such features were identified considering dragon fruits’ appearance and physical characteristics. Shakil et al. [17] noted that dragon fruits’ shape and surface imperfections indicate their quality, which correlates with the earlier parameters. The presence of spines and coloration are indicators of maturity, the fruit’s shape uniformity suggests size, and variations in color and texture are used to distinguish defects. Image augmentation was employed to bolster the model’s resilience against overfitting. The augmentation process included geometric transformations like flipping, cropping, rotating, stretching, and zooming, along with color space adjustments, image mixing, random erasing, and applying kernel filters, among other techniques.

4. Results and Discussions

4.1. Evaluation Comparison

The system’s classification performance was evaluated by comparing its results with the classification by farmers (

Table 1. Evaluation of dragon fruits based on the prototype’s evaluation.

Evaluation of Dragon Fruits
Fruit Number/Trial	Prototype’s Evaluation
	Size	Maturity	Defects
1	Small	Mature	No presence of defects
2	Unknown	Immature	Presence of Defects
3	Small	Mature	No presence of defects
4	Unknown	Immature	Presence of Defects
5	Large	Mature	No presence of defects
6	Large	Mature	No presence of defects
7	Medium	Mature	No presence of defects
8	Unknown	Immature	No presence of defects
9	Medium	Mature	No presence of defects
10	Small	Mature	No presence of defects
11	Medium	Mature	Presence of Defects
12	Medium	Mature	Presence of Defects
13	Large	Mature	Presence of Defects
14	Medium	Mature	No presence of defects
15	Large	Mature	No presence of defects
16	Unknown	Immature	No presence of defects
17	Medium	Mature	Presence of Defects
18	Medium	Mature	Presence of Defects
19	Unknown	Immature	No presence of defects
20	Unknown	Immature	Presence of Defects
21	Medium	Mature	No presence of defects
22	Unknown	Immature	Presence of Defects
23	Unknown	Immature	Presence of Defects
24	Unknown	Immature	Presence of Defects
25	Unknown	Immature	Presence of Defects
26	Unknown	Immature	No presence of defects
27	Unknown	Immature	Presence of Defects
28	Unknown	Immature	No presence of defects
29	Unknown	Immature	No presence of defects
30	Unknown	Immature	No presence of defects

and

Table 2. Evaluation of dragon fruits based on the farmer’s evaluation.

Evaluation of Dragon Fruits
Fruit Number/Trial	Farmer’s Evaluation
	Size	Maturity	Defects
1	Small	Mature	No presence of defects
2	Medium	Immature	Presence of Defects
3	Small	Mature	No presence of defects
4	Medium	Immature	No presence of defects
5	Large	Mature	No presence of defects
6	Large	Mature	No presence of defects
7	Medium	Mature	No presence of defects
8	Small	Mature	No presence of defects
9	Medium	Mature	No presence of defects
10	Small	Mature	No presence of defects
11	Medium	Mature	Presence of Defects
12	Medium	Mature	Presence of Defects
13	Large	Mature	Presence of Defects
14	Large	Mature	No presence of defects
15	Large	Mature	No presence of defects
16	Medium	Immature	No presence of defects
17	Large	Mature	Presence of Defects
18	Medium	Mature	Presence of Defects
19	Medium	Immature	No presence of defects
20	Large	Immature	Presence of Defects
21	Medium	Immature	No presence of defects
22	Small	Immature	Presence of Defects
23	Medium	Immature	Presence of Defects
24	Medium	Immature	Presence of Defects
25	Medium	Immature	Presence of Defects
26	Small	Immature	No presence of defects
27	Large	Immature	Presence of Defects
28	Medium	Immature	No presence of defects
29	Small	Immature	No presence of defects
30	Medium	Immature	No presence of defects

). This comparative analysis was conducted to validate the system’s accuracy and effectiveness in classifying dragon fruits.

4.2. Statistical Analysis

We conducted a descriptive analysis based on the results presented in Table 1 and Table 2. The overall accuracy was calculated by comparing the ratio of correctly classified fruits with that of the farmer’s classification (1).

$$\%Accuracy=|{\displaystyle \frac{T_C}{T_F}}|$$

(1)

where T_C is the total number of fruits correctly classified by the system and T_F is the total number of fruits.

Immature dragon fruits were labeled “unknown” in size. Due to their ongoing growth, the current size measurements were not effective. “Overall matching characteristics” refers to the system’s classification as matched to the farmer’s (

Table 3. Statistical analysis results.

Parameters	Classification	Total Number of Fruits (TF)	Total Number of Fruits Correctly Classified by the System (TC)	Accuracy
Maturity	Mature	15	14	93.33%
	Immature	15	14
Size	Small	7	7	93.33%
	Medium	15	14
	Large	8	7
Defect	Presence of Defects	12	12	96.67%
	No presence of defects	18	17
Overall Matching of Characteristics	Matched	30	25	83.33%
	Not Matched	30	5	16.67%

).

Table 4. Statistical analysis results table.

Classification Category	Accuracy
Size	93.33%
Maturity	93.33%
Defects	96.67%
Overall	83.33%

shows the resulting accuracies for each category of dragon fruit used in this study. The size and maturity showed an accuracy of 93.33%, while defect classification resulted in 96.67% accuracy. The overall accuracy was 83.33%. The developed system in this study performed well in the classification of defective dragon fruit and effectively and reliably classified dragon fruit in terms of their maturity, size, and defects.

5. Conclusions and Recommendations

We developed a visual identification system to assess the maturity, size, and defects of dragon fruits. By employing EfficientNet and YOLOv8, the system classified the fruit’s maturity, size, and defects accurately. The size, maturity, and defects of dragon fruits were recorded and compared with those estimated by farmers utilizing traditional observation methods. The comparison results showed similarities between the results of the two methods. It is necessary to implement a measuring system to indicate the optimal distance between the camera and the object. The dataset needs to be expanded, and advanced algorithms need to be used to optimize the classification capabilities of the system. The output reports also need to be improved.