Next Article in Journal
Assessment of Biometric Parameters and Health of Canna’s Cultivars as Plant Useful in Phytoremediation of Degraded Agrocenoses
Previous Article in Journal
Internet and Computers for Agriculture
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Agriculture
Volume 13
Issue 1
10.3390/agriculture13010156
agriculture-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Oil Palm Fresh Fruit Bunch Ripeness Detection Methods: A Systematic Review

by
Jin Wern Lai
,
Hafiz Rashidi Ramli
*,
Luthffi Idzhar Ismail
and
Wan Zuha Wan Hasan
Department of Electrical and Electronic Engineering, Faculty of Engineering, Universiti Putra Malaysia, Serdang 43400, Selangor, Malaysia
*
Author to whom correspondence should be addressed.
Agriculture 2023, 13(1), 156; https://doi.org/10.3390/agriculture13010156
Submission received: 28 October 2022 / Revised: 23 November 2022 / Accepted: 23 November 2022 / Published: 8 January 2023
(This article belongs to the Section Agricultural Technology)
Browse Figures
Versions Notes

Abstract

:
The increasing severity of the labour shortage problem in the Malaysian palm oil industry has created a need to explore other avenues for harvesting oil palm fresh fruit bunches (FFBs) such as through autonomous robots’ deployment. However, the first step in using an autonomous system to harvest FFBs is to identify which FFBs have become ripe and are ready to be harvested. In this work, we reviewed previous and current methods of identifying the maturity of fresh fruit bunches as found in the literature. The different methods were then compared in terms of the types of sample data used, sensor modalities, and types of classifiers used with a particular focus on the feasibility of each method for on-field application. From the 51 papers reviewed, which include a total of 11 unique approaches, it was found that the most feasible method for detecting ripe FFBs in the field is a combination of computer vision and deep learning. This system has the advantages of being a noncontact approach that is low cost while also being able to operate in real time with high accuracy.

1. Introduction

Malaysia is one of the biggest palm oil producing countries in the world. The palm oil industry is a significant contributor to the country’s gross domestic product (GDP), and its development is still on an upward trend. The palm oil companies have more than a million hectares of plantation land to produce fresh fruit bunches (FFBs). These bunches contain the precious palm oil, and they are harvested when they have ripened. Therefore, several rules and guidelines were developed in order to achieve the maximum oil extraction rate (OER) from the FFBs when they are processed in the mills. The first step in maximising oil extraction is making the correct selection of FFBs to be harvested. This is accomplished by referring to the guidelines from the Malaysian Palm Oil Board (MPOB). On the oil palm estates, the FFBs can only be harvested once the oil palm trees reach maturity three years after planting. The field workers will harvest the FFBs during the 10–14 days of the harvesting interval. The harvesters will search for oil palm trees with a certain number of detached fruitlets that have dropped to the ground. This signifies that there are ripe FFBs on the tree. The workers will then visually identify the FFBs that are ripe based on the colour of the FFBs as well as the sockets that are formed when the fruitlets drop to the ground. FFBs that meet the correct criteria are considered to have ripened and should be harvested. The harvested FFBs will then be collected and transported to the palm oil mill for oil extraction [1]. In general, the FFBs are delivered to the mill within 24 h after harvesting to ensure that the quality of the fruit is maintained. However, this is not guaranteed due to factors such as unpredictable thunderstorms during the afternoon and other logistical incidents. This can cause the FFBs to become overripe as they are processed in the mill which will result in oil that is of reduced quality.
The oil palm industry requires technologically advanced equipment to minimise dependence on labour, especially with the labour shortage crisis faced in recent years. Efforts to implement state-of-the-art technology for harvesting the FFBs such as mobile robots that can operate autonomously are becoming increasingly essential for maintaining and increasing productivity levels. In addition, such innovations could also be used to eliminate the possibility of human error in decision-making tasks that affect productivity [2].
The first step in realising an autonomous FFB harvesting system is to develop a system that would be to be able to accurately and objectively identify ripened FFBs that are ready for harvest. In the last decade, numerous methods of objectively identifying the maturity of FFBs have been proposed by researchers to help ensure that the FFBs harvested have reached the proper maturity stage. In this paper, we present a comprehensive review of the various systems that are able to detect the maturity of FFBs with an emphasis on examining the feasibility and readiness of the systems developed to be used on-field. This is because it is in the interest of the industry to address the labour shortage issue with the aforementioned technologies as soon as possible. The inclusion criteria for papers reviewed are:
(a)
Published between 2012 to 2022.
(b)
The written language is English only
(c)
Found using the following combination of keywords in Google Scholar: oil palm, fresh fruit bunch, freshness, mature, maturity ripeness, grading, assessment, classification, detection, identification.
(d)
The papers found underwent a final filtering process where only the papers that were presenting the results of a proposed method for detecting the ripeness of (exclusively) oil palm FFB were selected for this review.
The following is the structure of the current work: Section 2 introduces the popular learning models and algorithms for detecting the maturity of FFBs. The description of the adopted approach for the collection and categorisation of the presented works is discussed in Section 3, and prospects in the domain are highlighted. Finally, the findings of the review are summaried in Section 4.

2. Fresh Fruit Bunch Ripeness Evaluation Methods

Generally, there are two types of methods to analyse the ripeness of FFBs, destructive methods and nondestructive methods. The destructive methods require physical contact in ways that affect the integrity of the FFBs and severely reduce the amount of oil that can be extracted from the tested FFBs. Meanwhile, the nondestructive techniques can involve noncontact features that are either visual or nonvisual. Nonvisual methods include using physicochemical, electrical, magnetic, and electromagnetic properties to determine fruit maturity such as fluorescence sensor, microwave sensor, inductive sensor, and thermal sensor. However, most nondestructive methods are visual, in which the selected sensor can evaluate through the morphology, colour, and other physical characteristics of the fruit to identify their maturity stages through the analysis of specific modalities such as colour vision, light detection and ranging, spectral image, and near-infrared spectroscopy.

2.1. Nondestructive Methods

2.1.1. Colour Vision

In the past, the maturity of FFB was determined by human observation. The observation process required labour cost, and also misclassifications could occur. Therefore, image analysis using colour vision was more suitable for screening FFBs because it is commercial and easy to handle. After the image is captured, inspection begins using image processing software and data acquisition software. Usually, a colour image forms and appears that combines the red, green, and blue (RGB) tristimulus values. Hence, the FFBs are captured using image processing, and the image captured undergoes analysis to evaluate ripeness by RGB colour [3]. Recently, to enhance the performance of maturity classification on FFB, different segmentation methods were proposed using colour vision systems. Septiarini et al. [4] implemented the Otsu method to segmentize the area of the oil palm fruit from the background in the image and obtain an iterative threshold. Septiarini et al. [5] implement the Canny algorithm combined with several morphology operations to remove the noise in the image.
To increase the accuracy of FFB maturity recognition, advanced artificial intelligence tools such as machine learning or deep learning are required [6]. For example, images are collected and preprocessed to train under the artificial neural network (ANN), and the ANN learns the features from those images [7,8]. Moreover, the support vector machine (SVM) and naïve Bayes classifier are applied on maturity grading after colour images are captured [9].
Furthermore, a multispectral system was proposed for evaluating the ripeness grading. The multispectral system was able to select specific optical filters with different wavelengths for penetrating objects or blocking noise. Therefore, the internal components and properties in the fruit could be discovered. Using the hyperspectral camera to capture the image could verify the amount of water content in the oil palm fruitlets. The multispectral system could perform even higher spatial resolution and image quality compare to the traditional system [10].

2.1.2. Light Detection and Ranging (LiDAR)

Generally, LiDAR is a remote sensing technology that is applied in precision agriculture, archaeology, and other applications. The use of LiDAR has the potential in the classification of FFB maturity since LiDAR can acquire the actual distance and intensity from the target object. The LiDAR sensor emits laser light to the targeted object and receives the reflected light signal to calculate the distance between the sensor and the object. LiDAR also calculates the intensity of the object, which is the key to distinguishing the ripeness stage of FFB [11]. Hashim et al. [12] verified the accuracy of using LiDAR sensors in FFB maturity. In the experiment, LiDAR Lite v2 sensor was used to experiment in the laboratory and on-field. The LiDAR Lite v2 emitted a pulse of light with a 905 nm wavelength, and the reflected signal was received and transmitted back. The data were used to plot the graphical representation of the point cloud 3D mapping representing different maturity stages. Husin et al. [13] also used LiDAR on FFB maturity classification. In the experiment, LiDAR Lite v3 was used, and it received the reflected light signal to calculate the mean of return intensity. The researchers enhanced the use of LiDAR technology by creating a distribution map based on received latitude and longitude location. This will pinpoint the location of the ripe FFBs that are ready for harvesting.

2.1.3. Optical Sensor—Spectral Image

Several different wavelengths can be used to capture FFB images. The ultimate goal of multispectral image analysis is to examine the image intensity at various wavelengths especially in the ultraviolet (UV), visible light, and infrared spectra. One of the key benefits of this examination is that more information detail can be investigated; for instance, the object’s chemical components can be retrieved. The hyperspectral camera is used to take multispectral images in order to measure the maturity level of palm oil FFB. The relative water content in palm oil FFBs can be determined using these images [10]. In our studies, the hyperspectral camera is not the only appliance for building an optical sensor system. Others researchers used several LEDs with different wavelengths as their source light to capture images of the FFBs for determining the maturity stage [14]. Moreover, the FFB image data were collected using a handheld four-band active sensor [15]. Four narrow-band (active optic) light sources and reflectance sensor components of various wavelengths were used in the system: two in the visible area (at 570 and 670 nm), one in the red-edge region (750 nm), and one in the near-infrared region (870 nm). For four bands, the spectral band width was around 50 nm. Each spectral area indicated distinct qualities that could be used to classify palm oil FFB maturity.

2.1.4. Near-Infrared Spectroscopy (NIR)

NIR is applied in the agriculture industry. The advantages of this method are nondestructive, and both physical and chemical substances can be measured. NIR spectroscopy analyses the chemical composition of fruit such as aqueous mixture, sugar level, firmness, and acidity. This method fills the gap in classifying FFB maturity. Traditionally, the harvester may make mistaken analyses of ripeness based on visual evaluation of features such as shape, size, and colour. Usually, the maturity stage of fruit can be identified by wavelength between 750 nm to 2500 nm. The electromagnetic spectrum in the near-infrared region can penetrate the organic substance because it has low absorption and low reflection [16]. Through the process, the substance properties allow for distinguishing the maturity grades of fruit, as the chemical composition of fruit changes significantly from unripe to ripe. Silalahi et al. [17] pointed out NIR spectroscopy as an alternative method for grading the ripeness of FFB by analysing the chemical composition of fruits. The samples were scanned under a near-infrared sensor with wavelengths of 1108–2500 nm. The test results showed the ripening of mesocarp changes depending on the oil content and moisture content. In addition, NIR spectroscopy was also applied to grading the ripeness of FFB combined with using a genetic algorithm neural network [18]. The variables that are used in the genetic algorithm are provided from PCA in the NIR spectral data. The reliability of the genetic algorithm model is based on statistical measurements such as mean absolute error (MAE), root mean squared error (RMSE), and the accuracy of the classification.

2.1.5. Raman Spectra

Raman spectroscopy has been used to determine the maturity and freshness of tomatoes [19] and citrus fruits [20] by detecting the molecular vibrations of carotenoids and other chemical components found in the fruit’s skin. Both a confocal Raman spectrometer and a portable Raman spectrometer were used to demonstrate these findings. A Raman-based device measures inelastic scattering from a compound’s surface, which is then used as a molecular fingerprint. A leaf-clip-based Raman probe successfully proved to provide real-time monitoring for early identification of nutritional insufficiency. Recently, Raj et al. [21] used Raman spectroscopy to determine the maturity of oil palm fruitlets. The organic characteristics of the exocarp of oil palm fruitlets were studied using a confocal Raman spectroscopy device. They led to the realization that carotene content is one of the most important factors in determining the maturity of oil palm fruits. They showed that the Raman spectrum’s carotene peak may be further deconvoluted and processed to identify four carotenoid components: carotene, lycopene, lutein, and neoxanthin. These elements and their derivations are used as features in machine-learning-based classification algorithms to automatically identify the maturity state of oil palm fruits.

2.1.6. Fluorescence Sensing

The use of fluorescence sensing on FFB classification was introduced by Hazir et al. [22]. The researchers measured the content of flavonoids and anthocyanin in the plant cell system using fluorescence methods. The flavonoids and anthocyanin are the phenolic content in the plant system that protect them from insect attack, and the phenolic content is important for attracting animals for seed dispersal. The researchers applied the fluorescence based on the phenolic content in the fruit. When UV light hits the epidermis, the flavonoids absorb the UV light. At the same time, the chlorophyll from mesophyll emits near-infrared fluorescence on the plant leaf. Meanwhile, anthocyanins are produced in the epidermis by absorbing the green light. Therefore, the phenolic content in FFBs can be measured to identify different maturity stages on the FFB after fluorescence scanning. Hazir et al. [23] used fluorescence sensors to classify the ripeness of FFB. The researchers investigated the implementation of the blue-to-red fluorescence ratio (BRR_FRF) as a parameter to differentiate the maturity of FFB. The researchers used the Multiplex*3 sensor in the experiment. The Multiplex*3 sensor functions under four types of lights, UV, blue, green, and red. The sensor has three photodiodes for detecting purpose on the fluorescence emission. The bandwidths of photodiodes emission are blue-green (590 nm), red (685 nm), and far-red (735 nm). The data are collected and measured. The BRR_FRF can be obtained by dividing the blue-green emission signal by the far-red emission signal. The data were obtained and trained by classification tree using SPSS 16.0.

2.1.7. Microwave Method

Microwave methods can be applied to measure the ripeness of FFB. Yeow et al. [24] developed a low-cost coaxial moisture sensor to classify the maturity stage of oil palm fruit by determining its moisture content. This method was applicable because the water molecules in the fruit are sensitive when exposed to the microwave. This is because the water molecules have the intention to absorb microwave energy. When the microwave sensor emitted light to the targeted oil palm fruitlet, the light was interrupted. The receiver observed the changes in the light and measured moisture content. The microwave sensor detected the moisture content significantly under the frequency range from 300 MHz to 300 GHz. The moisture content in the fruitlet had the highest amount during the unripe stage and the lowest amount when overripe. This is due to the water content diminished and oil content produced after the anthesis in the mesocarp. Using a microwave, the relative complex permittivity ɛr to the agriculture product is shown in equation 1:
ɛr = ɛr′ − j ɛr′′
The variable ɛr′ is the dielectric constant, and ɛr′′ is the dielectric loss factor. Both ɛr′ and ɛr′′ are related to the moisture content because it absorbs the microwave energy. The dielectric constant affects the electric field distribution. Additionally, the energy absorption of the material is induced by the loss factor. The relative permittivity of water and oil also has a huge difference; their relative permittivity is 80 and 2.5 respectively. Hence, the microwave sensor can detect a slight difference in resonant frequency [25].

2.1.8. Inductive Sensing

Inductive sensing technology was first proposed by [26]. In their study, they tested with the frequency from 100 Hz to 100 MHz on unripe and ripe oil palm fruitlets. The result showed a significant difference between unripe and ripe oil palm fruitlets regarding the inductance and resonant characteristics. In addition, after testing with the variation in coil diameter, the single air coil with 24 mm coil diameter showed obvious differences between unripe and ripe fruitlets based on the resonant frequency. Harun et al. [27] continued their study on inductive sensing. They investigated the inductive sensor replacing the single air coil to the dual flat-type shape of air coil and found that it improved the sensitivity up to 167% of the determination of FFB ripeness. Lately, the ripeness of FFB can be proved by resonance frequency against time as the result showed it has a linear relationship. Additionally, the moisture content of the fruitlets reduced in a negative logarithm function was investigated. The analysis showed that the resonance frequencies acquired for unripe and ripe fruitlets are around 8.5 MHz and 9.8 MHz [28]. The inductive sensing method demonstrates the potential for determining the FFB ripeness. The design structure of the inductive sensor is improving to obtain more sensitive and precise results. Aliteh et al. [29] proposed using the triple flat-type air coil inductive sensor to identify the maturity of FFB. The researchers found that the coil with a fixed number of turns with different lengths was more sensitive because the inductive sensor is of sufficient length to attach to the fruit surface area. Currently, the inductive-based concept was implemented in the application for determining the ripeness of FFB. An inductive sensor system was developed and applied in the field [30].

2.1.9. Laser Light Backscattering

Laser-light backscattering was lately proposed to classify the ripeness of FFB and applied for determining the maturity of such as apricot [31], banana [32], and macaw oil palm [33]. Laser-light backscattering can determine the textural and mechanical properties of the fruit. The physical properties in the fruit can be acquired by tuning the wavelength of the laser diode. In addition, the backscattering profile is correlated to the scattering properties and absorption of the fruit [34]. The laser-light backscattering is supported by Utom et al. [35], who discovered that chlorophyll act as a photoreceptor. It absorbs light with a wavelength in the region of 400 to 700 nm. Therefore, the carotenoids and chlorophyll contents can be an indicator to classify the maturity of FFB. In the past, this method was implemented in the different agricultural crops for monitoring fruit quality. For example, Babazadeh et al. [36] applied laser light backscattering imaging to distinguish the potatoes that only contained α-solanine taxicant. Ali et al. [37] developed a backscattering images system to classify seeded and seedless watermelons. In 2020, Ali et al. [38] combined backscattering imaging and computer vision to determine the maturity level of FFB. The authors used discriminant analysis and quadratic discriminant analysis to carry out the classification with the combination of backscattering parameters and RGB values. The experimental research demonstrated this method achieved above 85% accuracies for classifying the ripeness of oil palm FFB.

2.1.10. Thermal Sensing

The maturity grading of FFB using thermal vision was first proposed by Fauziah et al. [39]. The authors found that the traditional FFB harvesting method was inappropriate due to the environmental condition and animal disturbances. In addition, the FFB harvesting with long-range detection and nondestructive method was usually affected by lighting environment. Therefore, the authors introduced using thermal vision since the lighting has no explicit effect on the thermal principle’s outcomes. The object temperature emission was used to carry out the testing procedure. The oil content had an effect on the thermal characteristics (heat) of the object. The oil content of the fruit increases as it ripens: the higher the ripeness level, the less water content and the higher the oil content. This phenomenon could be seen using a thermal camera’s image. This camera is monochromatic, meaning it cannot observe the colours but can detect the strength of radiation. The strength of the radiation is directly proportional to the ripeness degree of the fruit, as seen in the picture extraction outcomes. The picture extraction results are the elements of redness, greenish, and bluish, which are transformed into temperatures. The FFB surface temperature can be utilised to predict greater ripeness. As the fruit ripens, the heat level rises. The temperature will fall after optimum ripeness. The authors further improved the thermal vision method implemented in a device. They discovered that two quality parameters had higher coefficients of determination related to FFB surface temperature: carotene content and oil and moisture ratio. Therefore, a linear and multiple regression model could be employed to predict the ripeness of oil palm FFB at the pre-harvest stage using a thermal imaging system. This method can also be used to estimate the time it will take to harvest the oil palm FFB [40].

2.2. Destructive Methods

Fruit Battery

The estimation of ripeness of FFB using fruit battery was first proposed by Minakata et al. [41]. The authors designed the fruit battery using copper and zinc electrode. These metal electrodes pricked through the surface of oil palm fruitlets, and ionization occurred. Zinc experiences oxidation by leaving two electrons as zinc has a higher level of activity series compared with copper. The copper electrode received electrons from the zinc electrode. The electron will combine with a hydrogen ion since the fruitlets contain moisture, and hydrogen gas is produced. The half-equations are shown below. A small current produced due to the movement of electrons had a potential difference in the mesocarp.
This is the half equation of the zinc and hydrogen equations [41]:
Zn → Zn2+ + 2e−,
2H+ + 2e− → H2
A load of electromotive force generated in the mesocarp confines in the proportion of moisture in oil palm fruitlets. The moisture and lipid content in unripe fruitlets occupy 80.1% and 5.9% of content percentage, which has a significant difference compared with a ripe fruitlet. The moisture content converts to lipid content during the maturation of the fruitlet. The ripe fruitlet has only 24.7% and 58.3% content [28].
The characteristic of moisture and lipid content in the fruitlet become an indicator for designating the maturity stage of oil palm fruitlets. For instance, Misron et al. [42] implemented the concept of fruit battery in an application to inspect the maturity of FFB at the mill. The authors proposed a fruit battery sensor with a charging concept to gain steady-state condition, which is to avoid inaccurate results from unstable load voltage. Moreover, the authors proposed the sensor by implementing different adequate parameters. The parameters included load resistance, charging voltage, and charging time. These parameters were tested with a variant of data from the experiment to deliver important information. An investigation found that the sensitivity performance increased where the load resistance, charging voltage, and charging time increased. Additionally, the fruit battery sensor improved the accuracy of screening the maturity stage of FFB when combined with computer vision [43].

3. Discussion

A total of 48 articles were studied in this review. Each article describes a detection method for determining the ripeness of fresh fruit bunches, and each detection method has the same parts. The first part involves using a specific type of sensor, e.g., camera— optical or inductive—to test on a specific part of the FFB, i.e., fruitlets or entire fruit. Second, the data captured by the sensor then underwent feature selection or extraction processing such as PCA before being fed to a classifier. Third and lastly, the classifier provided a final decision on whether the FFB was ripe or otherwise based on the input it received. The classifier could be classically handcrafted (e.g., K-Nearest Neighbor), or it could be more modern (e.g., convolutional neural network).

3.1. Source of Sample Data

Throughout the literature, the sample data used varied between the different types of classification maturity methods. The sample data are the source material that is used to perform the maturity analysis. Both FFBs and oil palm fruitlets can be tested to differentiate the maturity stages through feature extraction. However, the sample data relies on the limitation of the sensor application itself. For example, the fruit battery method requires electrodes to prick the oil palm fruitlet for measurement. For other methods such as computer vision, LiDAR sensor, and optical sensor, researchers are working with both FFB and oil palm fruitlets in their studies. These methods enable the information to be processed regardless of the capture distance and morphography of the object. Table 1 summarizes the material data experimented. In the majority of the previous work, the sample data (i.e., FFB/fruitlets or both) are collected from the plantation and then brought into the laboratory or testing environment for analysis. Only a few of the published works used sample data that were collected and tested on-field. This will be explained in more detail in a later section.

3.2. Sensors and Modalities

Figure 1 illustrates the number of articles categorised according to the modalities of the applied sensors. Almost half of the articles (22) used colour vision to capture the data from the tested FFB for further determination on its maturity, making it the most popular method. This can be attributed to the fact that the hardware requirements for this method is relatively low cost. However, one of the main challenges in using colour vision for detection is that it is not able to account for partial ripeness conditions. In some cases, different parts of the FFB reach maturity at different intervals. This could mean that an FFB appears to be ripe on one side but is actually unripe in other parts. If the image captured was on the side that appeared ripe, then the FFB would be misclassified as a ripe FFB.
In the other articles, new methods were proposed that use different features that cannot be detected by colour imaging, such as fruit battery, thermal sensor, and Raman spectra. These new discoveries offer potentially innovative solutions for the palm oil industry to implement FFB maturity classification systems.
As mentioned previously, the methods for the maturity classification of FFB are categorised into nondestructive and destructive. Nondestructive methods refer to methods that measure the FFB maturity stages without damaging the fruit and affecting its oil output. In this study, all of the methods used were nondestructive, except for the fruit battery method as this method uses two different types of metal electrodes to prick the mesocarp. The maturity classification is then performed based on the studied electrolyte reaction. The fruit battery method focuses on the inspection of FFB at the mills’ reception [42]. According to the current practice, FFBs require an inspection upon arrival at the mill before it is further processed.

3.3. Classifier Comparisons and Trends

Once the features are extracted, they are fed into a classifier to produce the decision on the maturity level of the tested FFB. In recent years, artificial intelligence (AI)-based classifiers have greatly increased in popularity. Figure 2 shows the distribution of the papers reviewed (in terms of total number of papers) according to the type of classifier model used to classify the maturity stages of the FFB. The analysis shows that most of the studies used artificial neural network (ANN) to perform FFB maturity determination. It also seems that the ANN model is an applicable and reliable tool since it has still been employed in this sector recently. The ANN model is adopted widely in different types of extracted features such as images from colour-based, thermal-based, and hyperspectral-based systems. Meanwhile, the CNN and SVM models are the second and third most used classifiers. It shows that these two methods are allowing for the possibility of integration and becoming applicable tools.
Table 2 shows the machine learning models and algorithms applied sorted by year. The ML models were used the most in the year 2021, when there were eight papers published on the topic of determining the maturity of FFB. This shows that the tasks of classifying FFB maturity are catching up with the latest technology trends.
However, from the authors’ point of view, the CNN model has high applicability to detect and classify maturity of FFB compare with others ML models even though ANN has been around longer and is therefore more established. The ANN model is studied universally in the maturity classification of FFB due to the training input is diversified. The feature extraction can be applied to the input neuron as it can transform into a numerical form. However, CNN is an effective approach for extracting a hierarchical representation of the input data that is invariant to transformations and scaling. The architecture of CNN is linked with a series of layers, it consists of convolutional layers, pooling layers, and fully connected or classification layers. The convolutional layer obtains image features by striding across the input image and producing a feature map. The various filter generates different feature maps that serve as feature detectors. To ensure diverse information is extracted completely, multiple convolutional layers are applied. The pooling layer trims the dimension of the feature maps and provides invariance to distortion. The classification procedure is carried out via s fully connected layer.
An accurate CNN model allows for performing complex tasks. The model recognises an object by evaluating a classifier for that specific object at multiple positions and scales in a test image. A predicted bounding box will appear in the scene once the target object is detected. In addition, a deep CNN model can perform detection with numerous classes of objects. The deep CNN model includes multiple convolutional layers and pooling layers. The enormous of feature maps could help to identify more features in an image like edges and lines. Famous pre-train CNN model such as AlexNet [67], ResNet [68], and DenseNet [69]. The autonomous harvesting system would take this feasibility of the CNN model to detect others object such as stalk and frond.
Meanwhile, the enhancement of the graphic processing unit (GPU) accelerates the computer graphics workload to achieve the CNN model operating in real-time. The advent of “You Only Look Once” (YOLO) proposed by Redmon et al. [70] aims to work on real-time object detection. YOLO computes multiple bounding boxes and class probabilities for those boxes are simultaneously predicted by a single neural network for each box. YOLO directly improves detection performance while training on complete images. Comparing the unified framework and conventional object identification methods offers several advantages. Hereafter, the application of YOLO in a variety of object detection tasks expanded greatly when YOLOv3 was introduced. This was later improved by Redmon and Farhadi [71]. The YOLOv3 was also implemented to detect and classify the maturity of FFB [66]. The result of the YOLOv3 algorithm shows its capability to detect and differentiate the levels of FFB maturity.

3.4. Readiness for On-Site Application

Most of the methods mentioned earlier are still in the experimental stage and more studies are being carried out to determine the method’s effectiveness in FFB ripeness detection. Those methods may be further refined for applications involving detection in the mill after harvesting or in-field at the oil palm plantation. Furthermore, most of the experiments were conducted within the laboratory setting. This provides a controlled environment eliminating real-world issues caused by occlusion and a noisy background to extract the features of FFB or fruitlet. The results of experimental analysis derived from a perfect condition may not be applicable in the plantation estate. In addition, environmental factors such as weather and topography present a challenge for the sensor to acquire important information accurately. Image-based methods have the capability to be applied in the oil palm estate and mill, as it is feasible because it can be analysed to include or erase the background noise dealing with image processing methods. From the 12 methods that have been reviewed, only three have been used for on-site testing which are computer vision, LiDAR, and inductive sensors. Table 3 summarizes the detection methods conducted in laboratory and field settings.
Through the research, the authors review the methods to classify the FFB’s maturity stage. It brings insight into the advantage and constraints of all the methods proposed. However, to revolute the traditional harvesting process to autonomous in the oil palm estate. It delivers the harvester and grader job transformation to reduce the workload and avoid misclassification. Detection and classification of the FFB become an imperative role to implement an autonomous harvesting system. The computer vision method is recommended as it is comprehensive to employ in the system. It does not have constraints and obligates within a specific range to extract the features in a fixed scene.
Based on the current research landscape, the most feasible detection system as in the one that is closest to being ready for deployment in the plantation is the system which combines colour vision sensing with a CNN classifier. Colour vision sensing has the advantage of being minimal in cost compared to the LiDAR sensing method since cameras are common and cheap whereas LiDARs are typically expensive. The cameras also do not need any additional set up, unlike induction sensors, and they can also operate from a greater distance. This is why they combine and work so well with CNN algorithms such as YOLO that can process images with impressive speed, making them suitable for real-time operation. The potential of this approach is high and can be improved further by expanding the dataset over time, evolution of CNN algorithms and hyperparameter optimisation.

4. Conclusions

Over the past few decades, traditional methods are still mainly applied to harvesting the FFB. It is valuable to enlighten the FFB harvesting process. Harvesting FFB autonomously could be a possible trend to solve problems such as labour shortages. In this paper, a method of classifying and detecting FFB ripeness is reviewed. The method of using colour vision is encouraging to implement in the harvest process. This is because the system setup is relatively lower cost since no expensive equipment is required. Based on the review conducted, it is a nondestructive method that will not harm or injure the fruitlet on FFB through the sensing process. Furthermore, it is highly feasible as it can be successfully utilised either in the mill or in field estate, and it is not restricted to the use of FFB or fruitlet as a data source for analysis unless the useful colour information can be extracted. Moreover, the colour vision method can be fully utilised to detect the frond and stalk as well, since cutting the frond and stalk is part of the tasks in harvesting the FFB.
Today, highly accurate real-time object detection systems can be produced more easily since the graphic processing unit (GPU) and the artificial intelligent algorithm required have advanced greatly. The enhancement of the GPU presents many advantages for the deep-layer CNN model as it requires advanced GPU to calculate more feature maps and layers. Moreover, the improved GPU can deal with the massive amount of graphic information, which allows the deep-learning model to operate at a faster speed. Meanwhile, the algorithm can be invented in even more layers to find out insignificant features in the data given. The more features are identified, the more accurate it can be. In the end, the YOLO model is recommended because it can be operating accurately under real-time detection. The YOLO series models have been embedded in many applications including the agriculture sector. YOLO v3 has also been tested to detect the maturity of FFB. Therefore, YOLO v4 is introduced to detect the maturity of FFB as it is an upgraded version of YOLO v3.
There is a wide range of future potential projects in this field. One such project is the combination of AI-based systems with Internet of Things (IoT) systems. For example, in a future implementation, trees that are found to have ripe FFBs are automatically geotagged, and their location is shared immediately with the harvesters. This allows for more efficient harvesting since the harvesters can now know the precise location of the ripe FFB. The big data generated and stored in the cloud can then be used to produce a predictive model, which can accurately estimate when certain oil palm trees are expected to bear ripe fruits again. This can help harvesters to harvest the FFBs when they are at their peak ripeness, which should then produce the maximum OER.

Author Contributions

Conceptualization, methodology, software, validation, formal analysis, writing—review and editing, visualization, J.W.L. and H.R.R.; investigation, resources, data curation, writing—original draft preparation, J.W.L.; supervision, project administration, funding acquisition, H.R.R., L.I.I. and W.Z.W.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by UNIVERSITI PUTRA MALAYSIA (Putra Grant, GP-GPB/2021/9699100).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No data are used in the review paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chew, C.L.; Ng, C.Y.; Hong, W.O.; Wu, T.Y.; Lee, Y.Y.; Low, L.E.; Kong, P.S.; Chan, E.S. Improving Sustainability of Palm Oil Production by Increasing Oil Extraction Rate: A Review. Food Bioprocess Technol. 2021, 14, 573–586. [Google Scholar] [CrossRef]
  2. Khan, N.; Kamaruddin, M.A.; Sheikh, U.U.; Yusup, Y.; Bakht, M.P. Oil Palm and Machine Learning: Reviewing One Decade of Ideas, Innovations, Applications, and Gaps. Agriculture 2021, 11, 832. [Google Scholar] [CrossRef]
  3. Abdullah, M.Z.; Guan, L.C.; Mohamed, A.M.D.; Noor, M.A.M. Color Vision System for Ripeness Inspection Of. J. Food Process. Preserv. 2002, 26, 213–235. [Google Scholar] [CrossRef]
  4. Septiarini, A.; Hamdani, H.; Hatta, H.R.; Kasim, A.A. Image-based processing for ripeness classification of oil palm fruit. In Proceedings of the 2019 5th International Conference on Science in Information Technology (ICSITech), Yogyakarta, Indonesia, 23–24 October 2019; pp. 23–26. [Google Scholar] [CrossRef]
  5. Septiarini, A.; Hamdani, H.; Hatta, H.R.; Anwar, K. Automatic Image Segmentation of Oil Palm Fruits by Applying the Contour-Based Approach. Sci. Hortic. 2020, 261, 108939. [Google Scholar] [CrossRef]
  6. Fracarolli, J.A.; Adimari Pavarin, F.F.; Castro, W.; Blasco, J. Computer Vision Applied to Food and Agricultural Products. Rev. Cienc. Agron. 2020, 51, e20207749. [Google Scholar] [CrossRef]
  7. Fadilah, N.; Mohamad-Saleh, J.; Halim, Z.A.; Ibrahim, H.; Ali, S.S.S. Intelligent Color Vision System for Ripeness Classification of Oil Palm Fresh Fruit Bunch. Sensors 2012, 12, 14179–14195. [Google Scholar] [CrossRef]
  8. Shabdin, M.K.; Shariff, A.R.M.; Johari, M.N.A.; Saat, N.K.; Abbas, Z. A Study on the Oil Palm Fresh Fruit Bunch (FFB) Ripeness Detection by Using Hue, Saturation and Intensity (HSI) Approach. IOP Conf. Ser. Earth Environ. Sci. 2016, 37, 012039. [Google Scholar] [CrossRef]
  9. Sabri, N.; Ibrahim, Z.; Syahlan, S.; Jamil, N.; Mangshor, N.N.A. Palm Oil Fresh Fruit Bunch Ripeness Grading Identification Using Color Features. J. Fundam. Appl. Sci. 2018, 9 (Suppl. S4), 563. [Google Scholar] [CrossRef] [Green Version]
  10. Setiawan, A.W.; Prasetya, O.E. Palm Oil Fresh Fruit Bunch Grading System Using Multispectral Image Analysis in HSV. In Proceedings of the 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), Doha, Qatar, 2–5 February 2020; pp. 85–88. [Google Scholar] [CrossRef]
  11. Dzuljalal, A.; Seri, M. Development of Virescens Fresh Fruit Bunch Ripeness Prediction Using LiDAR for Smart Agriculture. In Proceedings of the 2021 IEEE Region 10 Symposium, Jeju, Republic of Korea, 23–25 August 2021. [Google Scholar]
  12. Hashim, F.H.; Mohd Zulkifli, Z.; Raj, T.; Huddin, A.B. A Rapid and Non-Destructive Method in Determining The Ripeness of Oil Palm Fresh Fruit Bunch (FFB). J. Kejuruter. 2018, 30, 93–101. [Google Scholar] [CrossRef]
  13. Husin, H.S.; Amar, N.; Bakar Sajak, A.A.; Sallehin Mohd Kassim, M. Distribution Map of Oil Palm Fresh Fruit Bunch Using LiDAR*. In Proceedings of the 2021 12th International Conference on Information and Communication Systems (ICICS), Valencia, Spain, 24–26 May 2021; pp. 4–9. [Google Scholar] [CrossRef]
  14. Setiawan, A.W.; Mengko, R.; Putri, A.P.H.; Danudirdjo, D.; Ananda, A.R. Classification of Palm Oil Fresh Fruit Bunch Using Multiband Optical Sensors. Int. J. Electr. Comput. Eng. 2019, 9, 2386–2393. [Google Scholar] [CrossRef]
  15. Ben Saeed, O.M.; Sankaran, S.; Shariff, A.R.M.; Shafri, H.Z.M.; Ehsani, R.; Alfatni, M.S.; Hazir, M.H.M. Classification of Oil Palm Fresh Fruit Bunches Based on Their Maturity Using Portable Four-Band Sensor System. Comput. Electron. Agric. 2012, 82, 55–60. [Google Scholar] [CrossRef]
  16. Arunkumar, M.; Rajendran, A.; Gunasri, S.; Kowsalya, M.; Krithika, C.K. Non-Destructive Fruit Maturity Detection Methodology—A Review. Mater. Today Proc. 2021. [Google Scholar] [CrossRef]
  17. Silalahi, D.D.; Reaño, C.E.; Lansigan, F.P.; Panopio, R.G.; Bantayan, N.C.; Davrieux, F.; Caliman, J.P.; Yuana, Y.Y.; Sudarno. Near Infrared Spectroscopy: A Rapid and Non-Destructive Method to Assess the Ripeness of Oil Palm (Elaeis Guineensis Jacq.) Fresh Fruit. J. Near Infrared Spectrosc. 2016, 24, 179–190. [Google Scholar] [CrossRef]
  18. Silalahi, D.D.; Reaño, C.E.; Lansigan, F.P.; Panopio, R.G.; Bantayan, N.C. Using Genetic Algorithm Neural Network on Near Infrared Spectral Data for Ripeness Grading of Oil Palm (Elaeis Guineensis Jacq.) Fresh Fruit. Inf. Process. Agric. 2016, 3, 252–261. [Google Scholar] [CrossRef] [Green Version]
  19. Trebolazabala, J.; Maguregui, M.; Morillas, H.; de Diego, A.; Madariaga, J.M. Portable Raman Spectroscopy for an In-Situ Monitoring the Ripening of Tomato (Solanum Lycopersicum) Fruits. Spectrochim. Acta Part A Mol. Biomol. Spectrosc. 2017, 180, 138–143. [Google Scholar] [CrossRef]
  20. Nekvapil, F.; Brezestean, I.; Barchewitz, D.; Glamuzina, B.; Chiş, V.; Cintă Pinzaru, S. Citrus Fruits Freshness Assessment Using Raman Spectroscopy. Food Chem. 2018, 242, 560–567. [Google Scholar] [CrossRef]
  21. Raj, T.; Hashim, F.H.; Huddin, A.B.; Hussain, A.; Ibrahim, M.F.; Abdul, P.M. Classification of Oil Palm Fresh Fruit Maturity Based on Carotene Content from Raman Spectra. Sci. Rep. 2021, 11, 18315. [Google Scholar] [CrossRef]
  22. Hazir, M.H.M.; Shariff, A.R.M.; Amiruddin, M.D. Determination of Oil Palm Fresh Fruit Bunch Ripeness-Based on Flavonoids and Anthocyanin Content. Ind. Crops Prod. 2012, 36, 466–475. [Google Scholar] [CrossRef]
  23. Hazir, M.H.M.; Shariff, A.R.M.; Amiruddin, M.D.; Ramli, A.R.; Iqbal Saripan, M. Oil Palm Bunch Ripeness Classification Using Fluorescence Method. J. Food Eng. 2012, 113, 534–540. [Google Scholar] [CrossRef]
  24. Yeow, Y.K.; Abbas, Z.; Khalid, K. Application of Microwave Moisture Sensor for Determination of Oil Palm Fruit Ripeness. Meas. Sci. Rev. 2010, 10, 7–14. [Google Scholar] [CrossRef] [Green Version]
  25. You, K.Y.; Wee, F.H.; Lee, Y.S.; Abbas, Z.; Lee, K.Y.; Cheng, E.M.; Khe, C.S.; Jamlos, M.F. A Review of Oil Palm Fruit Ripeness Monitoring Using Microwave Methods in Malaysia. IOP Conf. Ser. Mater. Sci. Eng. 2020, 767, 012007. [Google Scholar] [CrossRef] [Green Version]
  26. Harun, N.H.; Misron, N.; Sidek, R.M.; Aris, I.; Ahmad, D.; Wakiwaka, H.; Tashiro, K. Investigations on a Novel Inductive Concept Frequency Method for the Grading of Oil Palm Fresh Fruit Bunches. Sensors 2013, 13, 2254–2266. [Google Scholar] [CrossRef]
  27. Harun, N.H.; Misron, N.; Sidek, R.M.; Aris, I.; Wakiwaka, H.; Tashiro, K. Dual Resonant Frequencies Effects on an Induction-Based Oil Palm Fruit Sensor. Sensors 2014, 14, 21923–21940. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Misron, N.; Aliteh, N.A.; Harun, N.H.; Tashiro, K.; Sato, T.; Wakiwaka, H. Relative Estimation of Water Content for Flat-Type Inductive-Based Oil Palm Fruit Maturity Sensor. Sensors 2017, 17, 52. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Aliteh, N.A.; Misron, N.; Aris, I.; Sidek, R.M.; Tashiro, K.; Wakiwaka, H. Triple Flat-Type Inductive-Based Oil Palm Fruit Maturity Sensor. Sensors 2018, 18, 2496. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Sinambela, R.; Mandang, T.; Subrata, I.D.M.; Hermawan, W. Application of an Inductive Sensor System for Identifying Ripeness and Forecasting Harvest Time of Oil Palm. Sci. Hortic. 2020, 265, 109231. [Google Scholar] [CrossRef]
  31. Mozaffari, M.; Sadeghi, S.; Asefi, N. Prediction of the Quality Properties and Maturity of Apricot by Laser Light Backscattering Imaging. Postharvest Biol. Technol. 2022, 186, 111842. [Google Scholar] [CrossRef]
  32. Zulkifli, N.; Hashim, N.; Abdan, K.; Hanafi, M. Application of Laser-Induced Backscattering Imaging for Predicting and Classifying Ripening Stages of “Berangan” Bananas. Comput. Electron. Agric. 2019, 160, 100–107. [Google Scholar] [CrossRef]
  33. Anderson, G.C.; de Carvalho, P.F.d.A.; Roberto, B.A.J.; Sérgio, Y.M.; Luís, M.N.G. Determination of Macaw Fruit Harvest Period by Biospeckle Laser Technique. African J. Agric. Res. 2017, 12, 674–683. [Google Scholar] [CrossRef] [Green Version]
  34. Lockman, N.A.; Hashim, N.; Onwude, D.I. Laser-Based Imaging for Cocoa Pods Maturity Detection. Food Bioprocess Technol. 2019, 12, 1928–1937. [Google Scholar] [CrossRef]
  35. Utom, S.L.; Mohamad, E.J.; Ameran, H.L.M.; Kadir, H.A.; Muji, S.Z.M.; Rahim, R.A.; Pusppanathan, J. Non-Destructive Oil Palm Fresh Fruit Bunch (FFB) Grading Method Using Optical Sensor. Int. J. Integr. Eng. 2018, 10, 35–39. [Google Scholar] [CrossRef] [Green Version]
  36. Babazadeh, S.; Ahmadi Moghaddam, P.; Sabatyan, A.; Sharifian, F. Classification of Potato Tubers Based on Solanine Toxicant Using Laser-Induced Light Backscattering Imaging. Comput. Electron. Agric. 2016, 129, 1–8. [Google Scholar] [CrossRef]
  37. Mohd Ali, M.; Hashim, N.; Bejo, S.K.; Shamsudin, R. Laser-Induced Backscattering Imaging for Classification of Seeded and Seedless Watermelons. Comput. Electron. Agric. 2017, 140, 311–316. [Google Scholar] [CrossRef]
  38. Mohd Ali, M.; Hashim, N.; Abdul Hamid, A.S. Combination of Laser-Light Backscattering Imaging and Computer Vision for Rapid Determination of Oil Palm Fresh Fruit Bunches Maturity. Comput. Electron. Agric. 2020, 169, 105235. [Google Scholar] [CrossRef]
  39. Fauziah, W.K.; Makky, M.; Santosa; Cherie, D. Thermal Vision of Oil Palm Fruits under Difference Ripeness Quality. IOP Conf. Ser. Earth Environ. Sci. 2021, 644, 012044. [Google Scholar] [CrossRef]
  40. Makky, M.; Cherie, D. Pre-Harvest Oil Palm FFB Nondestructive Evaluation Method Using Thermal-Imaging Device. IOP Conf. Ser. Earth Environ. Sci. 2021, 757, 012003. [Google Scholar] [CrossRef]
  41. Minakata, K.; Tashiro, K.; Wakiwaka, H.; Kobayashi, K.; Misrom, N.; Aliteh, N.A.; Nagata, H. Proposal of Fruit Battery Method for Estimating Oil Palm Ripeness. In Proceedings of the 2018 12th International Conference on Sensing Technology (ICST), Limerick, Ireland, 4–6 December 2018; pp. 399–402. [Google Scholar] [CrossRef]
  42. Misron, N.; Azhar, N.S.K.; Hamidon, M.N.; Aris, I.; Tashiro, K.; Nagata, H. Effect of Charging Parameter on Fruit Battery-Based Oil Palm Maturity Sensor. Micromachines 2020, 11, 806. [Google Scholar] [CrossRef] [PubMed]
  43. Aliteh, N.A.; Minakata, K.; Tashiro, K.; Wakiwaka, H.; Kobayashi, K.; Nagata, H.; Misron, N. Fruit Battery Method for Oil Palm Fruit Ripeness Sensor and Comparison with Computer Vision Method. Sensors 2020, 20, 637. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Septiarini, A.; Hatta, H.R.; Hamdani, H.; Oktavia, A.; Kasim, A.A.; Suyanto, S. Maturity Grading of Oil Palm Fresh Fruit Bunches Based on a Machine Learning Approach. In Proceedings of the 2020 5th International Conference on Informatics and Computing (ICIC), Gorontalo, Indonesia, 24 December 2020; pp. 19–22. [Google Scholar] [CrossRef]
  45. Sabri, N.; Ibrahim, Z.; Isa, D. Evaluation of Color Models for Palm Oil Fresh Fruit Bunch Ripeness Classification. Indones. J. Electr. Eng. Comput. Sci. 2018, 11, 549–557. [Google Scholar] [CrossRef]
  46. Tuerxun, A.; Mohamed Shariff, A.R.; Janius, R.; Abbas, Z.; Mahdiraji, G.A. Oil Palm Fresh Fruit Bunches Maturity Prediction by Using Optical Spectrometer. IOP Conf. Ser. Earth Environ. Sci. 2020, 540, 012085. [Google Scholar] [CrossRef]
  47. Tzuan, G.T.H.; Hashim, F.H.; Raj, T.; Baseri Huddin, A.; Sajab, M.S. Oil Palm Fruits Ripeness Classification Based on the Characteristics of Protein, Lipid, Carotene, and Guanine/Cytosine from the Raman Spectra. Plants 2022, 11, 1936. [Google Scholar] [CrossRef] [PubMed]
  48. Azmi, M.H.I.M.; Hashim, F.H.; Huddin, A.B.; Sajab, M.S. Correlation Study between the Organic Compounds and Ripening Stages of Oil Palm Fruitlets Based on the Raman Spectra. Sensors 2022, 22, 7091. [Google Scholar] [CrossRef] [PubMed]
  49. Zolfagharnassab, S.; Shariff, A.R.B.M.; Ehsani, R.; Jaafar, H.Z.; Aris, I.B. Classification of Oil Palm Fresh Fruit Bunches Based on Their Maturity Using Thermal Imaging Techniques. Agriculture 2022, 12, 1779. [Google Scholar] [CrossRef]
  50. Fadilah, N.; Saleh, J.M.; Ibrahim, H.; Halim, Z.A. Oil Palm Fresh Fruit Bunch Ripeness Classification Using Artificial Neural Network. In Proceedings of the 2012 4th International Conference on Intelligent and Advanced Systems (ICIAS2012), Kuala Lumpur, Malaysia, 12–14 June 2012; Volume 1, pp. 18–21. [Google Scholar] [CrossRef]
  51. Fadilah, N.; Mohamad-Saleh, J. Color Feature Extraction of Oil Palm Fresh Fruit Bunch Image for Ripeness Classification. In Proceedings of the 13th International Conference on Applied Computer and Applied Computational Science (ACACOS’14), Kuala Lumpur, Malaysia, 23–25 April 2014; pp. 51–55. [Google Scholar]
  52. Bensaeed, O.M.; Shariff, A.M.; Mahmud, A.B.; Shafri, H.; Alfatni, M. Oil Palm Fruit Grading Using a Hyperspectral Device and Machine Learning Algorithm. IOP Conf. Ser. Earth Environ. Sci. 2014, 20, 012017. [Google Scholar] [CrossRef]
  53. Alfatni, M.S.M.; Shariff, A.R.M.; Abdullah, M.Z.; Marhaban, M.H.; Shafie, S.B.; Bamiruddin, M.D.; Saaed, O.M.B. Oil Palm Fresh Fruit Bunch Ripeness Classification Based on Rule-Based Expert System of ROI Image Processing Method Results. IOP Conf. Ser. Earth Environ. Sci. 2014, 20, 012018. [Google Scholar] [CrossRef]
  54. Taparugssanagorn, A.; Siwamogsatham, S.; Pomalaza-Ráez, C. A Non-Destructive Oil Palm Ripeness Recognition System Using Relative Entropy. Comput. Electron. Agric. 2015, 118, 340–349. [Google Scholar] [CrossRef]
  55. Patkar, G.; Anjaneyulu, G.S.G.N.; Mouli, P.V.S.S.R.C. Palm Fruit Harvester Algorithm for Elaeis Guineensis Oil Palm Fruit Grading Using UML. In Proceedings of the 2015 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), Madurai, India, 10–12 December 2015. [Google Scholar] [CrossRef]
  56. Alfatni, M.S.M.; Mohamed Shariff, A.R.; Bejo, S.K.; Ben Saaed, O.M.; Mustapha, A. Real-Time Oil Palm FFB Ripeness Grading System Based on ANN, KNN and SVM Classifiers. IOP Conf. Ser. Earth Environ. Sci. 2018, 169, 012067. [Google Scholar] [CrossRef]
  57. Fahmi, F.; Palti, H.; Emerson, S.; Suherman, S. Oil Palm Fresh Fruit Bunch Ripeness Classification Using Back Propagation and Learning Vector Quantization. IOP Conf. Ser. Mater. Sci. Eng. 2018, 434, 012066. [Google Scholar] [CrossRef]
  58. Ibrahim, Z.; Sabri, N.; Isa, D. Palm Oil Fresh Fruit Bunch Ripeness Grading Recognition Using Convolutional Neural Network. J. Telecommun. Electron. Comput. Eng. 2018, 10, 109–113. [Google Scholar]
  59. Alfatni, M.S.M.; Mohamed Shariff, A.R.; Ben Saaed, O.M.; Albhbah, A.M.; Mustapha, A. Colour Feature Extraction Methods for Real Time System of Oil Palm Fresh Fruit Bunch Maturity Grading. IOP Conf. Ser. Earth Environ. Sci. 2020, 540, 012092. [Google Scholar] [CrossRef]
  60. Herman, H.; Susanto, A.; Cenggoro, T.W.; Suharjito, S.; Pardamean, B. Oil Palm Fruit Image Ripeness Classification with Computer Vision Using Deep Learning and Visual Attention. J. Telecommun. Electron. Comput. Eng. 2020, 12, 21–27. [Google Scholar]
  61. Septiarini, A.; Sunyoto, A.; Hamdani, H.; Kasim, A.A.; Utaminingrum, F.; Hatta, H.R. Machine Vision for the Maturity Classification of Oil Palm Fresh Fruit Bunches Based on Color and Texture Features. Sci. Hortic. 2021, 286, 110245. [Google Scholar] [CrossRef]
  62. Melidawati; Cherie, D.; Fahmy, K.; Makky, M. Non Destructive Evaluation Quality of Oil Palm Fresh Fruit Bunch (FFB) (Elaeis Guineensis Jack) Based on Optical Properties Using Artificial Neural Network (ANN). IOP Conf. Ser. Earth Environ. Sci. 2021, 644, 012032. [Google Scholar] [CrossRef]
  63. Suharjito; Elwirehardja, G.N.; Prayoga, J.S. Oil Palm Fresh Fruit Bunch Ripeness Classification on Mobile Devices Using Deep Learning Approaches. Comput. Electron. Agric. 2021, 188, 106359. [Google Scholar] [CrossRef]
  64. Patkar, G.; Morajkar, S. Feature Selection Approach for Oil Palm Fruit Grading Expert System. In Proceedings of the 6th International Conference on Recent Trends on Electronics, Information, Communication & Technology (RTEICT), Bangalore, India, 27–28 August 2021; pp. 670–674. [Google Scholar] [CrossRef]
  65. Junos, M.H.; Mohd Khairuddin, A.S.; Thannirmalai, S.; Dahari, M. An Optimized YOLO-Based Object Detection Model for Crop Harvesting System. IET Image Process. 2021, 15, 2112–2125. [Google Scholar] [CrossRef]
  66. Selvam, N.A.M.B.; Ahmad, Z.; Mohtar, I.A. Real Time Ripe Palm Oil Bunch Detection Using YOLO V3 Algorithm. In Proceedings of the 2021 IEEE 19th Student Conference on Research and Development (SCOReD), Kota Kinabalu, Malaysia, 23–25 November 2021; pp. 323–328. [Google Scholar]
  67. Krizhevsky, A. One Weird Trick for Parallelizing Convolutional Neural Networks. arXiv 2014, arXiv:1404.5997. [Google Scholar]
  68. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar] [CrossRef] [Green Version]
  69. Iandola, F.; Moskewicz, M.; Karayev, S.; Girshick, R.; Darrell, T.; Keutzer, K. DenseNet: Implementing Efficient ConvNet Descriptor Pyramids. arXiv 2014, arXiv:1404.1869. [Google Scholar]
  70. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar] [CrossRef]
  71. Redmon, J.; Farhadi, A. YOLOv3: An Incremental Improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
Agriculture 13 00156 g001 550
Figure 1. The number of articles reviewed based on different detection methods.
Figure 1. The number of articles reviewed based on different detection methods.
Agriculture 13 00156 g001
Agriculture 13 00156 g002 550
Figure 2. The number of articles categorised by different machine learning models and algorithms.
Figure 2. The number of articles categorised by different machine learning models and algorithms.
Agriculture 13 00156 g002
Table
Table 1. Type of sample data used with every method.
Table 1. Type of sample data used with every method.
MethodType of Sample DataReference
Color visionFresh fruit bunches[8,44,45]
Oil palm fruitlets[4]
LiDAR sensorFresh fruit bunches[11,13]
Oil palm fruitlets[12]
NIRFresh fruit bunches[17]
Oil palm fruitlets[18]
Hyperspectral cameraFresh fruit bunches[10]
Optical sensorFresh fruit bunches[35]
Optical spectrometerFresh fruit bunches[46]
Multiband optical sensorFresh fruit bunches[14]
Optical sensor systemFresh fruit bunches[15]
Raman spectraOil palm fruitlets[21,47,48]
Fluorescence sensorFresh fruit bunches[22,23]
Inductive sensorOil palm fruitlets[28,29]
Fresh fruit bunches[30]
Laser-light backscattering and computer visionFresh fruit bunches[38]
Thermal visionFresh fruit bunches[39]
Thermal imagingFresh fruit bunches[40,49]
Fruit batteryOil palm fruitlets[41,42]
Fruit battery and computer visionOil palm fruitlets[43]
Table
Table 2. Machine learning models and algorithms applied to classify the FFB maturity stage.
Table 2. Machine learning models and algorithms applied to classify the FFB maturity stage.
YearML Model and AlgorithmSensorReference
2012ANN, PCAColour vision system[7]
2012C&RTFluorescence sensor[23]
2012ANNColour image[50]
2014ANN, PCAColour image[51]
2014ANNHyperspectral based system[52]
2014KNN, ANN, SVMColour vision[53]
2015Kullback-Leibler distanceColour vision[54]
2016ANNColour vision[8]
2016PCA, LDANear Infrared (NIR)[17]
2016Genetic algorithm neural networkNear Infrared (NIR)[18]
2016Unified modeling language (UML)Colour vision[55]
2018SVM, Naïve BayesColour vision[9]
2018KNN, ANN, SVMColour image[56]
2018Backpropagation, LVQColour image[57]
2018CNN, AlexNet, SVMColour vision[58]
2019SVMColour vision[4]
2019LDAOptical sensor—multiband[14]
2019Regression analysisFruit battery[41]
2020SVMFruit battery[43]
2020LDAColour vision[44]
2020Kstar algorithmOptical—spectrometer[46]
2020ANNColour vision[59]
2020DenseNet, ResAttColour vision[60]
2021KNNOptical- Raman spectroscopy[21]
2021ANNThermal vision system[39]
2021Linear regressionThermal imaging[40]
2021ANNColour vision[61]
2021ANNColour vision[62]
2021CNN—EfficientNetColour vision[63]
2021Fuzzy logic-[64]
2021CNN—YOLOv3Colour vision[65]
2022ANNRaman spectra[47]
2022Weighted KNN, Trilayered neural networkRaman spectra[48]
2022LDA, MDA, ANN, KNNThermal imaging[49]
2022CNN—YOLOv3Colour vision[66]
Table
Table 3. Location of method implemented on classification FBB maturity.
Table 3. Location of method implemented on classification FBB maturity.
MethodLocationReference
Computer visionPlantation estate[4,8,44,45]
LiDAR sensorPlantation estate[11,13]
Laboratory and plantation estate[12]
Near Infrared (NIR)Laboratory[17,18]
Optical spectrometerLaboratory[46]
Raman spectraLaboratory[21,47,48]
Fluorescence sensorLaboratory[22]
Inductive sensorLaboratory and plantation estate[28]
Laboratory[29]
Plantation estate[30]
Laser-light backscattering and computer visionLaboratory[38]
Thermal visionLaboratory[39]
Thermal imagingLaboratory[40,49]
Fruit batteryLaboratory[41]
Mill and laboratory[42]
Fruit battery and computer visionLaboratory[43]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lai, J.W.; Ramli, H.R.; Ismail, L.I.; Wan Hasan, W.Z. Oil Palm Fresh Fruit Bunch Ripeness Detection Methods: A Systematic Review. Agriculture 2023, 13, 156. https://doi.org/10.3390/agriculture13010156

AMA Style

Lai JW, Ramli HR, Ismail LI, Wan Hasan WZ. Oil Palm Fresh Fruit Bunch Ripeness Detection Methods: A Systematic Review. Agriculture. 2023; 13(1):156. https://doi.org/10.3390/agriculture13010156

Chicago/Turabian Style

Lai, Jin Wern, Hafiz Rashidi Ramli, Luthffi Idzhar Ismail, and Wan Zuha Wan Hasan. 2023. "Oil Palm Fresh Fruit Bunch Ripeness Detection Methods: A Systematic Review" Agriculture 13, no. 1: 156. https://doi.org/10.3390/agriculture13010156

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop