Development Challenges of Fruit-Harvesting Robotic Arms: A Critical Review

: Promotion of research and development in advanced technology must be implemented in agriculture to increase production in the current challenging environment where the demand for manual farming is decreasing due to the unavailability of skilled labor, high cost, and shortage of labor. In the last two decades, the demand for fruit harvester technologies, i.e., mechanized harvesting, manned and unmanned aerial systems, and robotics, has increased. However, several industries are working on the development of industrial-scale production of advanced harvesting technologies at low cost, but to date, no commercial robotic arm has been developed for selective harvesting of valuable fruits and vegetables, especially within controlled strictures, i.e., greenhouse and hydroponic contexts. This research article focused on all the parameters that are responsible for the development of automated robotic arms. A broad review of the related research works from the past two decades (2000 to 2022) is discussed, including their limitations and performance. In this study, data are obtained from various sources depending on the topic and scope of the review. Some common sources of data for writing this review paper are peer-reviewed journals, book chapters, and conference proceedings from Google Scholar. The entire requirement for a fruit harvester contains a manipulator for mechanical movement, a vision system for localizing and recognizing fruit, and an end-effector for detachment purposes. Performance, in terms of harvesting time, harvesting accuracy, and detection efﬁciency of several developments, has been summarized in this work. It is observed that improvement in harvesting efﬁciency and custom design of end-effectors is the main area of interest for researchers. The harvesting efﬁciency of the system is increased by the implementation of optimal techniques in its vision system that can acquire low recognition error rates.


Introduction
Using robotics to accomplish work has become desirable over the last two decades in every field, i.e., industry [1], medical [2], agriculture [3], and military [4].Many researchers have discussed the development challenges that arise in the field of agriculture for fruit and vegetable harvesting and other field operations, i.e., sowing, weeding, and spraying vegetables [5][6][7].However, with advancements in the Internet of Things (IoT), sensors, high-speed internet (4G/5G/6G), and robotic automation with required attachments, developments in agriculture are at their peak, and many operations, i.e., crop quality and quantity measurement, disease finding, the planting process, and harvesting are taken from types of machinery in developed countries [8][9][10][11].The word automation here refers to the equipment and devices that work in place of manual procedures.Globally, the demand for agricultural products is increasing rapidly.It is expected that, in the next 30 years, agriculture production needs to be increased by about 50% to meet food demands globally [12,13].
Many regions around the world are facing labor shortages in the agricultural sector [14].Robotic arms can help address this issue by automating the harvesting process and reducing dependency on human labor, especially during peak harvesting seasons when the demand for workers is high.With the passage of time, the demand for manual farming is decreasing due to the unavailability of agricultural skilled labor and its shortage due to extreme weather conditions in the field [15].Sometimes, the reason behind labor shortages occurs due to the pandemic situation and its effect on worldwide travel restrictions [16][17][18][19].These acts may affect the availability of migrant laborers in addition to several other factors, such as people migrating from rural to urban life, lack of interest in farming, and traditional old farmers.As a result, a lot of fruit products are spoiled in the field, where harvesting relies on seasonal field workers [20].In some cases, manual harvesting affects farmers' health due to improper body posture, which can cause musculoskeletal disorders [21].Farmers also highlight the problem of working in a harsh environment, which can cause several health issues.Moreover, another disadvantage of manual harvesting is the high labor cost and inefficient working.Surveys from multiple companies show about 35-40% of the yield of citrus is spent on manual harvesting [22].Another study about cost calculation regarding the production of sweet cherries revealed that half of their production margin is utilized on their manual harvesting [23].
To overcome the issue of labor shortage, ensure labor health, and increase the productivity of agriculture in the current challenging environment, the promotion of research and development must be implemented in agriculture.Many agricultural products like potato, wheat, sugarcane, and corn ripen and are ready for harvesting in a single movement.However, some high-value crops, such as apples, sweet peppers, tomatoes, etc., do not ripen in a single movement [24].Harvesting crops that require diverse ripening stages is a laborious, repetitive, and costly process, making it a prime candidate for automation in agriculture.Apart from addressing labor shortages, automation also plays a crucial role in ensuring the quality of the harvested crop, making it fit for human consumption [25].Also, agricultural automation can harvest crops with high speed and precision, surpassing human capabilities.They can work tirelessly without getting fatigued, which leads to increased efficiency in the harvesting process [26].Significant research and development efforts have been dedicated to advancing the field of agricultural automation, but no satisfactory commercial solutions have been introduced so far.Numerous automated machines have been designed to alleviate the challenges faced in agriculture, but this review will focus specifically on the robotic arm harvester.
Every effort is made in this review paper to discuss all the technical challenges that may occur in the development of fruit-harvesting robotic arms, their needs, and their importance in current scenarios.Furthermore, each main component of fruit-harvesting robotic arms, along with their concerns and challenges, is discussed.

Recent Developments
Previous related work in the development of robotic arm harvesters is discussed in this section.Also, their performance and limitations available in the literature are considered.
In the early stages of fruit-harvesting robotic arms, the designs were rudimentary, with basic grippers and single-axis movements characterizing their mechanical form [27][28][29].These early arms lacked adaptability to different fruit shapes and sizes, leading to a quest for more sophisticated solutions [30].As the years progressed, a breakthrough came with the introduction of multi-axis movement, granting these robotic arms greater flexibility and reach [31].The incorporation of force sensors and vision systems marked another milestone, enabling the arms to delicately handle fruits and precisely locate them within their environment [32].However, it was the 2010s that saw a true revolution in gripper design.Adaptive grippers emerged, their flexible fingers and suction cups adapting seamlessly to a variety of fruit shapes [33][34][35][36].Soft grippers, employing compliant materials, further improved interactions, reducing the risk of damage.In the subsequent years, advanced materials and electromagnetic actuators transformed the landscape, making arms lightweight, energy-efficient, and precise.The hybrid gripper systems of the 2020s brought a new level of versatility, combining different gripping mechanisms for enhanced fruit handling.Simultaneously, artificial intelligence emerged as a key player, empowering these arms with real-time decision-making capabilities [37][38][39].Through specialized attachments and end-effectors, from gentle suctions for delicate fruits to cutting tools for stalked ones, these robotic arms evolved into versatile, indispensable tools in fruit harvesting [40][41][42].Ergonomics and safety considerations also received due attention, ensuring that these mechanical marvels not only excelled in function but also in user-friendly operation.The design of the fruit-harvesting robotic arm is one of continuous innovation, driven by a quest for precision, adaptability, and efficiency, ultimately reshaping the landscape of agricultural practices.
Researchers have made significant strides in addressing the challenge of localizing fruits in fruit-harvesting robotic arms since the last century [43].Before the 2000s, research in the field of fruit-harvesting robotic arms faced significant challenges in localizing fruits due to the limited availability of advanced technology [44].Their efforts primarily focused on rudimentary methods of fruit detection and localization.One of the earliest approaches involved basic computer vision techniques.Researchers developed algorithms to analyze 2D images of fruit-bearing trees [45][46][47].These algorithms relied on color-based segmentation and shape recognition to identify potential fruit locations [48,49].However, this method had limitations, especially in varying lighting conditions and with occluded or partially hidden fruits.
Another approach was the use of ultrasonic sensors [50,51].These sensors emitted high-frequency sound waves and measured the time taken for the waves to bounce back after hitting an object.Although they were effective in detecting obstacles, they were less precise in identifying individual fruits.Furthermore, some researchers explored the use of infrared sensors to detect temperature variations associated with ripe fruits [52].This approach, however, had limited success, as factors like ambient temperature and humidity could affect the readings.Researchers also experimented with mechanical contact-based systems.These systems involved probes or robotic arms that physically touched fruits to assess their ripeness [53].Although this approach could provide valuable information, it was not suitable for delicate fruits or those in high-density clusters.Initially, in the early 2000s, the focus was on basic computer vision techniques for fruit recognition.This involved developing algorithms to detect and identify fruits based on color, shape, and texture.As technology advanced, researchers began integrating more sophisticated sensing systems, such as multi-spectral imaging, which allowed for a more detailed analysis of fruit characteristics.This enabled the robots to distinguish between ripe and unripe fruits with higher accuracy.In the mid-2010s, the implementation of depth sensors marked a crucial advancement.These sensors provided precise information about the location of fruits in three-dimensional space, allowing for more accurate localization [54][55][56].Additionally, force feedback systems were introduced to ensure the gentle handling of delicate fruits during the harvesting process [38].A significant breakthrough came with the introduction of machine learning algorithms around 2013.These algorithms enabled the robotic systems to learn and adapt to different fruit varieties and environmental conditions, enhancing their ability to accurately identify and localize fruits [57][58][59][60].By 2015, researchers had achieved simultaneous multi-fruit harvesting capability, a milestone that greatly increased the efficiency of fruit harvesting robots [61].Furthermore, the development of robotic arms with multiple degrees of freedom allowed for more flexible and precise manipulation, further improving localization accuracy.In recent years, advancements in artificial intelligence (AI) have played a pivotal role.Integration of AI algorithms has enabled real-time assessment of fruit ripeness, allowing the robots to make decisions on the spot.Edge computing, introduced around 2021, further enhanced the speed and efficiency of AI processing directly on the robot, reducing reliance on external computational resources [62].Furthermore, the integration of LiDAR technology has provided the capability for 3D mapping of orchard terrain.This allows robots to navigate and localize fruits in complex and dynamic environments [63,64].Overall, researchers have employed a combination of computer vision, sensor technologies, machine learning, and AI algorithms to progressively refine the localization capabilities of fruit harvesting robots.These advancements have not only improved the efficiency and accuracy of fruit harvesting but also have the potential to revolutionize the agricultural industry by addressing labor shortages and increasing productivity.Figure 1 shows the timeline of the significant contributions to the design and development of fruit-harvesting robotic arms in decades.It can be seen that there has been a major contribution to its development in the recent decade due to its growing demand in the agricultural sector.A real-time image processing algorithm is used to determine the location of fruit, and their kinematic solution is obtained using C# [69].A mechanical custom-made end-effector is developed to detach the fruit, and the pressure required for the desired fruit is cal- An autonomous robotic arm developed by for apple harvesting that comprises a custom-made end-effector.A Pentium IV 2 GHz PC with 1 GB RAM, Panasonic VR006L robotic arm, tractor, generator, and touch panel PC with HMI are used in the manufacturing of a prototype.
Huang [65] developed a robotic arm with a path-planning plate form simulated in MATLAB.Their focus is to find a solution for the end-effector harvester to reach its destination with high precision by implementing an inverse kinematics technique.
Another low-cost prototype for a fruit harvester is designed by [66] using a stereo vision camera.The vision system is installed in the end-effector so that the updated information related to arm movement is sent to the controller for further necessary actions.The complete prototype is constructed using Minoru 3D for the vision system, DC gear motors, 3D printed arm parts using dimension SST 1200es, 3D printer, and an STM32F407VGT6 controller.
It is observed from the literature the main key issues are due to achieving low harvesting time, which was examined by [67].A low-cost, high-speed robotic apple harvester has been proposed, which boasts the ability to sense, plan, and harvest.The entire prototype is mounted on a John Deere Gator electric utility vehicle and comprises a manipulator, custom-made end-effectors, TOF camera, and color camera.The detection of objects (apples in this case) is achieved using circular Hough transformation (CHT) and blob analysis (BA) techniques.
An autonomous sweet pepper recognition and tracking system has been developed using a simulation approach [68].The simulation is carried out using MATLAB, V-REP, and ROS software for analysis.The algorithms are executed in MATLAB, and the real-time images captured by an external camera are filtered to remove color noise.The system's performance was evaluated using 78 images.
A real-time image processing algorithm is used to determine the location of fruit, and their kinematic solution is obtained using C# [69].A mechanical custom-made endeffector is developed to detach the fruit, and the pressure required for the desired fruit is calculated precisely by determining the tolerable pressure.Visual tasks are carried out using a two-dimensional camera with the addition of a depth sensor.The results were validated using 100 images, demonstrating an acceptable level of accuracy in fruit localization and harvesting.
A geometrical solution that can transform the exact location of peduncle coordinates into robotic arm joint functions using a Convolutional Neural Network (CNN) is validated [70].Kinect RGB-Depth camera is mounted on the robot for real-time image processing.The result validation shows a harvested success rate of about 52%.
Another robotic arm harvesting prototype based on deep learning techniques implemented for fruit harvesting and recognition are Point Net-based and Mobile-DasNet, respectively, which is designed in [71].The complete prototype is customized using a Universal Robotic manipulator UR5, self-design end-effector, intel D-435 depth camera, and NVIDIA GTX-1070 GPU.
Feng incorporated laser technology into an apple-harvesting robot [72].A laser vision system that allows the robot to accurately measure the distance to the target.This laser vision system enables the robot to perform a three-dimensional scan of the target scene, capturing detailed spatial geometry information and analyzing the relationship between the fruits and branches.
An integrated prototype is designed in [73], whose testing and validation are carried out in the greenhouse.The whole system is implemented by acquiring six Degree of Freedom (DOF) robotic arms, a Fotonic F80 depth camera, GPU, and PLC.The software counterpart in this model consists of C++ and ROS indigo on Ubuntu.Although attractive work is carried out, it has several drawbacks in terms of harvest success rate and detachment time.
A novel apple harvesting robot end effector that incorporates a pneumatic flexible actuator as a curved joint is created in [74].This innovative design allows for a substantial output force, enabling the end effector to firmly grasp the target fruit with excellent flexibility.This breakthrough development has the potential to significantly enhance the efficiency and effectiveness of apple harvesting processes.
A localization-based detection and fruit harvesting prototype is prepared in [75] with a custom cutting mechanism.The proposed system contains a Braccio robotic arm, depth camera, and Arduino Due controller.The software part is established using MATLAB and Arduino IDE environment.The prototype has several limitations in terms of performance, and the system is not fully automated because, during testing, it requires some manual modification.
A closed-range vision-based chili harvesting robot with a custom end-effector is designed in [52].The proposed manipulator detaches the chili using a suction and grasping mechanism.To minimize fruit damage, the detachment process is developed the same as manual harvesting.The prototype contains a laser diode, a laser receiver [76], five DOF manipulators, and a computer.The drawback of this design is a single trunk training model that works only on aligning a single trunk, so this type of prototype is not used for harvesting in a field environment.
Zhang designed a manipulator for an apple-picking robot and conducted experiments to study the control stability of the manipulator [77].The control area was divided into a stretching area and a harvesting area to ensure smooth operation.The trial showed that once the robot ranges the target, the required stabilization time is less than two seconds.This suggests that the manipulator's control system is stable and capable of quickly stabilizing once it reaches the harvesting area.
Another designed prototype for an automated mushroom harvester is proposed in [78].Special custom end-effectors are designed for the delicate control of mushrooms.The algorithm developed for localizing and identification of mushrooms is based on the calculated area of the mushroom top.The maximum diameter of the mushroom top harvested by the system is 75 mm.To solve the problem of low lightning conditions, a Phillips TLE 23Watt bulb is used near end-effectors to maintain a better environment for target identification by the vision system.The calculated damage rate claim by the author during the testing of the developed prototype is 3%.
Zhao has developed a robot for apple harvesting that features a 5 • series joint with an integrated arm that has both lifting and telescopic capabilities [79].The robot's performance was tested under controlled laboratory conditions, resulting in an 80% success rate, with a harvesting period of 15 s.
An Automated Robotic arm for the orange harvester was developed in [80].The vision system for fruit identification is composed of two cameras (ZED Stereo camera and A4tecch Webcam) and one ultrasonic sensor.A stereo camera is used to detect orange trees by applying a green detection algorithm.A webcam is used to serve the end-effector to reach the fruit, and an ultrasonic sensor is used for the distance measurement of the fruit from the end-effector.
The intelligent fruit-harvesting robot designed by Gu was equipped with advanced technologies such as autonomous navigation, computer vision, and robotic arm control [81].The robot was able to autonomously navigate through the orchard using sensors and algorithms to detect obstacles and plan its path.It could identify mature fruits using computer vision algorithms with a high recognition rate in the field of agricultural robotics.
Most of the above developments show that the basic aim of their work is the successful harvesting of fruit using an optimal detachment mechanism.Results claimed by the manufacturer in the previous developments are shown in Table 1.It is observed from previous work that harvesting time, harvesting success rate, and localization success rate are the main focused parameters that are discussed by the authors in the literature.The harvesting cycle is the period for the detachment of a single fruit that starts from the initial manipulator movement to the successful reach of the fruit to the container.The localization success rate is the ratio of the total number of available fruits and detected fruits by the vision systems.Harvesting success rate is the ratio of total number of fruits harvested and the number of fruits detected by the system.

System Requirements
In this section, the entire requirement for the development of fruit-harvesting robotic arms is discussed.The design contains a manipulator for mechanical movement, a vision system for localizing and recognition of objects, and an end-effector for detachment purposes.The development of a software platform is also an important part of the system that will manage mutual collaboration between components so that a well-defined workflow has been established.Data processing in software is dependent on manipulator coordinates, image information, and data from sensors.A basic flow chart for automated fruit harvesters working in a field environment is shown in Figure 2.
It is observed from previous work that harvesting time, harvesting success rate, and localization success rate are the main focused parameters that are discussed by the authors in the literature.The harvesting cycle is the period for the detachment of a single fruit that starts from the initial manipulator movement to the successful reach of the fruit to the container.The localization success rate is the ratio of the total number of available fruits and detected fruits by the vision systems.Harvesting success rate is the ratio of total number of fruits harvested and the number of fruits detected by the system.

System Requirements
In this section, the entire requirement for the development of fruit-harvesting robotic arms is discussed.The design contains a manipulator for mechanical movement, a vision system for localizing and recognition of objects, and an end-effector for detachment purposes.The development of a software platform is also an important part of the system that will manage mutual collaboration between components so that a well-defined workflow has been established.Data processing in software is dependent on manipulator coordinates, image information, and data from sensors.A basic flow chart for automated fruit harvesters working in a field environment is shown in Figure 2.   Working principle and their mechanical structure concern to each component is discussed in this section.However, their occurring challenges are considered in the next section.

Manipulator
For the successful harvesting of valuable fruit, a mechanical architecture that consists of a robotic manipulator and end-effectors is needed.The function of the manipulator is to guide the end-effector to reach the target, where the end-effector grasps it, and the manipulator brings fruit to the container [82].The entire workspace is known by the system so that they will navigate the fruit to the target without damaging the system and fruit.Parameters considered in the workspace depend upon plant height, system height, number of fruits, the distance of target fruit, and size of target fruit.
Another specification for the development of a manipulator is to consider the payload of the desired fruit, i.e., The manipulator design for watermelon and apple has different types of load capacity conditions.An easy way of understanding a manipulator is that it resembles a human arm [83].Where the human arm is related to a six DOF manipulator, each joint of the robotic arm is equivalent to the shoulder, arm, and wrist, as shown in Figure 3.
nipulator brings fruit to the container [82].The entire workspace is known by the sys so that they will navigate the fruit to the target without damaging the system and f Parameters considered in the workspace depend upon plant height, system height, n ber of fruits, the distance of target fruit, and size of target fruit.
Another specification for the development of a manipulator is to consider the pay of the desired fruit, i.e., The manipulator design for watermelon and apple has diffe types of load capacity conditions.An easy way of understanding a manipulator is th resembles a human arm [83].Where the human arm is related to a six DOF manipula each joint of the robotic arm is equivalent to the shoulder, arm, and wrist, as show Figure 3.Where the shoulder joint of the manipulator is attached to a fixed body instead moveable structure, and the remaining joints are found inside the robotic arm.Usu the fourth DOF of the robotic arm is used for wrist movement, the fifth for adjustmen rotation or orientation, and the sixth is for gripping [84].An essential amount of dr power is required for operating motors that are found inside the joints for their movem which is described by the manufacturer's power rating depending upon the moto quirement, weight of the manipulator, and type of work taken from the robotic arm.Ev DOF internally carries a motor inside its joint for mechanical movement.Heavy pay is lying on the second DOF, so a servo motor with high torque is especially required this joint [85].

Vision System
Vision system is an important component throughout all automated robotic 1syst known as Vision Guided Robotic system (VGR).In agriculture robotic harvesting, the ject (fruit in our case) is first identified by the vision system.Manipulators reach a ta based on the information coordinates provided by the vision system [86].For several ades, active research has been performed on the development of optimal visual p forms that will identify the fruit with high accuracy.Many algorithms are applied in m Where the shoulder joint of the manipulator is attached to a fixed body instead of a moveable structure, and the remaining joints are found inside the robotic arm.Usually, the fourth DOF of the robotic arm is used for wrist movement, the fifth for adjustment of rotation or orientation, and the sixth is for gripping [84].An essential amount of driven power is required for operating motors that are found inside the joints for their movement, which is described by the manufacturer's power rating depending upon the motor requirement, weight of the manipulator, and type of work taken from the robotic arm.Every DOF internally carries a motor inside its joint for mechanical movement.Heavy payload is lying on the second DOF, so a servo motor with high torque is especially required for this joint [85].

Vision System
Vision system is an important component throughout all automated robotic 1systems known as Vision Guided Robotic system (VGR).In agriculture robotic harvesting, the object (fruit in our case) is first identified by the vision system.Manipulators reach a target based on the information coordinates provided by the vision system [86].For several decades, active research has been performed on the development of optimal visual plate forms that will identify the fruit with high accuracy.Many algorithms are applied in many prototypes where accuracy in computational and sensing parameters is the first choice to achieve efficient results.The most important feature of fruit sensing is color-based identification [87,88].Several distinct algorithms based on segmentation were developed for apples, citrus, mangoes, tomatoes, pineapples, etc.For vision identification, researchers also used other techniques like Circular Hough Transform [89], a global mixture of Gaussians [90], Blob identification [91], and many more that can achieve a 70% to 95% success rate in a controlled environment.The development of efficient vision systems is the basic key feature needed to increase the success rate of harvesting devices.Different types of sensors are used for the identification of fruits, i.e., ultrasonic sensors, laser scanners, Light Detection and Ranging (LiDAR) sensors, stereo vision cameras, depth cameras, and RGB cameras.Different types of sensors used for robotic arm vision guiding are shown in Figure 4. also used other techniques like Circular Hough Transform [89], a global mixture of Gaussians [90], Blob identification [91], and many more that can achieve a 70% to 95% success rate in a controlled environment.The development of efficient vision systems is the basic key feature needed to increase the success rate of harvesting devices.Different types of sensors are used for the identification of fruits, i.Many industries have made special universal sensors that have multiple functionalities in a single module, i.e., a single module has the ability of depth plus RGB color sensing and maybe more [96].Using these sensors, different properties of plants and fruits, like fruit size, location, color, pose, and orientation, have been acquired.There are two types of VGR systems: arm-mounted camera VGR system and stationary camera mount VGR system.In the stationary camera mount system, the vision sensors are mounted on a base that is separate from the manipulator.This type of system requires an additional infrastructure that will monitor the field of view available in their range.The captured information has been sent to the software system.After receiving information from the software, they will calculate the position of the target using inverse kinematics [97].The manipulator reaches the target according to defining variable values and completes the desired task.Certain calibrations are required that will interpret the vision sensor position and manipulator position concerning each other.In the arm-mounted VGR system, the vision sensor is attached to the manipulator and repeatedly changes its position due to arm movement.This type of system does not require additional infrastructure design for the camera.During their task, the updated information is received by the software, so the system knows their current position in the 3D plane as compared to the initial point.In  [92], (B) stereovision camera [93], (C) ultrasonic sensor [94], and (D) depth camera sensor [95].
Many industries have made special universal sensors that have multiple functionalities in a single module, i.e., a single module has the ability of depth plus RGB color sensing and maybe more [96].Using these sensors, different properties of plants and fruits, like fruit size, location, color, pose, and orientation, have been acquired.There are two types of VGR systems: arm-mounted camera VGR system and stationary camera mount VGR system.In the stationary camera mount system, the vision sensors are mounted on a base that is separate from the manipulator.This type of system requires an additional infrastructure that will monitor the field of view available in their range.The captured information has been sent to the software system.After receiving information from the software, they will calculate the position of the target using inverse kinematics [97].The manipulator reaches the target according to defining variable values and completes the desired task.Certain calibrations are required that will interpret the vision sensor position and manipulator position concerning each other.In the arm-mounted VGR system, the vision sensor is attached to the manipulator and repeatedly changes its position due to arm movement.This type of system does not require additional infrastructure design for the camera.During their task, the updated information is received by the software, so the system knows their current position in the 3D plane as compared to the initial point.In this scenario, the distance of the fruit to the camera is first evaluated systematically and then, based on the displacement between the camera, gripper, and target available in the workspace, is estimated by scaling [98].
A monocular camera is used for vision purposes, and a stepper motor is used for manipulator movement [99].In this scenario, a depth camera is not required; the depth information is acquired through their optical phenomenon.The degree of blurriness is estimated to find the distance of an object in the plane.The accuracy of the system is not too good as compared to the stereovision camera-based system.Another design of a system based on a stereovision camera that can estimate the depth of an object can be found [100].The blob technique is used for extracting features of the image, and detection is carried out using each camera; segmentation occurs before delivering a command to further attached modules.The term "each camera image" refers to a stereovision camera that uses two cameras with a confined distance among them that work like human eyes.Results show better performance in accuracy and target size estimation as compared to monocular cameras.
An apple fruit picker that contains stereovision cameras has been developed [101].Their performance is validated under different conditions where the target ranges from 30 cm to 110 cm.Validation is carried out in two scenarios; first, a manual procedure is performed by placing a small object in the center of an apple that has an error estimate of approx.0.63% in distance.In the second scenario, the centroid of the apple is found through computation analysis that has an error of about 3.54%.For the detection of a ripened tomato, a structured light-based system was developed in [102].In this work, the concept of structured light is used for the localization of fruit, and the Cb layer is used for the estimation of its ripeness.An error of a few millimeters is spotted during system validation.As seen in the literature, most of the development process is based on stereovision cameras.Another robotic manipulator designed for harvesting lychees is also based on a stereovision system [103].This fruit picker is tested at the laboratory level for path planning and fruit detachment.The manipulator is developed in a virtual environment, and path planning kinematics is coded in Microsoft Visual C++.Most research papers reviewed up to this point are purely based on the arm-mounted camera technique or the stationary mount technique.In contrast, a novel technique is proposed that is based on the collaboration of both vision systems [104], where a stationary mount camera is responsible for the greater field of view of the workspace, and accuracy is gained from the arm-mounted camera.Testing of this technique was carried out in a citrus orchard, and the system obtained 95% accuracy.A prototype has been designed for detecting and picking small asparagus of 230 mm in height [105].The vision system consists of two slit lasers and an old TV camera.The reason behind the two lasers is that they controlled the desired height of asparagus plants.Another laser-camera system is designed to be used for the detection and detachment of apples.A machine vision scenario is used for apple detection, and laser sensors are used for fruit localizing [106].The accuracy of the system, as claimed by the author, is 100% in detection and has an accuracy of 90% in fruit detachment.

End-Effector
There are two types of fruit harvesting techniques: one is the bulk technique, and the second one is the fruit-by-fruit harvesting technique [107].The design of end-effectors does not matter in the bulk fruit-picking technique, where the detachment process is performed by a Rapid Displacement Actuator (RDA).Branches of trees with ripened fruit are detected by the vision system, and then by actuating RDA, the fruit is harvested and caught by another system for collection in the container.RDA is used to apply vibration to the branches of trees [108].However, this type of harvesting has many disadvantages that will be discussed in the subsequent section.The design of end-effectors has a had significant impact on individual fruit harvesting techniques.There is a lot of research work occurring on the custom shape of the harvester.So far, different types of cutting tools have been invented, such as scissor type, suction mechanism [109], and swallow harvester [110].
Normally, the suction-based approach is used for harvesting, and it is much more reliable and easy to design [73].This approach is achieved through a cap-type end-effector and a tube body connecting the cap of the sucker.By squeezing with air pressure, the target fruit is harvested and transferred to a storage container via a discharge tube.The second type of harvesting tool is usually made for fragile crops and requires complex engineering for its cutting mechanism.This type of effector consists of scissors that will cut the peduncle of the fruit and a gripper that will hold the fruit until it reaches the target containers.A specially designed trained system is required to detect the cutting point of the peduncle.Research efforts have focused more on the development of a cutting peduncle-based system than on another harvesting system [111].These types of systems for tomatoes and sweet peppers are developed in [112,113].Another type of end-effector works almost like a human hand that will reach the target and grasp it in its mechanical design, just like fingers.After grasping them, the fruit will be detached from the plant.However, certain statistics may be required to calculate the gripping and detachment force to avoid damaging fruit and plant [114].Force is applied using manipulator movement, and sometimes twisting the wrist will provide enough effort for their detachment.This type of harvesting technique is mainly applied to rigid plants and trees where fruit picking may cause some disturbance that will not affect them.So, in the development of swallow end-effectors, fruit shape, fruit size, fruit weight, and the plant's nature are important components.A vast majority of researchers work on the development of reliable end-effectors, as shown in Figure 5.
containers.A specially designed trained system is required to detect the cutting point of the peduncle.Research efforts have focused more on the development of a cutting peduncle-based system than on another harvesting system [111].These types of systems for tomatoes and sweet peppers are developed in [112,113].Another type of end-effector works almost like a human hand that will reach the target and grasp it in its mechanical design, just like fingers.After grasping them, the fruit will be detached from the plant.However, certain statistics may be required to calculate the gripping and detachment force to avoid damaging fruit and plant [114].Force is applied using manipulator movement, and sometimes twisting the wrist will provide enough effort for their detachment.This type of harvesting technique is mainly applied to rigid plants and trees where fruit picking may cause some disturbance that will not affect them.So, in the development of swallow endeffectors, fruit shape, fruit size, fruit weight, and the plant's nature are important components.A vast majority of researchers work on the development of reliable end-effectors, as shown in Figure 5.

Development Challenges
In this section, the hurdles faced in the development of automated fruit-harvesting robotic arms will be reviewed.Every component of the system suffers from a series of challenges.Challenges are dependent on the parameters cost, working environment, the efficiency of the system, the size of the prototype, types of sensors, implemented algorithm, harvesting time, manipulator material, and hardware.Adjusting one parameter in the name of improvement will result in compromising or reducing some other parameter.An overview of development challenges arising in different components of robotic arms is shown in Figure 6.

Development Challenges
In this section, the hurdles faced in the development of automated fruit-harvesting robotic arms will be reviewed.Every component of the system suffers from a series of challenges.Challenges are dependent on the parameters cost, working environment, the efficiency of the system, the size of the prototype, types of sensors, implemented algorithm, harvesting time, manipulator material, and hardware.Adjusting one parameter in the name of improvement will result in compromising or reducing some other parameter.An overview of development challenges arising in different components of robotic arms is shown in Figure 6.

Vision Challenges
The vision system for fruit-harvesting robotic arms encounters a myriad of challenges.These encompass varying lighting conditions, potential obstruction by leaves and branches, overlapping fruits, and the challenge of accurately detecting distant fruits.Natural variations in fruit color and shape, as well as occlusion by plant elements, further complicate recognition.Environmental factors like wind and rain introduce dynamic ele-

Vision Challenges
The vision system for fruit-harvesting robotic arms encounters a myriad of challenges.These encompass varying lighting conditions, potential obstruction by leaves and branches, overlapping fruits, and the challenge of accurately detecting distant fruits.Natural variations in fruit color and shape, as well as occlusion by plant elements, further complicate recognition.Environmental factors like wind and rain introduce dynamic elements, while seasonal changes in plant and fruit characteristics demand adaptability.Ensuring dust-free camera lenses, precise calibration, and real-time processing capabilities are imperative.Depth perception, resilience to environmental variations, and adaptability to diverse fruit types are also crucial.Additionally, the system's capacity for learning and generalization across different environments and fruit types is essential for its effectiveness and versatility in the field.
Identification of fruit by visual sensing is an important element of the system that is suffering from several challenges.Over several decades, significant work has been carried out in the identification of fruit, and many techniques have been developed, and implemented in the prototypes.The researcher claims to have achieved an acceptable number of results in the previous work.One the problems reported by the researchers is that of different lighting environments, affect the segmentation process and play a vital role in fruit identification [115][116][117].The different lighting conditions mean the designed work seems to look good and has better accuracy in the control lighting environment in laboratory conditions.However, these systems face problems in the orchard/field environment due to the presence of intense sunlight in the daytime and very dim light at night.These issues are addressed to some extent by harvesting the fruit at sunset or providing enough light during the night-time but will still reduce the efficiency of the system [118].The prototype is said to be highly efficient if it can work in every environment without depending on the limited time window.
Variable lighting problems arise due to the effect of unbalanced exposure on the image, which will disturb saturation and contrast.So, in these situations, the quality of the image may cause problems in fruit identification.An exposure fusion technique is developed in [119].This method produces a well-exposed picture by retaining just the best areas for every frame in various exposure-given images, as shown in Figure 7.
AgriEngineering 2023, 5, FOR PEER REVIEW 14 developed in [119].This method produces a well-exposed picture by retaining just the best areas for every frame in various exposure-given images, as shown in Figure 7.To calculate the optimum portions and weight, they merged the picture layer into a final image using visual quality characteristics such as sharpness, saturation, and wellexposedness.The use of this method resulted in an evenly focused view of the target with little shades.Another technique is proposed that will perform fruit identification independent of color properties; based on this work, the fruit is identified by a 3D geometry plane [120].
Like vision algorithm techniques, the selection of visual sensors is also an important To calculate the optimum portions and weight, they merged the picture layer into a final image using visual quality characteristics such as sharpness, saturation, and wellexposedness.The use of this method resulted in an evenly focused view of the target with little shades.Another technique is proposed that will perform fruit identification independent of color properties; based on this work, the fruit is identified by a 3D geometry plane [120].
Like vision algorithm techniques, the selection of visual sensors is also an important issue [121].Two types of vision scenarios are available in VGR, as described in Section 3.2.In stationary camera mount, the vision sensor is mounted on a fixed frame, also known as the method.This type of system has the advantage of knowing their fixed position as a reference point in the working area.The limitation of this system is the installation of an extra frame for the vision sensor and the second high-resolution camera required for object detection.The camera must be three-dimensional, with the ability to extract depth information.For better identification of fruit quality, the image for target tracking should not be lower than 5 Mp.The selection of position for the camera and manipulator mounting is also a challenging task; their position is identified in such a manner that the view of the camera is not interrupted during manipulator movement.Complex computation is required for the localization of fruit from a fixed point, and the availability of error in depth data from a fixed position is also higher.
Challenges faced in this scenario are tackled using an arm-mounted technique, also known as the eye-in-hand technique.In this scenario, the vision camera is fixed in the hand of the manipulator or on the end-effector [122].In this scenario, the distance of the target at every instance is monitored in a closed loop concerning the position of the end-effector and vision sensor.A low-cost two-dimensional camera is enough for vision purposes.Arm mounted solves many challenges that are faced in a stationary mount scenario, like the need for another frame for the camera, interference of camera and manipulator, and requirement for a high-quality camera.Along with these advantages, there are also limitations in the arm-mounted technique.A camera is mounted in the middle or at the top of the endeffector, which will affect the burden on the manipulator.Some manipulators could not tolerate the burden of heavily integrated cameras, as seen in the case of the Microsoft Kinect depth sensor used in [123], which is approximately 1.5 kg.So, in these types of scenarios, the implementation of a stationary camera mount system is optimal.

End-Effector Challenges
The selection of a harvesting technique is also a challenging task.In the bulk technique, a heavy mass of fruit is harvested in a short time by applying vibration to the plant or tree.This process was first adopted several years ago and gave a good output in time consumption [124], but it has several disadvantages reported by fruit growers that fruit and plant damage occur due to applied detachment force.However, this problem is very limited in big canopy trees and hardy fruits but has a high impact on fragile fruit and plants [125].Another problem in bulk harvesting is that the fruit detachment is carried out without the identification of single characteristics of fruit, so most of the harvested fruit is unripe.As a result, bulk harvesting of these products is not acceptable in the market as compared to manually harvested products due to damaged and unripened conditions.Active research is still in progress in this field to harvest the fruit at their maturity level, which is not a trivial process, but it will affect the harvesting time.Another method for harvesting is fruit-by-fruit harvesting or selective harvesting [126].Implementation of this technique also suffers from technical challenges like the selection of end-effector, type of vision system, and DOF of a robotic arm.In successful harvesting, the selection of an end-effector is an important factor.Three types of end-effectors are developed in selective harvesting: cutting end-effector, suction mechanism, and swallow end-effector.Commonly, a suction mechanism is used in most cases where the end-effector does not make any harsh contact with the plant, but physical contact occurs with fruit during harvesting.The problem facing the suction base technique is the need for another mechanism for suction that will burden extra costs on the system.Another limitation of this technique is damage to fragile plants and fruit.
A cutting mechanism is needed for harvesting fragile fruit and plants [127].An additional mechanism is attached with an end-effector that consists of a scissor-type cutting tool.When the end-effector reaches the target fruit, the actuator forces the cutting mechanism for peduncle cutting.Not only is the cutting mechanism needed, but the griping mechanism needed to hold the fruit until it reaches the manipulator does not reach the destination container.This mechanism is very useful in the case of sensitive plants and fruit, but the challenge in this technique is the identification of a cutting point.High-resolution cameras and complex algorithms are used in their development.These types of developed cutting mechanism end-effectors are also problematic in a cluster of fruit.Sometimes, after harvesting fruit, part of the peduncle may be stuck in the cutting tool; as a result, the cutter will not cut further peduncle perfectly [128].Before designing the cutting tool, a mathematical calculation is carried out for the desired applied force that depends on the strength of the peduncle, which is also a challenging task.Many developed prototypes based on the cutting mechanism use two power sources that will increase the cost of the system.An optimal solution is developed in [36]; a novel gripper is designed that will share a single power source for the fruit gripper and cutter.
Another type of end-effector is the swallowing end-effector, which is almost like a human hand.At the edge of the end-effector, a mechanism like the fingers of a human is attached that will pick the target fruit, almost like traditional manual harvesting.In the swallow-type end-effector, the detachment force burden is also experienced by the manipulator.The detachment is performed by grasping the fruit and applying the force by the manipulator's movement that will cause damage to the plant, fruit, and the manipulator's body.As a consequence of the above statements, the selection of a manipulator in the robotic arm is dependent on the type of task performed by the system.When the system is developing for hardy fruit and plants, the suction mechanism will be improved.The swallow technique is implemented for fragile fruit and hardy plants.However, a cutting mechanism is developed for that system where plant and fruit both are sensitive to force.

Manipulator Challenges
The selection of the number of DOFs in a robotic arm manipulator is also an important consideration in a design.The working feasibility of the robotic arm in a complex working environment is increased by increasing the DOF of the robotic arm so that extending to the target is much easier in that case.A better outcome is obtained by using kinematics simulations to control the manipulator precisely.The manipulator's kinematic approach concentrates on the movement within each link before considering any force that causes the movement [129].Every DOF denotes an important variable necessary to express the position of one link concerning the preceding link, adding to the complexity of a manipulator [130].
Additional constraints seem to be that the system needs to be able to operate continuously in its field environment, which might be either an open-field agricultural field environment or operating in lab testing.Outdoor-working robots will be subjected to more demanding operational circumstances.Rain, wind, dust, and extreme temperature variations may necessitate protection for actuators, sensors, and control systems.In lab testing, most of the operating tools are easily available, but in field operations, there is a requirement for additional generators, power sources, and sufficient lighting conditions.

Conclusions
The development of fruit-harvesting robotic arms presents a multifaceted set of challenges across three critical domains.In terms of vision challenges, the ability to accurately detect, locate, and assess the ripeness of fruits in complex and dynamically changing environments remains a challenging task.Overcoming this necessitates advanced sensor fusion techniques, robust computer vision algorithms, and real-time data processing capabilities.At the same time, in manipulator design, accommodating the diverse range of fruit sizes, shapes, and locations demands an adaptable kinematic structure coupled with precise control mechanisms.Furthermore, ensuring gentle yet effective fruit handling necessitates innovations in compliance and actuation systems.Lastly, end effector challenges require an intricate balance of delicacy and adaptability, requiring specialized materials, tactile sensors, and advanced computer vision algorithms.Addressing these challenges will be pivotal in realizing the full potential of fruit-harvesting robotic arms and revolutionizing agricultural practices for increased efficiency and sustainability.Advancement in technologies in the current era is merging into many fields, such as industries, the medical field, education systems, defense systems, navigation systems, and even domestic appliances.That makes work easier, lower cost, and faster than traditional approaches.Implementation of technologies in the field of agriculture also tends to be modernized with the advanced era.For several years, research has been performed in the field of agriculture to facilitate the former.Many commercial automated machines have been developed for harvesting crops that are ripening and harvested at the same time as potato, wheat, sugarcane, corn, etc.However, a robotic arm as a harvesting device for those agricultural products that require repetitive tasks for harvesting at different intervals is still being tested.The reason behind the delay in this effort is that there are several challenges in their development.These challenges appear in every component of the robotic arm and obstruct their development.The main components of automated robotic arms are vision systems, manipulators, and end-effectors.The main problem that arises in the vision system is the selection of a camera-fixing scenario and applied algorithm.The development of manipulators faces challenges in selecting the number of DOFs.The system is said to be highly efficient when the vision system identifies all the fruit in its field of view, the algorithm calculates their location, the manipulator reaches every target fruit, the end-effector detaches the fruit from the plant, and the manipulator successfully holds the fruit before transferring it to the fruit container.It is observed from previous work that the development of robotic arms is associated with their application.Factors like the working environment, fruit type, nature of the fruit, and the extent of the cost must be known by the developer.The effort made in this review article is to discover all the work that is necessary for consideration in the development of a robotic arm.

Future Works
Future advancements in the mechanical design domain of fruit-harvesting robotic arms will prioritize structural optimization for a balance of strength and weight (i.e., low weight and high strength).Innovations in articulation, linkage mechanisms, and kinematics will enhance maneuverability to navigate around complex orchard layouts.Integration of torque and force sensors, along with redundancy and safety features, will ensure precise and gentle fruit handling.Sealing mechanisms and protective enclosures can protect components from environmental factors, while vibration control techniques can minimize oscillations.Ergonomic considerations will streamline operation and maintenance, and advanced safety features will prevent accidents.Future strides in the vision systems of fruit-harvesting robotic arms will steer in a new era of precision.Advanced 3D cameras and machine vision technologies will be deployed to swiftly identify ripe fruits, assess their size, and determine their position and orientation.Additionally, multispectral imaging may play a pivotal role in distinguishing fruit from foliage, further enhancing accuracy.Integration of AI-powered algorithms for real-time decision-making will optimize picking trajectories.As for end effectors, the focus will shift towards adaptable, multi-modal designs.Grippers equipped with compliant materials and adjustable features will gently yet securely grasp fruits of varying shapes and sizes.Modular end-effector attachments will enable seamless transitions between different fruit types, ensuring versatility in harvesting operations.These advancements in vision systems and end effectors promise to revolutionize fruit harvesting with unprecedented speed and precision in the future.

Figure 1 .
Figure 1.Timeline showing significant contributions in fruit-harvesting robotic arms development.

Figure 1 .
Figure 1.Timeline showing significant contributions in fruit-harvesting robotic arms development.

Figure 3 .
Figure 3.Comparison of a human arm with a robotic arm.

Figure 3 .
Figure 3.Comparison of a human arm with a robotic arm.
e., ultrasonic sensors, laser scanners, Light Detection and Ranging (LiDAR) sensors, stereo vision cameras, depth cameras, and RGB cameras.Different types of sensors used for robotic arm vision guiding are shown in Figure 4.

Figure 7 .
Figure 7. Exposure-fused technique applied to an apple with variable lighting conditions.

Figure 7 .
Figure 7. Exposure-fused technique applied to an apple with variable lighting conditions.

Table 1 .
Overview of recent developments in automated fruit harvesting prototypes.