Development, Design, and Improvement of an Intelligent Harvesting System for Aquatic Vegetable Brasenia schreberi

Abstract

At present, there is a lack of effective and usable machinery in the harvesting of aquatic vegetables. The harvesting of most aquatic vegetables such as Brasenia schreberi relies entirely on manual labor, resulting in a high labor demand and labor shortages, which restricts the industrial development of aquatic vegetables. To address this problem, an intelligent harvesting system for the aquatic vegetable Brasenia schreberi was developed in response to the challenging working conditions associated with harvesting it. The system is composed of a catamaran mobile platform, a picking device, and a harvesting manipulator control system. The mobile platform, driven by two paddle wheels, is equipped with a protective device to prevent vegetable stem entanglement, making it suitable for shallow pond aquatic vegetable environments. The self-designed picking device rapidly harvests vegetables through lateral clamping and cutting. The harvesting manipulator control system incorporates harvesting posture perception based on the YOLO-GS recognition algorithm and combines it with an improved RRT algorithm for robotic arm path planning. The experimental results indicate that the intelligent harvesting system is suitable for aquatic vegetable harvesting and the improved RRT algorithm surpasses the traditional one in terms of the planning time and path length. The vision-based positioning error was 4.80 mm, meeting harvesting accuracy requirements. In actual harvest experiments, the system showed an average success rate of 90.0%, with an average picking time of 5.229 s per leaf, thus proving its feasibility and effectiveness.

1. Introduction

Aquatic vegetables refer to vascular plants that grow in freshwater and whose products can be used as vegetables, including lotus (Nelumbo nucifera), water bamboo (Zizania latifolia), Gorgon fruit (Euryale ferox), water chestnut (Eleocharis dulcis), water celery (Oenanthe javanica), water caltrop (Trapa natans), water spinach (Ipomoea aquatica), water shield (Brasenia schreberi), etc. [1]. Aquatic vegetable production is widespread in many regions of the world, especially in Asia. For example, China has a variety of aquatic vegetables, such as lotus, water bamboo, and water chestnut [2]. Aquatic vegetables are of great importance in meeting people’s vegetable supply, enriching spring, summer, and off-season vegetable markets. Aquatic vegetables grow in water and are widely distributed, with different external characteristics and growth patterns, making their management and harvesting very difficult. Some can be harvested in one-time operations, such as water chestnuts, while some require continuous selective harvesting, such as lotus seedpods. There are some machines for harvesting aquatic vegetables during one-time operations, such as water chestnut harvesting [2] and lotus root harvesting [3], but there are relatively few machines for harvesting aquatic vegetables using the selective harvesting mode.

Brasenia schreberi is an aquatic economic crop that is widely distributed across multiple regions in Asia, Europe, North America, and Australia [4,5]. The main edible part of Brasenia schreberi is the tender bud. The tender leaves can also be used for sauces, toothpaste, facial masks, and other health and beauty products. The mechanized harvesting of Brasenia schreberi is quite difficult as it grows in water and has relatively small leaves and buds. The leaves of Brasenia schreberi generally float on the surface of the water. The tender buds grow in the water, about 10–20 cm below the water surface. Brasenia schreberi can only be harvested selectively. The traditional operation is not only laborious, but also inefficient, requiring high labor intensity (the common operation is shown in Figure 1; Figure 1a shows a harvester picking in rain pants and Figure 1b shows picking using wooden barrels). At present, the harvesting of Brasenia schreberi primarily relies on manual picking. There are few mechanical harvesting methods. Xia et al. [6] proposed a manually controlled harvester for Brasenia schreberi, which can reduce the difficulty of picking in water but has not improved the efficiency and has not been widely adopted. From the perspective of labor supply, the acceleration of urbanization has led to a gradual decrease in the rural young labor force, resulting in increasing labor costs and the intensification of labor shortages. Moreover, Brasenia schreberi exhibits a relatively long harvesting period, which generally starts in April and can extend to October, following a multiple selective-harvest model. Traditional harvesting methods are not only laborious but also inefficient, with a high labor intensity. Therefore, the mechanization and intelligentization of aquatic vegetable harvesting operations are inevitable trends, and the establishment of a stable and efficient intelligent harvesting system has become a new challenge in the agricultural field.

In recent years, the exploration and application of harvesting robots in the agricultural field have been increasing [7]. A stable and integrated intelligent harvesting system can ensure harvesting efficiency while maintaining a high success rate, thereby reducing the labor intensity of operators. In an intelligent harvesting system for Brasenia schreberi, it is particularly critical to improve three aspects [8]. First, the mobile platform of the harvesting system needs to adapt to the specific working environment to ensure stability and operability under complex conditions. Second, the end effector requires a special design to achieve the rapid cutting of Brasenia schreberi stems, thereby improving the picking efficiency. Finally, the mechanical arm picking system based on machine vision needs to accurately identify Brasenia schreberi while maintaining a high positioning accuracy and operational efficiency.

As a crucial component of the harvesting system, the development of the mobile platform should consider the location. In recent years, due to the unique characteristics of aquatic environments, unmanned boats have emerged as common mobile platforms in water areas. Zhu et al. [9] adopted a monohull design powered by an electric propeller to create an unmanned water quality monitoring boat. Ge et al. [10] implemented a dual-float and paddle wheel drive scheme, combined with Beidou technology and inertial navigation equipment, to design an intelligent baiting boat for crab pond environments. Matsushita Yoshiki et al. [11] utilized an aluminum frame for the deck, two rubber floats for the hull, and two 200 W electric propellers to create a near-sea fish-attracting boat equipped with underwater light-emitting diodes. Liu et al. [12], addressing the need for fertilization and weeding in paddy fields, designed a shallow-draft unmanned operation boat that employed an above-water air propeller to protect young rice seedlings instead of the traditional underwater propeller. Ruan et al. [13] designed a multifunctional automatic feeding boat for river crabs based on their automatic feeding requirements, creating an intelligent control system and implementing a PID control algorithm to achieve automatic control of the feeding boat. From the studies mentioned above, it is evident that mobile platforms in aquatic environments typically feature lightweight structural designs to ensure good mobility, with the drive devices and sensor equipment varying significantly depending on the tasks. Considering the shallow depth and accumulation of silt at the bottom of the ponds where Brasenia schreberi grows [14], the existing mobile platform designs do not sufficiently adapt to this specific environment. Therefore, it is essential to design an appropriate mobile platform to facilitate the harvesting of Brasenia schreberi.

End effectors serve as a crucial link between the harvesting robot and the working object, and in the current field of fruit and vegetable harvesting, research on end effectors has gained significant attention. In fruit and vegetable harvesting operations, the drive method, material, and structure of the end effector often require a specific design tailored to the characteristics of the harvested object. Xie et al. [15] proposed a flexible harvesting end effector for cherry tomatoes, designed based on bionic principles. To minimize the damage caused to the surface of tomato fruits during harvesting, flexible rubber was utilized to imitate the tarsal chain joints of insects, achieving a 76% harvesting success rate. For harvesting harder and heavier objects, the load capacity of the drive device has received greater attention. The shape and contact conditions of the picking device influence the picking effect. Zhang et al. [16] designed an end effector for cherry tomato picking that increased the contact area and reduced the damage caused to fruit by incorporating a curved end. They also optimized the picking method and parameters. The picking mode further impacts the picking effect. Ji et al. [17] investigated the picking posture of a three-claw end for apple picking, comparing and analyzing the differences in vertical, horizontal, and circular breaking methods, ultimately obtaining an optimized picking mode that reduces stress on apples. Roshanianfard et al. [18] developed a five-finger end effector driven by a 220 V servo motor for pumpkin harvesting, achieving a harvesting success rate of 79% for pumpkin fruits with a radius range of 76.2–265 mm. To minimize the damage caused to fruit as much as possible, Li et al. [19] designed a rope-driven anthropomorphic end effector for the non-destructive harvesting of pears, achieving a 0% grasping damage rate through the use of a grasping force sensor. These studies have provided their own solutions for common types of fruits and have achieved certain results. However, the method of harvesting Brasenia schreberi is significantly different from that of terrestrial fruits and vegetables. The end effector for harvesting aquatic vegetables needs to operate underwater, grasping leaves and cutting stems. To complete this process as efficiently and accurately as possible, it is necessary to design a dedicated Brasenia schreberi-harvesting end effector.

The mechanical arm harvesting system based on machine vision Is the core of the intelligent harvesting system, which consists of two main components: the visual processing module and the mechanical arm control module. The primary function of the visual processing module is to capture real-time information about the appearance, position, and size of fruits or vegetables through image recognition and processing techniques, thereby providing essential data support for the operation of the mechanical arm. Numerous scholars have conducted in-depth research on this aspect [20]. Cao et al. [21] investigated underwater crab image acquisition and recognition to meet the monitoring needs of crabs. They employed a joint detection and instance segmentation network (JDSNet) to achieve the monitoring, recognition, and segmentation of underwater crabs. Ma et al. [22] proposed an improved YOLOv8 model for lotus seedpod segmentation, aiming to detect the maturity of lotus seeds and to estimate the picking pose. Zhan et al. [23] proposed a method for fruit recognition using a Deep Convolutional Neural Network (DCNN) that can quickly and accurately identify different types of ripe fruits with an average recognition accuracy of 94.16%. Lin et al. [24] utilized a Fully Convolutional Network (FCN) model to detect and segment pomegranate fruits, achieving an average accuracy of 89.3%. Singh et al. [25] utilized an end-to-end Fully Convolutional Neural Network to identify cotton, achieving an average precision of 95.16% with a fewer number of parameters. In terms of mechanical arm control planning, researchers continue to explore control methods in complex unstructured environments [26]. Kang et al. [27] proposed a novel path-planning algorithm (EDDS-bi-RRT) based on the RRT algorithm, adjusting the expansion direction and dynamic step size, and enabling the mechanical arm to achieve efficient path planning in various environments. Zhang et al. [28] introduced an HDRRT-Connect method for the motion planning of mango-harvesting robots, incorporating an adaptive target attraction strategy and achieving a harvesting success rate of 90%. In practical harvesting scenarios, both components are essential. It is crucial to reasonably construct a software and hardware platform that integrates target recognition algorithms and mechanical arm control algorithms with a depth camera and mechanical arm to achieve the automated harvesting of Brasenia schreberi.

Currently, research on harvesting systems for aquatic vegetables remains limited. This paper aims to design a feasible intelligent Brasenia schreberi-harvesting system to address the challenge of accurately and efficiently harvesting Brasenia schreberi in aquatic environments. As harvesting Brasenia schreberi buds in water is very challenging, we focus on harvesting Brasenia schreberi leaves on the water’s surface in this paper. To the best of our knowledge, this is the first attempt to develop automated machinery for Brasenia schreberi harvesting.

The main contributions of this paper are as follows:

• Designing an appropriate mobile platform to support the harvesting of Brasenia schreberi;
• Designing an end effector capable of underwater operation to perform the tasks of clamping and cutting Brasenia schreberi;
• Integrating target recognition algorithms and mechanical arm path-planning algorithms with a depth camera and mechanical arm to achieve efficient harvesting control.

The rest of this paper is structured as follows: Section 2 introduces the materials and methods adopted in this paper, including the design of major parts of the intelligent harvesting system and the construction of the harvesting manipulator control system. Section 3 introduces the experimental results and compares our method with the existing methods. Section 4 summarizes this paper and outlines the direction for future research.

2. Materials and Methods

2.1. An Overview of the Intelligent Harvesting System for Aquatic Vegetables

The intelligent Brasenia schreberi-harvesting system described in this study adopts a boat-like structure, as illustrated in Figure 2. The main structure consists of a dual-hull mobile platform, a mechanical arm harvesting system, and an end effector, with the mechanical designs based on SolidWorks 2023 software (Dassault Systèmes, Vélizy-Villacoublay, France). The mobile platform is 3.2 m in length and 1.6 m in width. This design ensures its stability, adaptability, and load-bearing capacity during navigation in ponds. The mobile platform is driven by two FDK060430C2 type servo motors, each with a rated power of 0.4 kW, resulting in a total power of 0.8 kW for the two motors. The rated speed of the motors is 3000 rpm, and through the action of a reducer, the maximum driving speed of the output shaft is adjusted to 100 rpm to meet the requirements for stable movement. To reduce the load and to ensure endurance, a 48 V 20 Ah lithium battery is used to power the entire system. The mechanical arm harvesting system consists of the mechanical arm FR5 (FAIRINO, Suzhou, China), a controller minipc-D3 (TexHoo, Guangzhou, China), and a depth camera D435 (Intel Corp, Santa Clara, CA, USA). The harvesting system integrates target recognition and localization algorithms with mechanical arm planning ones to achieve precise and efficient control. The end effector, designed autonomously according to the characteristics of aquatic vegetables, employs a lateral stem-clamping and cutting method to synchronize the clamping and cutting operations during harvesting, thereby simplifying the harvesting process.

2.2. Design of the Mobile Platform

2.2.1. Hull Type Selection

The hull design of the boat can be classified into two types: monohulls and multihulls [29]. Multihulls consist of two or more independent hulls connected by a rigid structure above the waterline, positioned at specific distances apart. Compared to monohulls, multihulls have a lower center of eccentricity, enabling them to maintain balance more easily. This feature is particularly beneficial in the management and harvesting operations of Brasenia schreberi, where multihulls offer significantly improved stability in both lateral and longitudinal directions compared to monohulls. Furthermore, catamarans (a type of multihull) exhibit greater adaptability in complex water conditions, demonstrating an excellent motion response performance even when fully loaded [30]. The mobile platform employs a dual-hull structure, with the bow hulls shaped in a conical form, which reduces resistance when moving in water, thereby minimizing energy consumption [31]. Considering the requirements for shallow draft and stability, this paper selects a catamaran design to adapt to the operational requirements of Brasenia schreberi ponds.

2.2.2. Hull Structure Design

To facilitate the efficient transfer of the harvesting system between different ponds, this paper adopts a three-segment structure for the frame, connected by pins and reinforced with stainless steel U-bolts, as shown in Figure 3. This structural design not only facilitates the rapid assembly and disassembly of the boat but also ensures its overall structural stability. Additionally, the design of the control cabinet fully considers the functionality, safety, reliability, and maintenance convenience of the electrical system. Appropriate electrical components are selected based on specific control requirements, and the wiring layout inside the control cabinet is meticulously designed to ensure the correctness, stability, and maintenance convenience of the electrical connections. Furthermore, the flat design of the hull surface facilitates the placement of necessary operational equipment.

2.2.3. Driving System Design

Brasenia schreberi generally grows in shallow ponds with a water depth of 30~50 cm. Compared to propeller drive, paddle wheel propulsion features a simpler structure design. This results in less friction between components and reduced energy dissipation, making it particularly advantageous for operation in shallow ponds. A simpler structure minimizes failure points, easing maintenance [32]. Additionally, the paddle wheel drive minimizes disturbance to the pond bottom, generating more uniform water flow, which helps maintain water quality. With fewer operational limitations in shallow waters, paddle wheel systems enhance the maneuverability of boats in confined areas. Therefore, paddle wheel drive is more suitable for shallow pond environments, making it a more environmentally friendly choice.

(1)

Synchronous belt design

The paddle wheel drive system consists of two identical units, with each comprising a drive motor, a reducer, a transmission device, a paddle wheel, and other connecting components. Since the motor cannot operate directly underwater, a synchronous belt transmission device is designed to keep the motor above the water surface while transmitting power to the paddle blades to achieve rotational motion. The synchronous belt transmission relies on the meshing between the teeth of the synchronous belt and the synchronous belt pulley, and achieves synchronous circumferential speed without relative sliding. When the synchronous belt pulley operates, there is a slight lateral thrust. To prevent the belt from slipping in the two-shaft transmission, one of the two belt pulleys must be equipped with a stop block. The designed structure is shown in Figure 4. Electric batteries are used as energy power to enhance the performance of the harvesting system and to reduce pollution.

In the driving system of the mobile platform, the synchronous belt driven by the servo motor can maintain high accuracy in the long-distance transmission process and has a low cost [33]. The design steps of the synchronous belt are as follows:

Step 1: calculate the design power $P_d$.

$P_d$ is calculated according to Equation (1).

$$P_d=K{P}_m$$

(1)

where $K$ is the working condition coefficient and $K=1.5$ can be obtained by referring to

Table 1. Coefficient table for working machine operation.

Load Characteristics	Intermittent Use 3~5 h per Day	General Use 8~10 h per Day	Continuous Use 16~24 h per Day
Stable Load	1.0	1.2	1.4
Slightly Variable Load	1.2	1.4	1.6
Highly Variable Load	1.3	1.5	1.7

, according to the performance and operating hours of the working machine.

Step 2: determine the model and pitch of the belt.

Substituting the data $P_m$ = 0.4 kW, the design power $P_d$ = 0.6 kW is obtained, and the belt model is determined as S5M. According to

Table 2. Pitch selection table.

Belt Code	Model	Pitch (mm)	Tooth Height (mm)	Belt Thickness (mm)	Root Thickness (mm)	Fillet Radius at Root	Fillet Radius at Tip
STPD/STS	S2M	2	0.76	1.36	1.30	0.20	0.20
	S3M	3	1.14	2.20	1.95	0.30	0.30
	S4.5M	4.5	1.71	2.81	2.93	0.45	0.45
	S5M	5	1.91	3.40	3.25	0.50	0.50
	S8M	8	3.05	5.30	5.20	0.80	0.80
	S14M	14	5.30	10.20	9.10	1.40	1.40

, the corresponding pitch $P_b$ = 5 mm.

Step 3: select the number of belt teeth.

According to the working requirements and trial calculation, the number of small belt teeth $Z_1=24$ and the number of large belt teeth $Z_2=40$ are selected.

Determine the pitch diameter of the pulley according to Equation (2) and Equation (3).

$$d_p={\displaystyle \frac{P_b{Z}_1}{\pi }}$$

(2)

$$D_p={\displaystyle \frac{P_b{Z}_2}{\pi }}$$

(3)

Substitute the data to get $d_p$ = 38.2 mm and $D_p$ = 63.66 mm.

Step 4: determine the theoretical pitch line length $L_p^{\prime }$ of the synchronous belt.

$$L_p^{\prime }=2a+{\displaystyle \frac{\pi \left(d_p+D_p\right)}{2}}+{\displaystyle \frac{{\left(D_p-d_p\right)}^2}{4a}}$$

(4)

In Equation (4), $a$ is the tentative shaft spacing, $d_p$ is the diameter of the small pulley, $D_p$ is the diameter of the large pulley, and $a$ is 220 mm; then, $L_p^{\prime }$ = 600.74 mm, and the standard pitch length closest to the calculated value $L_P$ = 600 mm is selected.

Step 5: calculate the transmission center distance C.

C is calculated according to Equation (5).

$$C={\displaystyle \frac{b+\sqrt[2]{b^2-8{\left(D_p-d_p\right)}^2}}{8}}$$

(5)

where $b=2L_p-\pi \left(D_p+d_p\right)$.

Step 6: determine the belt wide band $B_{\mathrm{W}}$.

$B_{\mathrm{W}}$ is determined by Equation (6).

$${B_w}^{\prime }={\displaystyle \frac{P_d}{P_s·K_m}}\times W_P$$

(6)

where $P_s$ is the reference transmission capacity, $K_m$ is the meshing compensation coefficient, and $W_P$ is the reference belt width.

Select the belt width $B_w$ which is closest to ${B_w}^{\prime }$ according to the principle of taking the larger one rather than the smaller one.

Finally, determine the parameters of the synchronous belt: S5M type, $P_b$ = 5 mm, $L_P$ = 600 mm, $B_w$ = 25 mm, Pulley: z1 = 24, $d_p$ = 38.2 mm, z2 = 40, $D_p$ = 63.66 mm; the transmission center distance C is 200 mm.

(2)

Structure design of paddle wheel

To meet the power demands of the boat, the structure of the paddle wheel needs to be reasonably designed. The paddle wheel types can be divided into a movable web one and a fixed web one. The primary distinction between the movable web type and the fixed one lies in whether the web plate (i.e., “paddle” on the wheel) is capable of movement.

The angle of the web plate in the movable web type can be adjusted on the axle. This design enables the web plate to alter its angle when entering or exiting the water surface, thereby reducing resistance and enhancing efficiency. The web plate of the fixed web type is secured to the axle and remains stationary. This design is simple and stable, with relatively low maintenance costs [34]. The fixed web type paddle wheel offers advantages such as high reliability, simple maintenance and good cost-effectiveness, making it the preferred choice for our boat. For the long-term underwater operation of the boat, polymethylmethacrylate with acid and alkali resistance and strong impact resistance is selected as the blade material.

As shown in Figure 4, the structural design of the paddle wheel follows the even layout of 8 paddle wheel blades, with the stiffener designed accordingly. This structure makes the paddle wheel drive more reliable.

The thrust of the paddle wheel is calculated according to Equations (7) and (8).

$$F_e=C_F{S}_F\rho n^2{D}^2$$

(7)

$$S_F=2bh$$

(8)

In Equation (7), $F_e$ represents the thrust of a single paddle wheel; $C_F$ is the coefficient of the propulsive force, which is taken as 3 in this paper; $n=1 \mathrm{r}/\mathrm{s}$ is the working speed of the paddle wheel; $\mathrm{h}$ is the distance from the bottom edge of the wheel to the waterline; $\mathrm{b}=0.225 \mathrm{m}$ is the width of the paddle wheel; $\rho =1000 \mathrm{kg}/{\mathrm{m}}^3$ is the water density; $D=0.6 \mathrm{m}$ is the diameter of the wheel; and $S_F$ is the hydraulic action area of the paddle wheel.

When fully loaded, the distance from the bottom edge of the paddle wheel blades to the waterline is 0.15 m; thus, $S_F$ is 0.0675 ${\mathrm{m}}^2$. Substituting the above data into Equation (7), the total thrust of the boat in an equilibrium state is 72.9 N.

The maximum speed we designed is 1.0 m/s, which was measured to be 1.1 m/s in practice.

2.2.4. Anti-Entanglement Device

A meshed anti-entanglement device is designed to avoid damaging the Brasenia schreberi stems and leaves, as shown in Figure 5.

2.3. Design of End Effector

Designing an end effector with an effective harvesting performance requires a detailed analysis of the characteristics of the harvesting object and the harvesting method [35]. Brasenia schreberi leaves typically float on the water surface, and the harvesting operation must be conducted at the corresponding underwater position, which requires the end effector to be capable of partial underwater operation [36]. The body of Brasenia schreberi is covered with a gelatinous substance, presenting a slippery characteristic [14], and measures must be taken during harvesting to prevent the leaves from slipping or being damaged. The main part harvested is the stem of Brasenia schreberi, and operations targeting this part must be swift and precise.

The end effector directly contacts the Brasenia schreberi, and its mechanical structure and harvesting method directly influence the quality of harvesting. Based on the harvesting requirements analyzed above, two design schemes for the harvesting method of the end effector are compared as follows.

(1)

Top-down enveloping harvesting; the principle is shown in Figure 6a. The harvesting end effector first positions itself above the target Brasenia schreberi, with all four fingers fully extended, then moves downward until it completely envelops the Brasenia schreberi leaf. After that, the fingers close, cutting the leaf off and completing the harvesting.

(2)

Lateral stem clamping and cutting harvesting; the principle is shown in Figure 6b. The harvesting end effector enters from the side of the short diameter of the Brasenia schreberi leaf, with the leaf positioned between the upper and lower fingers and the stem located at the stem placement area. After entering laterally to about 2/3 of the short diameter length, the lower finger joint of the end effector moves upward to close against the upper finger, lifting the leaf out of the water and cutting it off, thus completing the harvesting.

Comparing the two harvesting methods, it is evident that top-down enveloping harvesting is suitable for situations where the Brasenia schreberi leaves are sparsely distributed, the outline of individual leaves is quite clear, and there is no overlap between leaves [6]. Moreover, completely enveloping the leaves results in an oversized end effector. In contrast, the lateral stem clamping and cutting method has stronger adaptability. If the short diameter direction of the leaf can be roughly determined, the leaf can be detached from the complex underwater environment for harvesting. Therefore, this paper adopts the lateral stem clamping and cutting method for the design of the end effector.

The end effector designed in this paper consists of three parts: the motion generation mechanism, the cutting mechanism, and the clamping mechanism; this is shown in Figure 7. The steering gear driving the actuator is installed in the sealed box at the middle and upper position of the end effector. The lower part of the end effector consists of the clamping and cutting mechanism, which are moved by the force transmitted by the disc transmission mechanism. As there are no electrical components in the lower part, the end effector can function partially in water. According to the principle of the lateral stem clamping and cutting method described above, the end effector needs to perform two actions to complete the harvesting task: the clamping of the Brasenia schreberi leaf and the cutting of the Brasenia schreberi stem. These two actions have a sequential order and cannot be performed simultaneously.

The dimensions of the parts in the end effector need to be designed based on the dimensions of the Brasenia schreberi leaves.

At the Yingtan Longhushan Brasenia schreberi experimental base, 10 leaves were randomly selected for leaf size measurement. The measurement examples are shown in Figure 8. The front of Brasenia schreberi leaves is usually green, while the back is dark red. The results show that the long diameter range is 69.76~88.36 mm, with an average of 76.280 mm and a mean square deviation of 6.320 mm. The short diameter range is 43.10~55.42 mm, with an average of 49.056 mm and a mean square deviation of 4.134 mm. The stem of Brasenia schreberi is relatively small, ranging from 0.80 to 2.40 mm.

As shown in Figure 9, the blue outline represents the clamping follower, the red outline represents the cutting follower, the green outline represents the retaining pin, and the black line outline represents the driving disc. The design of the motion generation mechanism focuses on the drive disc, which facilitates the clamping of Brasenia schreberi through the actions of the clamping and cutting followers. According to the harvesting requirements, the clamping range is determined to be 30 mm, the diameter of the drive disc (Φ2) is 55 mm, and the diameter of the retaining pin (Φ3) is 5 mm. By applying the Pythagorean theorem, the distance d3 between the two endpoints of the arc segment of the locating slot Is calculated to be 40 mm, and the center arc diameter (Φ1) of the locating slot arc segment is 50 mm.

As shown in Figure 10, the cutting mechanism employs a Y-shaped connection, with the cutting follower serving as the driving component, and the motion stroke P_x is 20 mm. To meet assembly requirements and prevent interference, the length of the connecting rod $l_1$ is determined to be 36 mm, and the distance $l_3$ from the connection point of the connecting rod and the cutting follower to the connection point with the slider is 12 mm. The distance $l_2$ between the cutting blades is calculated to be 67.88 mm based on the structural dimensions and installation positions of the slider and the cutting blades.

The clamping mechanism consists of upper and lower pinch fingers, as shown in Figure 11a. The structural dimensions of the clamping mechanism are shown in Figure 11b. Successful clamping can be achieved when the finger extension length a is greater than two-thirds of the minor axis length of the leaf. As the short diameter of Brasenia schreberi is generally less than 60 mm, we select a = 40 mm, b = 85 mm, and c = 12 mm. To improve the clamping effect, sponge gaskets are added to the clamping fingers.

2.4. Machine Vision-Based Robotic Arm Harvesting System

To successfully implement autonomous harvesting operations, the various modules of the mechanical arm harvesting system must be highly integrated [37]. The designed mechanical arm harvesting system, as shown in Figure 12, primarily consists of three modules: perception, decision-making, and execution. The operation of the perception module is divided into two paths. One path captures video frames from the environment through the RGB imaging device of the depth camera and transmits them to the target recognition algorithm to identify Brasenia schreberi. The other path, utilizing the depth camera and the principle of three-dimensional imaging, completes the spatial positioning of the Brasenia schreberi target. The decision-making module is responsible for evaluating and processing the overall state of the system, including judging whether the harvesting target obtained by the perception module is within the robot’s working range, checking the status of the end effector, and pre-planning the path for the mechanical arm. When the decision-making module determines that the harvesting conditions are met, the system enters the operation execution phase. At this point, the mobile platform remains stationary, the end of the mechanical arm moves to the target point, and the end effector completes the clamping and cutting operations.

2.4.1. Perception Module

Depth cameras represent the current mainstream sensing solution for three-dimensional machine vision, offering a cost-effective alternative. The selected image acquisition device is the D435 depth camera, which not only captures RGB information from the environment but also employs the Time-of-Flight (ToF) principle to obtain depth information for each pixel. This camera outputs RGB images at 1920 × 1080 pixels and depth images at 640 × 480 pixels, meeting the system’s requirements for image acquisition accuracy.

Compared to traditional methods, deep learning algorithms can adaptively learn features from images at a deeper level, demonstrating superior generalization capabilities when handling complex scenes. In fact, in the field of target recognition and localization, numerous effective deep learning methods have been proposed; these can be divided into two-stage and one-stage algorithms based on different implementation steps. The representative of two-stage algorithms is the RCNN algorithm proposed by Girshick et al. [38], which, as one of the earlier target detection methods, generates region proposals in the first stage and then performs classification and recognition in the second stage. The advantage of this method is its high detection accuracy. The emerging one-stage algorithms are primarily represented by YOLO and SSD [39,40], which use convolutional neural networks to simultaneously obtain region proposals and target confidence levels, focusing on detection efficiency [41]. However, to date, the YOLO series algorithms have undergone several generations of optimization, achieving a balance between accuracy and efficiency. Therefore, we select the one-stage YOLO algorithm based on convolutional neural networks [42] as the target recognition algorithm for this study.

To deploy the algorithm on the main controller of the harvesting platform with limited computational power (Intel Core i5-8300H processor (Intel Corp, Santa Clara, CA, USA), 16 GB RAM and Ubuntu 18.04 OS) [43], the network structure design was improved based on the YOLOv5s, which has a smaller model size, as shown in Figure 13. This design aims to enhance the model’s operational efficiency while maintaining good detection performance.

The YOLO-GS network, which is the recognition algorithm used in our previous research [44], has undergone several key improvements based on the YOLOv5s network. A lightweight CSP cross-stage module C3-GS, integrated with a 3D attention mechanism, was designed to improve the model’s capability to extract and fuse the feature information of Brasenia schreberi in complex environments. The original CBS module was replaced with Ghost Conv [45] to decrease the number of model parameters and computational load, thereby reducing the computational cost. An additional 160 × 160 detection head was introduced to improve the recognition of multi-scale Brasenia schreberi targets. Finally, the Focal-EIoU loss function [46] was incorporated to optimize the target detection performance.

After recognition is completed, the two-dimensional coordinates of the target are mapped to the depth map to achieve the feature fusion of RGB and depth information. The generated target detection boxes are sorted according to the depth distance of the center point. Next, the Brasenia schreberi image surrounded by the target box is cropped from the image in sequence for processing. After applying Gaussian blur and Otsu thresholding, the contour is extracted and the ellipse is fitted. Finally, the short axis of the ellipse is calculated. The overall process is shown in Figure 14.

The next step is to map the short axis endpoints to the depth map, to obtain the three-dimensional coordinates of the short axis endpoints, and to calculate the direction vector in the camera coordinate system, using the point farther away from the X-axis as the starting point and the other point as the endpoint. Combined with the three-dimensional coordinates of the center point of the target box, the picking point coordinates containing attitude information are finally calculated.

2.4.2. Decision-Making Module

The decision-making module primarily accomplishes three functions: it performs coordinate transformation of the harvesting points, workspace judgment of the harvesting points, and the monitoring of the end effector’s status.

The prerequisite for implementing these functions is to establish the transformation relationship between the mechanical arm and the camera coordinate systems. The mechanical arm used in this paper is the 6-DOF FR5 mechanical arm introduced by FAOYIWEI Company. This series of mechanical arms is currently widely used in material handling and target-grasping operations. It provides APIs based on ROS (Robot Operating System) and can communicate and feedback with the ROS host system through protocols to control the motion of the mechanical arm in real time. Figure 15 illustrates the basic parameters and workspace of the FR5 mechanical arm.

On this basis, it is necessary to calculate the transformation relationships between the coordinate systems of each joint of the mechanical arm, which allows the integration of the arm’s motion with subsequent path planning, achieving pose description within the same coordinate system. The mechanical arm can be considered as a multi-link serial mechanism. Based on the structure of a 6-DOF mechanical arm, a D-H coordinate system is established, as shown in Figure 16. In the standard DH parameter method [47], the kinematic parameters of the mechanical arm links are described through the link length $a_i$, link twist angle ${\alpha }_i$, link offset $d_i$, and joint angle ${\theta }_i$.

The D-H parameters of the mechanical arm are shown in

Table 3. Robotic arm linkage parameters.

Linkage	${\mathit{\theta }}_{\mathit{i}}$	${\mathit{d}}_{\mathit{i}}$ (mm)	${\mathit{a}}_{\mathit{i}}$ (mm)	${\mathit{\alpha }}_{\mathit{i}}$
1	${\theta }_1$	152	0	π/2
2	${\theta }_2$	0	−425	0
3	${\theta }_3$	0	−395	0
4	${\theta }_4$	102	0	π/2
5	${\theta }_5$	102	0	−π/2
6	${\theta }_6$	100	0	0

.

According to the chain rule of coordinate system transformation, the transformation matrix from any two adjacent joint coordinate systems {i − 1} to {i} can be expressed as follows:

$${}_{i }{}^{i-1}T=\left[\begin{array}{cccc}\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_i& -\mathrm{s}\mathrm{i}\mathrm{n}{\theta }_i\mathrm{c}\mathrm{o}\mathrm{s}{\alpha }_i& \mathrm{s}\mathrm{i}\mathrm{n}{\theta }_i\mathrm{s}\mathrm{i}\mathrm{n}{\alpha }_i& a_i\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_i\\ \mathrm{s}\mathrm{i}\mathrm{n}{\theta }_i& \mathrm{c}\mathrm{o}\mathrm{s}{\theta }_i\mathrm{c}\mathrm{o}\mathrm{s}{\alpha }_i& -\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_i\mathrm{s}\mathrm{i}\mathrm{n}{\alpha }_i& a_i\mathrm{s}\mathrm{i}\mathrm{n}{\theta }_i\\ 0& \mathrm{s}\mathrm{i}\mathrm{n}{\alpha }_i& \mathrm{c}\mathrm{o}\mathrm{s}{\alpha }_i& d_i\\ 0& 0& 0& 1\end{array}\right]$$

(9)

Substitute the D-H parameters in Table 3 into Equation (9) to obtain the homogeneous transformation matrix between the links of the harvesting robotic arm. The transformation matrix is shown in Equations (10)–(15):

$${}_1{}^{0}T=\left[\begin{array}{cccc}\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_1& 0& \mathrm{s}\mathrm{i}\mathrm{n}{\theta }_1& 0\\ \mathrm{s}\mathrm{i}\mathrm{n}{\theta }_1& 0& -\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_1& 0\\ 0& 1& 0& 152\\ 0& 0& 0& 1\end{array}\right]$$

(10)

$${}_2{}^{1}T=\left[\begin{array}{cccc}\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_2& -\mathrm{s}\mathrm{i}\mathrm{n}{\theta }_2& 0& -425\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_2\\ \mathrm{s}\mathrm{i}\mathrm{n}{\theta }_2& \mathrm{c}\mathrm{o}\mathrm{s}{\theta }_2& 0& -425\mathrm{s}\mathrm{i}\mathrm{n}{\theta }_2\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]$$

(11)

$${}_3{}^{2}T=\left[\begin{array}{cccc}\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_3& -\mathrm{s}\mathrm{i}\mathrm{n}{\theta }_3& 0& -395\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_3\\ \mathrm{s}\mathrm{i}\mathrm{n}{\theta }_3& \mathrm{c}\mathrm{o}\mathrm{s}{\theta }_3& 0& -395\mathrm{s}\mathrm{i}\mathrm{n}{\theta }_3\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]$$

(12)

$${}_4{}^{3}T=\left[\begin{array}{cccc}\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_4& 0& \mathrm{s}\mathrm{i}\mathrm{n}{\theta }_4& 0\\ \mathrm{s}\mathrm{i}\mathrm{n}{\theta }_4& 0& -\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_4& 0\\ 0& 1& 0& 102\\ 0& 0& 0& 1\end{array}\right]$$

(13)

$${}_5{}^{4}T=\left[\begin{array}{cccc}\mathrm{c}\mathrm{o}\mathrm{s}{\theta }_5& 0& -\mathrm{s}\mathrm{i}\mathrm{n}{\theta }_5& 0\\ \mathrm{s}\mathrm{i}\mathrm{n}{\theta }_5& 0& \mathrm{c}\mathrm{o}\mathrm{s}{\theta }_5& 0\\ 0& -1& 0& 102\\ 0& 0& 0& 1\end{array}\right]$$

(14)

$${}_6{}^{5}T=\left[\begin{array}{cccc}cos{\theta }_6& -sin{\theta }_6& 0& 0\\ sin{\theta }_6& cos{\theta }_6& 0& 0\\ 0& 0& 1& 100\\ 0& 0& 0& 1\end{array}\right]$$

(15)

Multiply the transformation matrices of each link ${}_{i }{}^{i-1}T$ $(i=1,2,3\dots ,n)$ to obtain the kinematic equation of the robotic arm:

$$\left.{}_6{}^{0}T={}_1{}^{0}T{}_2{}^{1}T{}_3{}^{2}T{}_4{}^{3}T{}_3{}^{4}T{}_6{}^{5}T=\left[\begin{array}{cc}R& P\\ 0& I\end{array}\right.\right]=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& p_x\\ r_{21}& r_{22}& r_{23}& p_r\\ r_{31}& r_{32}& r_{33}& p_z\\ 0& 0& 0& 1\end{array}\right]$$

(16)

with:

$$\left\{\begin{array}{l}{r}_{11}=c_6\left(c_1{c}_{234}{c}_5+s_1{s}_5\right)-c_1{s}_{234}{s}_6;\\ r_{21}=c_6\left(s_1{c}_{234}{c}_5-c_1{s}_5\right)-s_1{s}_{234}{s}_6;\\ r_{31}=s_{234}{c}_6{c}_3+c_{234}{s}_6;\\ r_{12}=-s_6\left(c_1{c}_{234}{c}_5+s_1{s}_5\right)-c_1{s}_{234}{c}_6;\\ r_{22}=-s_6\left(s_1{c}_{234}{c}_5-c_1{s}_5\right)-s_1{s}_{234}{c}_6;\\ r_{32}=-s_{234}{s}_6{c}_5+c_{234}{c}_6;\\ r_{13}=-c_1{c}_{234}{s}_5+s_1{c}_5;\\ r_{23}=-s_1{c}_{234}{s}_5-c_1{c}_5;\\ r_{33}=-s_{234}{s}_5;\\ p_x=100\left(-c_1{c}_{234}{s}_5+s_1{c}_5\right)+102c_1{s}_{234}-395c_1{c}_{23}+102s_1-425c_1{c}_2;\\ p_y=100\left(-s_1{c}_{234}{s}_5-c_1{c}_5\right)+102s_1{s}_{234}-395s_1{c}_{23}-102c_1-425s_1{c}_2;\\ p_z=-100s_{234}{s}_5-102c_{234}-425s_2-395s_{23}+152;\end{array}\right.$$

where:

$$s_i=sin\left({\theta }_i\right),$$

$$c_i=cos\left({\theta }_i\right),$$

$$s_{ij}=s\left({\theta }_i+{\theta }_j\right),$$

$$c_{ij}=c\left({\theta }_i+{\theta }_j\right),$$

$$i=1,2,\dots ,6,$$

$$j=1,2,\dots ,6.$$

Equation (16) describes the pose of the end effector of the mechanical arm in the base coordinate system of the mechanical arm. In the subsequent hand–eye calibration process, it is only necessary to establish the coordinate relationship between the end effector and the camera. This allows the use of the end effector coordinates as an intermediate variable to unify the camera coordinate system with the mechanical arm’s base coordinate system.

In the realm of robotic vision, hand–eye calibration is predicated on the foundational principles of coordinate transformation. Its primary function is to ascertain the relationship between the base coordinate system of the robotic arm and that of the camera, thereby integrating the image coordinates identified by the camera into the base coordinate system of the robotic arm. Depending on the camera’s position during the calibration process, hand–eye calibration methodologies are categorized into two distinct types: when the camera is rigidly affixed to the end effector of the robotic arm, the configuration is referred to as “eye-in-hand”; conversely, when the camera is stationary in a position external to the robotic arm, it is termed “eye-to-hand” [48].

During the process of image acquisition, to circumvent the inaccuracies that arise from the necessity of repeatedly repositioning a calibration board mounted on a tripod, the “eye-to-hand” calibration approach is employed. As depicted in Figure 17, the camera is secured at an optimal external location relative to the robotic arm, while the calibration board is affixed to the arm’s terminus. Throughout the robotic arm’s motion, the positional and orientational relationships between the end effector and the calibration board are maintained invariant.

During the calibration configuration process, the camera captures the mechanical arm’s end effector moving to two arbitrary poses with the calibration board attached, leading to the following equation:

$${}_{End}{}^{Base}T_1 \times {}_{Base }{}^{Camera}T_1 \times {}_{Camera}{}^{Object }T_1 = {}_{End}{}^{Base}T_2 \times {}_{Base }{}^{Camera}T_2 \times {}_{Camera}{}^{Object }T_2 = {}_{End }{}^{Object}T$$

(17)

After matrix transformation, the equation can be simplified to the following:

$${}_{End}{}^{Base}T_2^{-1} \times {}_{End}{}^{Base}T_1 \times {}_{Base }{}^{Camera}T_1 = {}_{Base }{}^{Camera}T_2 \times {}_{Camera}{}^{Object }T_2 \times {}_{Camera}{}^{Object }T_1^{-1}$$

(18)

This problem can be transformed into solving the homogeneous equation AX = XB. In the transformation formula above, ${}_{End}{}^{Base}T$ represents the transformation relationship between the mechanical arm’s base coordinate system and the end effector coordinate system, which can be obtained from the previously calculated coordinate relationships; ${}_{End }{}^{Object}T$ represents the transformation relationship between the recognized object (calibration board) and the end effector, which is easily determined since they are rigidly connected; ${}_{Camera}{}^{Object }T$ represents the transformation relationship between the calibration board and the camera, obtained through the hand–eye calibration process; the unknown part of the equation is ${}_{Base }{}^{Camera}T$, the transformation relationship between the camera coordinate system and the mechanical arm’s base coordinate system, which can thus be determined.

When the coordinates of the harvesting points in the camera coordinate system, transformed by the hand–eye matrix into the mechanical arm’s base coordinate system, do not exceed the mechanical arm’s limit space, the harvesting points obtained by the perception module are deemed to meet the harvesting criteria.

The state information of the end effector mainly includes its position in the mechanical arm’s base coordinate system and its open/closed status. The former is obtained through the previously calculated kinematic equations of the mechanical arm, while the latter is determined by reading the angle of the servo motor.

2.4.3. Execution Module

The key point of the execution module lies in the harvesting planning of the mechanical arm. When planning the harvesting of Brasenia schreberi, several special situations must be considered.

Firstly, since the harvesting environment is underwater, it is essential to avoid the complete submersion of the harvesting device to reduce costs and technical difficulties. On one hand, as mentioned earlier, the motor of the end effector is positioned in the upper half. On the other hand, a feasible operating area is pre-planned to ensure that the clamping and cutting mechanism can operate underwater and accurately reach the Brasenia schreberi on the surface, while preventing electronic components from being submerged. This approach can reduce the pressure on waterproof design and enhance the stability and service life of the device.

Secondly, in practical testing, due to the slippery nature of Brasenia schreberi leaves, changes in the posture of the end effector can easily cause the leaves to slip, affecting the success rate of the harvesting task. Therefore, during harvesting, it is essential to adjust the end effector’s posture promptly according to the short axis direction obtained by the perception module, ensuring that the Brasenia schreberi leaves are fully accommodated within the operating range of the end effector. During the process of harvesting and moving to the storage area, it is crucial to maintain a specific posture to clamp the leaves, preventing slips and ensuring the overall success rate of the process.

Thirdly, the harvesting system needs to be equipped with perception and positioning devices such as radar or GPS modules to assist in harvesting. Due to the limited space on the boat, some devices inevitably fall within the operating range of the mechanical arm. Any collision could not only damage the devices and disrupt the harvesting process but also pose safety risks. Therefore, when planning the mechanical arm’s movement, it is essential to establish obstacle models of the existing devices to prevent collisions.

Lastly, the planning of the harvesting path must not overlook the requirement for continuous harvesting. Brasenia schreberi often grows in patches, and a well-planned harvesting path can significantly improve efficiency, reduce redundant movements in the mechanical arm, and ensure a smooth transition from one area to another. This enables the mechanical arm to quickly and continuously perform multiple harvesting actions, meeting the demands of large-scale operations while ensuring the quality of the Brasenia schreberi and achieving efficient harvesting.

In the field of mechanical arm path planning control, the RRT algorithm proposed by LaValle [49] is well-known. The core idea of this algorithm is to generate a rapidly exploring random tree that explores a collision-free feasible path through continuous random sampling. The steps are as follows:

(1)

Set the start point of the random tree as q_init.

(2)

Generate a random sample point q_rand in the map.

(3)

Find the leaf node q_near on the random tree that is closest to q_rand.

(4)

Extend the fixed step size u from q_near towards q_rand to generate a temporary node q_temp.

(5)

Perform collision detection on q_temp. If a collision with an obstacle occurs, discard the node q_rand and repeat step (2). Otherwise, retain q_temp as a new node q_new and add it to the random tree.

(6)

Determine if q_new has reached the goal point q_goal or satisfies the constraints. If so, terminate the algorithm. Otherwise, continue expanding.

(7)

Repeat steps (2) to (6) until the goal point is found or the iteration limit is reached. Select the shortest path from multiple iterations as the final output.

The expansion principle of the rapidly exploring Random Tree (RRT) is illustrated in Figure 18.

To tackle these issues, the manipulator’s planning effectiveness is enhanced via an improved RRT-based method. This optimized approach integrates both fixed and dynamic step sizes. Initially, when far from the target, a large fixed step size is adopted to facilitate rapid expansion. As the target approaches, the algorithm switches to a dynamic step size, adjusting it based on the current position and the distance to the goal point. This hybrid strategy retains the low computational load of the fixed step size while leveraging the advantages of the dynamic step size, such as there being fewer path points and smoother paths near the target. The dynamic step size is calculated using the following formula:

$$\text{step\_size}=max(min\text{\_step\_size},min(\text{base\_step\_size},\text{distance\_to\_goal}\left)\right)$$

(19)

Finally, the random tree is bidirectionally expanded, from both the start and goal points, with a greedy strategy. This guides the tree from the target to expand toward the start point’s $q_{new}$, enhancing the search efficiency and forming an effective picking path-planning method.

2.5. Two-Dimensional Simulation Experiments of the RRT Algorithm

To verify the generality of the algorithm [26], path planning experiments are conducted on a 100 × 100 grid map in a 2D environment using MATLAB 2023b. The start point is the lower left corner (10,10), and the end point is the upper right corner (90,90). The experimental platform is equipped with an Intel Core i7-13790F processor (Intel Corp, Santa Clara, CA, USA) and 16 GB RAM, and runs on Windows 11 OS [50].

This paper first simulates the original algorithm and then successively introduces the enhanced dynamic step size and two-way expansion strategies. To mitigate algorithmic randomness, 100 repeated tests are executed for each case under the specified conditions. The simulation scenario for each case, as displayed in Figure 19, starts with green and ends with red. The black zones signify obstacles, the white zones denote the expandable areas, and the red line indicates the last path generated. The planning success rate is summarized based on the experimental results, including key performance metrics such as the planning time, number of path nodes, and path length.

Simulated experiments are also conducted on other common manipulator path-planning algorithms [49,51,52], and the results are comprehensively compared with the improved algorithm. The simulation scenarios are shown in Figure 20.

2.6. Three-Dimensional Simulation Experiments of RRT Improved Algorithm

Simulation experiments are also conducted in a three-dimensional setting, using a 100 × 100 × 100 3D map for path planning. Spheres in the map are randomly generated to act as scene obstacles. The path-planning process starts from [5, 5, 5] and ends at [95, 95, 95]. Figure 21 shows the simulation scenarios for different algorithms.

2.7. Simulation Experiments of Robotic Arm

To verify the simulation effects of the improved RRT algorithm proposed in this paper when integrated with a real robotic arm, we conducted simulations within the ROS robotic system, utilizing Moveit and its plugin interfaces to achieve obstacle avoidance motion planning for the robotic arm in the three-dimensional space, with the results displayed through Rviz. The simulation framework for robotic arm obstacle avoidance is illustrated in Figure 22, where collision detection is performed using the OBB detection method in the FCL plugin. The KDL motion solver is configured with forward and inverse kinematics equations, and the OMPL motion planner is employed to conduct comparative experiments with both the basic RRT algorithm and the improved RRT algorithm proposed in this paper. The experimental platform is configured with an Intel Core i5-8300H processor (Intel Corp, Santa Clara, CA, USA), 16 GB RAM and Ubuntu 18.04 OS [50].

The simulation scenario for robotic arm obstacle avoidance is illustrated in Figure 23. In this figure, the yellow arm indicates the initial pose of the robotic arm, while the white arm denotes the target pose of the robotic arm. Obstacles in the scene are depicted in various colors, and the simulation is presented in Rviz.

2.8. Localization Function Experiments

Based on the natural growth conditions of Brasenia schreberi, localization function experiments were conducted in the laboratory (as shown in Figure 24). Using measurement tools, the actual position information of the Brasenia schreberi (based on the base coordinate system of the manipulator) was pre-measured. Then, the target recognition algorithm and visual positioning program of the perception module were employed to identify and locate the Brasenia schreberi, obtaining its position information. This position information was subsequently transmitted to the mechanical arm controller through inverse kinematics calculations, enabling the end effector of the mechanical arm to reach the target position. The localization function experiments were carried out by comparing the actual measured position information with that reached by the end of the manipulator.

2.9. Practical Picking Experiments

The actual picking experiments were primarily conducted to validate the end effector and the intelligent Brasenia schreberi-harvesting system proposed in this paper. To comprehensively assess the performance of the designed system, multiple series of picking experiments were meticulously carried out on Brasenia schreberi samples with diverse sizes and postures. The single picking time and average success rate were selected as the key evaluation indexes, as these could accurately reflect the efficiency and reliability of the harvesting system.

The specific process of the picking experiment is shown in Figure 25. First, set an initial position for the mechanical arm, and utilize the depth camera to recognize and determine the coordinates of the Brasenia schreberi in the camera coordinate system. According to the coordinate transformation relationship between the mechanical arm’s base coordinate system and the camera coordinate system obtained through hand–eye calibration, convert the pose of the Brasenia schreberi to the pose in the mechanical arm’s base coordinate system. Then, employ the improved trajectory planning algorithm to plan a series of joint angle values for the mechanical arm to move from the initial position to the harvesting point. Next, transmit the generated joint angles sequentially to the robot control cabinet, which controls the harvesting mechanical arm to rotate to the specified position. Finally, control the end effector to execute clamping and cutting actions. The actions are executed in the following order: The Brasenia schreberi-harvesting mechanical arm swiftly moves from the initial position to the pre-harvesting position, then gradually approaches the target Brasenia schreberi-harvesting position; the end effector performs the harvesting action, and after harvesting, it returns to the storage area.

Ten experiments were conducted on Brasenia schreberi in real harvesting scenarios following the above process, and the size (short diameter), harvesting time, and success of each experiment were recorded.

3. Results and Discussion

3.1. Analysis of Two-Dimensional Simulation Experiment Results

In the two-dimensional environment, the simulation results of the original algorithm and its improved strategy are shown in

Table 4. Ablation study on manipulator path-planning algorithms.

Algorithm	Planning Time/s	Number of Path Nodes	Path Length/cm	Success Rate/%
RRT	3.92	64	189.00	98%
RRT + DynamicStep	1.92	33	162.60	99%
RRT + Bidirectional	1.24	51	169.84	99%
RRT + Bidirectional + DynamicStep	0.77	31	154.08	100%

.

The planning time of the improved algorithm is only 0.77 s, which is approximately 80% shorter than that of the traditional RRT algorithm (3.92 s). This demonstrates that by combining bidirectional expansion and a dynamic step size strategy, the algorithm can explore the search space more efficiently and find the feasible path more quickly. The improved algorithm generates 31 path nodes, the fewest among all algorithms. This not only simplifies the path, but also means that the robot needs to process less node information when executing tasks, thereby reducing the storage and computing costs. From the perspective of path length, its path length is 154.08 cm, which is about 18.4% shorter than that of the RRT algorithm. The shorter path means that the robot can reach the target point faster and reduce energy consumption during movement. Finally, the success rate of the improved algorithm is 100%, which is the highest among all algorithms. This indicates that through the improved strategy, the algorithm exhibits greater reliability and stability in complex environments, consistently finding a feasible path.

In summary, the improved algorithm significantly boosts the efficiency and quality of path planning by integrating bidirectional expansion with a dynamic step size. The former accelerates path discovery by allowing simultaneous searches from both the start and target points, while the latter fine-tunes the step size based on the distance between the current node and target point, further improving the search efficiency. These enhancements ensure that the improved algorithm excels in its planning time, path node count, path length, and success rate, making it ideal for scenarios where the efficiency and quality of path planning are crucial.

The comprehensive comparison results for RRT*, RRT-Connect, the Goal-Biased RRT algorithm and our improved algorithm are shown in

Table 5. Comparison of manipulator path-planning algorithms in 2D environment.

Algorithm	Planning Time/s	Number of Path Nodes	Path Length/cm	Success Rate/%
RRT*	4.88	20	149.02	98%
RRT-Connect	0.90	45	161.86	99%
Goal-Biased RRT	2.92	54	159.00	99%
Improved Algorithm	0.77	31	154.08	100%

.

The improved algorithm’s planning time is merely 15.8% of RRT*’s, 85.6% of RRT-Connect’s, and 26.4% of Goal-Biased RRT’s. This significant reduction in planning time clearly shows that the improved algorithm can find paths much more efficiently than the other three. In terms of the number of path nodes, the improved algorithm also shows its advantage. It generates just 68.9% of the nodes required by RRT-connect and 57.4% of those needed by Goal-Biased RRT. This means that the path planned by the improved algorithm is simpler and more direct. What is more, the improved algorithm’s path length is approximately 4.8% shorter than RRT-Connect’s and 3.1% shorter than Goal-Biased RRT’s. So, the improved RRT algorithm can quickly create a shorter path with fewer nodes, improving the efficiency of path planning.

3.2. Analysis of Three-Dimensional Simulation Experiment Results

A comparison of the simulation results for RRT, RRT-connect, Goal-Biased RRT and the improved algorithm in a three-dimensional environment is shown in

Table 6. Comparison of simulation effect of common algorithms in 3D environment.

Algorithm	Planning Time/s	Number of Path Nodes	Path Length/cm	Success Rate/%
RRT	6.63	68	201.00	96%
RRT-Connect	1.75	59	203.21	100%
Goal-Biased RRT	5.16	69	204.00	99%
Improved Algorithm	0.48	25	182.41	100%

.

The planning time of the improved algorithm is only 0.48 s, which is merely 7.2% of RRT’s, 27.4% of RRT-Connect’s and 9.3% of Goal-Biased RRT’s, demonstrating the efficient processing ability of the improved algorithm. Simultaneously, the number of path nodes in the improved algorithm is only 25, making the path more concise and direct. This is 63.2% less than that of RRT, 57.6% less than that of RRT-Connect, and 63.8% less than that of Goal-Biased RRT. The number of nodes is significantly reduced, reducing the computing requirements. In addition, its path length is 182.41 cm, which is 6.2% shorter than RRT’s, 10.2% shorter than RRT-Connect’s and 10.6% shorter than Goal-Biased RRT’s, contributing to a shorter travel distance that further saves time. In 100 experiments, its planning success rate reached 100%, matching RRT-Connect and exceeding the 96% of RRT and 99% of Goal-Biased RRT, indicating its high reliability.

3.3. Analysis of Simulation Results of Robotic Arm

In the scenario shown in Figure 23, 50 experiments were conducted with RRT and the improved algorithm proposed in this paper. The time and success rate for each planning instance were recorded. The average planning time and path length were calculated, as shown in

Table 7. Comparison of RRT and the improved algorithm.

Algorithm	Planning Time (s)	Path Length (mm)	Success Rate
RRT	0.403	4103	94%
Improved Algorithm	0.192	2935	98%

. Figure 26 illustrates the comparison of a certain running track between the RRT and the improved algorithm.

The improved algorithm demonstrates a significant reduction in planning time and path length, while also achieving a higher success rate compared to the standard RRT algorithm. This indicates that the improvements made to the RRT algorithm enhance its efficiency and effectiveness in obstacle avoidance tasks.

From Figure 26, it can be observed that the path length of the improved algorithm proposed in this paper is shorter than that of the basic RRT algorithm. As shown in Table 7, the improved algorithm reduces the planning time by 52.4% and the path length by 28.5% compared to the basic RRT algorithm, achieving the expected results.

3.4. Analysis of Experimental Results of Localization Function

A comparison of the results of the localization function experiments is shown in

Table 8. Visual localization experiment results.

Serial Number	Actual Measured Position/mm	End Effector Reported Position/mm	Localization Error/mm
1	(516, 168, −80)	(512, 172, −80)	5.66
2	(392, 220, −77)	(395, 221, −74)	4.36
3	(421, 195, −81)	(425, 191, −83)	6.00
4	(559, −156, −87)	(561, −156, −84)	3.61
5	(625, 110, −73)	(628, 109, −70)	4.36

.

As indicated in Table 8, the localization errors of the five datasets were summed up and then averaged. The average localization error of the positioning module of the harvesting system was determined to be 4.80 mm. An analysis of the causes revealed that the visual processing module is the primary contributor to the error. During the visual recognition process, there is a certain deviation between the identified center point and the actual one. In addition, the depth camera acquires depth information with millimeter-level errors. However, since the end effector has a clamping range of 12 mm, the average error is significantly smaller than this range. This indicates that the localization accuracy meets the requirements for the end effector to perform clamping and cutting operations.

3.5. Analysis of Experimental Results of Practical Picking

The results of ten practical picking experiments are shown in

Table 9. Practical picking experiment results.

Number	Diameter (mm)	Picking Time (s)				Result
		Recognition Time (s)	Planning Time (s)	Execution Time (s)	Total Time (s)
1	44.46	0.034	0.19	4.95	5.174	T
2	45.82	0.041	0.18	5.12	5.341	T
3	47.32	0.033	0.23	4.97	5.233	T
4	51.08	0.036	0.20	4.71	4.946	T
5	50.52	0.038	0.18	5.41	5.628	T
6	50.58	0.029	0.19	4.72	4.939	T
7	43.10	0.045	0.22	5.27	5.535	T
8	47.52	0.030	0.22	5.05	5.300	T
9	54.74	0.032	0.19	4.84	4.972	T
10	55.42	0.044	0.25	4.93	5.224	F

.

The experimental results demonstrated that the picking success rate was significantly higher for Brasenia schreberi with a diameter ranging from 40 to 60 mm, achieving an average success rate of 90%. The harvesting system exhibited remarkable proficiency in swiftly and accurately completing the identification, planning, and execution phases, with an average picking time of 5.229 s per leaf. Notably, in the 10th experiment, where the short stem of the leaves was 55.42 mm, a picking failure occurred. A detailed analysis revealed that during this experiment, the visual processing module’s positioning deviation caused the Brasenia schreberi’s center point to shift closer to the outer side. Coupled with the loosening of the end effector’s fastening screw, which induced slight vibrations, this resulted in the stem slipping out of the lower finger gap and triggered cutting failures. To resolve these issues, enhancing the robotic arm’s motion control strategy and the end effector’s design is essential. Additionally, further refining the visual positioning system will improve its adaptability in special situations and elevate the harvesting success rate. Regarding time consumption, the identification and planning phases have been optimized and maintained within a reasonable timeframe. However, the relatively long execution time remains a critical aspect for future research. By systematically analyzing the factors influencing the execution time, including the manipulator’s speed and the end effector’s efficiency, and implementing targeted enhancements, the efficiency of the entire picking process can be further improved.

4. Conclusions

This paper designed and implemented an intelligent Brasenia schreberi-harvesting system based on machine vision, which consists of a mobile platform, an end effector, and a mechanical arm harvesting system based on machine vision. The mobile platform adopts a catamaran structure equipped with a paddle wheel drive device that is suitable for shallow pond environments and offers good stability and mobility. The end effector uses a lateral stem clamping and cutting method, and is designed with a motion generation mechanism, a clamping mechanism, and a cutting mechanism to achieve the rapid clamping and cutting of Brasenia schreberi. The mechanical arm harvesting system based on machine vision uses the YOLO-GS network for Brasenia schreberi target recognition and positioning, and an improved algorithm for mechanical arm path planning, achieving efficient and accurate harvesting. The experimental results indicate that the improved algorithm surpasses the traditional RRT algorithm in both planning time and path length, improving the obstacle avoidance efficiency and path planning effectiveness of the mechanical arm. The average positioning error of the visual positioning module is 4.80 mm, which meets the requirements for actual harvesting. In actual harvesting experiments, the system achieved an average harvesting success rate of 90% and an average harvesting time of 5.229 s, demonstrating the feasibility and effectiveness of the system.

Some progress has been made in the intelligent harvesting of Brasenia schreberi in this study; for example, the preliminary construction of the harvesting system has been completed and the automatic intelligent harvesting of Brasenia schreberi has been achieved based on this improved harvesting path-planning algorithm. However, due to a lack of experience, there is still room for improvement in the harvesting system. In the current end effector, its vibration may cause the Brasenia schreberi to fall off during the picking process. In the future, the design of the actuator can be further refined, such as by introducing a bionic design or adaptive control strategies to improve its adaptability to different sizes of Brasenia schreberi. In the experiment, it was found that the execution time and storage time are relatively long. In the future, the joint control strategy of the manipulator could be further improved to increase the movement speed and efficiency of the manipulator. Although the improved algorithm performs well in terms of time efficiency and path quality, there is still room for improvement. In the future, reinforcement learning or deep learning algorithms can be combined to train for special scenarios during operation to further optimize the real-time adaptability of path planning [7]. The intelligent harvesting system in this study is primarily designed for Brasenia schreberi. In the future, we can attempt to extend it to the harvesting of other aquatic vegetables or land crops and migrate the completed intelligent harvesting system in order to promote the universal application of agricultural mechanization.