Design, Integration, and Experiment of Transplanting Robot for Early Plug Tray Seedling in a Plant Factory

Abstract

In the context of plant factories relying on artificial light sources, energy consumption stands out as a significant cost factor. Implementing early seedling removal and replacement operations has the potential to enhance the yield per unit area and the per-energy consumption. Nevertheless, conventional transplanting machines are limited to handling older seedlings with well-established roots. This study addresses these constraints by introducing a transplanting workstation based on the UR5 industrial robot tailored to early plug tray seedlings in plant factories. A diagonal oblique insertion end effector was employed, ensuring stable grasping even in loose substrate conditions. Robotic vision technology was utilized for the recognition of nongerminating holes and inferior seedlings. The integrated robotic system seamlessly managed the entire process of removing and replanting the plug tray seedlings. The experimental findings revealed that the diagonal oblique-insertion end effector achieved a cleaning rate exceeding 65% for substrates with a moisture content exceeding 70%. Moreover, the threshold-segmentation-based method for identifying empty holes and inferior seedlings demonstrated a recognition accuracy surpassing 97.68%. The success rate for removal and replanting in transplanting process reached an impressive 95%. This transplanting robot system serves as a reference for the transplantation of early seedlings with loose substrate in plant factories, holding significant implications for improving yield in plant factory settings.

1. Introduction

The plant factory is an advanced form of protected agriculture [1] that allows for control over climatic factors such as light, carbon dioxide levels, temperature, and humidity, creating optimal conditions for plant growth [2,3]. This form of agriculture offers advantages such as high yield, superior quality, continuous production, efficient resource utilization, and reduced susceptibility to environmental fluctuations [4,5,6], allowing it to produce more products in less space [7]. Plug tray seedling technology is extensively employed in plant factories, primarily due to its high germination rates, uniform growth, seed-saving benefits, and compatibility with mechanized operations [8]. However, the development of plant factories is constrained by drawbacks such as high energy consumption [9,10], high per-unit area costs [11], and limited space, which is not conducive to the installation of large-scale automated equipment [10]. Moreover, the suboptimal quality of the seeds, imprecise seeding within the plug trays, unfavorable temperature and humidity conditions, and insufficient nutritional provision, along with the presence of plant diseases and pests, collectively contribute to a suboptimal germination rate ranging between 80% and 95% [12,13] for seedlings cultivated in plug trays [14] and subsequently lead to yield losses in both the per-unit energy consumption and per-unit area. This, in turn, exacerbates the operational costs of plant factories.

To achieve the operations of removing and replanting seedlings, researchers have developed various forms of end effectors, mainly including the plug-in and clamping type [15,16,17], oblique insertion type [18,19,20,21], and deformed sliding needle type [22,23]. The current end effectors have found wide applications in transplanting seedlings at relatively advanced growth stages. This is due to the fact that mature seedlings possess well-established root systems, ensuring effective substrate cohesion [24]. However, early-stage seedlings, characterized by underdeveloped root systems and loose substrate, especially for nongerminating holes, have no root system to consolidate the substrate, presenting challenges in effective grasping. The existing end effectors used in transplanting machines are primarily designed for well-developed root systems and compacted substrate blocks [21,25,26,27] in greenhouses, so they may not be suitable for handling younger seedlings grown in loose substrate in plant factories.

In the domain of inferior seedling identification, researchers have undertaken extensive investigations. Tong [28,29] employed the watershed segmentation algorithm to partition leaves and subsequently computed the area and perimeter of individual leaves within tray seedlings. Xu [13] designed a visual system for the early differentiation of underdeveloped watermelon seedlings, utilizing phenotype detection and machine learning. This system relied on two early characteristics, seedling height and leaf area, to evaluate their growth status, resulting in an 84% discrimination accuracy. Wen [30] proposed a combination of threshold segmentation and morphology algorithms for image segmentation. These studies only introduce the recognition methods but fail to provide detailed information on how to convert recognition results into robot motion control coordinates.

In plant factories, unfit seedlings consume expensive LED light sources and occupy limited planting space, which is a great waste. The removal and replanting of unfit seedlings at the early stage represent a pivotal opportunity to optimize the utilization of energy and planting space. This approach translates into a noteworthy increase in the production per unit area and per-unit energy consumption, ranging from 5% to 20%. Moreover, the large size of automated equipment, exemplified by the transplanters commonly utilized in greenhouses, poses a challenge in terms of the limited space within plant factories. Given the scarcity of planting space in plant factories and the resultant spatial constraints on the installation of large automated equipment, there arises a need for the development of transplanters tailored to plant factory environments. Such smart transplanters would not only enhance the automation rates but also serve to replace manual labor, thereby contributing to a reduction in labor costs.

As mentioned above, the existing end effectors, the identification and positioning of underdeveloped seedlings, and the overall dimensions of the transplanting machines used in greenhouses are not well suited to plant factory environments. This article aims to solve the above problems and develop a prototype of an entire transplanting robot system suitable for a plant factory.

In this article, a diagonal oblique insertion type end effector was designed to achieve a stable grasp on loose substrate. At the same time, a prototype transplant robot containing the end effector was also designed. Robotic vision technology was utilized for the recognition of empty holes and inferior seedlings. This method was also integrated into the robot system. The performance and efficiency of the proposed robotic transplanting system for plug tray seedlings were evaluated using extensive experiments.

The remainder of this paper is organized as follows: Section 2 introduces the materials we used and the method we proposed in detail. The experimental results are reported in Section 3. In Section 4, we provide a detailed discussion of this transplanting robot and point out the problems to be solved in further study. Finally, Section 5 concludes the paper.

2. Materials and Methods

2.1. Materials

2.1.1. Robot System Setup

• Structural composition

The robot transplanting system consists of a UR5 robot, end effector, camera, computer, air compressor, and other components, as shown in Figure 1e. The camera model is Logi-C270 (Logitech Corporation, Taiwan, China). The 6 DoF (degree of freedom) cooperative robotic arm UR5 (Universal Robots, Odense, Denmark) with a diagonal oblique-insertion end effector is applied for grasping and replanting the plug tray seedlings. The partial parameters of the primary devices adopted in this research are listed in

Table 1. The partial parameters of the main devices (camera and robot manipulator) adopted in this research.

Camera			Robot Manipulator
Feature	Parameter	Unit	Feature	Parameter	Unit
Model	Logi-C270	-	Model	UR5	-
Type	RGB camera	-	DoF	6	-
Resolution	720 P/30 fps	-	Work range	850	mm
FoV	60	°	Weight	18.4	kg
Sensor	CMOS	-	Load (kg)	5	kg
Interface	USB2.0	-	Pose Repeatability	±0.1	mm

in detail, including the camera and the robot.

2.

Analysis of robot workspace

The workspace of the robot extends 850 mm from the base joint. However, as the robot stretches out, the knee-joint effect can give high forces in the radial direction (away from the base) but also low speeds. Similarly, the short leverage arm, when the tool is close to the base and moving tangentially (around) the base, can cause high forces, but also at low speeds. Therefore, when robots move in these areas of the workspace, as shown in Figure 1a, there is a risk of pinch injuries. To ensure personnel safety, these two areas should be avoided when setting up the plug trays. In the production of protected agriculture, 72-hole plug trays are widely used [30,31,32,33], and research has shown that the 72-hole plug tray is more conducive to cultivating high-quality tomato seedlings while reducing seedling cultivation energy consumption [34]. Therefore, the 72-hole plug tray was chosen as the seedling cultivation tray. The plug tray consists of 12 rows and 6 columns, with an external size of 540 mm × 280 mm, made of PVC. The distance between the plug trays and the robot base was 205 mm so that the end effector could reach all positions of the supply and target plug trays in a more suitable posture, and the robot was always in the safe working area (the green region shown in Figure 1a) during movement. The “Eye on Arm” method for camera installation was adopted, which ensures that the camera position is always unaffected when the system position changes. We set the shooting position to 650 mm directly above the target plug tray so that the camera could capture the complete plug tray. The positions of the plug tray and the camera for image acquisition are shown in Figure 1b.

3.

End effector control

The gripper lifting cylinder and the seedling grabbing cylinder are actuated by two single-acting solenoid valves. These solenoid valves are connected to the digital output points 1 and 2 of the UR5 robot controller. This configuration eliminates the need for an additional controller to regulate the movements of the cylinders, as shown in Figure 1c. The two cylinders of the end effector are controlled by the solenoid valves KA1 and KA2. The air circuit connection diagram of the end effector is shown in Figure 1d, by sending the “set_digital_out (1 or 2, True)” command to the robot to activate the solenoid valves KA1 and KA2. Conversely, we reset the solenoid valves by sending the “set_ digital_out (1 or 2, False)” command. Using a wireless router, we established a wireless connection between the robot and the computer.

4.

Control system

All the framework for robotic control was run and tested on the MECHREVO X8Ti-A computer (MECHREVO Corporation, Beijing, China) with AMD R7 4800 CPU, NVIDIA GeForce RTX 2060 GPU, 16 GB memory (DDR4-3200 MHz), and 512 GB hard disk. All the control software was run on a Windows 10 64-bit operating system. The RGB camera was connected to the computer. The wireless data transmission between the robot and the computer was achieved through socket technology based on the TCP/IP protocol. This wireless control of the robot facilitated convenient debugging and experimentation processes.

Based on the devices and parameters mentioned above, a robotic plug tray seedling transplanting system was set up for image acquisition and system integration. The system model is shown in Figure 1e. Moreover, we also made a prototype of the transplanting robot, as shown in Figure 1f.

2.1.2. Design of End Effector

In order to ensure that the end effector can grasp as much substrate as possible, a diagonal oblique-insertion-type end effector was designed. Moreover, this gripper structure enables seedlings to grasp from diagonal corners of square holes, keeping the gripper furthest away from the seedling positioned at the center of the hole during planting. This design effectively minimizes the risk of damaging the seedling leaves with the gripper. The external contour of the gripper matches the inner wall of the tray’s hole, and the dimensions of the hole are shown in Figure 2a, with a height of 40 mm. When designing the gripper, a portion of 10 mm was cut from the diagonal of the hole using the hole model without the bottom, dividing the hole into two parts, which served as the initial contours of the two grippers. Then, the upper mounting components were designed to complete the design of the shovel-shaped gripper, as shown in Figure 2b. By appropriately trimming the contour of the shovel-shaped gripper, a fork-shaped gripper can be obtained, as shown in Figure 2c. In subsequent studies, the grasping performance of these two structural grippers will be compared. The end effector consists mainly of a vertical lifting cylinder, gripper cylinder, lifting guide rail, chute plate, mounting plate, ripper guide rail, gripper, push rod, etc., as shown in Figure 2d. By combining a guide groove plate with a guide rail slider mechanism at a certain angle, the lifting motion of the cylinder was converted into a diagonal insertion motion of the gripper, and the substrate was clamped diagonally from the hole.

The end effector transplanting the plug seedlings is as follows: First, the end effector moves to a position above the target hole of the supply tray, as shown in Figure 3a. Second, the lifting cylinder pushes the gripper down, making the lower end of the gripper close to the surface of the substrate, as shown in Figure 3b. Again, the piston rod of the gripper cylinder extends, pushing the gripper to tilt downwards along the diagonal edge of the hole to insert into the plug seedling substrate at a certain depth, as shown in Figure 3c. Then, the lifting cylinder retracts and drives the gripper to rise, removing the seedling with substrate from the hole, as shown in Figure 3d. In this way, the end effector completes the task of grasping the seedling, and the process of planting the seedling is opposite to the grasping process, as shown in Figure 3e–h.

The hole shape of the plug tray is an inverted pyramid. To achieve the insertion of the gripper along the hole wall of the plug tray, the angle of the guide rail is required to be consistent with the angle of the hole wall, and when the gripper cylinder moves from the top dead center to the bottom dead center, the gripper should move just above the hole to the bottom of the hole, as shown in Figure 4a. To achieve such a motion effect, it is necessary to calculate the inclination angle of the guide groove plate. The motion analysis of the gripper is shown in Figure 4b. When the piston rod of the gripper cylinder moves from A₁ to A₂, the gripper moves from B₁ to B₂, and there is the following relationship:

$$\left\{\begin{array}{c}{h}_1+h_4=h_0+h_2\\ L_3=L_1-L_2\\ h_1=L_1\tan \alpha \\ h_2=L_2\tan \beta \\ L_3=h_4\cot \beta \end{array}\right.$$

(1)

By solving the system of equation group 1, we can determine that

$$\mathsf{\alpha }={\tan }^{-1}\left(\left(\left(h_0-h_4\right)\tan \beta \right)/h_4\right)$$

(2)

In these formulas, $h_0=60$, represents the cylinder stroke, $h_4=40$, represents the lifting distance of the gripper, and $\beta =70.71°$, which is the inclination angle of the hole wall (guide rail inclination angle). Substituting $h_0$, $h_4$, and β into the above equation yields $\alpha =55°$. A statistical analysis of the seedlings’ height with the age under 15 days determined that the plant height was less than 80 mm. Therefore, the stroke of the lifting cylinder was set to 100 mm to ensure that the gripper would not damage the seedlings during its movement.

2.1.3. Seedlings for Experiment

In this study, the target crop for the transplanting operation is the Big Red Cooperation 903 tomato (Shanghai Hongqiao Tianlong Seed Industry Co., Ltd., Shanghai, China, variety registration number: GPD Tomato (2018) 310450) seedlings. The substrate was prepared by mixing mud carbon and perlite in a volume ratio 3:1, and then a tiny amount of water was sprayed to moisten the substrate. After the substrate was loaded, we used a hole-pressing template to press circular sowing holes with a diameter of 1.0 cm to 1.5 cm and a depth of about 1.0 cm to 1.5 cm on each plug, then planted tomato seeds in the hole. The seeds were covered with vermiculite on the surface.

By recording the daily growth of tomato seedlings, it can be observed that when the seedlings reached the 8th day of growth, the cotyledons had flattened, allowing for the identification of inferior seedlings based on leaf size. On the 8th day, the true leaves began to emerge, and the seedling leaves had reached the edge of the cells. On the 9th day, true leaves had visibly grown, and there was minimal leaf overlap between adjacent seedlings, which does not significantly affect the recognition of inferior seedlings. On the 11th day, most seedling leaves had extended beyond the cell boundaries, causing mutual overlap and instances where leaves obstructed inferior seedlings or empty cells, leading to potential recognition errors. Additionally, older seedlings have larger leaves, making them more susceptible to damage when grasped by the transplanting machine, potentially affecting their growth. Therefore, it is determined that seedlings aged 8–10 days are the most suitable for transplanting. Subsequent transplanting experiments in this study exclusively used seedlings within this age range.

2.2. Methods

2.2.1. Grasping Test Method

The gripping action on the substrate is a process involving the interaction between the gripper and the substrate. Within this process, the gripper’s structural form and the substrate’s moisture content emerge as pivotal factors influencing the gripping effectiveness. The design of the gripper structure determines how it engages with the substrate, while the moisture content of the substrate impacts the intermolecular forces within the substrate. Therefore, this study aims to assess the influence of these two variables, gripper structure and substrate moisture content, on the gripping performance.

According to the designed 3D model, we manufactured a prototype of the end effector. The shovel-type and fork-type grippers were fabricated using metal 3D printing technology, and the material was 316 L stainless steel. The temperature in the testing environment was maintained at 20–26 °C, with a humidity range of 800–1200 ppm. The air pressure of the cylinders was set at 0.45 MPa. Two types of grippers, shovel-type and fork-type, were used in the experiment. The grasping speed was set at 0.4 m/s, and the lifting speed was 0.2 m/s. The experimental parameters are shown in

Table 2. Experimental parameters.

Feature	Parameter	Unit
Nursery temperature	20–26	°C
Nursery humidity	800–1200	ppm
Air pressure of cylinders	0.45	MPa
Grasping speed	0.4	m/s
Lifting speed	0.2	m/s
Drying temperature	80	°C

.

The process of the grasping test is shown in Figure 5, including seedlings planting, substrate grasping test, wet substrate weighing, substrate drying, and dry substrate weighing. The plug tray and container were numbered. We measured the weight of the cup, the weight of the substrate grabbed by the gripper, the sum of the weights of the gripper and residual substrate, and the total weight of the substrate (including the grabbed substrate, residual substrate, and scattered substrate), and denoted them as $m_0$, $m_1$ $m_2$, and $m_3$, respectively. Then, we placed the substrate into an oven to dry. After drying the substrate, we measured its dry weight $m_4$. The moisture content was calculated according to the following formula:

$$m_c=\left(m_3-m_0\right)/\left(m_4-m_0\right)\times 100\%$$

(3)

We used $M_g$ to represent the percentage of the substrate grabbing, and then the substrate capture percentage could be calculated based on this formula:

$$M_g=\left(m_1-m_0\right)/\left(m_3-m_0\right)\times 100\%$$

(4)

2.2.2. Visual Recognition Method

Our study focuses on the early-stage tray seedlings planted in plant factories, where the seedlings typically have 2–3 leaves, the leaf area is relatively small, and there is minimal leaf overlap during this stage. This characteristic makes leaf segmentation achievable through conventional computer vision methods. The traditional computer vision approach is effective in achieving good recognition results and cost-effective, making it conducive to the widespread adoption and application of the equipment. Hence, we opted for traditional computer vision methods to recognize the undesirable seedlings in this study.

The complete process of inferior seedling identification includes image preprocessing, grayscale processing, threshold segmentation, pixel value analysis, and result output. The complete image processing algorithm flowchart is shown in Figure 6.

• Image preprocessing: Firstly, the rembg algorithm, an open-source tool available on GitHub, was employed for background removal in the seedling tray images. This algorithm, built upon the u2net deep learning model, exhibits rapid and precise background elimination capabilities. Compared to the original image in Figure 7a, the image without background using the rembg algorithm has less noise, as shown in Figure 7b. Due to discrepancies in the camera’s installation position and angle, image skewness was observed. To mitigate the impact of image skewness on recognition accuracy, a perspective transformation was applied to rectify the images. We superimposed a grid on both the original and the corrected images, with grid cells matching the size of the holes, as shown in Figure 7c,d. It became evident that the corrected image aligned better with the grid, indicating improved alignment and accuracy.
• Grayscale processing: Two grayscale algorithms were compared for image processing. The first one utilizes the grayscale algorithm provided by OpenCV with the formula 0.114 ×b + 0.587 × g + 0.299 × r. The second one is the Excess green (ExG) algorithm with the formula: 2 ×g − b − r. The processed images and grayscale histogram images are shown in Figure 8a–d. By comparing the results of these two methods (Figure 8a and Figure 8b), it is evident that the ExG algorithm effectively reduces the prominence of the tray and substrate in the image and provides better extraction of green leaves. The pixel-value histograms of the images processed by the ExG algorithm (Figure 8c) show more significant differentiation than the grayscale algorithm provided by OpenCV (Figure 8d), which is more favorable for subsequent leaf segmentation processes.
• Threshold segmentation: Gaussian filtering was applied to the grayscale image to reduce noise. Subsequently, we used both adaptive threshold and Otsu’s method [35] to binarize the denoised grayscale image. As shown in the results, the image processed using the adaptive threshold method (Figure 9a) still contains a significant amount of substrate that was not effectively removed, resulting in considerable noise. Otsu’s method (Figure 9b) produced better segmentation results than the adaptive threshold method. Therefore, we chose to use Otsu’s method for threshold in our further analysis.
• Pixel value analysis and result output: Following the previously mentioned method, a grid matching the size of the tray’s holes was overlaid on the binarized image. Then, the pixel values in each grid were counted. The qualification of seedlings was determined by setting an appropriate pixel threshold. These pixel values were visualized using Matplotlib, as shown in Figure 10a, where more giant bubbles represented higher pixel values. Green bubbles represented healthy seedlings, red bubbles represented inferior ones, and red hollow circles indicated pixel values of 0, signifying the absence of seedlings in those regions. Finally, the processing results were displayed, as shown in Figure 10b, and the row and column data for unhealthy seedlings were output as arrays for robot control.

By following this comprehensive workflow, the identification of inferior seedlings can be carried out effectively, providing valuable insights for quality control and optimization of seedling production.

2.2.3. Robot Positioning Method

The UR5 robotic arm, chosen for its widespread industrial application, is a 6-degree-of-freedom robot capable of reaching any position within its reachable space. Given that the transplantation task in this study essentially corresponds to a planar grasping problem for the robot, the robot’s degrees of freedom are sufficient to meet the operational requirements.

In existing research, it is common practice to employ threshold segmentation to extract the contour of the plug tray from the image. Subsequently, statistical analysis is conducted on the pixel intensity peaks along the horizontal and vertical directions to determine the plug tray boundary. This boundary information is then utilized to calculate the centroid, which aids in precise positioning of the individual cell within the plug tray [13,36]. This method’s positioning accuracy depends on the contour’s extraction effect. However, the process of boundary extraction can be affected by factors such as camera internal parameters, noise, lighting variations, and background conditions, which may lead to incomplete boundary extraction. Consequently, this could result in inaccuracies in positioning. Simultaneously, their research lacks pertinent data regarding positioning accuracy, and reported experiments involving grasp tests using positional information are absent. This paper introduces a three-point teach-in positioning method, assesses the positioning accuracy of this approach, and conducts seedling thinning and supplementation experiments utilizing the acquired positioning information. The robotic arm achieves precise positioning of the cell cavities. The specific methodology is outlined as follows.

• Use the plug tray positioning block to limit the target tray and the seedling supply tray to fixed positions, ensuring that the positions of the two trays relative to the UR5 robot base remain unchanged during removing and replanting operations.
• Number the hole positions of the tray in row M and column N, and manually teach the robot to move to the three corner positions of the tray (as shown in Figure 11) at $P_{\mathrm{1,1}}\left(x_{\mathrm{1,1}},y_{\mathrm{1,1}}\right)$ , $P_{1, N}\left(x_{1,N},y_{1,N}\right)$, and $P_{M,1}\left(x_{M,1},y_{M,1}\right)$, and record the coordinate values of the robot at these three points in the world coordinate system of the robot.
• Due to installation errors causing misalignment between the coordinate axes of the robot system and the edges of the plug tray in the robot’s coordinate system, vector methods can be employed to calculate the position coordinates of target points. The visual system identifies the inferior seedling according to the method in Section 2.2.2 after the image of the plug tray seedlings is obtained. If the position of the inferior seedling is in row i and column j, as shown in Figure 11a, the vectors satisfy the following relationships:

$$\left\{\begin{array}{c}\overrightarrow{P_{\mathrm{1,1}}{P}_{i,j}}=\overrightarrow{P_{\mathrm{1,1}}{P}_{1,j}}+\overrightarrow{P_{1,j}{P}_{i,j}}\\ \overrightarrow{P_{\mathrm{1,1}}{P}_{1,j}}=\left(j-1\right) \overrightarrow{P_{\mathrm{1,1}}{P}_{i,N}} / \left(N-1\right)\\ \overrightarrow{P_{1,j}{P}_{i,j}}=\left(i-1\right) \overrightarrow{P_{\mathrm{1,1}}{P}_{M,1}} / \left(M-1\right)\end{array}\right.$$

(5)

By substituting the coordinates of each point into Equation (5), we can derive the calculation formula Equation (6) for the coordinates of the inferior seedling’s position.

$$\left\{\begin{array}{c}{x}_{i,j}=x_{\mathrm{1,1}}+\left(j-1\right)\left(x_{1,N}-x_{\mathrm{1,1}}\right)/\left(N-1\right)+\left(i-1\right)\left(x_{M,1}-x_{\mathrm{1,1}}\right)/\left(M-1\right)\\ y_{i,j}=y_{\mathrm{1,1}}+\left(i-1\right)\left(y_{M,1}-y_{\mathrm{1,1}}\right)/\left(M-1\right)+\left(j-1\right)\left(y_{1,N}-y_{\mathrm{1,1}}\right)/\left(N-1\right)\end{array}\right.$$

(6)

• After calculating the coordinate data of the hole position, the computer sends the position data to the UR5 robot controller through wireless communication, thereby controlling the robot to eliminate inferior seedlings sequentially according to the planned path.

A scaled plan view of the plug tray was created using CAD, incorporating a central crosshair at the center of each hole. The drawing was printed, affixed to the surface of the plug tray, and placed on the positioning block. A positioning pointer tool was designed, 3D-printed, and mounted on the end effector, as shown in Figure 11b. Following the TCP positioning method outlined in Section 2.2.3, the coordinates of other holes were calculated based on the coordinates of three corner points. The robot was then directed to move the end effector to other holes, and the positioning error was recorded by observing the pointer’s scale line position. Accounting for the softness of the plug tray and potential errors in each placement, we tested five different trays, measuring the positioning error for ten holes after each tray. The results indicate that the maximum positioning error in the x-direction is 1.00 mm and in the y-direction it is 1.50 mm, with average positioning errors of 0.44 mm in the x-direction and 0.59 mm in the y-direction. Hence, we adjusted the gripper spacing to ensure insertion into the substrate from a position 2 mm away from the edge of the hole, guaranteeing no interference with the plug tray.

This method effectively mitigates the errors arising from seed positioning discrepancies during sowing, the incline of the plug tray, and potential camera distortions during image recognition. The localization error is primarily contingent upon the extent of deformation incurred during the tray placement and the dimensional variations across diverse plug trays. Considering that plug trays conform to standard industrial specifications, resulting in a high level of consistency in dimensions, this approach inherently guarantees increased localization accuracy. Consequently, the method effectively satisfies the positioning requirements for transplantation procedures.

3. Results

3.1. Grasping Effect

After the experiment, we plotted the data of the percentage of grasping weight for the two types of grippers at different moisture levels on a scatter plot, as shown in Figure 12. Based on the results, we can draw the following conclusions.

With the increase in moisture content, the grasping percentage also increases. When the moisture content is 70% to 75%, both types of grippers achieve their maximum grasping percentages of about 65%. This is attributed to the fact that the cohesion within the substrate becomes stronger as the moisture content increases. However, as the moisture content rises, the grasping percentage decreases. This decrease can be attributed to the excessive moisture in the substrate, which results in a higher substrate weight and makes it more prone to scattering during the grasping process.

When the moisture content is below 65%, the shovel-type gripper has a higher grasping percentage than the fork-type gripper. This is because the shovel-type gripper has a closed contact surface with the substrate, preventing substrate scattering. In contrast, the fork-type gripper has gaps between its fork teeth, which can lead to substrate scattering. However, when the moisture content exceeds 65%, the increased cohesion among the substrate particles causes them to stick together, making it less likely for them to scatter from the fork-type gripper’s teeth. Additionally, the fork-type gripper’s smaller contact area with the substrate during insertion makes it smoother to penetrate the substrate, resulting in a slightly higher grasping percentage than the shovel-type gripper.

Based on the analysis, it is recommended to choose a time shortly after tidal irrigation when the substrate has a higher moisture content for transplanting operations. This is because higher moisture levels facilitate better grip and handling of the seedlings during transplantation. Additionally, considering that the fork-type gripper has a smaller contact area with the substrate, it is more suitable for inserting into the soil. Therefore, selecting the fork-type gripper for the transplanting process would be advantageous. By following these recommendations, it is expected that the transplanting efficiency and success rate can be optimized, leading to improved overall performance in the seedling transplantation process.

3.2. Test of Inferior Seedling Recognition

According to the method described in Section 2.2.2, we tested the visual system’s recognition accuracy on 432 seedlings, with two trays each for seedlings aged 8 to 10 days. We counted the number of misidentified seedlings and recorded the experimental data in

Table 3. Data of recognition accuracy test.

Seedling Age/(Day)	NO. of Trays	Number of Seedlings	Number of Recognition Errors	Recognition Accuracy	Time Cost/(s/Tray)
8	1	72	3	95.83%	1.19
	2	72	1	98.61%	1.27
9	3	72	1	98.61%	1.20
	4	72	2	97.22%	1.20
10	5	72	2	97.22%	1.24
	6	72	1	98.61%	1.23

. By incorporating a timer into the recognition algorithm program, recognition time is recorded, and the average recognition time can be calculated. We can calculate that the average recognition accuracy of this algorithm is 97.68%, with an average recognition time of 1.22 s. This performance meets the requirements for robot transplanting operations.

The main reasons leading to recognition errors can be categorized into two situations. Firstly, when the leaves extend beyond the boundaries of the hole in the image, their pixel values tend to be relatively low due to being outside the bounding box. This may result in healthy seedlings being mistakenly identified as inferior ones. Secondly, the skewed growth of leaves can cause them to protrude into adjacent empty holes or holes containing inferior seedlings, increasing in pixel values within the adjacent hole’s bounding box. Consequently, empty holes or inferior seedlings may be incorrectly recognized as healthy seedlings.

Regarding the issue of leaves not being positioned at the center of the holes in the images, there are primarily two contributing factors. Firstly, during the manual seeding process, it may be the case that seeds were not accurately placed in the central position of the holes. Secondly, during the growth of seedlings, factors such as lighting conditions, temperature, and humidity in the growth environment can influence stem tilting, causing the leaves to deviate from the central position of the holes.

3.3. Removal and Replanting Test

3.3.1. Automatic Removal Test

Based on the result of the end effector test in Section 3.1, it is evident that the highest grasping percentage occurs at around 70–75% moisture content. Therefore, we conducted the experiments approximately 2 h after watering, when the moisture content was around 70%, to achieve the best grasping performance. Unlike the end effector test, where only the grasping performance of the end effector was tested during the actuation of the two cylinders, the removing test required the UR5 robotic arm to operate the end effector in coordination with the vision system, encompassing the entire process from recognition to substrate grasping and dropping.

An automatic removal test was conducted with four trays of tomato seedlings aged 8–10 days. During the test, the mass of the removed substrate and the total mass of the substrate were measured. The removal was considered successful when the percentage of removed substrate mass exceeded 60%, and there was no substrate dispersion during the motion of the UR5 robotic arm. The complete process of seedling removal was captured on video to determine the time needed to accomplish the task. The test data for the four trays of seedlings are presented in

Table 4. Data of removal test.

NO. of Trays	Number of Inferior Seedlings	Number of Successfully Removed Seedlings	Success Rate	Time Cost/(s)
1	14	14	100%	64.01
2	7	7	100%	38.81
3	13	13	100%	61.18
4	21	21	100%	96.04

. According to the test result, the removal success rate of the transplanting robot was 100%, with a removal efficiency of 12.69 seedlings per minute.

3.3.2. Automatic Replanting Test

The replanting experiment involved seedlings of the same age and substrate moisture content as the thinning experiment. During the experiment, the UR5 robotic arm controlled the end effector to complete the process, including grasping seedlings from the seedling supply tray, transporting them, and replanting them into the target tray. Throughout the replanting operation, if any of the following issues occurred, such as seedling grasping failure, seedling dropping during transportation, seedling misalignment during replanting, or damage to seedling leaves, the replanting was considered unsuccessful.

We conducted replanting experiments on five trays, each with 20 unqualified seedlings already removed. The entire process of replanting was recorded on video to determine the time needed to accomplish the seedling replanting task. The experimental data are presented in

Table 5. Data of replanting test.

NO. of Tests	Number of Inferior Seedlings	Number of Successfully Replanting Seedlings	Success Rate	Time Cost/(s)
1	6	6	100%	30.55
2	30	20	66.67%	152.28
3	22	15	68.18%	132.83
4	27	16	59.26%	158.62

. According to the results, the replanting success rate was 67.06%, with a replanting efficiency of 10.75 seedlings per minute.

By observing the experimental video, it was found that there was a suspended area due to incomplete support below the seedling supply tray. During grasping, the plug tray would bend downward and deform, resulting in insufficient insertion depth of the grippers, which led to the failure of seedling grasping. There are two main reasons for the aslant of seedlings replanting. One reason is that there is friction between the plug tray and the grippers, which causes the grippers to fail to return promptly. At this time, the lifting cylinder retracts and drives the grippers to rise, lifting the seedlings and causing them to tilt. This problem can be solved by changing the gripper cylinder to a double-acting cylinder and adding a magnetic switch to detect whether the gripper cylinder returns to its original position. The second reason is that the height of the seedling pushing rod is too high, which does not play a role in pushing and pressing the substrate during seedling release, resulting in the grippers lifting the seedling during the retraction process and causing it to tilt. This problem can be solved by increasing the length of the seedling pushing rod.

Based on the analysis of the reasons for failure, we optimized the transplanting system by adding bottom support, adjusting the length of the seedling push rod, and replacing the gripper cylinder with a double-acting cylinder. We retested 100 seedlings with the optimized system, and the test results are shown in

Table 6. Data of replanting test of the optimized system.

NO. of Trays	Number of Inferior Seedlings	Number of Successfully Replanting Seedlings	Success Rate	Time Cost/(s)
1	20	20	100%	127.96
2	20	18	90%	128.22
3	20	19	95%	128.23
4	20	18	90%	129.90
5	20	20	100%	128.16

. After optimization, the success rate of seedling replanting increased to 95%, and the seedling replanting efficiency reached 9.34 seedlings per minute, meeting the requirements of the transplanting operation.

3.3.3. Effect Analysis of Transplanting Operation

To assess the potential effects of the transplanting operation on seedling growth, we conducted a comparative growth experiment involving two groups of seedlings: one group was subjected to transplantation, while the other served as the control group. Each group consisted of 30 seedlings planted on opposite sides of a tray to ensure they were exposed to the same growth environment. We measured the height of the transplanted and control group seedlings every 3 days, continuously recording data up to the 12th day after transplantation, as shown in Figure 13a,b. Based on the information provided in Figure 13c, it is evident that both groups of seedlings exhibit a consistent growth trend, and at a significance level of 0.05, there is no significant difference in average plant height between the two groups of seedlings with the same growth days. Therefore, it can be concluded that the impact of transplantation operations on seedling growth is not significant.

4. Discussion

In this study, we developed a robotic transplanting system based on computer vision and met the basic needs of the plug tray seedlings transplanting process. However, some problems were exposed from the experiment, which still need to be solved within further study to optimize this prototype continuously. In this section, we will give a detailed discussion of these problems.

The influence of substrate moisture content and gripper design on the substrate grasping percentage was investigated in this article. However, it is important to acknowledge that additional factors may impact the grasping performance, such as gripper spacing, grasping force applied to the substrate, and the size of gripper teeth gaps. In subsequent research, a comprehensive approach could employ orthogonal experiments to analyze the combined effects of various influencing factors systematically. This would aid in identifying the optimal combination of gripper design parameters for achieving the best grasping performance.

Our experiments revealed that seedlings might exhibit stem tilting or be planted off-center within the cell, leading to instances where leaves protrude from the hole, thereby increasing the likelihood of misidentification and missing removal. To mitigate this, sowing machines could enhance the uniformity of seed placement within the tray plugs. A blade separation mechanical device can also be installed to separate the blades of adjacent seedlings in the hole from each other in order to eliminate the problem of mutual obstruction of the blades. Moreover, exploring deep learning techniques for recognizing and detecting inferior seedlings could provide more accurate results. Challenges may arise in determining target annotation and model construction to achieve improved recognition outcomes.

Experimental investigations have revealed two predominant failure scenarios within the transplantation process: unsuccessful seedling grasping and seedling tilting during replanting. Insufficient support strength beneath the supply tray was identified as a cause of seedling grasping failure, leading to plate deformation during grasping and inadequate insertion depth of the gripper into the tray plug. Adding support at the bottom of the tray has improved the success rate of seedling grasping. In the future, we can also consider optimizing the gripper structure to reduce the resistance when inserting it into the substrate. Seedling tilting was attributed to friction between the plug tray and the gripper, resulting in delayed retraction and subsequent upward movement of the gripper due to frictional resistance. By replacing the single-acting pneumatic cylinder with a double-acting one, we observed an improvement in the replanting performance. In the future, another enhancement could be achieved by replacing the push rods with substrate-pressing pneumatic cylinders. This modification would ensure that the substrate remains firmly pressed during the gripper’s seedling release, preventing any tilting when the gripper retracts.

In the experiments, there were no instances of seedling damage, which demonstrates that the diagonal insertion approach, employed by the diagonal oblique-insertion-type end effector, enables seedlings to grasp from diagonal corners of square holes, keeping the gripper furthest away from the seedling positioned at the center of the hole during planting. This design effectively minimizes the risk of damaging the seedling leaves with the gripper. Additionally, during the rapid movement of the robot after grasping the seedlings, there were no instances of substrate or seedling dislodgment. This indicates that the gripper structure provides excellent containment for the substrate, preventing any scattering.

Currently, commercially available transplanters are widely used in greenhouses. These transplanters exhibit high efficiency and stable operation. However, they are generally only suitable for transplanting seedlings with older age, well-developed root systems, and tightly wrapped substrates by roots. They are not suitable for transplanting early-stage seedlings in plant factories. Moreover, these transplanters are expensive, occupy a large space, and are difficult to install and deploy in the limited space of plant factories. We compared our research with existing commercial transplanters, as shown in

Table 7. Comparison with commercial transplanters.

Manufacturer/Model	Advantage	Disadvantage
TTA/FlexPlanter	Highly efficient and operationally reliable.	Unable to be used for the transplantation of early seedlings in loose substrate. Expensive and space-consuming, it cannot be used in plant factory environments.
Flier Systems/Plug Fixer	Highly efficient and operationally reliable.	Dedicated air-assisted seedling removal equipment is required. Expensive and space-consuming, it cannot be used in plant factory environments.
Viscon/Fix-O-Mat TIFS-IV	Highly efficient and operationally reliable.	Unable to be used for the transplantation of early seedlings in loose substrate. Expensive and space-consuming, it cannot be used in plant factory environments.
Ours	The end effector can grasp loose substrates, is cost-effective, occupies minimal space, and is suitable for use in plant factories.	Single end effector operation is employed, and there is room for further improvement in efficiency.

.

5. Conclusions

In the practical production process of plant factories, the efficient utilization of energy and planting area directly impacts operational benefit. Early removal and replanting operations for plug tray seedlings in plant factories significantly enhance yield per unit energy consumption and per unit area. However, existing transplanting machines cannot grasp the loose substrate of early seedlings. Addressing the challenge of grasping loose substrate, this study introduced a diagonal oblique-insertion end effector and developed a transplanting robot system for early-stage seedlings in plant factories.

Experimental results indicate that the optimal substrate moisture content for transplanting was 70% to 75%, and the fork-type gripper demonstrated the best substrate gripping performance. Additionally, the study explored visual recognition algorithms for identifying inferior seedlings and robot positioning algorithms. The end effector, visual recognition algorithms, and robot motion control were systematically integrated to enable the robot to autonomously recognize and remove unqualified seedlings and replant with healthy seedlings. Automatic transplanting tests revealed a recognition accuracy of 97.68% for inferior seedlings, a 100% success rate in seedling removal, an efficiency of 12.69 seedlings per minute for removal, a 95% success rate in seedling replanting, and an efficiency of 9.34 seedlings per minute for replanting.

In this study, a single end effector was employed for the transplanting operation, limiting the efficiency of the transplanting machine. In future research, enhancing transplanting efficiency could be achieved by either increasing the number of end effectors or optimizing the transplanting path.

In general, this research provides a novel approach for the early replacement of seedlings in plant factories, facilitating the removal of inferior seedlings and replanting healthy ones during the early growth stages. This method contributes to more efficient utilization of electric energy consumed by LED lighting and planting areas in plant factories, thereby enhancing the operational benefit of plant factory production.