Robot Operations for Pine Tree Resin Collection

Abstract

A robotic technology consisting of an industrial robot mounted on an autonomous rover used to tap slash pine trees and collect their oleoresin for processing is introduced, and the technological challenges related to the robotic operations are discussed in detail. Unlike the case of industrial automated manufacturing systems where the relative position between the tool and workpiece can be controlled within a few hundredths of a millimeter accuracy, when used in highly unstructured environments characteristic to forestry or agriculture, the positioning accuracy between the industrial robot and the target on which it operates can be much lower than the accuracy required for the operation of the industrial robot. The paper focuses on presenting the robotic operations necessary for drilling three converging boreholes in the pine tree, spraying the boreholes with chemicals, inserting a plastic tube with pre-attached collection bag in one borehole and inserting two plugs in other two boreholes. The challenges related to performing these robotic operations in conditions of large variations in the actual shape of the pine tree trunk and variations in the relative position between the robot and the pine tree after the autonomous vehicle positions itself in front of the tree are presented. The technical solutions used to address these challenges are also described. The strategies used to programmatically adjust the robot toolpath based on detection of the borehole entry points and on the measurement of the insertion force are presented.

1. Introduction

Industrial automated manufacturing systems using computer numerical control (CNC) machines and industrial robots are structured environments in which the relative position between the tool performing the manufacturing operation and the workpiece can be commonly controlled within a few hundredths of a millimeter accuracy. There has been proposed automated equipment consisting of industrial robots installed on autonomous rovers to perform operations in forestry and agriculture [1,2,3,4]. Forestry and agricultural environments can be semi-structured, as is the case of plantations [5,6,7,8,9], or highly unstructured, as is the case of wild forests [2,3]. In these applications, the autonomous rover identifies the targets, for example trees, approaching them within an operating distance after which the industrial robot performs specific operations on the target. In such unstructured environments, the positioning accuracy of the rover relative to the target is typically much lower than the submillimeter-scale accuracy required for the operation of the industrial robot. The reasons for which the positional accuracy of wheeled vehicles cannot be generally controlled at submillimeter-scale are the following: (i) the wheels are typically not actuated by stepper of servo motors—the only ones which can control their position at a fraction of a degree; (ii) the relatively large wheels radii (hundreds of millimeters) amplify the angular error of the actuating motor; (iii) the large manufacturing tolerances of the wheels and (iv) the soft and/or irregular terrains on which wheeled vehicles operate.

One such application is the robotic technology for tapping slash pine trees and collecting their oleoresin for processing using an autonomous robotic forest rover [1]. The technology consists of a forest rover equipped with an industrial robot (Figure 1) that identifies healthy, mature pine trees larger than 8 inch in diameter, approaches them and performs the robotic operations required for tapping the trees. This technology aims to replace the traditional labor-intensive manual work required in the collection of slash pine oleoresin [10] and to revitalize the once thriving naval stores industry in the southeast United States [11].

Secretion of oleoresin is an efficient defense-response inducer when pine trees are locally wounded. Based on this fact, there have been traditionally developed two tapping techniques for harvesting pine oleoresin [10,12,13]: (a) the bark removal method, in which bark streaks are periodically removed and the resin is collected in open containers, and (b) the borehole method, in which boreholes are drilled into the tree trunk and the oleoresin is collected in closed vessels attached to tubes inserted into the boreholes.

This paper demonstrates for the first time the feasibility of the operations required for tapping pine trees by an industrial robot using the borehole method and represents a first step in demonstrating the fully autonomous harvesting process.

Traditionally, the work required for tapping slash pine trees and collecting their oleoresin using the borehole method includes the following activities: (i) identifying slash pine trees having a trunk of at least 8 inches in diameter at chest height; (ii) drilling three converging boreholes in the tree trunk at angles that allow the flow of oleoresin due to gravity; (iii) manually spraying the three holes with resin flow stimulating chemicals and the pine tree trunk with insecticide; (iv) manually tapping one of the holes with a polymer pipe connected to a collection vessel and (v) manually capping the remaining two holes to prevent them from desiccating and the turpentine from evaporating. These activities occur at the beginning of the oleoresin harvesting season when a worker typically taps hundreds of pine trees daily. State-of-the-art attempts to automate the process consist of hydraulic drills and spray nozzles attached to a delivery vehicle such as a tractor or bobcat driven and actuated by a human operator [14]. Besides drilling and spraying the holes, the rest of the operations are performed manually. To the best of our knowledge, there have been no previous attempts to fully automate the process using the borehole method. However, we notice the demonstration of harvesting latex from rubber trees in China using a robot mounted on an autonomous rover using the bark removal method [15].

The demonstration of the automated technology requires addressing two major but unrelated challenges: (a) the demonstration of the industrial robot’s capability to perform the tapping operations in conditions of large variations in the actual shape of the pine tree trunk and variations in the relative position between the robot and the tree after the rover places itself in front of it and (b) the demonstration of the rover’s capability to identify mature slash pine trees and to navigate towards them.

This paper focusses on presenting the challenges related to performing the robotic operations required for tapping slash pine trees and collecting their oleoresin for processing in conditions of large variations in the shape of the tree trunk and in the relative position between the industrial robot and the tree after the rover on which the robot is mounted reaches the destination, as well as the technical approach used to address these challenges. The paper presents the experimental setup, the strategy and the results used to demonstrate the robotic operations. The robotic operation cycle includes loading the spindle from the tool stand and drilling three converging boreholes in the pine tree at angles that allow the flow of oleoresin due to gravity; replacing the spindle with a nozzle and spraying the boreholes with resin flow stimulating chemicals and insecticide; replacing the nozzle with a 3-fingers gripper, picking a plastic tube with pre-attached collection bag and tapping one borehole, then picking two plugs and capping the other boreholes. The strategy for identifying mature slash pine trees and the rover’s autonomous navigation will be presented elsewhere.

2. Materials and Methods

2.1. Experimental System

The robot operations were demonstrated in the laboratory using a Fanuc LR Mate 200iD robot with R-30iB controller mounted on a fixed test stand built in house (Figure 2) and equipped with a TSS tool stand (ATI Industrial Automation) for automated tool changing, a fixture for attaching a pine tree log, a compressor and safety panels.

The laboratory experiments were intended to demonstrate the capability of the industrial robot to perform the manufacturing operations required for tapping slash pine trees in conditions of inherent low positional accuracy between the robot and the pine tree on which it operates after the autonomous rover reaches the target. The laboratory experiments were performed under controlled conditions that replicated the operating conditions in the field and using real pine tree logs. The variations in the pine tree log diameter and in the relative position between the log and the industrial robot were variables controlled in the laboratory experiments. The manufacturing operations were considered successful if the robot proved capable of drilling three converging 6-inch-deep holes without jamming or breaking the cutting tool, replacing the cutting tool with a nozzle, spraying the drilled holes and the tree trunk with liquids, replacing the nozzle with a robotic hand and using it to successfully insert a tube in one of the holes and two plugs in the remaining holes. The insertions had to be snug and obtained with a sufficient force to prevent the separation or air penetration into the holes. The experiments were also used to acquire information for further process optimization and minimization of the operation cycle.

The robot tools were designed and integrated in house and consist of the following: (i) a three-fingers robotic hand (Figure 3) for picking up a PVC tube with pre-attached collection bag and two plugs and for inserting them in the boreholes; (ii) a spindle with a cutting tool (Figure 4) for drilling three converging boreholes and (iii) a two-nozzle sprayer (Figure 5) for spraying the boreholes and the tree trunk.

The robotic hand (Figure 3) consists of a DPZ+ 50-1 three-fingers centric gripper with SWA011 adapter plate (Schunk), a QC11 quick connect with tooling interface plate (ATI Industrial Automation) and three fingers fabricated in house using additive manufacturing on a Stratasys J750 Polyjet 3D printer. Two different ratio blends of VeroBlackPlus^TM rigid photopolymer with a Shore hardness value of 83–86 (scale D) and Agilus30^TM flexible photopolymer with a Shore hardness value of 30–35 (scale A) were used in a single printing process to fabricate the robot fingers having structured mechanical properties. A ratio of 90%/10% VeroBlackPlus^TM/Agilus30^TM was used to obtain the rigid fingers and a ratio of 20%/80% was used to obtain the soft fingertips. The prototype fingers thus obtained were capable to withstand the insertion forces, provided a good grasp of the tubes and prevented their slippage during the insertion process.

The drilling tool (Figure 4) consists of a 500 W, 24–50 VDC spindle attached to an aluminum angle plate, cutting tool, QC11 quick connect and tooling interface plate (ATI Industrial Automation).

The spraying tool (Figure 5) consists of two nozzles attached to an angle plate, QC11 quick connect and tooling interface plate (ATI Industrial Automation).

The robot end-of-arm is equipped with an Axia80 EtherNet force/torque sensor (ATI Industrial Automation) attached through an interface plate and a QC11 master quick connect for changing the tools. The sensor communicates with a roboRIO (National Instruments) real time target on which the LabView programs have been deployed. roboRIO communicates with the Fanuc R-30iB controller through digital input/outputs (DI/DO).

2.2. Challenges and Strategies to Address Them

While trivial to human operators, there are significant challenges when executing the operations required for tapping slash pine trees by an industrial robot. These challenges derive from using a 6-degrees of freedom (DOF) serial link manipulator with all revolute joints having a limited payload (7 kg in the case of Fanuc LR Mate 200iD) and rigid joints. The first significant challenge consists in the requirement to programmatically align the axis of insertion (for example, the axis of the tube) with the axis of the borehole and then to move the tool/tube along this axis. A second significant challenge derives from the large variations in both the shape of the pine tree trunk and in the relative position between the robot and the tree after the rover positioned itself in front of it. This implies that the position of the borehole entry point that needs to be drilled is not known a priori, and therefore needs to be determined programmatically. This is particularly important during the process of inserting the tube with collection bag or the two plugs in the boreholes, since misestimation of its position will lead to either an incomplete insertion or may result in breaking the tool. A third challenge consists in the requirement to insert the tube and plugs in the borehole with a predetermined force.

The first challenge is addressed by predefining tool frames associated to each of the three tools using the tree-point method [16] and predefining user frames oriented in the direction of the boreholes that need to be drilled, sprayed or tapped but having axes parallel to the tool frames (Figure 6). In this case, all tool frames are parallel to each other and have the Z-axis perpendicular to the robot end-of-arm (EOA), and the user frame has axes parallel to the tool frames and Z-axis aligned to the axis of the borehole. Each robot operation is programmed to move the robot EOA using the tool frame associated to the loaded tool and the user frame associated to the borehole that needs to be drilled, sprayed or tapped. This method ensures that the axis of the tool/tube is oriented along the axis of the borehole and that it is precisely inserted along it, minimizing thus the chance of being jammed.

The tool attached to the robot EOA is programmed to move along its axis, but the entry point of the borehole is not known a priori. This is particularly important during the process of inserting the tube with collection bag or the two plugs, since misestimation of its position will lead to either an incomplete insertion or may result in breaking the tool. This problem was addressed by attaching an Axia80 EtherNet force/torque sensor (ATI Industrial Automation) to the robot EOA to identify the moment when the tip of the drilling tool touches the tree surface. The position and orientation of the tool center point (TCP) relative to the robot’s world frame is recorded when a component of the measured force or torque exceeds a threshold value. Recording of the TCP pose during robot operation is possible using global position registers (PR[]) of Fanuc’s programming language. The position and orientation components of a PR[] can be recorded or changed programmatically. To identify a meaningful change in the value of any force/torque components when the drilling tool touches the tree, experiments were run using various tools and feed rates.

A typical result for the T_Z component of the torque is shown in Figure 7. Note that the magnitudes of the force/torque components depend both on the type of tool and on the drilling feed rate.

Data shown in Figure 7 were acquired at a rate of 10 Hz. The threshold absolute value for the T_Z component of the torque indicating the moment when the cutting tool touches the tree was further considered to be 2.5 Nm.

The overall system architecture for identifying the borehole entry point is shown in Figure 8. Axia80 sensor communicates through ethernet cable to a roboRIO real time target (National Instruments) on which the LabView program was deployed. The program scans the value of the T_Z torque component and turns on a digital output (DO1) when |T_Z| ≥ 2.5 Nm. DO1 of the roboRIO target is connected to DI101 of the R-30iB robot controller.

The robot subroutine for drilling runs a loop in which the TCP of the drill moves in linear motion in increments of 1 mm along the axis of the cutting tool and scans the value of DI101. The loop is repeated until the TCP touches the tree which is indicated by DI101 turning on. In this moment, the pose of the TCP (this will be the borehole entry point) is recorded in PR [17], and the drill proceeds to cut a borehole of 152 mm (6 inch). The block diagram of the subroutine is shown in Appendix A. The algorithm used to programmatically identify the starting point of the borehole and to cut it is applied to all three boreholes.

We note two alternative methods typically used in industrial robotic applications when cylindrical components must be inserted in holes during assembly processes: the use of machine vision to locate the hole entry point [17,18,19,20] or the use of an end-effector with compliance system [21,22,23,24,25,26,27] that compensates for misalignments and/or limited robot accuracy. It has been concluded that for the present application, machine vision would not identify the holes with sufficient positional accuracy due to the irregular shape of the hole contour (irregular bark texture). Furthermore, the method adopted in this work proved sufficiently robust and accurate to make the use of compliance systems unnecessary.

Using the procedures shown above for controlling the trajectory and orientation of the tools during operations and for identifying the borehole entry point enables the robot to precisely insert the tube with the collection bag, or the plugs in the boreholes. The challenge to perform these last operations consists in controlling the insertion force and avoiding an incomplete insertion or breaking the tool. This challenge is addressed by using the Axia80 EtherNet force/torque sensor to control the moment the 3-finger gripper opens to release the tube/plugs when the insertion force, F_Z reaches a threshold value. The F_Z component of the force was monitored during a cycle starting with the moment when the robot releases the loaded tool, loads the 3-finger gripper to the EOA, picks a tube and inserts it in the borehole (Figure 9).

The threshold value of the force was progressively increased until a strong insertion was obtained, and the possibility of the gripper to trigger open before the cycle ended was eliminated. An insertion force of −125 N was selected.

Axia80 sensor communicates the instant values of the force/torque components to roboRIO on which the LabView program was deployed. The program scans the value of F_Z and turns on a digital output (DO0) when F_Z ≤ −125 N. DO0 of the roboRIO target is connected to DI102 of the R-30iB robot controller (Figure 8). The LabView project for monitoring the insertion force and for triggering the output to the robot controller uses a binary shared variable, DO0, and has virtual instruments (VIs) deployed on the driver station computer for monitoring the force (Figure 10a) and on roboRIO target for triggering the output to the robot controller (Figure 10b).

The robot subroutine for insertion operations runs a loop which inserts the tube in the borehole using a linear motion in increments of 1 mm and scans the value of DI102. The loop is repeated until DI102 turns on, moment at which a robot output, RO2, opens the gripper fingers to release the tube, and the gripper withdraws. The block diagram of the robot program used for insertion operation is shown in Appendix B.

3. Results and Discussion

The strategies described in the previous section to programmatically identify the points of insertion and to control the toolpaths and the insertion force proved successful and robust. The full operation cycle for tapping slash pine trees has been successfully demonstrated. It was found that the operation cycle time is limited by vibrations of the robotic arm induced by cutting forces during the drilling operations.

To optimize the drilling operation, different cutting tools were tested for drilling 1.25-inch diameter, 6-inch-deep holes in the pine tree, including a 1.25-inch diameter auger with centering tip; a 1.25-inch diameter spade drill with centering tip; a 3/8-inch diameter, 2-flute ball mill and a 1.25-inch diameter, 4-flute end mill modified by adding a self-locking centering tip. In the latter case, a 0.2-inch diameter, ¾-inch deep hole was initially cut in the center face of the end mill, followed by tapping a ¼”-20 LH thread. A 5-axes Sodick AD 35L sinker electrical discharge machine (EDM) with graphite/Ti tools was used for both operations. A centering pin with ¼”-20 LH male thread was machined from SS 316 steel on a Haas ST10 CNC lathe and attached to the 1.25-inch end mill. When the 3/8-inch ball mill was used, the toolpath was modified by adding circular motions to obtain 1.25-inch-diameter holes. In the case of each cutting tool, the experimental drilling operations started with maximum spindle rotations per minute (RPM) and feed rate. Failures consisted in jamming the tool due to vibrations in the robot’s 4th joint that generated chatter. In case of failure, the cutting feed rate was progressively decreased to lower the cutting forces. It has been concluded that all the cutting tools with the exception of the 1.25-inch spade drill with centering tip generated excessive cutting forces that resulted in jamming the tool. The optimum cutting time for drilling one hole using the 1.25-inch spade drill was approximately 7 min, which resulted in a total operation cycle time of approximately 25 min. To further optimize the operation cycle time, either a different, stronger robot could be used, or the drilling operations could be performed by adding linear slides with cutting spindles attached on them to replace the robot drilling.

To optimize the insertion operation, experiments were performed running the robot program with progressively increased motion speed and motion termination types. Figure 11 shows the force component F_Z during the tube insertion in the borehole.

When using FINE motion termination (the robot stops after each motion instruction), a limiting insertion speed was reached beyond 30 mm/s. The insertion speed could be increased by using CNT100 motion termination (the robot does not stop after each motion instruction). However, the higher the insertion speed, the more likely it is to exceed the threshold of the insertion force and the greater the danger of the gripper colliding with the tree. A robot motion speed of 100 mm/s with CNT100 motion termination was selected for the operation.

4. Conclusions

In this paper we demonstrate for the first time the feasibility of the manufacturing operations necessary for tapping slash pine trees and collecting their oleoresin for processing using the borehole method by an industrial robot. This represents a first step in the demonstration of a fully autonomous technology intended to revive the naval stores industry in southeast United States.

The paper discusses the challenges related to performing the robotic operations in conditions of large variations in the shape of the tree trunk and in the relative position between the industrial robot and the tree after the rover on which the robot is mounted reaches the destination, as well as the technical approach used to address these challenges. We present the experimental setup, the strategy and the results used to demonstrate the robotic operations. The robotic operation cycle includes loading a spindle from a tool stand and drilling three converging boreholes in the pine tree at angles that allow the flow of oleoresin due to gravity; replacing the spindle with a nozzle and spraying the boreholes with resin flow stimulating chemicals and insecticide; replacing the nozzle with a 3-fingers gripper; and picking a plastic tube with pre-attached collection bag and tapping one borehole and then picking two plugs and capping the other boreholes.

The first challenge consists in the requirement to programmatically align the axis of insertion (for example, the axis of the tube) with the axis of the borehole and then to move the tool/tube along this axis without jamming it. This challenge is addressed by predefining tool frames associated to each of the three tools and predefining user frames oriented in the direction of the boreholes that need to be drilled, sprayed or tapped but having axes parallel to the tool frames. In this case, all tool frames are parallel to each other and have the Z-axis perpendicular to the robot EOA, and the user frames have axes parallel to the tool frames and the Z-axis aligned to the axis of the borehole. This method ensures that the axis of the tool/tube is oriented along the axis of the borehole and that it is precisely inserted along it, minimizing thus the chance of being jammed. A second significant challenge derives from not knowing a priori the position of the borehole entry points and therefore from the need to determine them programmatically. This challenge was addressed by attaching a force/torque sensor to the robot EOA to identify the moment when the tip of the drilling tool touches the tree surface. The position and orientation of the TCP relative to the robot’s world frame is recorded when a component of the measured force or torque exceeds a threshold value. Recording programmatically the TCP pose during robot operation is thus possible using global position registers of Fanuc’s programming language. A third challenge consists in the requirement to insert the tube and plugs in the boreholes with a predetermined force. This challenge was addressed by using the force/torque sensor to control the moment the 3-finger gripper opens to release the tube/plugs when the insertion force reaches a threshold value.

The full operation cycle for tapping slash pine trees was successfully demonstrated in the laboratory under controlled conditions that replicate the operation conditions in the field. It was found that the operation cycle time is limited by vibrations of the robotic arm induced by cutting forces during the drilling operations. The optimum cutting time for one hole was approximately 7 min, which resulted in a total operation cycle time of approximately 25 min. To further optimize the operation cycle time, either a different, stronger robot can be used, or the drilling operations could be performed by adding linear slides with cutting spindles attached on them to replace the robot drilling. The full manufacturing operation cycle remains to be validated in field.

5. Patents

The U.S. Patent Application No. 17/331,124 (ref. [1]) which discloses the technology presented in this article has been filed with US Patent and Trademark Office on 26 May 2021.