Prediction Model for the Oscillation Trajectory of Trellised Tomatoes Based on ARIMA-EEMD-LSTM

Abstract

Second-order damping oscillation models are incapable of precisely predicting superimposed and multi-fruit collision-induced oscillations. In view of this problem, an ARIMA-EEMD-LSTM hybrid model for predicting the oscillation trajectories of trellised tomatoes was proposed in this study. First, the oscillation trajectories of trellised tomatoes under different picking forces were captured with the aid of the Nokov motion capture system, and then the collected oscillation trajectory datasets were then divided into training and test subsets. Afterwards, the ensemble empirical mode decomposition (EEMD) method was employed to decompose oscillation signals into multiple intrinsic mode function (IMF) components, of which different components were predicted by different models. Specifically, high-frequency components were predicted by the long short-term memory (LSTM) model while low-frequency components were predicted by the autoregressive integrated moving average (ARIMA) model. The final oscillation trajectory prediction model for trellised tomatoes was constructed by integrating these components. Finally, the constructed model was experimentally validated and applied to an analysis of single-fruit oscillations and multi-fruit oscillations (including collision oscillations and superposition oscillations). The following experimental results were yielded: Under single-fruit oscillation conditions, the prediction accuracy reached an RMSE of 0.1008–0.2429 mm, an MAE of 0.0751–0.1840 mm, and an MAPE of 0.01–0.06%. Under multi-fruit oscillation conditions, the prediction accuracy yielded an RMSE of 0.1521–0.6740 mm, an MAE of 0.1084–0.5323 mm, and an MAPE of 0.01–0.27%. The research results serve as a reference for the dynamic harvesting prediction of tomato-picking robots and contribute to improvement of harvesting efficiency and success rates.

1. Introduction

Tomato is one of the most extensively cultivated vegetables of the Solanaceae family in the world. In China, the planting area ratio of greenhouse tomato cultivation to traditional tomato cultivation is 3:1. Greenhouse cultivation has become the prevailing method of tomato production because it markedly enhances space utilization and fruit quality [1,2,3]. However, the flexible elasticity of hanging vine ropes induces persistent nonlinear oscillations of trellised tomatoes during harvesting. Traditional harvesting robots, devoid of efficient oscillation prediction algorithms, are frequently compelled to cease operations until the fruit reaches a state of repose. This predicament severely curtails their operational efficiency [4,5]. Although recent studies have made progress in machine vision and path planning, the real-time prediction of the pose of dynamically oscillating fruits still faces considerable challenges [6]. Consequently, there is an urgent need for a high-precision oscillation prediction trajectory model.

Dynamic harvesting technology is one of the most effective approaches to improving the efficiency and success rate of robotic harvesting [7,8,9]. At present, common dynamic harvesting methods include vision-based dynamic localization and physics-based predictive harvesting. Among them, vision-based dynamic localization has been the most extensively researched [10,11]. Visual sensing and deep learning were combined to achieve automatic segmentation of fruits, trunks, branches, and trellis ropes [12,13]. On this basis, a visual system for a sweet pepper harvesting robot was studied by integrating perception with the planning of harvesting operations [14]. In the study conducted by Lv et al. [15], a novel approach was proposed for the rapid localization and picking of oscillating fruits by an apple-picking robot. Their approach dynamically captured images of oscillating fruits to identify and extract centroid coordinates, used fast Fourier transform (FFT) modeling to obtain oscillation periods, and calculated the prismatic joint stroke velocity based on depth data. As a result, the fruits were grasped at equilibrium positions, and their experiments achieved a picking success rate of 84%. Concurrently, Yang et al. [16] proposed a video analysis-based method for tracking the trajectories of oscillating fruits, as well as an automatic fruit target recognition method based on the Luma, Chroma blue, Chroma red (YCbCr) color space. By applying the Meanshift algorithm, the researchers tracked the oscillatory trajectories and subsequently derived the velocity and acceleration curves of the fruits from the tracking results, which laid the groundwork for determining the optimal harvesting time. During the picking process, when the target fruit is subjected to vibrations, the visual recognition system will experience processing latency, which frequently brings about unsuccessful picking. Machine vision dynamic tracking necessitates a rapid response from the robotic arm; however, the oscillatory movement of fruits exhibits a reciprocating motion, which forces the robotic arm to cover excessive and unnecessary distances during tracking. Such an inefficient movement may result in reduced efficiency or even tracking failure. The second approach involves the construction of a second-order damped system physical model for the purpose of predicting oscillation trajectories. For instance, the spring-mass-damped model of the grape fruit-stem system, utilized by Zhang et al. [17], essentially constituted a multi-degree-of-freedom second-order system. The oscillatory behavior of this model was characterized by its resonant response and the damping-induced attenuation of vibrations. Similarly, the flexible body model of the coffee fruit-stem system, analyzed by Villibor et al. [18], could also be regarded as a multi-degree-of-freedom second-order system for predicting oscillation patterns. This model elucidated the resonance characteristics of the system during oscillation, as well as the influences of frequency and amplitude on the response. However, they did not establish a connection between these findings and the actual motion state of the fruit. The oscillations of tomatoes can be categorized into two types, i.e., single-fruit oscillations and multi-fruit oscillations (including superimposed oscillations and collision oscillations). Single-fruit oscillations have been shown to present oscillation patterns analogous to those observed in second-order systems. However, given the variability in tomato mass and vine thickness, the employment of a second-order system for prediction may lead to inaccuracies.

To address this limitation, a growing number of researchers have applied machine learning and data-driven algorithms to motion prediction problems. For example, missile trajectories are often characterized by trajectory variation and abrupt changes. Traditional mathematical modelling methods may fail to accurately predict trajectories. In the study by Sun et al. [19], a long short-term memory network (LSTM)-based pipeline trajectory prediction model was constructed. This model represented the target altitude, velocity, and direction of velocity as analytical functions incorporating linear decay terms and amplitude-decaying sine terms. By dynamically updating parameter estimates, the model achieved trajectory prediction for hypersonic gliding targets and provided velocity vector information for interceptor guidance. Meanwhile, Gao et al. [20] proposed a CORR-CNN-BiLSTM-Attention model, which integrated convolutional neural network (CNN) feature extraction, bidirectional long short-term memory (BiLSTM) temporal analysis, attention mechanisms, and recursive correction of radar return data, to achieve high-precision prediction of missile trajectories and launch points. Xu and Wu [21] put forward an autoregressive integrated moving average (ARIMA)-based tracking algorithm, namely the ARIMA-AT prediction model, for autonomous underwater vehicles (AUVs). In comparison to these motion patterns, tomato oscillations manifest a state in which linear and nonlinear components are superimposed. Consequently, the direct application of trajectory prediction algorithms designed for the aforementioned systems to tomato oscillation prediction would result in ineffective predicted trajectories.

In light of the intricate trajectories of oscillatory movements observed in greenhouse trellised tomatoes, researchers have conducted some beneficial explorations: Motivated by the efficacy of ARIMA in modelling and predicting non-stationary time series [22,23] and the aptitude of LSTM in anticipating nonlinear trends, researchers decomposed ARIMA and LSTM with the aid of the empirical mode decomposition (EMD) or ensemble empirical mode decomposition (EEMD) techniques. Wang et al. [24] developed the EMD/EEMD-ARIMA hybrid model for realizing long-term runoff prediction. This model decomposed the original hydrological time series into components with distinct temporal characteristics by EMD or EEMD and then forecasted each component individually using ARIMA. Moreover, Yan et al. [25] proposed an airflow velocity prediction hybrid model combining seasonal autoregressive integrated moving average (SARIMA), EEMD, and LSTM. Their prediction model demonstrated high prediction accuracy.

In summary, existing visual recognition and physical models for dynamic localization are insufficient for predicting fruit motion, as they lack a detailed exploration into the oscillation trajectories of fruits and thus fail to provide an effective method for dynamic prediction during robotic harvesting. To overcome these limitations, an ARIMA-EEMD-LSTM hybrid model for predicting the oscillation trajectories of greenhouse trellised tomatoes was proposed in this study. The EEMD decomposition process was responsible for segmenting oscillation signals into multiple intrinsic mode function (IMF) components. The high-frequency and low-frequency IMF components were predicted by using the LSTM model and the ARIMA model respectively, and the final oscillation trajectory prediction model for trellised tomatoes was constructed by integrating these IMF components. Ultimately, comprehensive oscillation data of trellised tomatoes under various experimental conditions, including single-fruit oscillations and multi-fruit oscillations (collision oscillations and superimposed oscillations), were collected with the aid of a motion capture system, and the accuracy of the proposed oscillation prediction model was verified through training and test sets. The proposed model provides a theoretical foundation for dynamic prediction and intelligent harvesting of tomatoes by robotic systems. This research represents the first attempt to model the oscillation trajectory of tomatoes on living plants, bridging a gap in existing research. Moreover, it offers valuable insights for the mechanized dynamic harvesting of other fruits and contributes to the advancement of modern agricultural robotics.

2. Materials and Methods

2.1. Analysis of the Oscillation Characteristics of Trellised Tomatoes

The experimental subject selected for this study is the ‘Dutch No. 9’ tomato variety, which is widely cultivated in Jiangsu Province, China. As displayed in Figure 1, the trellised tomato vines have an average height of 1.8–2.0 m and a diameter of about 10.0 mm. Each vine is bound and twined around a trellising rope, which is secured to a rigid upper beam via an iron hook. The lower end of the trellising rope is wrapped and tied to the base of the tomato stem and anchored to a water pipe positioned on the soil surface. An extension allowance of 15.0–20.0 mm is left to permit natural movement. The overall fixation method of the tomato plant is illustrated in Figure 1a,b. A single mature tomato fruit weighs 500–700 g and measures about 60.0 mm in diameter on average. Thanks to the adoption of the single-stem pruning technique, each plant produces 5–6 fruit clusters, with each cluster bearing 1–6 tomatoes. According to statistical analysis on fruit distribution per cluster based on a population of 2800 trellised tomato plants per mu (about 666.67 m²), 4 and 5 tomatoes per cluster are the most frequent counts, accounting for 45% and 27% of the total, respectively; clusters with 3 tomatoes represent 15% of the total; and the rest two categories each constitute less than 5%.

Tomatoes, as members of the Solanaceae family and Dicotyledonous plants, show the characteristics typical of such plants. The fruits grow in clusters, each bearing multiple tomatoes with varying degrees of ripeness, as depicted in Figure 2. Growth nodes are usually observed at the base of the fruit (Figure 2c). Since the tensile strength at node A in Solanaceae plants varies in different growth stages, tomatoes are typically harvested from node A in the semi-ripe stage. The distance from the terminal point C of the upper branch to the terminal point B of the tomato petiole generally ranges from 15 mm to 25 mm. During robotic harvesting, contact between adjacent tomatoes may occur, potentially leading to fruit collisions.

2.2. Prediction Model for the Oscillation Trajectories of Trellised Tomatoes

The ARIMA model boasts high accuracy in forecasting linear trends, but it performs poorly when dealing with nonlinear dynamics. In contrast, the LSTM model excels at forecasting nonlinear trends [26,27,28], which effectively compensates for the limitations of ARIMA in nonlinear prediction. The oscillation characteristics of tomatoes can be categorized into single-fruit and multi-fruit oscillations (including superimposed and collision oscillations). Single-fruit oscillations feature stable frequency and linear behavior, suitable for the ARIMA model; superimposed oscillations feature unstable frequencies and nonlinear trends, better captured by the LSTM model; and multi-fruit collision oscillations combine both linear and nonlinear characteristics. Finally, in order to solve the inability of the traditional second-order damped oscillation prediction model to accurately predict multi-fruit superimposed oscillations and collision oscillations, EEMD-based optimization is incorporated in the classification process. Leveraging the complementary strengths of these three approaches, this study proposes an ARIMA-EEMD-LSTM hybrid model. The specific steps of the algorithm are as follows.

Step 1: data preparation. The original dataset $Y_t$ of trellised tomato oscillation trajectories, collected by the Nokov motion capture test platform, is read by the XINGYING 3.4.0.4088 software, divided into the training and test subsets, and subsequently normalized for model processing.

Stage 2: EEMD decomposition. The original dataset $Y_t$ of trellised tomato oscillation trajectories is decomposed into a sum of several IMFs and a residual component $r(t)$ by EEMD. The EEMD process is expressed as Equation (1):

$$Y_t={\displaystyle {\sum }_{i=1}^{n}IM{F}_i(t)+r(t)}$$

(1)

where $IM{F}_i(t)$ is the IMF components corresponding to different frequency bands of the original dataset, and $r(t)$ is the residual component containing minor trend features that remain after the extraction of IMFs. After decomposition, the IMF components can be obtained and then visualized.

Step 3: component classification. The energy and cumulative energy of each IMF component are calculated. By setting the critical point of cumulative energy to 0.8, components with cumulative energy below 0.8 are classified as high-frequency components, while those with cumulative energy equal to or above 0.8 are classified as low-frequency components.

Step 4: model prediction. High-frequency components are predicted by the LSTM model, while low-frequency components are predicted by the ARIMA model.

(a) The ARIMA prediction model is employed to predict the linear portion of a time series through autoregressive and moving average processes. The linear trend and periodicity of the data are characterized by the autoregressive order ($p$), difference order ($d$), and moving average order ($q$). The mathematical expression of the ARIMA ($p$, $d$, $q$) prediction model for handling linear trends is given as Equation (2):

$$\varphi (B){(1-B)}^d{Y}_t=\theta (B){\epsilon }_t$$

(2)

where $Y_t$ is the original dataset of oscillation displacements of trellised tomatoes; $d$ is the difference order, which stabilizes the time series of the ARIMA prediction model; and ${\epsilon }_t$ is the residual term that presents the statistical characteristics of white noise after model fitting.

Equation (3) is an autoregressive polynomial.

$$\varphi (B)=1-{\varphi }_{1}B-{\varphi }_2{B}^2-\cdots -{\varphi }_p{B}^p$$

(3)

Equation (4) is a moving average polynomial.

$$\theta (B)=1+{\theta }_{1}B+{\theta }_2{B}^2+\cdots +{\theta }_q{B}^q$$

(4)

The optimal parameter combination for the ARIMA prediction model is selected as ($p$, $d$, $q$) = (1, 0, 0). Next, the ARIMA mathematical model is trained using this configuration and applied in a rolling prediction manner to estimate the low-frequency components.

(b) The following section will examine the LSTM prediction model. The LSTM network is employed to process the residual sequence subsequent to ARIMA model prediction, namely, the nonlinear components not encompassed by the linear model. Thanks to its gating mechanism (input, forget, and output gates), the LSTM network is able to remember and process long-term and short-term dependencies and capture complex nonlinear features. The mathematical formulations of the input, forget, and output gates are given in Equations (5)–(7):

$$i_t=\sigma (W_i\left[h_{t-1},x_t\right]+b_i)$$

(5)

$$f_t=\sigma (W_f\left[h_{t-1},x_t\right]+b_f)$$

(6)

$$o_t=\sigma (W_o\left[h_{t-1},x_t\right]+b_o)$$

(7)

where $i_t$ is the process value of the input gate; $x_t$ is the input value at time $t$; $\sigma$ is the sigmoid function; $W_i\left[h_{t-1},x_t\right]$ and $b_i$ are the weight matrix and bias vector of the input gate for the $i$-th input window, respectively. Similarly, $f_t$ is the process value of the forget gate at time $t$. The forget gate, which acts on the previous memory state, allows neurons to selectively remember or forget past information. $W_f\left[h_{t-1},x_t\right]$ and $b_f$ are the weight matrix and bias vector of the forget gate for the $f$-th input window, respectively. $o_t$ is the process value of the output gate at time $t$, and $W_o\left[h_{t-1},x_t\right]$ and $b_o$ are the weight matrix and bias vector of the output gate for the $o$-th input window, respectively.

The cell state update $\widetilde{C_t}$ and output $C_t$ values of the input gate are calculated using Equations (8) and (9):

$${\stackrel{\sim }{C}}_t=\tanh (W_C\left[h_{t-1},x_t\right]+b_C)$$

(8)

$$C_t=f_t\odot C_{t-1}+i_t\odot \widetilde{C_t}$$

(9)

where $\widetilde{C_t}$ is the updated cell state at time $t$; $C_t$ is the output value at the current cell state; $\tanh$ is the $\tanh$ function; $\odot$ is the Hadamard product; $W_C\left[h_{t-1},x_t\right]$ and $b_C$ are the weight matrix and bias vector corresponding to the update state of the $c$-th input gate unit.

The output value $h_t$ of the output gate at time $t$ is computed as Equation (10):

$$h_t=o_t\odot \tanh (C_t)$$

(10)

All predicted values corresponding to the input windows are then calculated, and model training options are configured. At last, rolling prediction with the LSTM prediction model was performed on high-frequency IMF components.

The final predicted value $\stackrel{\wedge }{Y_t}$ is obtained by summing the predictions from both the ARIMA and LSTM models, as expressed in Equation (11):

$${\stackrel{\wedge }{Y}}_t={\displaystyle {\sum }_{i=1}^{n_1}ARIMA(IM{F}_i)}+{\displaystyle {\sum }_{j=n_1+1}^{n}LSTM(IM{F}_j)}+ARIMA(r)$$

(11)

where $n_1$ is the number of low-frequency IMF components predicted by the ARIMA model; $ARIMA(\cdot )$ and $LSTM(\cdot )$ are the corresponding prediction functions. Finally, the performance of the model is assessed by calculating metrics such as root mean square error (RMSE), mean absolute percentage error (MAPE), and mean absolute error (MAE), and predictions are compared against actual values to analyze prediction errors.

The overall control method and steps are illustrated in Figure 3. The collected datasets are imported into MATLAB R2023b for simulation, and the trajectory prediction is conducted using the ARIMA-EEMD-LSTM hybrid model. The datasets are normalized initially and then subjected to EEMD decomposition for the purpose of obtaining IMF components. Afterwards, the energy of each IMF component is calculated. Components with cumulative energy below 0.8 are classified as high-frequency components and predicted by the LSTM model with rolling prediction. Those with cumulative energy equal to or above 0.8 are treated as low-frequency components and predicted by the ARIMA model with rolling prediction. Eventually, the LSTM algorithm prediction on high-frequency components and the ARIMA algorithm prediction on low-frequency components are superimposed to generate the oscillation trajectory prediction. The algorithm flowchart is shown in Figure 3.

2.3. Test Instruments and Equipment

The oscillation trajectory acquisition system for trellised tomatoes, based on the ARIMA-EEMD-LSTM hybrid model, uses the Nokov motion capture system (Beijing, China) (Figure 4a,b). By attaching markers to the trellised tomato fruits, their movements are tracked in real time by multiple infrared cameras. The oscillation trajectories of these marker balls are processed with the aid of XINGYING software (Version 3.4.0.4088) and converted into precise oscillation trajectory datasets for the trellised tomato fruits. The results make it possible to accurately measure the movement trajectory of the soft movement state of the tomato vines. The motion capture camera utilized in this study is provided in Figure 4c, and its performance parameters are listed in

Table 1. Primary parameters of motion capture camera equipment.

Model	Resolution	Number of Pixels	Frequency (Hz)	Delay (ms)	Field of View (°)	Lens (mm)
Mars2H	2048 × 1088	2.2 MP	380	2.8	56.3 × 43.7	8

. The initial step in acquiring the original datasets is to establish the coordinate systems of the motion capture system and the harvesting site (Figure 4a,d). It is noteworthy that the system consists of eight Mars2H motion capture cameras. The eight cameras are positioned at four corners, with two cameras at each corner. The distance between adjacent corners is 4 m. Each camera is mounted on a stand at a height of 3 m. The trellised tomato plants are placed at the center of the test site and secured with ropes to a 2-m-high stand. Each tomato is marked with a marker ball.

2.4. Evaluation of Trajectory Metrices

In the prediction of picking-induced oscillations, the objective is to enable the model to achieve more accurate and rapid prediction while minimizing the number of known data points. Accordingly, the present study employs the RMSE [29], MAE [30], and MAPE [31] to evaluate the prediction accuracy of the model.

(1) The RMSE is a frequently employed metric for quantifying the discrepancy between numerical values. Its formula is given as Equation (12):

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}{\displaystyle {\sum }_{i=1}^n{(y_i-{\widehat{y}}_i)}^2}}(\mathrm{mm})$$

(12)

where $n$ denotes the number of original datasets of trellised tomato oscillation trajectories; $y_i$ represents the actual value of the $i$-th original dataset; and ${\widehat{y}}_i$ is the predicted value from the ARIMA-EEMD-LSTM hybrid model. A smaller RMSE value is indicative of higher prediction accuracy of the model.

(2) The MAE is defined as the average of the absolute errors between predicted and actual values. Its formula is given as Equation (13):

$$MAE={\displaystyle \frac{1}{n}}{\displaystyle {\sum }_{i=1}^n\left|y_i-{\widehat{y}}_i\right|}(\mathrm{mm})$$

(13)

where $n$ denotes the number of original datasets of trellised tomato oscillation trajectories; $y_i$ represents the actual value of the $i$-th original dataset; and ${\widehat{y}}_i$ is the predicted value from the ARIMA-EEMD-LSTM hybrid model. A smaller MAE value marks higher prediction accuracy of the model.

(3) The MAPE is a metric used for quantifying the average relative error between the predicted and actual values. Its formula is given as Equation (14):

$$MAPE={\displaystyle \frac{1}{n}}{\displaystyle {\sum }_{i=1}^n\left|\frac{y_i-{\widehat{y}}_i}{y_i}\right|}\times 100\%$$

(14)

where $n$ denotes the number of original datasets of trellised tomato oscillation trajectories; $y_i$ represents the actual value of the $i$-th original dataset; ${\widehat{y}}_i$ is the predicted value from the ARIMA-EEMD-LSTM hybrid model; and the absolute value $\left|{\displaystyle \frac{y_i-{\widehat{y}}_i}{y_i}}\right|$ stands for the absolute percentage error of the $i$-th original dataset. A smaller MAPE value corresponds to higher prediction accuracy of the model.

3. Test Results and Analysis

3.1. Prediction of Single-Fruit Oscillation Trajectory

Datasets of single-tomato oscillation trajectory were collected by using the oscillation trajectory acquisition system (Figure 5). A horizontal picking force of 13.25 N was applied to tomato B to pick it, while the oscillation trajectory data of tomato A along the X-axis were acquired. The oscillation trajectory of tomato A is provided in Figure 5a. A total of 1488 datasets were collected over a time period from 0 s to 24.76 s. Initially, the datasets were divided into five subsets with different training intervals and corresponding prediction time intervals (

Table 2. Prediction accuracy of single-tomato oscillation trajectory on the X-axis at different prediction times.

Training Time (s)	Prediction Time (s)	RMSE (mm)	MAE (mm)	MAPE
0.0–10.0	10.0–24.76	0.2429	0.1840	0.06%
0.0–12.5	12.5–24.76	0.1814	0.1398	0.05%
0.0–15.0	15.0–24.76	0.1456	0.1098	0.04%
0.0–17.5	17.5–24.76	0.1205	0.0871	0.02%
0.0–20.0	20.0–24.76	0.1008	0.0751	0.01%

). The accuracy of the oscillation prediction trajectory was found to vary with the prediction time. As can be observed in Table 2, the lowest prediction accuracy was attained with 602 data points in the training set, a training set time of 0–10 s, and a predicted trajectory time of 10–24.76 s. The evaluation metrics for the X-axis trajectory were as follows: an RMSE of 0.2429 mm, an MAE of 0.1840 mm, and an MAPE of 0.06%. The highest prediction accuracy was achieved with 1202 data points in the training set, a training set time of 20.00 s, and a predicted trajectory time of 20.00–24.76 s. The corresponding metrics were as follows: an RMSE of 0.1008 mm, an MAE of 0.0751 mm, and an MAPE of 0.01%. Through experiments, it was determined that the prediction accuracy for oscillation trajectories is directly proportional to the number of training data collected. Nevertheless, the number of training data collected is contingent on the robotic picking efficiency. In light of the actual picking conditions and the precision requirements of the prediction model, the parameters for EEMD decomposition were set as follows: noise intensity 0.2, ensemble number 50, maximum iteration count 500, maximum IMF count 15, and cumulative energy threshold for component classification 0.8. The LSTM network structure was constructed, comprising an input layer with 30 units, an LSTM layer with 50 units, a Dropout layer with a rate of 0.2, a fully connected layer with 1 unit, and a regression layer. The ARIMA parameter combination ($p$, $d$, $q$) was set to (1, 0, 0). After meticulous tunning of parameters, the optimal solution for oscillation prediction was ultimately attained.

3.2. Prediction of Multi-Fruit Oscillation Trajectory Subjected to Different Picking Forces

Most clusters of trellised tomatoes bear 3–6 tomatoes that are nestled together, and the number and weight of fruits per cluster vary. Resultantly, the oscillation trajectories for multi-fruit clusters are far more complex and variable. Unlike the oscillation of a single fruit, which bears a resemblance to second-order damped systems, the prediction of multi-fruit oscillations poses a considerable challenge to the ARIMA-EEMD-LSTM hybrid model. It requires low-frequency ARIMA prediction components and high-frequency LSTM prediction components to adjust automatically. In Figure 6, the time ranges for the displayed oscillation and prediction trajectories are 0–9.1 s and 5.0–9.1 s, respectively. As given in

Table 3. Prediction accuracy of multi-fruit oscillation trajectory on the X-axis subjected to different picking forces.

Prediction Time (s)	Harvesting Capacity (N)	RMSE (mm)	MAE (mm)	MAPE
5.0–9.1	15.36	0.6466	0.5323	0.26%
5.0–9.1	12.42	0.3852	0.2770	0.11%
5.0–9.1	14.15	0.3008	0.2364	0.08%
5.0–9.1	11.59	0.1521	0.1084	0.01%
5.0–9.1	15.62	0.6740	0.5323	0.27%

, distinct picking forces (F₁, F₂, F₃, F₄, and F₅) were applied to tomatoes A and B, their values being 13.36° N, 12.42° N, 14.15° N, 11.59° N, and 15.62° N, respectively. The application of picking forces was realized by the front hook of the tension gauge and maintained for a duration of 1.0–2.0 s. When F₁ was applied parallel to the horizontal direction, tomato A was not detached. The oscillation and prediction trajectories of tomato C were recorded, as provided in Figure 6a. The amplitude and frequency of the oscillation trajectory declined gradually. The oscillation trajectory prediction combined a low-frequency ARIMA component of 0.6 and a high-frequency LSTM component of 0.4, which were then superimposed. In the case where F₂ was applied at a 30° angle below the horizontal direction, the oscillation trajectory of tomato A changed little initially, followed by a rapid drop in amplitude and frequency in the period 5–7 s and a gradual stabilization in the period 7–9.1 s (Figure 6b). Therefore, the oscillation trajectory prediction algorithm utilized a low-frequency ARIMA component of 0.9 and a high-frequency LSTM component of 0.1, and the predicted results of the combined components were then integrated. The direction of F₃ formed a 45° angle with the horizontal downward direction. The oscillation and prediction trajectories of tomato D are displayed in Figure 6c. It was observed that both the frequency and amplitude of the oscillation changed progressively. In the prediction period from 5.0 s to 9.1 s, the amplitude decreased slowly while the frequency remained relatively stable. Consequently, the oscillation trajectory prediction algorithm utilized a low-frequency ARIMA component of 0.8 and a high-frequency LSTM component of 0.2. When F₄ was applied at a 60° angle below the horizontal direction, the oscillation and prediction trajectories of tomato E were acquired (Figure 6d). The amplitude of the oscillation trajectory varied slightly, while the frequency varied notably. The oscillation trajectory prediction algorithm incorporated a low-frequency ARIMA component of 0.3 and a high-frequency LSTM component of 0.7. The direction of F₅ formed a 90° angle with the horizontal downward direction. The oscillation and prediction trajectories of tomato F are given in Figure 6e. Evidently, both the amplitude and frequency changed substantially. The oscillation trajectory prediction algorithm consisted of a low-frequency ARIMA component of 0.1 and a high-frequency LSTM component of 0.9, and the final prediction was produced by superimposing these components. For oscillations involving two tomato fruits (Figure 6a,b), the prediction accuracy was relatively low, as evidenced by the following X-axis evaluation metric values: an RMSE of 0.6466 mm, an MAE of 0.5323 mm, and an MAPE of 0.26%. For clusters of four fruits (Figure 6c–e), superior prediction accuracy was achieved, as evidenced by an RMSE of 0.1521 mm, an MAE of 0.1084 mm, and an MAPE of 0.01%. Detailed results for all five experiments are listed in Table 3. The experimental findings suggested a correlation between the amplitudes and frequencies of the multi-fruit oscillation trajectories and the magnitudes of the applied picking forces. That is, larger picking forces induced greater variations in amplitude and frequency.

3.3. Prediction and Analysis of Multi-Fruit Collision Vibration Trajectory

During the harvesting process of trellised tomatoes, multiple tomatoes often collide with each other (Figure 7). When a harvesting force F₁₁ of 15.86 N was applied (by the front hook of the tension gauge for a duration of 1.0–2.0 s) to tomato A at a 45° angle with the horizontal downward direction, tomato F collided once with tomato G, and the oscillation trajectory of tomato G was recorded (Figure 7a). As can be seen from the trajectory curve, an initial spatial collision occurred between tomatoes F and G at the onset of the applied F₁₁, resulting in a transient fluctuation in the trajectory curve, after which no further collisions were observed. The oscillation and prediction trajectory times covered the time intervals of 0–18.93 s and 9.0–18.93 s, respectively. The prediction trajectory curve exhibited relatively small amplitudes and stable frequency variations. The trajectory prediction was achieved through the utilization of a low-frequency ARIMA prediction component of 0.7 and a high-frequency LSTM prediction component of 0.3. Finally, the two components were superimposed to obtain the comprehensive prediction trajectory curve. The evaluation of the X-axis trajectory in the experiment yielded an RMSE of 0.2930 mm, an MAE of 0.2269 mm, and an MAPE of 0.02%. Concurrently, multiple collisions were observed among tomatoes A, B, C, D, and E, and the oscillation and prediction trajectories of tomato B were recorded (Figure 7b). As revealed by the oscillation trajectory, tomato B collided multiple times with tomatoes A, C, D and E. The oscillation and prediction trajectories corresponded to the time intervals of 0–9.1 s and 5.0–9.1 s, respectively. The oscillation of the trajectory curve posed a huge challenge to the algorithm. The proportions of the low-frequency ARIMA prediction component and the high-frequency LSTM prediction component continuously adjusted automatically. Specifically, the former adjusted between 0.1 and 0.3, while the latter adjusted between 0.7 and 0.9. The final prediction curve was obtained by superimposing the dynamically adjusting components. The evaluation results for the X-axis trajectory showed an RMSE of 0.4077 mm, an MAE of 0.3575 mm, and an MAPE of 0.05%. During multi-fruit oscillations, the prediction accuracy weakened as the number of collisions grew, and the trajectory changes were rather complex.

3.4. Prediction of Oscillation Trajectories of Trellised Tomatoes Subjected to Consecutive Picking Forces

During harvesting, a subsequent picking force may be applied before previous oscillations have fully attenuated, thereby producing consecutive forces on the tomato and superimposed oscillation trajectories. As demonstrated in Figure 8, an initial picking force of 13.55 N was applied to tomato E. At 5 s, a second picking force F₁₂ of 13.11 N was applied at a 45° upward angle relative to the horizontal direction. Resultantly, collisions occurred among tomatoes A, C, and E. The oscillation trajectory dataset for tomato A comprised 1706 data points, spanning an oscillation trajectory time of 0–18.93 s and a prediction time of 10.0–18.93 s. As shown in Figure 8a, the collision between tomato A and tomato B caused fluctuations in the trajectory curve. Experimental observations revealed that the oscillation amplitude rose due to the superposition of two consecutive picking forces. In contrast, the oscillation amplitude of the prediction curve was relatively minor, and its frequency change was relatively stable. In the trajectory prediction model, the low-frequency ARIMA component accounts for 80%, while the high-frequency LSTM component makes up 20%. Finally, the low-frequency ARIMA component and the high-frequency LSTM component were combined to obtain the comprehensive prediction trajectory curve. The evaluation metrics of the X-axis curve presented an RMSE of 0.3290 mm, an MAE of 0.2706 mm, and an MAPE of 0.04%. The above experimental results suggested that the oscillation amplitude of tomato trajectories is superimposed after the application of two consecutive picking forces, accompanied by the occurrence of multi-fruit collision oscillations and trajectory jitter.

When three picking forces F₁₃ of 14.85 N (initially), 13.72 N (at 7 s), and 13.15 N (at 17 s) were applied to tomato G at an upward angle of 30° with the horizontal direction, the oscillation trajectory dataset for tomato B was collected from 0 s to 27.22 s (Figure 8b), and the predicted trajectory time spanned from 17.0 s to 27.22 s. No notable collisions were observed between tomatoes B and D. However, the oscillation amplitude of tomato B grew cumulatively during the application of three consecutive picking forces, and its oscillation frequency also rose, which posed a considerable challenge to the trajectory prediction algorithm. Meanwhile, the contributions of the low-frequency ARIMA prediction component and the high-frequency LSTM prediction component are adjusted automatically, varying in the ranges of 0.2–0.4 and 0.6–0.8, respectively. The final predicted trajectory was obtained by superimposing the adjusted components. The evaluation metrics for the X-axis curve included an RMSE of 0.4609 mm, an MAE of 0.3291 mm, and an MAPE of 0.05%. The experimental findings suggested that during the application of three consecutive picking forces, the oscillation amplitude varied remarkably, and the oscillation frequency also changed rapidly, which led to a reduction in the prediction accuracy.

Both collision and the superposition of picking forces would cause oscillatory trajectory fluctuations in tomatoes, characterized by unstable frequencies, sudden amplitude changes, and accelerated vibration. Compared with collisions, the superposition of consecutive picking forces leads to a sudden increase in oscillation amplitude. Collisions, however, do not cause amplitude amplification but result in frequent jittering; larger collision frequency is accompanied by stronger vibrations and more evident changes in frequency and amplitude. Trellised tomatoes typically grow in clusters of four to five fruits. Empirical observations confirmed that, under continuous application of multiple picking forces, the oscillation amplitude increases, and the collision oscillation trajectories of multi-fruit tomatoes show noticeable jitter.

3.5. Comparison of Prediction Accuracy for Single-Fruit and Multi-Fruit Oscillation Trajectories Among Different Models

The prediction accuracy of five different models for single-fruit and multi-fruit oscillation trajectories is summarized in

Table 4. Comparison of prediction accuracy for single-fruit and multi-fruit oscillation trajectories among five different models.

Experiment Category	Model	Training Time (s)	Prediction Time (s)	RMSE (mm)	MAE (mm)	MAPE
Single-fruit oscillation test	RF	0–15.0	15–24.76	0.5014	0.4164	0.13%
	ARIMA	0–15.0	15–24.76	0.2042	0.1634	0.07%
	LSTM	0–15.0	15–24.76	0.1935	0.1526	0.07%
	EMD-LSTM	0–15.0	15–24.76	0.1902	0.1502	0.06%
	ARIMA-EEMD-LSTM	0–15.0	15–24.76	0.1862	0.1431	0.04%
Multi-fruit oscillation test	RF	0–8.0	8–18.93	1.0956	0.7843	0.06%
	ARIMA	0–8.0	8–18.93	0.5245	0.3968	0.04%
	LSTM	0–8.0	8–18.93	0.4610	0.3845	0.04%
	EMD-LSTM	0–8.0	8–18.93	0.4112	0.3215	0.03%
	ARIMA-EEMD-LSTM	0–8.0	8–18.93	0.3864	0.2865	0.02%

. The models include ARIMA, LSTM, EMD-LSTM, random forest (RF) [32], and ARIMA-EEMD-LSTM models. For single-fruit oscillation trajectory prediction, a dataset comprising 1488 data points was collected. The training set time and the oscillation trajectory prediction time ranged from 0 s to 15.0 s and from 15.0 s to 24.76 s, respectively. Among the five models assessed, the proposed ARIMA-EEMD-LSTM hybrid model achieved the highest prediction accuracy. For multi-fruit oscillation trajectory prediction, a dataset comprising 1706 data points was collected. The training set time and the oscillation trajectory prediction time lay in the ranges of 0–8.0 s and 8.0–18.93 s, respectively. Among the five different prediction models, the ARIMA-EEMD-LSTM hybrid model again yielded the highest prediction accuracy. In light of the preceding experiments, it can be concluded that the ARIMA-EEMD-LSTM hybrid model fulfilled the requisite practical accuracy criteria for oscillation trajectories. This finding provides a theoretical foundation for enhancing the operational efficiency of tomato-harvesting robots.

4. Conclusions

In this study, an ARIMA-EEMD-LSTM hybrid model for predicting the oscillation trajectories of trellised tomatoes was proposed. First, an experimental motion capture system was constructed to capture the oscillation trajectory datasets of trellised single-fruit and multi-fruit tomatoes under varying picking forces. Afterwards, the EEMD method was employed to decompose oscillation signals into multiple IMF components, among which the high-frequency and low-frequency components were predicted by the LSTM model and the ARIMA model, respectively. The final oscillation trajectory prediction model for trellised tomatoes was constructed by integrating these components. Ultimately, the accuracy of the constructed model was experimentally verified.

(1) The proposed model achieved superior oscillation trajectory prediction accuracy for trellised tomatoes in comparison to other models. For single-fruit oscillations, the prediction accuracy reached an RMSE of 0.1008–0.2429 mm and an MAE of 0.0751–0.1840 mm. For multi-fruit oscillations, the prediction accuracy reached an RMSE of 0.1521–0.6740 mm and an MAE of 0.1084–0.5323 mm. These findings confirm that the proposed prediction model for the oscillation trajectories of trellised tomatoes meets the accuracy requirements of practical applications, which provides a theoretical basis for dynamic prediction and efficiency improvement in tomato-harvesting robots.

(2) The ARIMA-EEMD-LSTM hybrid model was employed to analyze single-fruit oscillations and multiple-fruit oscillations (including collision oscillations and superimposed oscillations). The analysis confirmed the inherent regularity of oscillation trajectories of trellised tomatoes and verified the prediction accuracy of the hybrid model. Moreover, the influences of trellised tomato fruit distribution, multi-fruit collision, picking force superimposition, data collection training set, and prediction time on oscillation trajectory prediction accuracy were examined. The MAPE ranged from 0.01% to 0.06% for single-fruit oscillations, and it varied between 0.01% and 0.027% for multi-fruit oscillations. These results prove the feasibility and accuracy of the proposed hybrid model and demonstrate its efficacy in predicting oscillation trajectories of trellised tomatoes during harvesting. Moreover, this study can also serve as a reference for the mechanized dynamic harvesting of other fruits.