Research on the Prediction of Insertion Resistance of Wheel Loader Based on PSO-LSTM

Abstract

Insertion resistance is the resistance caused by a pile to a wheel loader when the latter inserts into the pile. It is significant to clarify the insertion resistance to avoid wheel slippage, increase additional energy consumption, and protect the wheel loader during the insertion process. To address the problem that current methods cannot accurately obtain the insertion resistance magnitude and insertion resistance variation trend, we propose a composite model based on the particle swarm optimization (PSO) algorithm and the long short-term memory (LSTM) neural network. Firstly, the Pearson correlation coefficient method is used to test the parameters related to insertion resistance. Following this, the hyperparameters in the LSTM are optimized by PSO. Finally, different proportions of training sets are set in PSO-LSTM and compared with LSTM. The experimental data are selected from gravel sample groups and sand sample groups consisting of insertion depths of 600 mm, 800 mm, and 1000 mm. The results show that PSO-LSTM has higher prediction accuracy, better robustness, stability, and generalization ability compared with LSTM. In PSO-LSTM, when the proportion of the training set is 80%, the average relative errors are 2.28%, 1.57%, and 1.53% for the gravel sample group and 1.14%, 0.71%, and 0.60% for the sand sample group.

1. Introduction

A wheel loader is an important piece of construction machinery that is widely used in construction, mining, and transportation activities, performing tasks such as scooping, loading, transporting, and dumping [1]. The main form of operation is that the wheel loader is inserted into the pile at a certain speed, and then the bucket is turned by the action of the lift arm, the tilt level, and the connecting rod to shovel the pile, and then the lift arm is lifted to transfer the material. During the insertion phase, the resistance of the bucket increases rapidly with the increasing insertion depth. During this process, it will be accompanied by a huge energy consumption [2]. Therefore, it is important to predict the magnitude and trend of the resistance of the bucket during the insertion phase for the realization of resistance reduction and an energy reduction in wheel loaders.

The study of the insertion resistance of wheel loader buckets has been a typical issue for a long time. In the 1980s, researchers proposed an empirical formula of resistance for insertion resistance, which is the most widely used method at present [3]. The formula can roughly estimate the magnitude of insertion resistance to a certain extent, but there is a problem of poor accuracy because many empirical coefficients in the formula need to be chosen manually. In response to the shortcomings of the empirical formula, some scholars have analyzed the resistance to the bucket using the method of soil mechanics. Among them, resistance analysis based on Coulomb Theory and Rankine Theory has been widely used [4,5,6]. Compared with the traditional empirical formula, the resistance calculation based on the soil mechanics method is more accurate. The shortcoming of the soil mechanics method is that the calculation process is complicated. In recent years, a bucket-resistance-prediction method based on the agent model has gradually become mainstream, and the advantage of this is that it relies on the establishment of an agent model between the mechanical motion of the wheel loader and the resistance, and then obtains the resistance value through the change in mechanical motion, which largely reduces the error caused by the manual selection of coefficients. Wang et al. [7] developed a static model of bucket working resistance and a kinematic model of the working trajectory of a wheel loader, from which the key measurement parameters of working resistance were obtained. Based on the static and kinematic models, the corresponding working trajectories and working resistance were fitted based on the indirect measurement parameters. The analysis results showed that the minimum deviation of the resistance was 0.05%, the maximum deviation was 19.59%, and the average deviation was 7.07%. Madau et al. [8] analyzed the forces during the shoveling operation of the wheel loader, established the dynamics and kinematics equations of the whole wheel-loader system, and then established a dynamic model for bucket resistance estimation and verified that the resistance error was less than 1%.

Although agent models have many advantages, their accuracy depends on dynamic or kinematic models. These models are the result of theoretical analysis, which differs from the actual operating conditions of the bucket and does not reflect the trend of resistance. This problem has been effectively solved by the advent of artificial neural networks (ANN), which are non-parametric models that learn from training data to obtain predicted values of output variables. There have been many successful applications of ANN in predicting insertion resistance. Park et al. [9] constructed a neural network with parameters such as the diameter of the pile, the insertion depth, the insertion energy, and the soil properties around the pile as the input variables and the insertion resistance as the output variable, and verified the feasibility of using neural networks to predict insertion resistance. Huang et al. [10] used a BP neural network to predict the resistance to the curved blade and selected the cutting angle, forward turning angle, and cutting speed of the blade as input variables, and the accuracy of the prediction results for the resistance reached 94%. Zhang et al. [11] constructed a fuzzy neural network to predict the traction resistance of tractor tillage and optimized the structure of the fuzzy neural network through a density-based noise application spatial clustering algorithm, and the average relative error of the predicted resistance was 4.36%. Although the above resistance models achieved good prediction accuracy, they all predicted the special point of insertion depth, ignored the characteristics of resistance change in time sequence, and could not achieve a prediction for the trend of resistance change.

For the characteristics of bucket insertion resistance, which is a timeseries and nonlinear, this study chooses to use the long short-term memory neural network (LSTM) as the basic model. LSTM is a classical method of deep learning [12,13], which addresses the problem of gradient disappearance or gradient explosion of recurrent neural networks [14,15]. LSTM can ascertain future data changes by historical timeseries data features, and has been widely used in many fields. Meanwhile, the introduction of the dropout layer in LSTM can effectively prevent overfitting in the training process [16]. However, LSTM is highly sensitive to the number of hidden layer units, the initial learning rate, and the dropout probability, and different parameters will have a large impact on the prediction performance. The best number of hidden layer units, initial learning rate, and dropout probability are not the same for different training sets and samples [17]. To address the shortcomings of LSTM, this paper combines particle swarm optimization (PSO) to optimize the above parameters. The PSO algorithm has the characteristics of fast searching speed, high efficiency, and easy convergence [18,19]. The model, by combining LSTM with PSO, has been applied to many fields. Ji et al. [20] used a variational modal decomposition method to decompose the propagation loss of electromagnetic waves, and combined PSO and LSTM to achieve the prediction of electromagnetic wave propagation loss in ocean tropospheric ducts. Zhao et al. [21] used LSTM as the basic model, combined with empirical modal decomposition and PSO, to achieve prediction for electric power fluctuations. The model improved the basis for improving the generator’s fluctuation compensation capability. Ren et al. [22] developed a PSO-LSTM model to estimate the state-of-charge of lithium-ion batteries, and the error was within 5%. Yang et al. [23] developed a glacier runoff prediction model by PSO-LSTM, and they used the Muzati River basin as the study object and achieved good prediction accuracy. Peng et al. [24] analyzed and determined the input layer using the random forest method and the Pearson correlation coefficient method. The PSO-LSTM model was used to predict the ammonia concentration in a piggery and the R² achieved 0.9487. Song et al. [25] constructed a PSO-LSTM model to predict short-term passenger flow in subway stations, and the prediction accuracy of PSO-LSTM is higher compared to LSTM. At the same time, the reasonable selection of characteristic data that can represent the trend of resistance changes assumes a high priority. Liao et al. [26] proposed a method to calculate the resistance of an excavator by analyzing the relationship between the hydraulic cylinder, excavator bucket, and resistance. The resistance obtained by this method is closer to the actual value and can effectively reflect the dynamic process of the actual resistance during the excavation operation as well as the transient change in the actual resistance and the impact state. This provides a new idea for predicting the insertion resistance of wheel loaders.

In response to the above problems and discoveries, this study proposes a PSO-LSTM model to predict the magnitude and change trend of bucket insertion resistance. Three sets of experiments with different insertion depths of 600 mm, 800 mm, and 1000 mm were conducted in each of two typical piles, gravel and sand. The data relating to cylinder pressure, wheel loader displacement, and resistance were collected by sensors and measurement equipment in the wheel loader. A PSO-LSTM model with cylinder pressure data and insertion depth as the input layer and insertion resistance as the output layer was constructed. Different proportions of data were set as the training set, and good prediction results were achieved.

2. Experimental Data Acquisition and Analysis

2.1. Experimental Principle and Design

To effectively collect the relevant parameters in the insertion process of the wheel loader, an automatic operating platform of the wheel loader is constructed in this paper. The experimental data required in this paper include cylinder pressure, insertion depth, and insertion resistance. The cylinder pressure can be measured by the sensors installed on the lift cylinder and the tilt cylinder. The insertion depth is approximately equal to the travel distance of the wheel loader and is measured by a fifth wheel gauge hinged to the rear of the wheel loader. The insertion resistance is measured by three sets of shaft-pin-type sensors and obtained by force synthesis. Meanwhile, the engine speed, the displacement of the lift cylinder, and the tilt cylinder during the insertion process are recorded. The recording frequency of the above data was 50 Hz, which means once every 0.002 s.

The experimental site was selected on empty and flat ground, and two kinds of materials, gravel and sand, were selected as shown in Figure 1. Gravel is a typical non-uniform, high-density, and large-particle material, while sand is a uniform, low-density, and small-particle material, both of which have high representativeness. Three levels of insertion depth were chosen, 600 mm, 800 mm, and 1000 mm.

2.2. Insertion Depth

Insertion depth is a key factor in determining the magnitude of insertion resistance. In this paper, the real insertion distance of the wheel loader is obtained by a fifth wheel gauge hinged to the rear of a wheel loader. Due to the limited accuracy of the fifth wheel gauge, the interval between two adjacent data measured by it is 9 mm, so the curve of the whole insertion distance shows a stepped-up trend. The insertion depth of gravel and sand of 1000 mm is shown in Figure 2. To exclude the influence of speed on insertion resistance and to ensure that the insertion speed of the wheel loader is maintained as uniformly as possible. In this paper, the engine speed is set to 1500 rpm, and the initial speed is about 3.42 km/h. Meanwhile, to ensure that the bucket always remains horizontal during insertion. In this paper, the displacements of the lift cylinder and the tilt cylinder during insertion are monitored, where the initial displacement of the lift cylinder is about 1411 mm and the initial displacement of the tilt cylinder is about 1402 mm, and when the displacement changes within −1 mm to 1 mm, it can be considered that the bucket remains horizontal. The engine speed and cylinder displacement for the insertion of gravel and sand with a depth of 1000 mm are shown in Figure 3, respectively. As can be seen from Figure 3, the engine speed is maintained at about 1500 rpm, but there will be a step up and down. The displacement of the lift cylinder and the tilt cylinder in the gravel remains stable, while the lift cylinder and the tilt cylinder in the sand show a smaller decrease, in which the change magnitude of the displacement of the lift cylinder is about 0.6 mm, while the change magnitude of the tilt cylinder is about 0.1 mm, which meets the requirement of keeping the bucket horizontal.

2.3. Resistance

The wheel loader insertion resistance comes from the combined force, consisting of three shaft-pin-type sensors. Taking the forward direction of the wheel loader as the positive direction, the shaft-pin-type sensor at the lift arm and bucket on the right side of the wheel loader is noted as P1, the shaft-pin-type sensor at the lift arm and bucket on the left side of the wheel loader is noted as P2, and the shaft-pin-type sensor at the connecting rod and bucket is noted as Q, as shown in Figure 4. Due to the influence of the structure of the wheel loader, these three sensors are not installed parallel to the horizontal direction, but at a certain angle to the horizontal direction, as shown in Figure 5. The angle between the Y-axis direction and horizontal direction of P2 is 70°; Q is in the same direction as P2, but its installation angle is different from P2, and the angle between its Y-axis direction and the horizontal direction is 43.85°; and P1 is in the opposite direction of P2 because it is installed on the other side of the wheel loader.

The resistance of gravel and sand in the X direction and Y direction measured by P1, P2, and Q, respectively, for an insertion depth of 600 mm is shown in Figure 6. Among them, P1 and P2 have almost the same magnitude but opposite directions in the X direction, where P2 is negative and P1 is positive. The resistance of gravel has a greater variation, while the resistance of sand has a smoother variation. The partial force in the Y direction is always positive and has almost the same trend, but the resistance value of P2 is slightly larger than P1 due to the uneven force on the left and right sides of the bucket, which is more obvious in the sand. As for Q, although its installation direction is the same as P2, its force direction is opposite to P1 and P2, and the bucket force schematic is shown in Figure 7. Therefore, the resistance values in the X direction and Y direction of Q are negative, and the absolute value of resistance in the Y direction is always higher than that in the X direction.

Since the installation angle of P1, P2, and Q are not parallel to the horizontal direction, the resistance direction of the three shaft-pin-type sensors needs to be corrected to make the direction horizontal. F_YQ, F_XQ, F_YP, and F_XP are the resistance values measured by the shaft-pin-type sensors, and F_QX and F_PX are the resistance values after correction. The calculation formula is shown in Equation (1).

$$\{\begin{array}{l}{F}_{PX1}=F_{XP1}\cos \alpha -F_{YP1}\sin \alpha \\ F_{PX2}=F_{XP2}\cos \alpha -F_{YP2}\sin \alpha \\ F_{QX}=F_{XQ}\cos \alpha -F_{YQ}\sin \alpha \end{array}$$

(1)

Following this, the insertion resistance of the bucket, F, can be expressed as:

$$F=F_{PX1}+F_{PX2}-F_{QX}$$

(2)

To verify the reliability of the insertion resistance, a comparison with the resistance calculated by the empirical formula is chosen in this paper. The empirical formula for insertion resistance is shown in Equation (3) [3], and the influence coefficients and parameters are shown in

Table 1. Parameters of the empirical formula.

Type	K1	K2	K3	K4	B (cm)	L (cm)
Gravel	0.75	0.12	1.1	1.4	290	Insertion depth
Sand	0.5	0.1

[27].

$$F_c=9.8K_1{K}_2{K}_3{K}_{4}B{L}^{1.25}$$

(3)

where K₁ is the influence coefficient of pile looseness; K₂ is the influence coefficient of pile type; K₃ is the influence coefficient of pile shape and height; K₄ is the influence coefficient of the bucket; B is the width of the bucket; L is the insertion depth.

The comparison between the insertion resistance calculated by the empirical formula and the actual insertion resistance is shown in Figure 8. Among them, the actual insertion resistance of gravel fluctuates from the calculated resistance, while the actual insertion resistance of sand is higher than the calculated resistance. This is related to the environment in which the experiment was conducted, which has high humidity due to the climate. The cohesion and adhesion of the sand increase due to the humidity, which makes the resistance to the bucket greater [28,29]. The two different types of materials exhibit different characteristics of resistance. The insertion resistance of gravel shows a greater randomness and a greater variation in resistance. In contrast, the insertion resistance of sand changes more smoothly and increases gradually. Moreover, the value of resistance generated by each insertion is unique, and the empirical formula does not accurately reflect the trend of insertion resistance.

For the soil mechanics method, the pile develops a virtual surface and a slip surface as the bucket moves inside the pile. This breaks the material pile into three parts, as shown in Figure 9. Area A is the material inside the bucket, area B is the material being shoveled, and area C is the unshoveled material. The wheel loader insertion process is divided into two phases. These include the pre-insertion phase and the late-insertion phase. In these two phases, the forces on the bucket are not the same, as shown in Figure 10, and the forces are calculated as shown in Equation (4). A number of these parameters need to be selected empirically, some of them need to be determined experimentally, and others need to be measured instantly. Meanwhile, the definition of the two phases is not clear. The above reasons lead to the insertion resistance, which is difficult to calculate according to the soil mechanics method, and the insertion resistance is not accurate compared to the real resistance.

$$\{\begin{array}{l}{F}_1=k\rho gSH\cos \alpha \\ F_2=\mu F_4\cos \beta +\mu F_3\cos (\phi -\beta )\\ F_3=-F_p\\ F_4=Mg\end{array}$$

(4)

where F₁ is the resistance of the bucket tip and edge; k is the resistance influence coefficient; ρ is the density of material; g is the acceleration of gravity; S is the shovel area; H is the vertical height of the bucket tip and the pile; α is the angle between the bucket movement direction and the horizontal direction; F₂ is the friction force exerted on the bucket by the material inside the bucket; μ is the friction coefficient; φ is the angle of internal friction; β is the angle between the bucket tip and the horizontal direction; F₃ is the force generated by the slip of the pile on the bucket; F_p is the Coulomb pressure, and it is shown in the Equation (5); F₄ is the gravitational force exerted by the material inside the bucket; M is the weight of the material in the bucket.

$$F_p=\frac{\rho H^2}{2{\sin }^2\epsilon \sin (\epsilon -\delta )}·\frac{{\sin }^2(\phi +\epsilon )}{{\left[1+\sqrt{\frac{\sin (\phi +\delta )\sin (\phi -\gamma )}{\sin (\epsilon -\delta )\sin (\epsilon +\gamma )}}\right]}^2}$$

(5)

where ε is the angle between the virtual surface and the vertical direction; γ is the angle of repose of the pile; δ is the angle between F_p and the normal vector of the virtual surface, and the angle is acute.

2.4. Cylinder Pressure

The movement of the bucket of the wheel loader is the result of the joint action of the lift arm, tilt level, and connecting rod, in which the movement of the lift arm is governed by the lift cylinder, while the movement of the tilt level and connecting rod is governed by the tilt cylinder. The diagram of the lift cylinder is shown in Figure 11, and the structure of the tilt cylinder is similar to that of the lift cylinder. Both the lift cylinder and the tilt cylinder consist of a large chamber and a small chamber. The pressure of the large chamber of the lift cylinder, the pressure of the small chamber of the lift cylinder, the pressure of the large chamber of the tilt bucket, and the pressure of the small chamber of the tilt bucket are recorded by the sensors, as shown in Figure 12.

The cylinder pressure data for gravel and sand at an insertion depth of 600 mm are shown in Figure 13. The pressure of the large chamber of the tilt cylinder and the pressure of the small chamber of the tilt cylinder have a high degree of symmetry in terms of variation. The pressure of the large chamber of the lift cylinder gradually increases with the insertion depth, while the pressure of the small chamber will show a smaller decrease. For gravel materials, the pressure data of both the lift cylinder and the tilt cylinder show large fluctuations, while for sand materials, the pressure data changes more smoothly.

To verify the degree of correlation between cylinder pressure data and insertion resistance, the Pearson correlation coefficient was chosen to calculate the correlation between cylinder pressure and resistance for gravel and sand at different insertion depths. The calculation formula is shown in Equation (6), and the calculation results are shown in

Table 2. Pearson correlation coefficient.

Cylinder		Gravel			Sand
	600 mm	800 mm	1000 mm	600 mm	800 mm	1000 mm
Tilt cylinder large chamber	0.8958	0.9231	0.8866	0.9802	0.9795	0.9968
Tilt cylinder small chamber	−0.936	−0.9716	−0.9161	−0.9770	−0.9118	−0.8337
Lift cylinder large chamber	0.9708	0.9876	0.9830	0.9953	0.9957	0.9952
Lift cylinder small chamber	−0.7687	−0.8494	−0.6946	−0.7552	−0.5794	−0.7176

. The closer the correlation coefficient is to 1, the higher the positive correlation between the two variables, and the closer it is to −1, the higher the negative correlation between the two variables [30]. When the absolute value of the correlation coefficient is greater than 0.8, it can be considered to have a strong correlation. The large chamber of the tilt cylinder, the small chamber of the tilt cylinder, and the large chamber of the lift cylinder all possess a strong correlation with the insertion resistance. The correlation between the small chamber of the lift cylinder and the insertion resistance is relatively low, but the absolute value of the correlation coefficient is basically above 0.7, and the lowest correlation coefficient appears in the sand with an insertion depth of 800, and the correlation is only −0.5794.

$$R=\frac{{\displaystyle \sum _{i=1}^n(X_i}-\overline{X})(Y_i-\overline{Y})}{\sqrt{{\displaystyle \sum _{i=1}^n(X_i}-\overline{X}{)}^2}\sqrt{{\displaystyle \sum _{i=1}^n(Y_i}-\overline{Y}{)}^2}}$$

(6)

where X and Y are random variables; $\overline{X}$ and $\overline{Y}$ are the mean of the random variables.

3. LSTM and PSO

3.1. LSTM Structure and Principle

LSTM is a special kind of recurrent neural network. Compared with traditional recurrent neural networks, LSTM introduces the structure of the gate into its structure. Therefore, it can show a good ability to deal with timeseries regression and classification tasks [31,32]. Its structure is shown in Figure 14.

For a traditional neural network, there is only one input, x_t. In the LSTM, there are two sets of inputs, the input vector x_t of the current unit and the output vector h_t−1 of the previous unit, and a series of complex processes is performed by the unit state C_t. C_t as a critique of the LSTM consists of the forget gate f_t, the input gate i_t, and the output gate o_t [33].

• Forget gate: The role of the forget gate is to choose to retain or discard part of the information; it receives the information of x_t of the current unit and h_t−1 of the previous unit. In the internal state C_t, a number 0–1 is generated, where 0 represents complete discard and 1 represents complete retention.

  $$f_t=\sigma (W_f·h_{t-1}+W_f·x_t+b_f)$$

  (7)
• Input gate: The role of the input gate is to determine which information is retained internally and to ensure that critical information can be saved. The values to be updated are first determined by the input gate i_t, and then a candidate state $C_t$ is determined by the input gate i_t.

  $$i_t=\sigma (W_i·h_{t-1}+W_i·x_t+b_i)$$

  (8)

  $$C_t=\tanh (W_c·h_{t-1}+W_c·x_t+b_c)$$

  (9)

The unit state is then updated by the forget gate f_t, the input gate i_t, the candidate unit state $C_t$, and the previous unit state C_t−1 to obtain the current unit state C_t.

$$C_t=f_t·C_{t-1}+i_t·C_t$$

(10)

Output gate: The role of the output gate is to specify the value that needs to be output, control the current unit state C_t passed to the activation function and output the output vector h_t.

$$o_t=\sigma (W_o·h_{t-1}+w_o·x_t+b_o)$$

(11)

$$h_t=o_t·\tanh (C_t)$$

(12)

where w is the weight, b is the bias, σ is the sigmoid activation function, and tanh is the hyperbolic tangent function, which are, respectively, defined as follows:

$$\sigma =\frac{1}{1+\exp (-x)}$$

(13)

$$\tanh (x)=\frac{\exp (x)-\exp (-x)}{\exp (x)+\exp (-x)}$$

(14)

To address the issue of overfitting in the LSTM training process, a dropout layer is introduced in this paper. The main principle is to make some LSTM units temporarily fail in each round of training [34,35], to ensure that a different neural network is trained in each iteration. The dropout probability of the dropout layer is a number from 0 to 1. The dropout probability is the probability of the temporary failure of units. By introducing the dropout layer, the original neural network structure is turned into a sparse-shaped network as shown in Figure 15. This prevents the data from being repeatedly trained by the same network and greatly improves the issue of overfitting in LSTM.

3.2. PSO Optimization Process

PSO is an optimization algorithm for population intelligence in the field of computational intelligence, and the algorithm is derived from the study of bird predation behavior [36]. Each particle in the algorithm corresponds to a potential solution to the problem, and each particle is characterized by three metrics: velocity, position, and fitness value. Among them, the velocity determines the direction and displacement of the particle’s next movement and is influenced by itself as well as by other particles. The position describes the current location of the particle. PSO calculates the particle fitness value and compares it with the particle historical best fitness value p_best(i), and the global best fitness value g_best. The velocity v_i and position x_i of the particle are continuously updated during the iterative process to achieve the global optimum. The velocity and position of the particle are updated as shown in Equations (15) and (16). The pseudo code of PSO is shown in Algorithm 1.

$$v_i^{k+1}=w{v}_i^k+c_1{r}_1(p_i^k-x_i^k)+c_2{r}_2(g_{best}^k-x_i^k)$$

(15)

$$x_i^{k+1}=x_i^k+v_i^{k+1}$$

(16)

where w is the inertia coefficient, k is the number of current iterations, c is the acceleration factor and is kept non-negative, and r is a random number [0, 1].

Algorithm 1 PSO pseudo code

1: Inputs: Population size n; Number of iterations m; w; c1; c2

2: Outputs: gbest

3: begin

4:        for i =1 to n do begin

5:             initialize (vi); initialize (xi);

6:       end;

7:       for i = 1 to m do begin

8:           for i =1 to n do evaluate (xi);

9:           for i =1 to n do update (pbest, gbest);

10:           for i =1 to n do begin;

11:               execute Equation (15);

12:               execute Equation (16);

13:           end;

14:       end;

15: end.

3.3. PSO-LSTM Model

Some parameters in the LSTM need to be set manually, such as the initial learning rate, the number of hidden layer units, and the dropout probability, and there are no clear selection criteria for these parameters. At present, the selection of parameters for LSTM often relies on the experience of previous researchers or the results of multiple experiments. Different parameter settings have a large impact on the model prediction performance; therefore, it is important to obtain reasonable parameters to improve the LSTM prediction performance. In this paper, we establish a PSO-LSTM model which combines the advantages of PSO and LSTM.

The initial learning, the number of hidden layer units, and the dropout probability of LSTM are optimized by PSO. The process of PSO-optimized LSTM is shown in Figure 16. The input layer is the cylinder pressure data and the actual insertion depth, where the cylinder pressure data includes the lift cylinder large chamber pressure, the lift cylinder small chamber pressure, the tilt cylinder large chamber pressure, and the tilt cylinder small chamber pressure. The output layer is the insertion resistance. The initial parameters of PSO are shown in

Table 3. Initial parameters of PSO.

Parameters	Value
Population size	5
Number of iterations	30
Inertia factor w	0.8
Acceleration factor c1	1.5
Acceleration factor c2	1.5

, and the range of PSO-LSTM optimization parameters is shown in

Table 4. PSO-LSTM optimization parameters.

Parameters	Lower Limit	Upper Limit	Minimum Speed	Maximum Speed
Initial learning rate	0.0001	0.1	−0.001	0.001
Number of hidden layer units	100	800	−2	2
Dropout probability	0	1	−0.01	0.01

.

4. Results and Discussion

4.1. Results and Analysis

The experimental data were obtained from the gravel sample groups and the sand sample groups with insertion depths of 600 mm, 800 mm, and 1000 mm. A total of 50%, 60%, 70%, and 80% of the data were used as the training set, and the remaining 50%, 40%, 30%, and 20% of the data were used as the test set, respectively. To compare the prediction performance of PSO-LSTM, it was chosen to compare it with conventional LSTM, and the relevant parameters of the two model groups were set as shown in

Table 5. Parameters of LSTM and PSO-LSTM.

Parameters	LSTM	PSO-LSTM
Solver	Adam	Adam
Number of iterations	300	300
The number of generations of learning-rate attenuation	50	50
The learning-rate attenuation rate	0.1	0.1
Initial learning rate	0.01	Adaptive optimization
Number of hidden layer units	400	Adaptive optimization
Dropout probability	0.5	Adaptive optimization

.

In this paper, the root mean square error (RMSE), the average relative error (ARE), and the goodness of fit, R², are selected as the indexes to evaluate the prediction performance. The smaller the RMSE and ARE, the smaller the deviation between the predicted value and the true value, and the higher the prediction accuracy. The closer the value of R² is to 1, the higher the degree of fit and the better the prediction result. Its calculation formula is as follows:

$$RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{(y_i^{*}-y_i)}^2}}$$

(17)

$$ARE=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|\frac{y_i-y_i^{*}}{y_i}\right|}$$

(18)

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^n{(y_i-y_i^{*})}^2}}{{\displaystyle \sum _{i=1}^n{(y_i^{}-{\overline{y}}_i)}^2}}$$

(19)

where y_i is the true value, $y_i^{*}$ is the predicted value, ${\overline{y}}_i$ is the average of the true value, and n is the number of samples.

The prediction results of the LSTM and PSO-LSTM models are shown in Figure 17, Figure 18 and Figure 19, respectively, for the insertion of gravel samples with depths of 600 mm, 800 mm, and 1000 mm.

The prediction performance indexes of LSTM and PSO-LSTM for gravel sample groups with insertion depths of 600 mm, 800 mm, and 1000 mm are shown in

Table 6. LSTM and PSO-LSTM prediction performance of the gravel sample groups.

Insertion Depth	The Proportion of Training Set	LSTM			PSO-LSTM
		RMSE	ARE/%	R2	RMSE	ARE/%	R2
600 mm	50%	4.46	8.50	0.777	2.26	4.24	0.942
	60%	3.46	6.75	0.846	1.69	3.12	0.963
	70%	5.16	7.57	0.574	1.98	3.25	0.937
	80%	4.51	7.18	0.380	1.44	2.28	0.936
800 mm	50%	4.71	7.76	0.851	2.75	4.76	0.949
	60%	3.26	5.62	0.894	2.47	4.25	0.939
	70%	3.17	5.27	0.807	2.23	3.49	0.905
	80%	1.83	2.56	0.933	1.06	1.57	0.977
1000 mm	50%	9.71	7.45	0.726	4.17	4.42	0.949
	60%	6.40	5.16	0.871	2.89	4.22	0.973
	70%	6.98	5.77	0.833	1.88	4.42	0.987
	80%	5.90	5.03	0.776	1.80	1.53	0.979

, and the optimized parameters of PSO-LSTM are shown in

Table 7. PSO-LSTM optimization parameters for the gravel sample groups.

Insertion Depth	The Proportion of Training Set	Initial Learning Rate	Number of Hidden Layer Units	Dropout Probability
600 mm	50%	0.0059	490	0.557
	60%	0.0029	234	0.643
	70%	0.0012	129	0.474
	80%	0.0009	414	0.515
800 mm	50%	0.0024	624	0.312
	60%	0.0019	665	0.206
	70%	0.0031	658	0.616
	80%	0.0034	725	0.435
1000 mm	50%	0.0014	633	0.528
	60%	0.0009	797	0.800
	70%	0.0016	678	0.701
	80%	0.0024	502	0.699

.

The prediction performance of the LSTM and PSO-LSTM models for inserting sand samples with depths of 600 mm, 800 mm, and 1000 mm are shown in Figure 20, Figure 21 and Figure 22.

The prediction performance indexes of LSTM and PSO-LSTM for sand sample groups with insertion depths of 600 mm, 800 mm, and 1000 mm are shown in

Table 8. LSTM and PSO-LSTM prediction performance for the sand sample groups.

Insertion Depth	The Proportion of Training Set	LSTM			PSO-LSTM
		RMSE	ARE/%	R2	RMSE	ARE/%	R2
600 mm	50%	1.04	2.23	0.955	0.56	1.32	0.987
	60%	3.66	8.42	0.284	0.50	1.16	0.987
	70%	1.71	4.22	0.687	0.87	2.11	0.918
	80%	1.18	2.82	0.183	0.55	1.14	0.922
800 mm	50%	2.53	5.13	0.879	1.77	3.44	0.941
	60%	3.07	6.14	0.645	0.92	1.52	0.968
	70%	2.31	4.37	0.569	0.74	1.08	0.955
	80%	2.19	4.06	0.004	0.47	0.71	0.955
1000 mm	50%	6.33	9.90	0.702	2.64	4.35	0.948
	60%	4.29	6.89	0.799	2.00	3.37	0.956
	70%	1.81	2.85	0.933	0.96	1.43	0.981
	80%	1.97	2.87	0.718	0.51	0.60	0.981

, and the optimization parameters of PSO-LSTM are shown in

Table 9. PSO-LSTM optimization parameters for the sand sample groups.

Insertion Depth	The Proportion of Training Set	Initial Learning Rate	Number of Hidden Layer Units	Dropout Probability
600 mm	50%	0.0081	395	0.465
	60%	0.0097	414	0.259
	70%	0.0048	208	0.255
	80%	0.0012	633	0.504
800 mm	50%	0.0086	368	0.743
	60%	0.0016	631	0.487
	70%	0.0009	651	0.554
	80%	0.0012	619	0.218
1000 mm	50%	0.0011	625	0.500
	60%	0.0022	768	0.495
	70%	0.0021	756	0.453
	80%	0.0082	647	0.653

.

In the LSTM model, when the training samples are fewer, it is easy to create an error accumulation, and the later predicted values of the timeseries show a large deviation from the actual values. This is more obvious in the gravel sample groups, where the AREs of insertion depths of 600 mm, 800 mm, and 1000 mm are greater than 7% when the training set is 50%. In contrast, this value is relatively low in the sand sample groups, but the ARE is as high as 9.9% in the sample with an insertion depth of 1000 mm. The prediction accuracy is further improved with the increase in the number of training samples. When the training set reaches 80%, the ARE is only 2.56% in the sample with 800 mm insertion depth in the gravel experiment groups, but 7.18% in the sample with 600 mm insertion depth. While in the sand experiment groups, only the ARE of the sample with the 800 mm insertion depth reaches 4.06%, and the AREs of the other two groups are below 2.87%. In general, the LSTM model has poor robustness and low stability, which cannot meet the requirements of actual applications.

In the PSO-LSTM model, the prediction accuracy of all samples is improved over some range, even when the training set is sparser. The maximum value of ARE for the three samples in the gravel sample groups is 4.76% when the training set is 50%. When the training set is 80%, the maximum value of ARE is 2.28%. In the sand sample groups, when the training set is 50%, the maximum value of ARE for the three samples is 4.35%. When the training set is 80%, the maximum value of ARE is 1.14% and the minimum value is only 0.60%. Meanwhile, the R² of both the gravel sample groups and the sand sample groups with different proportions of training sets is greater than 0.9, which indicates that the predicted values fit the actual values better. Moreover, the RMSE values also showed a significant decrease compared with the LSTM model.

Combining the data in Table 7 and Table 9, it can be seen that different initial learning rates, the number of hidden layer units, and dropout probability have a large impact on the prediction results. The initial learning rate set in the LSTM model is 0.01, which is a high value, and the reasonable setting interval should be between 0.001 and 0.003, as shown in Table 7 and Table 9. For the number of hidden layer units, the prediction performance is better when the setting is above 500. As for the dropout probability, there is no obvious setting method, which is related to the changing trend of the sample data and other parameter settings.

When the proportion of the training set is 80%, the resistance predicted by PSO-LSTM, the resistance calculated by empirical formula, and the true resistance for the gravel sample groups and the sand sample groups are shown in Figure 23.

Both in the gravel sample groups and in the sand sample groups, it is obvious that there is a large deviation between the resistance calculated by the empirical formula and the true resistance. In particular, the insertion resistance of the wheel loader is unique each time, which makes it difficult to obtain a more accurate resistance by empirical formula. The resistance predicted by PSO-LSTM is closer to the true resistance and can reflect the trend of resistance changes well, which is not available for the empirical formula.

4.2. Discussion

In summary, the PSO-LSTM model developed in this study can predict the magnitude and trend of insertion resistance well, even when the training data are fewer, and considerable prediction results can be obtained. Moreover, the PSO-LSTM model has good stability, robustness, and generalization performance for various piles and insertion depths, which can reflect the change in insertion resistance of wheel loaders. This study provides a basis for subsequent research on the reduction in energy consumption of wheel loaders, energy-efficient operation, reasonable trajectory planning of shoveling, and operation in special operating environments.

The research in this paper also has certain shortcomings. The PSO-LSTM model built in this paper is optimized only for the initial learning rate, the number of hidden layer units, and dropout probability; however, the parameters affecting the prediction performance also include the maximum number of iterations, the number of generations of learning rate attenuation and the learning rate attenuation rate, etc., which can be tried to optimize these parameters. At the same time, increasing the number of LSTM layers can prevent the overfitting situation to a certain extent. In this paper, only a single-layer LSTM model is constructed from the perspective of computational efficiency, and changing the single-layer LSTM into a two-layer LSTM can be considered in the future. Meanwhile, only two typical piles, gravel and sand, are selected for experiments in this paper, and other kinds of piles are not tried, and conducting experiments in other special piles to verify the prediction performance of the model can be considered.

5. Conclusions

In this paper, a PSO-LSTM model is established and tested on gravel sample groups and sand sample groups with insertion depths of 600 mm, 800 mm, and 1000 mm. The results show that the PSO-LSTM model can predict the insertion resistance and its change trend during the insertion of wheel loaders with good accuracy, and the main conclusions are as follows:

• This paper proposes using the insertion depth and cylinder pressure as the input layer, and the correlation between cylinder pressure and insertion resistance is examined by the Pearson correlation coefficient. Among them, the absolute values of Pearson correlation coefficients of the pressure values of the large chamber of the tilt cylinder, the small chamber of the tilt cylinder, and the large chamber of the lift cylinder are greater than 0.8 in both the gravel sample group and the sand sample group. The absolute values of the Pearson correlation coefficients of the majority of them are greater than 0.9. The absolute values of the Pearson correlation coefficients of the pressure of the small chamber of the lift cylinder are relatively low, but they are basically above 0.7, which proves that the correlation between the cylinder pressure data and the insertion resistance is high.
• In this paper, we compare the prediction performance of the LSTM model and the PSO-LSTM model for insertion resistance. Experiments were conducted with 50%, 60%, 70%, and 80% of samples as training sets, respectively, in three different insertion depths of gravel sample groups and sand sample groups. The results show that in the LSTM model, the prediction accuracy gradually decreases due to the accumulation of errors, which is more obvious when the training-set samples are small. In contrast, in the PSO-LSTM model, the prediction performance of the model is greatly improved by optimizing the initial learning rate, the number of hidden layer units, and the dropout probability. In the gravel sample groups and sand sample groups with different proportions of training sets, both RMSE and ARE are significantly reduced. The values of R² are all greater than 0.9, and the fit between the predicted and actual values is high.
• In the PSO-LSTM model, when the training set was 80% of the total data, the AREs of the gravel sample groups with insertion depths of 600 mm, 800 mm, and 1000 mm were 2.28%, 1.57%, and 1.53%, respectively. In contrast, the AREs in the sand sample groups were 1.14%, 0.71%, and 0.60%, respectively. In general, the prediction accuracy of the sand sample groups is higher than that of the gravel sample groups, which is related to the pile characteristics and the mechanism of resistance generation. The insertion resistance of sand changes more smoothly, while the insertion resistance of gravel changes in a more fluctuating manner, which greatly increases the difficulty of predicting the insertion resistance for gravel.