Cooperative Carrying Control for Multi-Evolutionary Mobile Robots in Unknown Environments

Abstract

This study provides an effective cooperative carrying and navigation control method for mobile robots in an unknown environment. The manager mode switches between two behavioral control modes—wall-following mode (WFM) and toward-goal mode (TGM)—based on the relationship between the mobile robot and the unknown environment. An interval type-2 fuzzy neural controller (IT2FNC) based on a dynamic group differential evolution (DGDE) is proposed to realize the carrying control and WFM control for mobile robots. The proposed DGDE uses a hybrid method that involves a group concept and an improved differential evolution to overcome the drawbacks of the traditional differential evolution algorithm. A reinforcement learning strategy was adopted to develop an adaptive WFM control and achieve cooperative carrying control for mobile robots. The experimental results demonstrated that the proposed DGDE is superior to other algorithms at using WFM control. Moreover, the experimental results demonstrate that the proposed method can complete the task of cooperative carrying, and can realize navigation control to enable the robot to reach the target location.

1. Introduction

Mobile robot control has been widely used in several applications, such as navigation, obstacle avoidance, path planning, and cooperative transport. To enhance the robot control quality, Cupertino [1] adopted a fuzzy controller. The fuzzy controller possesses robustness and an anti-noise ability; the controller can identify and calculate signals with uncertainties. However, to design an applicable fuzzy controller, designers must spend a considerable amount of time analyzing the experimental input and output data of a mobile robot. Thus, machine-learning technology has gradually attracted considerable research attention. Zhu and Yang [2] and Rusu et al. [3] used supervised learning methods to adjust the parameters of the if–then rules in fuzzy neural networks by training data. The disadvantages of supervised learning [2,3] are that it is difficult to collect training data in advance and obtain precise training data.

Recently, reinforcement learning [4] has been widely used in control applications for mobile robots. The method not only can automatically construct a complete fuzzy neural network in the absence of precise training data but also adjust the parameters of the system through the machine leaning algorithm to complete the navigation task. Therefore, designing a mobile robot using evolutionary computing in unknown environments has become a topic of interest. Hsu and Juang [5] implemented a wall-following control using reinforcement learning. A fitness function was used to evaluate a robot’s movement in an unknown environment, and an optimal fitness value was adopted as the reinforcement signal. Anish and Parhi [6] used sensors to measure the distance between a robot and an obstacle. In [6], measured distances were used as inputs, and steering angle and speed were used as outputs for the adaptive network-based fuzzy inference system controller. Juang and Chang [7] adopted four sonar sensors and used the data from these sensors as input signals. In this application, the speeds of the left and right wheel are the outputs of the fuzzy neural network controller. The mobile robot was trained in the training environment by adjusting the parameters of the fuzzy neural network controller. Although the aforementioned methods can be used to successfully complete the navigation task, their performances are not optimal. Because the robot is in a real environment, the input signal contains uncertainties due to noise interferences from the sensors.

Although some researchers [8,9] have used the type-1 fuzzy set to solve uncertain problems, the control performance in the real environment is not optimal. Therefore, Kim and Chwa [10] used an interval type-2 fuzzy set for solving uncertain problems. The membership values of fuzzy sets provide a footprint of uncertainty (FOU) to deal with uncertainties. Thus, the controller design is more flexible and has improved error tolerance. Castillo and Melin [11] reduced the computational complexity of an interval type-2 fuzzy system.

Recently, researchers have used evolutionary algorithms for solving the parameter optimization problem, such as particle swarm optimization (PSO) [12], ant colony optimization (ACO) [13], differential evolution (DE) [14], the artificial bee colony (ABC) algorithm [15], and bacterial foraging optimization (BFO) [16]. In recent years, DE has been widely used in various fields [17,18] and has the advantages of a simple structure, reduced parameter setting requirements, and superior problem solving ability. However, the traditional DE method has a disadvantage, in that it can easily become trapped in a local optimal solution. To eliminate this disadvantage, an improved DE is proposed for solving mobile robot control problems.

The major contributions of this paper are described as follows. First, an efficient interval type-2 fuzzy neural controller (IT2FNC) based on dynamic group differential evolution (DGDE) was designed to implement the carrying control and wall-following mode (WFM) control for mobile robots. In the proposed IT2FNC, a functional link neural network (FLNN) with a nonlinear combination input was added in consequent part of a fuzzy rule. Second, the proposed DGDE used a hybrid method that involves a group concept and an improved DE to overcome the drawbacks of easily trapped into local optimal in the traditional DE algorithm. Second, a manager mode was developed to assist mobile robots in navigation control. Third, the manager mode switched to WFM or toward-goal mode (TGM), based on the relationship between the mobile robot and the unknown environment. Fourth, the proposed control method can complete the task of cooperative carrying and can realize navigation control to enable the robot to reach a target location. Moreover, the experimental results demonstrated that the proposed DGDE learning algorithm is superior to other algorithms at using WFM control.

2. Mobile Robot Specifications

In this study, the e-puck mobile robot developed by the Ecole Polytechnique Fédérale de Lausanne, was adopted, as displayed in Figure 1a. The mobile robot has been widely used in studies, such as in embedded computing, signal processing, inter-robot communication, robot control, feature extraction from images and sounds, swarm intelligence, and cooperative behavior in robotics.

The e-puck mobile robot is a two-wheeled mobile robot with an axle diameter of 4.12 cm and a maximum speed of 15 cm/s. The infrared sensor has a sensing range of approximately 360° and is symmetrical. The sensor performs tasks such as object detection, distance detection, and obstacle avoidance. The sensors S₀–S₃ that are on the right side of the robot are mounted at 10°, 45°, 90°, and 135°, respectively. Each sensor can detect a distance between 1 and 6 cm, as shown in Figure 1b.

3. Proposed Type-2 Fuzzy Controller Based on an Evolutionary Algorithm

An IT2FNC was proposed to realize wall-following control. The associated DGDE learning algorithm can be used to adjust the parameters of the IT2FNC.

3.1. Interval Type-2 Fuzzy Neural Controller

In this section, the structure of the IT2FNC is introduced. Figure 2 displays the structure of the IT2FNC. $X_1,\cdots ,X_{\mathrm{n}}$ represents the input of IT2FNC, whereas $Y_{Left}$ and $Y_{Right}$ represent the left and right wheel speed of the robot, respectively. To reduce the computational complexity of the order reduction during defuzzification, this study adopted the centers of sets (COS) [11,19] to conduct the order reduction process. A functional link neural network (FLNN) with a nonlinear combination input was added in consequent part of a fuzzy rule [20]. Figure 2 presents the five-layer structure of an IT2FNC. The IT2FNC consists of an input layer, a membership function layer, a firing layer, a consequent layer, and an output layer. The if–then rule can be expressed as follows:

$$\begin{gathered} \mathrm{Rule}j:\mathrm{IF}{x}_1\mathrm{is}{\tilde{A}}_{1j}\mathrm{and}{x}_2\mathrm{is}{\tilde{A}}_{2j}\dots \mathrm{and}{x}_i\mathrm{is}{\tilde{A}}_{ij}\dots \mathrm{and}{x}_n\mathrm{is}{\tilde{A}}_{nj} \\ \mathrm{THEN}{y}_j={{\displaystyle \sum }}_{k=1}^M{\omega }_{kj}{\varnothing }_k, \\ ={\omega }_{1j}{\varnothing }_1+{\omega }_{2j}{\varnothing }_2+\dots +{\omega }_{Mj}{\varnothing }_M, \end{gathered}$$

(1)

where $n$ is the number of inputs, $x_i$ represents the ith input, $y_j$ denotes the output of the jth fuzzy rule, ${\tilde{A}}_{1j},{\tilde{A}}_{2j},\dots ,{\tilde{A}}_{nj}$ represent the interval type-2 fuzzy sets, ${\omega }_{kj}$ is the link weight, ${\varnothing }_k$ represents the basis trigonometric function, and $M$ is the number of basis functions.

The five-layer structure of the IT2FNC is described as follows:

Layer 1 (input layer): This layer only imports the input data into the next layer:

$$u_i^{\left(1\right)}=x_i.$$

(2)

Layer 2 (membership function layer): This layer performs the fuzzification. Each node in this layer defines an interval type-2 fuzzy set, as displayed in Figure 3. The Gaussian primary membership function has an uncertainty mean $\left[m_{ij1},m_{ij2}\right]$ and standard deviation ${\sigma }_{ij}$ and is expressed as follows:

$$u_{ij}^{\left(2\right)}=exp\left(-\frac{{\left[u_i^{\left(1\right)}-m_{ij}\right]}^2}{{\sigma }_{ij}^2}\right)\equiv N(m_{ij},{\sigma }_{ij};u_i^{\left(1\right)}),m_{ij}\in \left[m_{ij1},m_{ij2}\right].$$

(3)

The membership degree of the Gaussian primary membership function $u_{ij}^{\left(2\right)}$ is called the footprint of uncertainty (FOU) and is expressed as the upper bound ${\overline{u}}_{ij}^{\left(2\right)}$ and the lower bound ${\underset{\_}{u}}_{ij}^{\left(2\right)}$. The membership degree is expressed as follows:

$${\overline{u}}_{ij}^{\left(2\right)}\left(u_i^{\left(1\right)}\right)=\{\begin{array}{cc}N\left(m_{ij1},{\sigma }_{ij};u_i^{\left(1\right)}\right),\hfill & if{u}_i^{\left(1\right)}<m_{ij1}\\ 1,\hfill & if{m}_{ij1}\le u_i^{\left(1\right)}\le m_{ij2},\\ N\left(m_{ij2},{\sigma }_{ij};u_i^{\left(1\right)}\right),\hfill & if{u}_i^{\left(1\right)}>m_{ij2}\end{array}$$

(4)

and

$${\underset{\_}{u}}_{ij}^{\left(2\right)}\left(u_i^{\left(1\right)}\right)=\{\begin{array}{c}N\left(m_{ij2},{\sigma }_{ij};u_i^{\left(1\right)}\right),if{u}_i^{\left(1\right)}\le \frac{m_{ij1}+m_{ij2}}{2}\hfill \\ N\left(m_{ij1},{\sigma }_{ij};u_i^{\left(1\right)}\right),if{u}_i^{\left(1\right)}>\frac{m_{ij1}+m_{ij2}}{2}\hfill \end{array}.$$

(5)

The output of each node is represented by the following interval: $\left[{\overline{u}}_{ij}^{\left(2\right)},{\underset{\_}{u}}_{ij}^{\left(2\right)}\right]$.

Layer 3 (firing layer): Each node is a rule node and uses an algebraic product operation to obtain the firing strengths ${\overline{u}}_j^{\left(3\right)}\mathrm{and}{\underset{\_}{u}}_j^{\left(3\right)}$ of each rule node. The firing strength of each rule node is defined as follows:

$${\overline{u}}_j^{\left(3\right)}={\displaystyle \prod }_i{\overline{u}}_{ij}^{\left(2\right)}\mathrm{and}{\underset{\_}{u}}_j^{\left(3\right)}={\displaystyle \prod }_i{\underset{\_}{u}}_{ij}^{\left(2\right)},$$

(6)

where ${{\displaystyle \prod }}_i{\overline{u}}_{ij}^{\left(2\right)}$ and ${{\displaystyle \prod }}_i{\underset{\_}{u}}_{ij}^{\left(2\right)}$ represent the firing strength of the interval’s upper bound and lower bound, respectively.

Layer 4 (consequent layer): Interval type-2 fuzzy sets are reduced to interval type-1 fuzzy sets through a type-reduction operation. The traditional type-2 order reduction method is based on highly complex calculation. Therefore, the centers of sets (COS) method [11] is adopted for implementing the reduction process and is described as follows:

$$Y\left(x\right)=\left[y_l,y_r\right]=\frac{{{\displaystyle \int }}_{a^1}^{}\cdots {{\displaystyle \int }}_{a^M}^{}{{\displaystyle \int }}_{f^1\in \left[{\underset{\_}{f}}^1,{\overline{f}}^1\right]}^{}\cdots {{\displaystyle \int }}_{f^M\in \left[{\underset{\_}{f}}^M,{\overline{f}}^M\right]}^{}1}{\frac{{{\displaystyle \sum }}_{i=1}^M{f}^i{a}^i}{{{\displaystyle \sum }}_{i=1}^M{f}^i}},$$

(7)

$${\underset{\_}{u}}_{}^{\left(4\right)}=y_l=\frac{{{\displaystyle \sum }}_{j=1}^R{\underset{\_}{u}}_j^{\left(3\right)}\left({{\displaystyle \sum }}_{k=1}^M{\omega }_{kj}{\varphi }_k\right)}{{{\displaystyle \sum }}_{j=1}^R{\underset{\_}{u}}_j^{\left(3\right)}},$$

(8)

and

$${\overline{u}}_{}^{\left(4\right)}=y_r=\frac{{{\displaystyle \sum }}_{j=1}^R{\overline{u}}_j^{\left(3\right)}\left({{\displaystyle \sum }}_{k=1}^M{\omega }_{kj}{\varphi }_k\right)}{{{\displaystyle \sum }}_{j=1}^R{\overline{u}}_j^{\left(3\right)}},$$

(9)

where ${{\displaystyle \sum }}_{k=1}^M{\omega }_{kj}{\varphi }_k$ represents a nonlinear combination of FLNN inputs, ${\omega }_{kj}$ represents the link weight, and ${\varnothing }_k$ represents the functional expansion of FLNN inputs. The functional expansion is based on basis trigonometric functions and defined as follows:

$$\left[{\varphi }_1,{\varphi }_2,\cdots ,{\varphi }_M\right]=\left[x_1,\sin \left(\pi x_1\right),\cos \left(\pi x_1\right),\cdots ,x_n,\sin \left(\pi x_n\right),\cos \left(\pi x_n\right)\right],$$

(10)

where $\mathrm{M}=3\times \mathrm{n}$ is the number of basis functions and n is the number of inputs.

Layer 5 (output layer): The node output of layer 5 is defuzzified by computing the average of ${\overline{u}}_{}^{\left(4\right)}$ and ${\underset{\_}{u}}_{}^{\left(4\right)}$. The crisp value y is obtained as follows:

$$y=\frac{{\overline{u}}^{\left(4\right)}+{\underset{\_}{u}}^{\left(4\right)}}{2}=u^{\left(5\right)}.$$

(11)

3.2. Proposed DGDE

DE has the advantages of fast convergence and simple implementation. However, some problems, such as low precision and becoming easily trapped into local optima, are encountered when complex problems are solved. Therefore, an efficient DGDE algorithm is proposed to overcome the shortcomings of traditional DE. The steps of DGDE are described as follows:

Step 1: Initialization and coding

By setting the parameter vectors of DE and randomly initializing the target vector $X_{i,G}$, the mathematical model is expressed as follows:

$$X_{i,G}=\left[x_{i,G}^1,x_{i,G}^2,\cdots ,x_{i,G}^D\right]$$

(12)

where $i=1,2,\cdots ,NP$; $NP$ is the number of population; $X_{i,G}$ represents the ith parameter vector in Gth generation, and $D$ is the number of dimensions.

The proposed DGDE is used to adjust the parameters of the IT2FNC. All the parameters in the IT2FNC are coded into one vector. Each vector represents an IT2FNC. The adjustable parameters in each IT2FNC are the uncertainty mean $m_{ij}$, standard deviation ${\sigma }_{ij}$, displacement value of the uncertainty mean $d_{ij}$, and link weight ${\omega }_{kj}$. Where the $d_{ij}=m_{ij2}-m_{ij1}>0$. The coding format is presented in Figure 4.

Step 2: Vector group

The fitness values of all parameter vectors are sorted in descending order. The initial group number of all vectors is set to zero (Figure 5).

The computation of the fitness values for the individuals in the population will be discussed later, see Equations (24)–(30). The highest fitness value of the vectors is set as the new group leader. This implies that the group number is updated from zero to one, as shown in Figure 6. On the basis of the average distance difference and the average fitness difference between these ungrouped vectors (i.e., group number 0) and the group leader, the threshold value of similarity comprises the threshold values of fitness and distance:

$$DI{S}^g={{\displaystyle \sum }}_{i=1}^{NP}{{\displaystyle \sum }}_{j=1}^D\sqrt{{\left(L_j^g-X_j^i\right)}^2},if{X}_{}^i\mathrm{is}\mathrm{ungrouped},$$

(13)

$$FI{T}^g={{\displaystyle \sum }}_{i=1}^{NP}\left|Fit\left(L_{}^g\right)-Fit\left(X_{}^i\right)\right|,if{X}_{}^i\mathrm{is}\mathrm{ungrouped},$$

(14)

$$Average\_Distance\left(ADI{S}^g\right)=\frac{DI{S}^g}{NI},$$

(15)

$$Average\_Fitness\left(AFI{T}^g\right)=\frac{FI{T}^g}{NI},$$

(16)

where $D$ represents the encoded dimension, $NP$ denotes the number of parameter vectors, $L_j^g$ is the jth dimension of the gth group leader, $NI$ is the total number of ungrouped vectors, and $ADI{S}^g$ and $AFI{T}^g$ represent the distance threshold value and the fitness threshold value of the gth group, respectively.

The difference of the distance values $\left(Di{s}^i\right)$ and the difference of the fitness values $Fi{t}^i$ between the ungrouped vectors and the leader vectors are calculated as follows:

$$Di{s}^i={{\displaystyle \sum }}_{j=1}^D\sqrt{{\left(L_j^g-X_j^i\right)}^2},$$

(17)

$$Fi{t}^i=\left|Fit\left(L^g\right)-Fit\left(X^i\right)\right|$$

(18)

If the $Di{s}^i<ADI{S}^g$ and $Fi{t}^i<AFI{T}^g$ conditions are satisfied for a vector, then the vector is similar to the gth group leader. Therefore, these vectors are grouped together and the group number is updated to $g$. Otherwise, the vectors are not grouped together. Figure 7 shows that the vectors X_2,G and X_i,G are similar to the 1st group leader.

Next, if any ungrouped vectors exist, the ungrouped vector with the highest fitness value is set as the group leader in a new group. Figure 8 presents the vector X_3,G from the ungrouped vectors that has the highest fitness value and is set as a new group leader (i.e., the 2nd group leader). The grouping process is completed when no ungroup vectors exist.

Step 3: Mutation

Two new mutation methods are proposed for improving the disadvantages of the traditional DE algorithms. The modified formula is expressed as follows:

Mutation method 1:

$$U_{i,G+1}=X_{rL,G}+F\left(X_{r1,G}-X_{r2,G}\right)$$

(19)

Mutation method 2:

$$U_{i,G+1}=X_{best,G}+F\left(X_{rL,G}-X_{r1,G}\right)+F\left(X_{r2,G}-X_{r3,G}\right)$$

(20)

where $U_{i,G+1}=\left[u_{i,G+1}^1,u_{i,G+1}^2,\cdots ,u_{i,G+1}^D\right],F$ is the mutation weight factor. Moreover, the weight factor is set to 0.5 as a general rule, $X_{best,G}$ is the best fitness vector, and $X_{rL,G}$ is a random leader selected from all the group leaders. Because the traditional DE method easily falls into local optima, this study employed the random leader as the base vector to increase the search ability effectively, as presented in Equation (19). In Equation (20), the best vector is set as the base vector, and three random vectors and one random leader vector are used. The mutated vector in Equation (20) revolves around the best vector and enhances the search ability in the solution space.

Step 4: Recombination

The recombination operation crosses the mutation vector with the target vector and generates a new vector $V_{i,G+1}$. The definition of $V_{i,G+1}$ is expressed as follows:

$$V_{i,G+1}=\left[v_{i,G+1}^1,v_{i,G+1}^2,\cdots ,v_{i,G+1}^D\right]$$

(21)

$$v_{i,G+1}^j=\{\begin{array}{c}{u}_{i,G+1}^j,ifran{d}^j\left(0,1\right)\le CR\\ x_{i,G+1}^j,otherwise\end{array}j=1\dots D$$

(22)

where $ran{d}^j\left(0,1\right)$ represents random values between zero and one in jth dimension, and CR is the crossover rate. The higher CR value represents the higher similarity between the vector and the mutation vector.

Step 5: Selection

The fitness values of vectors are evaluated for selecting the target vectors in the next generation. If the fitness value of a vector is less favorable than the current target vector, the target vector will remain in the next generation. The selection operation is described as follows:

$$X_{i,G+1}=\{\begin{array}{ll}{V}_{i,G+1},ifFit\left(V_{i,G+1}\right)& >Fit\left(X_{i,G}\right)\\ X_{i,G},& otherwise\end{array}$$

(23)

The flowchart of DGDE is presented in Figure 9.

3.3. Wall-Following Control of Mobile Robots

Recently, some researchers have proposed the wall-following control of mobile robots for using reinforcement learning. Jhang et al. [21] used an interval type-2 recurrent fuzzy cerebellar model articulation controller (IT2RFCMAC). They adopted a Takagi–Sugeno–Kang (TSK) in the consequent part in IT2RFCMAC. The TSK is a linear function and simple implementation. However, in complex problems, nonlinear functions are better in terms of performance than linear functions. Therefore, in this study a nonlinear functional link neural network of the proposed IT2FNC was used as the consequent part to improve control performance. The proposed IT2FNC that based on DGDE is demonstrated. Wall-following control was utilized by the IT2FNC to control the mobile robot. A reinforcement learning strategy was used to adjust the controller parameters of the proposed IT2FNC. The block diagram for the wall-following control of the mobile robot is presented in Figure 10. The four input signals of the proposed IT2FNC are the $S_0$, $S_1$, $S_2$, and $S_3\mathrm{distances}$, which are measured by the infrared sensor. The outputs of the IT2FNC are the rotational speeds V_L and V_R of the two wheels.

To allow mobile robots to be used in different environments, the training environment in this study featured straight lines, corners, right-angled corners, and slope lines. Figure 11 presents the 1.7 × 1.6 m² training environment.

To avoid collision with obstacles and deviation from the wall during the wall-following control learning process, three terminal conditions of wall-following control learning were specified:

• If the total moving distance of the mobile robot was larger than the predefined maximal distance of the training environment, the mobile robot successfully moved in a circular path in an unknown environment.
• The mobile robot collided with the wall when the measured distance from any infrared sensor was less than 1 cm, as displayed in Figure 12a.
• The mobile robot deviated from the wall when the measured distance $S_2$ was greater than 6 cm, as displayed in Figure 12b.

The proposed DGDE was used to train the parameters of the IT2FNC. Each vector in the DGDE represents a solution of an IT2FNC. When any terminal condition during the wall-following control learning process is satisfied, a fitness function was used to evaluate the performance of the mobile robot. The proposed fitness function comprises three sub-fitness functions—$S{F}_1$, $S{F}_2$, and $S{F}_3$. $S{F}_1$, $S{F}_2$, and $S{F}_3$ represent the total moving distance, the distance between the robot and the wall, and the degree of parallelism between the robot and the wall, respectively. The three sub-fitness functions are defined as follows:

(1)

$S{F}_1$: If the moving distance $R_{dis}$ was greater than the predefined value $R_{stop}$, the robot successfully moved around a circular path in the training environment and set $R_{dis}=R_{stop}$. The sub-fitness function $S{F}_1$ is defined as follows:

$$S{F}_1=R_{stop}-R_{dis}$$

(24)

(2)

$S{F}_2$: The goal of the wall-following control was to maintain a fixed distance between the side of the robot and the wall. Therefore, the sub-fitness function $S{F}_2$ is defined as the average of $WD\left(t\right)$, where $WD\left(t\right)$ represents the distance between the side of the robot and the wall at each time step, and is defined as follows:

$$WD\left(t\right)=\left|S_2\left(t\right)-d_{wall}\right|$$

(25)

$$S{F}_2=\frac{{{\displaystyle \sum }}_{t=1}^{T_{stop}}WD\left(t\right)}{T_{stop}}$$

(26)

where $d_{wall}$ is the pre-defined fixed distance ($\mathrm{i}.\mathrm{e}.,d_{wall}$ = 4 cm), as presented in Figure 13a. $T_{stop}$ is the total number of time steps in a learning process. If the robot remains at a fixed distance from the wall, the $S{F}_2$ value is equal to zero.

(3)

$S{F}_3$: This sub-fitness function was used for evaluating the degree of parallelism between the robot and the wall. If the robot was parallel to the wall, the angle $\theta$ between the robot and wall was 90°. On the basis of the law of cosines, $x\left(t\right)$ must have the same value as that of $R{S}_2$, as presented in Figure 13b.

$$R{S}_1=r+{\delta }_1,R{S}_2=r+{\delta }_2$$

(27)

$$x\left(t\right)=\sqrt{R{S_1}^2+R{S_2}^2-2R{S}_{1}R{S}_2\cos (45°)}$$

(28)

where $r$ is the radius of the robot; ${\delta }_1$ and ${\delta }_2$ represent the distance between the sensor 1 and the wall and that between sensor 2 and the wall, respectively, and $S{F}_3$ represents the average value of the degree of parallelism during movement. If the robot is parallel to the wall, $S{F}_3$ is equal to zero.

$$S{F}_3=\frac{{{\displaystyle \sum }}_{t=1}^{T_{stop}}\left|R{S}_2-x\left(t\right)\right|}{T_{stop}}$$

(29)

Therefore, the proposed fitness function is defined as follows:

$$F(·)=\frac{1}{1+\left(S{F}_1+S{F}_2+S{F}_3\right)}$$

(30)

Figure 14 shows the block diagram of learning process of wall-following control. Each solution represents an IT2FNC controller, which is evaluated by the fitness function. Additionally, it automatically adjust parameters of IT2FNC using evolutionary strategies. The best solution will be replaced when a better solution exits in each generation.

3.4. Experimental Results of the Wall-Following Control

To verify the effectiveness of the proposed method, the performance of the WFM controller while using the proposed DGDE-1 (mutation method 1) and DGDE-2 (mutation method 2) were compared with the performance of the WFM control while using other methods. Each method was evaluated 10 times to verify the stability of each algorithm.

The initial parameters of the DGDE are the number of the population (NP), crossover rate, generation, weighting factor of mutation, and number of fuzzy rules, as presented in

Table 1. Initial parameters of DGDE.

NP	CR	F	Generation	Rule
30	0.9	0.5	3000	5,6,7

. Moreover, we considered different fuzzy rule numbers for performance evaluation.

Table 2. Performance evaluation of different fuzzy rule numbers. STD: standard deviation.

	Number of Rules	DGDE-1			DGDE-2
Fitness Value		5	6	7	5	6	7
Best		0.932	0.961	0.948	0.949	0.962	0.953
Worst		0.865	0.891	0.821	0.908	0.919	0.891
Average		0.903	0.933	0.911	0.923	0.942	0.913
STD		0.017	0.012	0.200	0.012	0.009	0.016
Number of successful runs		10	10	10	10	10	10

presents the performance evaluation results of different fuzzy rule numbers. The IT2FNC with six fuzzy rules was more efficient than those with five and seven fuzzy rules.

Table 3. Performance comparison of various algorithms in the wall-following control behavior.

	Evaluation Items	Fitness Value				Number of Success Runs	Computation Time (H:M:S)
Algorithms		Best	Worst	Average	STD
DGDE-1		0.961	0.891	0.933	0.012	10	5:01:39
DGDE-2		0.962	0.919	0.942	0.009	10	4:38:56
JADE [22]		0.950	0.860	0.911	0.029	10	10:11:03
Rank-DE [23]		0.958	0.867	0.922	0.025	10	18:21:05
DE [14]		0.941	0.262	0.786	0.184	8	1:03:38
PSO [12]		0.947	0.206	0.738	0.257	7	5:49:45
ABC [15]		0.932	0.354	0.735	0.149	8	2:57:23

presents the performance evaluations of different algorithms. In this table, the performance indexes include the best fitness function, the worst fitness function, the average fitness function, the standard deviation (STD), the number of successful runs, and computation time for one training. The number of successful runs is the number of times the robot moved successfully around a circular path in the training environment. Figure 15 presents the learning curves of the WFM control when various evolutionary algorithms are used. The proposed DGDE achieved superior fitness values and successful runs than other methods. On the other hand, the proposed method also compares with the IT2RFCMAC controller [21].

Table 4. Performance comparison of different network architecture.

	Evaluation Items	IT2FNC		IT2RFCMAC [23]
Algorithms		DGDE-1	DGDE-2	DGDE
Best		0.961	0.962	0.925
Worst		0.891	0.919	0.868
Average		0.933	0.942	0.906

shows that the proposed IT2FNC with nonlinear functional link neural network performs better than IT2RFCMAC with linear TSK architecture [21].

To verify the WFM control performance of different learning algorithms, a training environment and two unknown testing environments were created, as presented in Figure 16, Figure 17 and Figure 18. The testing environment in Figure 17 focuses on many difficult and large curves, whereas the other testing environment in Figure 18 focuses on many right angle curves. The fitness value was used to evaluate the WFM, and the detailed comparison results are presented in

Table 5. Fitness value of various algorithms in the testing environments.

	Evaluation Items	Fitness Value
Algorithms		Training Environment	Testing Environment 1	Testing Environment 2
DGDE-1		0.961	0.901	0.864
DGDE-2		0.962	0.899	0.872
JADE [22]		0.950	0.895	0.862
Rank-DE [23]		0.958	0.874	0.789
DE [14]		0.941	fail	fail
PSO [12]		0.947	0.828	0.721
ABC [15]		0.932	fail	fail

.

4. Cooperative Carrying and Navigation Control of Multi-Evolutionary Mobile Robots

In this section, the cooperative carrying and navigation control of multi-evolutionary mobile robots is discussed. Figure 19 shows that the distance between two robots $R_d$ was set to 15 cm and a rectangular object was placed on the two robots. The front and rear robots represent the leader and follower, respectively. In the experiments, the leader explored the front environment, and the follower assisted the leader in lifting objects to achieve obstacle avoidance and prevent objects being dropped during cooperative carrying.

4.1. Wall-Following Control of the Cooperative Carrying Method

A dual controller is proposed for the cooperative carrying of two mobile robots. An auxiliary controller was incorporated in the follower robot to learn the WFM of cooperative carrying. The block diagram is presented in Figure 20. The auxiliary controller contained five input signals and two output signals. The inputs are the sensed distances by the follower’s sensors ($S_0$, $S_1$, $S_2$, and $S_3$) and the distances between two robots ($R_d$). The outputs are the rotational speeds V_L and V_R of the two wheels.

To implement cooperative carrying in an unknown environment, the training environment featured straight lines, smooth curves, U-shaped curves, and continuous curves to train the follower’s auxiliary controller. Figure 21 displays the 1.5 × 1.4 m² training environment.

To avoid collision with obstacles and objects being dropped during the cooperative carrying learning process, five terminal conditions were specified:

(1)

If the measured distance from one of the sensors in the follower robot is less than 1 cm, the follower robot collides with the obstacles.

(2)

If the measured distance of the sensor $S_2$ is higher than 6 cm, the follower robot deviates from the wall.

(3)

If the measured distance between the leader robot and follower robot $R_d$ is less than 10 cm or higher than 20 cm, the leader and follower robots are inferred to be very close or very far.

(4)

If the measured distance of the sensor is less than the height $R_h$ of the robot, the object is dropped by robots.

(5)

If the measured distance $S_{wall}$ of the sensor is less than 1 cm or greater than 7.5 cm, the object approaches the wall or deviates from the wall.

When one of the aforementioned conditions is satisfied, the cooperative carrying of the robots has failed. If the robots engaging in cooperative carrying can successfully move around a circular path in the training environment, the auxiliary controller completes the training process. The time step of the robot moving in the training environment is defined as the fitness function $F(·)$:

$$F(·)=T_{stop}$$

(31)

4.2. Navigation Control of Cooperative Carrying

This paper also proposes an effective navigation control method for cooperative carrying in unknown environments. The manager mode automatically selects between the TGM and WFM on the basis of the relative position of the mobile robot and the target location.

(1) Toward-Goal Mode

In the navigation control of the unknown environment, the robot used infrared sensors to detect the object. In order to turn towards the goal position, the mobile robot calculates the angle difference ${\mathsf{\theta }}_{TG}$ between the current direction of the robot and the target direction, as presented in Figure 22:

$${\mathsf{\theta }}_{TG}={\mathsf{\theta }}_{Robot}-{\mathsf{\theta }}_{Goal}$$

(32)

where ${\mathsf{\theta }}_{Robot}$ is the angle between the mobile robot and the x axis and ${\mathsf{\theta }}_{Goal}$ is the angle between the goal and the x axis in inertial coordinate system.

To avoid objects being dropped during the cooperative carrying process, both the follower and leader move in the same direction and at the same speed, as displayed in Figure 23.

(2) Manager Mode

The robot is divided into three zones—$O_1$, $O_2$, and $O_3$, as displayed in Figure 24. In the process of TGM, the manager mode switches to the left WFM or the right WFM on the basis of the zone of the robot $O_i$ and on the basis of which sensor $S_i$ (i = 0, 2, 3, …,7) detected an obstacle.

• Left wall-following control:

  (i)

  The goal direction is located at $O_1$, and $S_7$ or $S_6$ detects obstacles.

  (ii)

  The goal direction is located at $O_2$, and $S_7$, $S_6$, and $S_5$ detect obstacles.

• Right wall-following control:

  (i)

  The goal direction is located at $O_1$, and $S_0$ or $S_1$ detects obstacles.

  (ii)

  The goal direction is located at $O_3$, and $S_0$, $S_1$, and $S_2$ detect obstacles.

Before switching to the WFM, the manager mode determines which robot is closest to the obstacle. If the leader robot encounters an obstacle, it will execute a prerotation process. This aim of this process is to enable the two robots to approach the wall for a smooth navigational control, as displayed in Figure 25.

The leader robot turns in order to be positioned parallel to the wall and records the angle of rotation ${\mathsf{\theta }}_L$. Then, this message is sent to the follower robot. The follower robot turns to $\mathsf{\pi }-{\mathsf{\theta }}_L$°, maintains a fixed distance from the leader robot, and moves toward the obstacle. The manager mode of the two robots switch to WFM until the pre-rotation process of the follower robot is completed. The pre-rotation process is presented in Figure 26.

If the follower robot encounters an obstacle, the manager mode switches to the WFM; this process is displayed in Figure 27. If the target direction is located in $O_1$ of the follower robot and the distance between the sensor on right side $S_1$ (the sensor of left side $S_6$) and the wall is greater than 6 cm, the object being held by the two robots is inferred to have passed the obstacle. Then, the manager mode switches to the TGM, as displayed in Figure 28.

4.3. Experimental Results of Cooperative Carrying Control

In this subsection, the proposed method is used to verify the success of cooperative carrying control in unknown environments. Figure 29 demonstrates that the robots complete the wall-following control of cooperative carrying in the training environment. Moreover, to verify the performance of navigation control, two different test environments were created for testing whether the robots successfully accomplished cooperative carrying and navigation control. The experimental results of the two test environments are presented in Figure 30. In this experiment, the average distance (RD) between the two robots and the average distance (FWD) between the follower robot and the wall were evaluated. The results are presented in

Table 6. Evaluation of the cooperative carrying performance.

	Evaluation Items	Training Environment		Testing Environment 1		Testing Environment 2
Algorithms		RD (cm)	FWD (cm)	RD (cm)	FWD (cm)	RD (cm)	FWD (cm)
DGDE-1		16.18	3.81	16.54	3.61	17.03	3.42
DGDE-2		15.43	3.96	15.76	3.89	17.16	4.23

. If the RD is large, the two robots (i.e., the leader robot and the follower robot) are not at a suitable distance during the cooperative transport control, and the object drops easily. However, if the FWD is very large or very small, the robots pass the curves with poor efficiency, and the object is easily moved and dropped.

5. Conclusions

Aiming at the navigation control for cooperative carrying in an unknown environment, this study proposed an IT2FNC based on a dynamic group differential evolution to realize the carrying control and WFM control for mobile robots. At the same time, the developed DGDE learning algorithm adopts dynamic grouping and local search methods, which enhance the search ability and convergence stability of the traditional DE method, and is used to adjust IT2FNC parameters. On this basis, manager mode is established to assist mobile robots in navigation control. The manager mode automatically selects between the WFM or the TGM base on the relative position between the mobile robot and the target location. In addition, the pre-rotation mechanism was employed to accomplish cooperative carrying control. In the training process, the best fitness function, the worst fitness function, the average fitness function, the standard deviation (STD), the number of successful runs, and the computation time of the proposed DGDE were 0.962, 0.919, 0.942, 0.009, 10, and 4:38:56, respectively. Although DE and ABC are shorter than the proposed method at the computation time, DE and ABC had only eight successful runs during 10 runs. Experimental results revealed that the proposed method achieved a superior WFM performance than other methods and successfully accomplished the navigation control of cooperative carrying to target locations in unknown environments. Since the trained controller using reinforcement learning needs to take a long time, future research work needs to implement the proposed learning algorithm on chip to improve the learning speed.