# Optimising Robot Swarm Formations by Using Surrogate Models and Simulations

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- 1.
- The study, training, and testing of six surrogate models to predict the behaviour of a UAV swarm in formation.
- 2.
- A hybrid EA (HEA) to be used as the optimisation algorithm for the parameters of our formation system, which combines simulations and predictions to balance efficiency and accuracy.
- 3.
- The evaluation of the optimised formation swarm in 150 unseen scenarios comprising up to 30 UAVs in terms of accuracy and stability.

## 2. Related Works

## 3. Proposal

#### 3.1. Distributed Formation Algorithm^{3} (DFA^{3})

^{3}(DFA

^{3}) [7] was designed to arrange robots at the vertices of a convex polyhedron surrounding a central point of interest, e.g., a rogue drone trespassing a restricted area. Each UAV calculates its relative orientation and distance to the rest of UAVs based on the beacon signals received from each swarm member. This formation algorithm does not rely on any localisation system, such as GPS, and it works on dynamic scenarios, as the UAV positions are calculated with respect to the other UAVs and to the rogue drone (RD) using attracting and repelling forces to achieve a stable equilibrium. Figure 1a shows fourteen UAVs surrounding a central rogue drone and the attracting/repelling forces between them, while Figure 1b shows the attracting/repelling forces between the central rogue drone and the other UAVs. Only forces involving UAVs i, j, and k were explicitly named as examples, to make sure the figures are comprehensible. As the UAVs move, these forces change their orientation and intensity until the final stable positions are achieved. Hence, each UAV does not have a fixed final position in the formation that is known in advance. In our experiments, the central object is tracked using its own radio signal. However, other methods can be used such as LIDAR (light detection and ranging) or images from onboard cameras.

^{3}is detailed in Figure 2 and its pseudocode can be found in [7]. Each UAV executes the same algorithm using the swarm’s optimal parameters and formation radius, i.e., the desired distance to the rogue drone ${D}_{CENTRE}$, which is a constant value. Once the vector $\overrightarrow{r}=\{{r}_{x},{r}_{y},{r}_{z}\}$ is initialised, a calculation of the forces with respect to the other UAVs is performed based on the received beacons and the given distance threshold ${D}_{THRESHOLD}$. In the next step, the same calculation is performed, taking into account the rogue drone at the centre of the desired spherical formation, using the values of ${D}_{MIN}$ and the extra intensity ${F}_{CENTRE}$. The calculated inclination $\theta $ and azimuth $\varphi $ are finally obtained from the resulting vector $\overrightarrow{r}$ to be used as the new moving direction (in 3D space) for the UAV.

#### 3.2. Formation Fitness

#### 3.3. ARGoS Simulations and Scenario Modelling

^{3}is executed onboard each UAV and was parameterised using the aforementioned formation parameters, i.e., ${D}_{THRESHOLD}$, ${D}_{MIN}$, ${F}_{CENTRE}$, and $SPEED$. Obtaining a stable formation depends on the values of these parameters, requiring an optimisation process which takes into account the distance to the centre (${D}_{CENTRE}$) and the number of UAVs. In this article, we propose the study of swarms of three, five, ten, fifteen, and thirty UAVs, tripling our previous studies. This can only be possible if we use surrogate models to replace the costly simulations.

#### 3.4. Realistic Simulations vs. Surrogate Models

^{3}, we observed that the whole process was taking too long for large swarms (720 h for 30 runs optimising a swarm of 10 UAVs), limiting the number of UAVs we were able to use. Therefore, in this article, we study the use of surrogate models [25] to speed up the evaluation of the formation parameters, allowing not only having more robots in the swarm (we plan to reach 30 UAVs), but also allowing more accurate optimisations by increasing the number of evaluations and improving the optimisation algorithm’s solutions.

#### 3.5. Surrogate Models

#### 3.5.1. Gaussian Processes (GPs)

#### 3.5.2. Artificial Neural Network (ANN)

## 4. Optimisation Approach

## 5. Experiments and Results

^{3}, the selected surrogate model, and the ARGoS simulations to evaluate the individuals of the HEA. Finally, the optimal parameters are used to test our formation algorithm on 30 unseen scenarios per case study to address its robustness. The source code of the DFA

^{3}, the problem instances, surrogate models, and datasets are available at https://gitlab.uni.lu/adars/dfa3 (accessed on 9 May 2023).

#### 5.1. Case Studies and Scenarios

^{3}running onboard, keeping the parametrised speed ($SPEED$). Since the same repelling forces between UAVs prevent them from being too close to each other, no extra collision avoidance algorithm was needed, providing the swarm parameters are optimal, allowing it to work as intended.

#### 5.2. Experimental Setup

#### 5.3. Data Collection

#### 5.4. Surrogate Training

#### 5.5. Surrogate Testing

#### 5.6. Evolutionary Optimisation

#### 5.7. Robustness Evaluation

_{UAV}), and the distance to the rogue drone at the centre (D

_{RD}) for all the scenarios in which a successful formation was achieved. It can be seen that the measured D

_{RD}was always closer to the desired ${D}_{CENTRE}$, i.e., 3 m for 3, 5, and 10 UAVs; 4 m for 15 UAVs; and 5 m for 30 UAVs. The distance between UAVs (D

_{UAV}) showed little variations, representing equally spaced UAVs in the formation, except for the most difficult case, i.e., 30 UAVs, although the variation is still lower than 13%. We have observed that for swarms of 10, 15, and 30 UAVs, there were some formation attempts that failed: 9, 13, and 10. Observing more than 50% successful formations is a good result, as the HEA has optimised only one particular scenario, in contrast to the 30 tested in this section. An increase in the robustness can be easily achieved by optimising not just one but several scenarios in parallel and using the average fitness values obtained to evaluate each swarm configuration, as has been previously observed in [7,24].

## 6. Conclusions

^{3}), and proposed six predictors, five based on Gaussian processes (GPs) and one based on an artificial neural network (ANN). Then, we have calculated two datasets using real ARGoS simulations. The first was used to train the predictors and the second was used to test their accuracy in terms of the median square error (MSE) and the computation time. After that, we optimised a new scenario (different initial UAV positions) using our proposed hybrid evolutionary algorithm (HEA) to obtain optimal configurations for each UAV swarm. Finally, we tested the best configurations achieved on 30 unseen scenarios per case study to evaluate the robustness of the calculated configurations.

^{3}, surrogate models, and ARGoS simulations plus the HEA worked on 79% of the new, unseen scenarios where it was tested, reducing optimisation times by 920% on average.

^{3}could be used for a variety of initial conditions.

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Brust, M.R.; Danoy, G.; Stolfi, D.H.; Bouvry, P. Swarm-based counter UAV defense system. Discov. Internet Things
**2021**, 1, 2. [Google Scholar] [CrossRef] - Chung, S.J.; Ahsun, U.; Slotine, J.J.E. Application of Synchronization to Formation Flying Spacecraft: Lagrangian Approach. J. Guid. Control. Dyn.
**2009**, 32, 512–526. [Google Scholar] [CrossRef] - Cardona, G.A.; Calderon, J.M. Robot Swarm Navigation and Victim Detection Using Rendezvous Consensus in Search and Rescue Operations. Appl. Sci.
**2019**, 9, 1702. [Google Scholar] [CrossRef] - Saeedi, S.; Trentini, M.; Seto, M.; Li, H. Multiple-Robot Simultaneous Localization and Mapping: A Review. J. Field Robot.
**2015**, 33, 3–46. [Google Scholar] [CrossRef] - Hauri, S.; Alonso-Mora, J.; Breitenmoser, A.; Siegwart, R.; Beardsley, P. Multi-Robot Formation Control via a Real-Time Drawing Interface. In Field and Service Robotics: Results of the 8th International Conference; Yoshida, K., Tadokoro, S., Eds.; Springer: Berlin/Heidelberg, Germany, 2014; pp. 175–189. [Google Scholar] [CrossRef]
- Pinciroli, C.; Trianni, V.; O’Grady, R.; Pini, G.; Brutschy, A.; Brambilla, M.; Mathews, N.; Ferrante, E.; Di Caro, G.; Ducatelle, F.; et al. ARGoS: A modular, parallel, multi-engine simulator for multi-robot systems. Swarm Intell.
**2012**, 6, 271–295. [Google Scholar] [CrossRef] - Stolfi, D.H.; Danoy, G. An Evolutionary Algorithm to Optimise a Distributed UAV Swarm Formation System. Appl. Sci.
**2022**, 12, 10218. [Google Scholar] [CrossRef] - Shahriari, B.; Swersky, K.; Wang, Z.; Adams, R.P.; de Freitas, N. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proc. IEEE
**2016**, 104, 148–175. [Google Scholar] [CrossRef] - Ong, Y.S.; Nair, P.B.; Keane, A.J. Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling. AIAA J.
**2003**, 41, 687–696. [Google Scholar] [CrossRef] - Yelten, M.B.; Zhu, T.; Koziel, S.; Franzon, P.D.; Steer, M.B. Demystifying Surrogate Modeling for Circuits and Systems. IEEE Circuits Syst. Mag.
**2012**, 12, 45–63. [Google Scholar] [CrossRef] - Cheng, S.; Prentice, I.C.; Huang, Y.; Jin, Y.; Guo, Y.K.; Arcucci, R. Data-driven surrogate model with latent data assimilation: Application to wildfire forecasting. J. Comput. Phys.
**2022**, 464, 111302. [Google Scholar] [CrossRef] - Poggi, C.; Rossetti, M.; Bernardini, G.; Iemma, U.; Andolfi, C.; Milano, C.; Gennaretti, M. Surrogate models for predicting noise emission and aerodynamic performance of propellers. Aerosp. Sci. Technol.
**2022**, 125, 107016. [Google Scholar] [CrossRef] - Westermann, P.; Evins, R. Surrogate modelling for sustainable building design—A review. Energy Build.
**2019**, 198, 170–186. [Google Scholar] [CrossRef] - Asher, M.J.; Croke, B.F.W.; Jakeman, A.J.; Peeters, L.J.M. A review of surrogate models and their application to groundwater modeling. Water Resour. Res.
**2015**, 51, 5957–5973. [Google Scholar] [CrossRef] - Alizadeh, R.; Allen, J.K.; Mistree, F. Managing computational complexity using surrogate models: A critical review. Res. Eng. Des.
**2020**, 31, 275–298. [Google Scholar] [CrossRef] - Bliek, L. A Survey on Sustainable Surrogate-Based Optimisation. Sustainability
**2022**, 14, 3867. [Google Scholar] [CrossRef] - Rosalie, M.; Kieffer, E.; Brust, M.R.; Danoy, G.; Bouvry, P. Bayesian optimisation to select Rössler system parameters used in Chaotic Ant Colony Optimisation for Coverage. J. Comput. Sci.
**2020**, 41, 101047. [Google Scholar] [CrossRef] - do Nascimento, R.G.; Fricke, K.; Viana, F. Quadcopter Control Optimization through Machine Learning. In AIAA Scitech 2020 Forum; American Institute of Aeronautics and Astronautics: Orlando, FL, USA, 2020. [Google Scholar] [CrossRef]
- Tang, Q.; Li, Y.; Deng, Z.; Chen, D.; Guo, R.; Huang, H. Optimal shape design of an autonomous underwater vehicle based on multi-objective particle swarm optimization. Nat. Comput.
**2020**, 19, 733–742. [Google Scholar] [CrossRef] - Lopes, H.J.; Lima, D.A. Evolutionary Tabu Inverted Ant Cellular Automata with Elitist Inertia for swarm robotics as surrogate method in surveillance task using e-Puck architecture. Robot. Auton. Syst.
**2021**, 144, 103840. [Google Scholar] [CrossRef] - Yue, H.; Medromi, H.; Ding, H.; Bassir, D. A novel hybrid drone for multi-propose aerial transportation and its conceptual optimization based on surrogate approach. J. Phys. Conf. Ser.
**2021**, 1972, 012103. [Google Scholar] [CrossRef] - Chandra, R.; Tiwari, A. Distributed Bayesian optimisation framework for deep neuroevolution. Neurocomputing
**2022**, 470, 51–65. [Google Scholar] [CrossRef] - Spiri Robotics. Spiri Mu. 2023. Available online: https://spirirobotics.com/ (accessed on 20 April 2023).
- Stolfi, D.H.; Danoy, G. Optimising Autonomous Robot Swarm Parameters for Stable Formation Design. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO’22, New York, NY, USA, 9–13 July 2022; pp. 1281–1289. [Google Scholar] [CrossRef]
- Cozad, A.; Sahinidis, N.V.; Miller, D.C. Learning surrogate models for simulation-based optimization. AIChE J.
**2014**, 60, 2211–2227. [Google Scholar] [CrossRef] - Seeger, M. Gaussian Processes For Machine Learning. Int. J. Neural Syst.
**2004**, 14, 69–106. [Google Scholar] [CrossRef] [PubMed] - Piironen, J. Gplite: General Purpose Gaussian Process Modelling. 2023. Available online: https://cran.r-project.org/package=gplite (accessed on 2 March 2023).
- Riedmiller, M.; Braun, H. A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, USA, 28 March–1 April 1993; Volume 1, pp. 586–591. [Google Scholar] [CrossRef]
- Fritsch, S.; Guenther, F.; Wright, M.N.; Suling, M.; Mueller, S.M. Neuralnet: Training of Neural Networks. 2023. Available online: https://CRAN.R-project.org/package=neuralnet (accessed on 2 March 2023).
- Holland, J.H. Adaptation in Natural and Artificial Systems; The MIT Press: Cambridge, MA, USA, 1992; p. 228. [Google Scholar] [CrossRef]
- Talbi, E.G. A Unified Taxonomy of Hybrid Metaheuristics with Mathematical Programming, Constraint Programming and Machine Learning. In Hybrid Metaheuristics; Talbi, E.G., Ed.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 3–76. [Google Scholar] [CrossRef]
- Goldberg, D.E.; Deb, K. A Comparative Analysis of Selection Schemes Used in Genetic Algorithms. Found. Genet. Algorithms
**1991**, 1, 69–93. [Google Scholar] [CrossRef] - Syswerda, G. Uniform Crossover in Genetic Algorithms. In Proceedings of the Proceedings of the 3rd International Conference on Genetic Algorithms, Fairfax, VA, USA, 1 June 1989; Morgan Kaufmann Publishers Inc.: San Francisco, CA, USA, 1989; pp. 2–9. [Google Scholar]
- Deb, K. Multi-Objective Optimization Using Evolutionary Algorithms; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2001. [Google Scholar]
- Lin, S. Computer solutions of the traveling salesman problem. Bell Syst. Tech. J.
**1965**, 44, 2245–2269. [Google Scholar] [CrossRef] - Benítez-Hidalgo, A.; Nebro, A.J.; García-Nieto, J.; Oregi, I.; Ser, J.D. jMetalPy: A Python framework for multi-objective optimization with metaheuristics. Swarm Evol. Comput.
**2019**, 51, 100598. [Google Scholar] [CrossRef] - Varrette, S.; Bouvry, P.; Cartiaux, H.; Georgatos, F. Management of an academic HPC cluster: The UL experience. In Proceedings of the 2014 International Conference on High Performance Computing & Simulation (HPCS), Bologna, Italy, 21–25 July 2014; pp. 959–967. [Google Scholar] [CrossRef]

**Figure 1.**Swarm of fourteen UAVs escorting the rogue drone at the centre. (

**a**) Attracting/repelling forces between UAVs. (

**b**) Attracting/repelling forces between UAVs in the swarm and the rogue drone. Only forces involving the UAV

_{i}, UAV

_{j}, and UAV

_{k}are represented.

**Figure 3.**Robot formation in the ARGoS simulator (swarm of five UAVs). Communications among swarm members are in blue. Rogue drone detections are in red.

**Figure 5.**Schema of the experiments proposed. First, data are collected from ARGoS simulations to train the surrogate models and test them. Second, the best surrogate model is used by the proposed hybrid EA to optimise the UAV swarm parameters. Finally, the optimal parameterisation is tested on a set of unseen scenarios to address the system robustness.

**Figure 6.**UAVs’ initial positions for the five case studies (100 scenarios per case study). The UAVs in each scenario are represented by a different colour.

**Figure 7.**UAVs’ final positions for the five case studies (scenarios ending in stable formations). The UAVs in each scenario are represented by a different colour.

# UAVs | # Scenarios | Dimensions | D_{CENTRE} |
---|---|---|---|

3 | 100 | 30 × 30 × 30 | 3 m |

5 | 100 | 30 × 30 × 30 | 3 m |

10 | 100 | 30 × 30 × 30 | 3 m |

15 | 100 | 30 × 30 × 30 | 4 m |

30 | 100 | 30 × 30 × 30 | 5 m |

**Table 2.**MSE values for the training process of predictors based on GPs and an ANN. Note that $gp\_lin$ (*) did not converge for five UAVs. Best values are in bold.

# UAVs | # Obs. | gp_lin | gp_sexp | gp_nn | gp_m32 | gp_m52 | ann |
---|---|---|---|---|---|---|---|

3 | 238 | 13.030 | 6.138 | 5.081 | 5.408 | 5.835 | 10.010 |

5 | 192 | * 11.665 | 4.059 | 3.643 | 3.720 | 3.898 | 5.295 |

10 | 118 | 7.542 | 2.236 | 0.995 | 1.759 | 1.926 | 2.525 |

15 | 119 | 12.042 | 5.083 | 4.486 | 4.747 | 4.980 | 4.398 |

30 | 86 | 16.882 | 6.224 | 5.735 | 5.926 | 6.187 | 6.220 |

**Table 3.**Elapsed training times in seconds for the six predictors. Note that $gp\_lin$ (*) did not converge for five UAVs. Best values are in bold.

# UAVs | # Obs. | gp_lin | gp_sexp | gp_nn | gp_m32 | gp_m52 | ann |
---|---|---|---|---|---|---|---|

3 | 238 | 0.457 | 1.211 | 2.012 | 1.021 | 1.128 | 45.934 |

5 | 192 | * 6.224 | 0.558 | 0.769 | 0.938 | 0.940 | 29.205 |

10 | 118 | 0.093 | 0.198 | 0.406 | 0.362 | 0.345 | 9.399 |

15 | 119 | 0.195 | 0.337 | 0.396 | 0.327 | 0.327 | 2.077 |

30 | 86 | 0.122 | 0.294 | 0.182 | 0.260 | 0.220 | 0.772 |

**Table 4.**MSE values for the predictions performed using the GP and ANN models compared with ARGoS simulations. Best values are in bold.

# UAVs | # Obs. | gp_lin | gp_sexp | gp_nn | gp_m32 | gp_m52 | ann |
---|---|---|---|---|---|---|---|

3 | 2220 | 12.564 | 6.596 | 6.199 | 6.279 | 6.399 | 8.943 |

5 | 1853 | 13.483 | 5.471 | 5.146 | 5.231 | 5.314 | 6.938 |

10 | 1096 | 11.680 | 3.977 | 3.966 | 3.841 | 3.872 | 5.319 |

15 | 923 | 15.797 | 4.635 | 4.349 | 4.455 | 4.563 | 4.879 |

30 | 728 | 20.714 | 6.258 | 6.004 | 6.115 | 6.250 | 8.622 |

**Table 5.**Average computing times in seconds for the six predictors compared with the corresponding ARGoS simulations (30 testing scenarios). Best values are in bold.

# UAVs | ARGoS | gp_lin | gp_sexp | gp_nn | gp_m32 | gp_m52 | ann | Speed-Up |
---|---|---|---|---|---|---|---|---|

3 | 6.601 | 0.190 | 0.190 | 0.190 | 0.191 | 0.190 | 0.122 | 54.1 |

5 | 10.169 | 0.191 | 0.191 | 0.195 | 0.190 | 0.191 | 0.124 | 82.0 |

10 | 24.342 | 0.190 | 0.190 | 0.188 | 0.190 | 0.190 | 0.124 | 196.3 |

15 | 83.414 | 0.191 | 0.191 | 0.191 | 0.190 | 0.189 | 0.125 | 667.3 |

30 | 446.939 | 0.189 | 0.194 | 0.193 | 0.191 | 0.192 | 0.124 | 3604.3 |

Mean: | 114.293 | 0.190 | 0.191 | 0.191 | 0.190 | 0.190 | 0.124 | 920.8 |

**Table 6.**Results from the optimisation of one new, unseen scenario for each case study. Best values are in bold.

# UAVs | Fitness | Time (Minutes) | Converged | ||||
---|---|---|---|---|---|---|---|

Mean | St Dev | Minimum | Mean | St Dev | Maximum | ||

3 | 2.804 | 0.625 | 1.217 | 13.570 | 0.781 | 15.332 | 100.0% |

5 | 6.770 | 8.245 | 3.833 | 29.429 | 2.185 | 32.924 | 96.7% |

10 | 6.033 | 0.370 | 5.348 | 85.214 | 10.352 | 115.564 | 100.0% |

15 | 26.699 | 19.390 | 8.055 | 157.386 | 42.317 | 225.209 | 60.0% |

30 | 10.356 | 0.887 | 9.419 | 1188.376 | 91.172 | 1449.161 | 100.0% |

**Table 7.**Fitness, distance between UAVs, and distance to the rogue drone in metres, all collected from the tests performed on 30 unseen scenarios per case study. Best values are in bold.

# UAVs | Fitness | D_{UAV} | D_{RD} | Formation | |||
---|---|---|---|---|---|---|---|

Mean | StDev | Mean | StDev | Mean | StDev | ||

3 | 3.673 | 0.698 | 5.116 | 0.104 | 3.058 | 0.001 | 100.0% |

5 | 5.602 | 1.032 | 4.765 | 0.030 | 3.070 | 0.002 | 100.0% |

10 | 7.119 | 0.625 | 4.178 | 0.083 | 3.011 | 0.001 | 70.0% |

15 | 10.790 | 0.492 | 5.758 | 0.009 | 4.098 | 0.001 | 56.7% |

30 | 11.156 | 0.886 | 5.783 | 0.727 | 4.991 | 0.012 | 66.7% |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Stolfi, D.H.; Danoy, G.
Optimising Robot Swarm Formations by Using Surrogate Models and Simulations. *Appl. Sci.* **2023**, *13*, 5989.
https://doi.org/10.3390/app13105989

**AMA Style**

Stolfi DH, Danoy G.
Optimising Robot Swarm Formations by Using Surrogate Models and Simulations. *Applied Sciences*. 2023; 13(10):5989.
https://doi.org/10.3390/app13105989

**Chicago/Turabian Style**

Stolfi, Daniel H., and Grégoire Danoy.
2023. "Optimising Robot Swarm Formations by Using Surrogate Models and Simulations" *Applied Sciences* 13, no. 10: 5989.
https://doi.org/10.3390/app13105989