# Genetic Programming Guidance Control System for a Reentry Vehicle under Uncertainties

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Genetic Programming for Control

#### 3.1. Inclusive Genetic Programming

#### 3.2. Inclusive Genetic Programming Settings

## 4. Test Case

#### 4.1. Reference Trajectory

#### 4.2. Uncertainty Model

## 5. Results

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Conflicts of Interest

## Abbreviations

AI | Artificial Intelligence |

CI | Computational Intelligence |

CSG | Cubic Spline Generalization |

EA | Evolutionary Algorithm |

FAC | Final Approach Corridor |

GA | Genetic Algorithm |

GP | Genetic Programming |

IAE | Integral of Absolute Error |

IC | Intelligent Control |

IGP | Inclusive Genetic Programming |

ML | Machine Learning |

MPC | Model Predictive Control |

NN | Neural Network |

RLV | Reusable Launch Vehicle |

## References

- Xie, Y.C.; Huang, H.; Hu, Y.; Zhang, G.Q. Applications of advanced control methods in spacecrafts: Progress, challenges, and future prospects. Front. Inf. Technol. Electron. Eng.
**2016**, 17, 841–861. [Google Scholar] [CrossRef] [Green Version] - Haibin, D.; Pei, L. Progress in control approaches for hypersonic vehicle. Sci. China Technol. Sci.
**2012**, 55, 2965–2970. [Google Scholar] [CrossRef] - Xu, B. Robust adaptive neural control of flexible hypersonic flight vehicle with dead-zone input nonlinearity. Nonlinear Dyn.
**2015**, 80, 1509–1520. [Google Scholar] [CrossRef] - Chai, R.; Tsourdos, A.; Savvaris, A.; Chai, S.; Xia, Y.; Philip Chen, C.L. Review of advanced guidance and control algorithms for space/aerospace vehicles. Prog. Aerosp. Sci.
**2021**, 122, 100696. [Google Scholar] [CrossRef] - Izzo, D.; Märtens, M.; Pan, B. A survey on artificial intelligence trends in spacecraft guidance dynamics and control. Astrodynamics
**2019**, 3, 287–299. [Google Scholar] [CrossRef] - Koza, J.R. Genetic Programming: On the Programming of Computers by Means of Natural Selection; MIT Press: Cambridge, MA, USA, 1992. [Google Scholar]
- Koza, J.; Keane, M.; Yu, J.; Bennett, F., III; Mydlowec, W. Automatic Creation of Human-Competitive Programs and Controllers by Means of Genetic Programming. Genet. Program. Evolvable Mach.
**2000**, 1, 121–164. [Google Scholar] [CrossRef] - Baeta, F.; Correia, J.; Martins, T.; Machado, P. TensorGP—Genetic Programming Engine in TensorFlow. In Applications of Evolutionary Computation. EvoApplications 2021; Springer: Cham, Switzerland, 2021. [Google Scholar]
- Wilson, C.; Marchetti, F.; Di Carlo, M.; Riccardi, A.; Minisci, E. Classifying Intelligence in Machines: A Taxonomy of Intelligent Control. Robotics
**2020**, 9, 64. [Google Scholar] [CrossRef] - Luo, Y.; Serrani, A.; Yurkovich, S.; Oppenheimer, M.W.; Doman, D.B. Model-predictive dynamic control allocation scheme for reentry vehicles. J. Guid. Control Dyn.
**2007**, 30, 100–113. [Google Scholar] [CrossRef] - Wang, J.; Cui, N.; Wei, C. Optimal Rocket Landing Guidance Using Convex Optimization and Model Predictive Control. J. Guid. Control Dyn.
**2019**, 42, 1078–1092. [Google Scholar] [CrossRef] - Bollino, K.P.; Ross, I.M. A pseudospectral feedback method for real-time optimal guidance of reentry vehicles. In Proceedings of the American Control Conference, New York, NY, USA, 9–13 July 2007; pp. 3861–3867. [Google Scholar] [CrossRef]
- Tian, B.; Fan, W.; Zong, Q. Real-Time Trajectory and Attitude Coordination Control for Reusable Launch Vehicle in Reentry Phase. IEEE Trans. Ind. Electron.
**2015**, 3, 1639–1650. [Google Scholar] [CrossRef] - Harl, N.; Balakrishnan, S.N. Reentry terminal guidance through sliding mode control. J. Guid. Control Dyn.
**2010**, 33, 186–199. [Google Scholar] [CrossRef] - Guo, Z.; Chang, J.; Guo, J.; Zhou, J. Adaptive twisting sliding mode algorithm for hypersonic reentry vehicle attitude control based on finite-time observer. ISA Trans.
**2018**, 77, 20–29. [Google Scholar] [CrossRef] - Wu, S.F.; Engelen, C.J.H.; Chu, Q.P.; Bubuška, R.; Mulder, J.A.; Ortega, G. Fuzzy logic based attitude control of the spacecraft X-38 along a nominal re-entry trajectory. Control Eng. Pract.
**2001**, 9, 699–707. [Google Scholar] [CrossRef] - Gao, J.; Shi, X.; Cheng, Z.; Xiong, J.; Liu, L.; Wang, Y.; Yang, Y. Reentry trajectory optimization based on Deep Reinforcement Learning. In Proceedings of the 31st Chinese Control and Decision Conference, CCDC 2019, Nanchang, China, 3–5 June 2019; pp. 2588–2592. [Google Scholar] [CrossRef]
- Kumar, G.N.; Sarkar, A.K.; Ahmed, M.S.; Talole, S.E. Reentry Trajectory Optimization using Gradient Free Algorithms. IFAC-PapersOnLine
**2018**, 51, 650–655. [Google Scholar] [CrossRef] - Duan, H.; Li, S. Artificial bee colony-based direct collocation for reentry trajectory optimization of hypersonic vehicle. IEEE Trans. Aerosp. Electron. Syst.
**2015**, 51, 615–626. [Google Scholar] [CrossRef] - Wu, Y.; Yan, B.; Qu, X. Improved Chicken Swarm Optimization Method for Reentry Trajectory Optimization. Math. Probl. Eng.
**2018**, 2018, 8135274. [Google Scholar] [CrossRef] [Green Version] - Vijay, D.; Bhanu, U.S.; Boopathy, K. Multi-objective genetic algorithm-based sliding mode control for assured crew reentry vehicle. Adv. Intell. Syst. Comput.
**2017**, 517, 465–477. [Google Scholar] [CrossRef] - Sushnigdha, G.; Joshi, A. Evolutionary method based integrated guidance strategy for reentry vehicles. Eng. Appl. Artif. Intell.
**2018**, 69, 168–177. [Google Scholar] [CrossRef] - Oh, C.K.; Barlow, G.J. Autonomous controller design for unmanned aerial vehicles using multi-objective genetic programming. In Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753), Portland, OR, USA, 19–23 June 2004; Volume 2, pp. 1538–1545. [Google Scholar] [CrossRef]
- Bourmistrova, A.; Khantsis, S. Genetic Programming in Application to Flight Control System Design Optimisation. In New Achievements in Evolutionary Computation; Books on Demand: Norderstedt, Germany, 2010. [Google Scholar]
- Verdier, C.F.; Mazo, M., Jr. Formal Controller Synthesis via Genetic Programming. IFAC-PapersOnLine
**2017**, 50, 7205–7210. [Google Scholar] [CrossRef] - Łapa, K.; Cpałka, K.; Przybył, A. Genetic programming algorithm for designing of control systems. Inf. Technol. Control
**2018**, 47, 668–683. [Google Scholar] [CrossRef] [Green Version] - Chiang, C.H. A genetic programming based rule generation approach for intelligent control systems. In Proceedings of the 3CA 2010—2010 International Symposium on Computer, Communication, Control and Automation, Tainan, Taiwan, 5–7 May 2010; Volume 1, pp. 104–107. [Google Scholar] [CrossRef]
- Marchetti, F.; Minisci, E.; Riccardi, A. Towards Intelligent Control via Genetic Programming. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 19–24 July 2020. [Google Scholar]
- Marchetti, F.; Minisci, E. A Hybrid Neural Network-Genetic Programming Intelligent Control Approach. In International Conference on Bioinspired Methods and Their Applications, Proceedings of the 9th International Conference, BIOMA 2020, Brussels, Belgium, 19–20 November 2020; Filipič, B., Minisci, E., Vasile, M., Eds.; Springer: Cham, Switzerland, 2020. [Google Scholar]
- Marchetti, F.; Minisci, E. Inclusive Genetic Programming. In Proceedings of the 24th European Conference on Genetic Programming, EuroGP 2021, Virtual Event, 7–9 April 2021; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2021. [Google Scholar]
- Fortin, F.A.; De Rainville, F.M.; Gardner, M.A.; Parizeau, M.; Gagńe, C. DEAP: Evolutionary algorithms made easy. J. Mach. Learn. Res.
**2012**, 13, 2171–2175. [Google Scholar] - Shir, O.M. Niching in Evolutionary Algorithms. In Handbook of Natural Computing; Rozenberg, G., Bäck, T., Kok, J.N., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; pp. 1035–1069. [Google Scholar] [CrossRef]
- Beyer, H.G.; Schwefel, H.P. Evolution strategies—A comprehensive introduction. Nat. Comput.
**2002**, 1, 3–52. [Google Scholar] [CrossRef] - Luke, S.; Panait, L. Fighting bloat with nonparametric parsimony pressure. In Parallel Problem Solving from Nature—PPSN VII; Guervós, J.J.M., Adamidis, P., Beyer, H.G., Schwefel, H.P., Fernández-Villacañas, J.L., Eds.; Springer: Berlin/Heidelberg, Germany, 2002; pp. 411–421. [Google Scholar]
- D’Angelo, S.; Minisci, E.; Di Bona, D.; Guerra, L. Optimization Methodology for Ascent Trajectories of Lifting-Body Reusable Launchers. J. Spacecr. Rocket.
**2000**, 37, 761–767. [Google Scholar] [CrossRef] - Marchetti, F.; Minisci, E.; Riccardi, A. Single-Stage to Orbit Ascent Trajectory Optimisation with Reliable Evolutionary Initial Guess. In Optimization and Engineering; Springer: Berlin, Germany, 2021; Available online: https://www.springer.com/journal/11081 (accessed on 2 August 2021).
- Bollino, K.P.; Ross, I.M.; Doman, D.D. Optimal nonlinear feedback guidance for reentry vehicles. In Proceedings of the Collection of Technical Papers—AIAA Guidance, Navigation, and Control Conference and Exhibit, Keystone, CO, USA, 21–24 August 2006; Volume 1, pp. 563–582. [Google Scholar] [CrossRef] [Green Version]
- ul Islam Rizvi, S.T.; He, L.; Xu, D. Optimal trajectory and heat load analysis of different shape lifting reentry vehicles for medium range application. Def. Technol.
**2015**, 11, 350–361. [Google Scholar] [CrossRef] [Green Version] - Pescetelli, F.; Minisci, E.; Maddock, C.; Taylor, I.; Brown, R.E. Ascent trajectory optimisation for a single-stage-to-orbit vehicle with hybrid propulsion. In Proceedings of the 18th AIAA/3AF International Space Planes and Hypersonic Systems and Technologies Conference, Tours, France, 24–28 September 2012; pp. 1–18. [Google Scholar]
- Zang, L.; Lin, D.; Chen, S.; Wang, H.; Ji, Y. An on-line guidance algorithm for high L/D hypersonic reentry vehicles. Aerosp. Sci. Technol.
**2019**, 89, 150–162. [Google Scholar] [CrossRef] - Wang, Z.; Grant, M.J. Autonomous entry guidance for hypersonic vehicles by convex optimization. J. Spacecr. Rocket.
**2018**, 55, 993–1006. [Google Scholar] [CrossRef]

**Figure 3.**Schematic of mutation operation. A randomly selected gene of an individual is mutated randomly.

**Figure 7.**Perturbed pressure and temperature profiles of one GP run for each different simulation. The continuous lines represent the successful cases and the dashed one the unsuccessful ones. The time starts at 100 s since the uncertainties were applied at 100 s.

**Figure 8.**Perturbed lift and drag profiles of one GP run for each different simulation. The continuous lines represent the successful cases and the dashed one the unsuccessful ones. The time starts at 100 s since the uncertainties were applied at 100 s.

**Figure 9.**Trajectories obtained as a results of the 20 GP runs in Simulation 0. Plot a shows an overview of the performed trajectories while plot b depicts a closeup on the final part of the trajectory to show which trajectory ended inside the FAC box and which did not. The black horizontal lines in plot b show the FAC bounds for $\theta $, $\lambda $ and h. UP refers to the different uncertainty profiles.

**Figure 10.**Trajectories obtained as a results of the 20 GP runs in Simulation 5. Plot a shows an overview of the performed trajectories while plot b depicts a closeup on the final part of the trajectory to show which trajectory ended inside the FAC box and which did not. The black horizontal lines in plot b show the FAC bounds for $\theta $, $\lambda $ and h. UP refers to the different uncertainty profiles.

**Figure 11.**Trajectories obtained as a results of the 20 GP runs in Simulation 10. Plot a shows an overview of the performed trajectories while plot b depicts a closeup on the final part of the trajectory to show which trajectory ended inside the FAC box and which did not. The black horizontal lines in plot b show the FAC bounds for $\theta $, $\lambda $ and h. UP refers to the different uncertainty profiles.

**Figure 13.**Median an standard deviations of the $Fi{t}_{1}$ values evaluated on the 20 different GP runs for each simulation.

**Figure 14.**Number of solutions that violated the constraints for each simulation and amount of the constraints violation. The left ordinate axis refers to the bar plots which shows the number of GP runs that violated the constraints. The right ordinate axis refers to the line plot and shows the mean constraints violation among the 20 GP runs for each simulation.

**Figure 15.**Lengths of successful and unsuccessful individuals for each simulation. The green dots represent the successful individuals while the red crosses the unsuccessful ones. (

**a**) depicts the first control parameter $\alpha $, and the second control parameter $\sigma $ is in (

**b**). In (

**a**) the continuous orange line represents the mean values of the lengths of the successful $\alpha $ individuals for each simulation, while in (

**b**) the dashed blue line represents the mean values of the lengths of the successful $\sigma $ individuals.

**Table 1.**Settings of IGP algorithm. The percentages near the mutation mechanisms refers to the probability of that mutation mechanism to be chosen when the mutation is performed. The selection criteria refers to the criteria used to select the individuals from the niches when performing crossover, mutation or 1:1 reproduction.

Population Size | 300 individuals |

Maximum Generations | 150 |

Stopping criteria | Reaching maximum number of generations or successful trajectory found |

Number of niches | 10 |

Crossover probability | 0.2 (+0.01 at every generation if $Fi{t}_{2}=0$) → 0.65 |

Mutation probability | 0.7 (−0.01 at every generation if $Fi{t}_{2}=0$) → 0.35 |

1:1 Reproduction probability | 0.1 |

Crossover selection criteria | Best, random |

Mutation selection criteria | Random |

1:1 Reproduction selection criteria | Best |

Evolutionary strategy | $\mu +\lambda $ |

$\mu $ | Population size |

$\lambda $ | Population Size $\times \phantom{\rule{4pt}{0ex}}1.2$ |

Number of Ephemeral constants | 2 |

Limit Height | 30 |

Limit Size | 50 |

Selection Mechanism | Inclusive Tournament |

Double Tournament fitness size | 2 |

Double Tournament parsimony size | 1.6 |

Tree creation mechanism | Ramped half and half |

Mutation mechanisms | Uniform (50%), Shrink (5%), |

Insertion (25%), Mutate Ephemeral (20%) | |

Crossover mechanism | One point crossover |

Primitives Set | $+,\phantom{\rule{4pt}{0ex}}-,\phantom{\rule{4pt}{0ex}}*,\phantom{\rule{4pt}{0ex}}add3,\phantom{\rule{4pt}{0ex}}tanh$, $psqrt$ |

$plog,\phantom{\rule{4pt}{0ex}}pexp,\phantom{\rule{4pt}{0ex}}sin,\phantom{\rule{4pt}{0ex}}cos$ | |

Fitness measures | $Fi{t}_{1}$, $Fi{t}_{2}$ |

States | $V,\chi ,\gamma ,\theta ,\lambda ,h$ |

Controls | $\alpha ,\sigma $ |

Initial | ${V}_{0}$ = 2600 m/s, |

Conditions | ${\chi}_{0}=$ 0 deg, |

${\gamma}_{0}$ = $-1.3$ deg | |

${\theta}_{0}$ = $-85$ deg, | |

${\lambda}_{0}$ = 30 deg, | |

${h}_{0}$ = 51 km, | |

Final | ${V}_{f}$ = 91.44 m/s, |

Conditions | ${\chi}_{f}$ = $-60$ deg, |

${\gamma}_{f}$ = $-6$ deg, | |

${\theta}_{f}$ = $-80.7112\pm \phantom{\rule{4pt}{0ex}}0.0014\phantom{\rule{4pt}{0ex}}$ deg, | |

${\lambda}_{f}$ = $28.6439\pm \phantom{\rule{4pt}{0ex}}0.0014\phantom{\rule{4pt}{0ex}}$ deg, | |

${h}_{f}$ = $609.6\pm \phantom{\rule{4pt}{0ex}}121.92\phantom{\rule{4pt}{0ex}}\mathrm{m}$ | |

Controls | $-2\phantom{\rule{4pt}{0ex}}\mathrm{deg}\le \alpha \le 40\phantom{\rule{4pt}{0ex}}$ deg |

Bounds | $-90\phantom{\rule{4pt}{0ex}}\mathrm{deg}\le \sigma \le 90\phantom{\rule{4pt}{0ex}}$ deg, |

Constraints | $-25\phantom{\rule{4pt}{0ex}}{\mathrm{m}/\mathrm{s}}^{2}\le {a}_{z}\le 25\phantom{\rule{4pt}{0ex}}{\mathrm{m}/\mathrm{s}}^{2}$, |

$q\le 40\phantom{\rule{4pt}{0ex}}\mathrm{kPa}$, | |

$\dot{Q}\le 4\phantom{\rule{3.33333pt}{0ex}}{\mathrm{MW}/\mathrm{m}}^{2}$ |

${h}_{c}$ | 121,920 m |

${M}_{c}$ | 10 |

${\alpha}_{c}$ | 40 deg |

${l}_{b,T}$ | 0.1 |

${u}_{b,T}$ | 0.5 |

${l}_{b,P}$ | 0.01 |

${u}_{b,P}$ | 0.5 |

${l}_{b,h}$ | 0.1 |

${u}_{b,h}$ | 0.2 |

${l}_{b,M}$ | 0.1 |

${u}_{b,M}$ | 0.2 |

${l}_{b,\alpha}$ | 0.1 |

${u}_{b,\alpha}$ | 0.2 |

${\mathit{l}}_{\mathit{b},\mathit{T}}$ | ${\mathit{u}}_{\mathit{b},\mathit{T}}$ | ${\mathit{l}}_{\mathit{b},\mathit{P}}$ | ${\mathit{u}}_{\mathit{b},\mathit{P}}$ | ${\mathit{l}}_{\mathit{b},\mathit{h}}$ | ${\mathit{u}}_{\mathit{b},\mathit{h}}$ | ${\mathit{l}}_{\mathit{b},\mathit{M}}$ | ${\mathit{u}}_{\mathit{b},\mathit{M}}$ | ${\mathit{l}}_{\mathit{b},\mathit{\alpha}}$ | ${\mathit{u}}_{\mathit{b},\mathit{\alpha}}$ | |
---|---|---|---|---|---|---|---|---|---|---|

Simulation 0 | 0.1 | 0.5 | 0.01 | 0.5 | 0.1 | 0.2 | 0.1 | 0.2 | 0.1 | 0.2 |

Simulation 1 | 0.11 | 0.55 | 0.011 | 0.55 | 0.11 | 0.22 | 0.11 | 0.22 | 0.11 | 0.22 |

Simulation 2 | 0.12 | 0.6 | 0.012 | 0.6 | 0.12 | 0.24 | 0.12 | 0.24 | 0.12 | 0.24 |

Simulation 3 | 0.13 | 0.65 | 0.013 | 0.65 | 0.13 | 0.26 | 0.13 | 0.26 | 0.13 | 0.26 |

Simulation 4 | 0.14 | 0.7 | 0.014 | 0.7 | 0.14 | 0.28 | 0.14 | 0.28 | 0.14 | 0.28 |

Simulation 5 | 0.15 | 0.75 | 0.015 | 0.75 | 0.15 | 0.3 | 0.15 | 0.3 | 0.15 | 0.3 |

Simulation 6 | 0.16 | 0.8 | 0.016 | 0.8 | 0.16 | 0.32 | 0.16 | 0.32 | 0.16 | 0.32 |

Simulation 7 | 0.17 | 0.85 | 0.017 | 0.85 | 0.17 | 0.34 | 0.17 | 0.34 | 0.17 | 0.34 |

Simulation 8 | 0.18 | 0.9 | 0.018 | 0.9 | 0.18 | 0.36 | 0.18 | 0.36 | 0.18 | 0.36 |

Simulation 9 | 0.19 | 0.95 | 0.019 | 0.95 | 0.19 | 0.38 | 0.19 | 0.38 | 0.19 | 0.38 |

Simulation 10 | 0.2 | 1.0 | 0.02 | 1.0 | 0.2 | 0.4 | 0.2 | 0.4 | 0.2 | 0.4 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Marchetti, F.; Minisci, E.
Genetic Programming Guidance Control System for a Reentry Vehicle under Uncertainties. *Mathematics* **2021**, *9*, 1868.
https://doi.org/10.3390/math9161868

**AMA Style**

Marchetti F, Minisci E.
Genetic Programming Guidance Control System for a Reentry Vehicle under Uncertainties. *Mathematics*. 2021; 9(16):1868.
https://doi.org/10.3390/math9161868

**Chicago/Turabian Style**

Marchetti, Francesco, and Edmondo Minisci.
2021. "Genetic Programming Guidance Control System for a Reentry Vehicle under Uncertainties" *Mathematics* 9, no. 16: 1868.
https://doi.org/10.3390/math9161868