# A NEAT Based Two Stage Neural Network Approach to Generate a Control Algorithm for a Pultrusion System

^{*}

## Abstract

**:**

## 1. Introduction—Neural Networks in Engineering. Just a Modern Buzzword?

## 2. Pultrusion Model and Neural Network Training

- The further an individual gets, the higher the fitness function result should be
- If a gate is missed, the fitness function should reflect the distance from the respective gate
- The fitness function should output the highest value when the curing target is reached

## 3. Discussion

## 4. Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

ANN | Artifcial Neural Network |

NEAT | NeuroEvolution of Augmenting Topologies |

FE | Finite Elements |

HPC | High Performance Computer |

## Appendix A. Hyperparamter NEAT-Model

[NEAT] | |

fitness_criterion | = max |

fitness_threshold | = 20,000.0 |

pop_size | = 250 |

reset_on_extinction | = 1 |

[DefaultGenome] | |

num_hidden | = 1 |

num_inputs | = 20 |

num_outputs | = 1 |

initial_connection | = partial_direct 0.5 |

feed_forward | = True |

compatibility_disjoint_coefficient = 1.0 | |

compatibility_weight_coefficient = 0.6 | |

conn_add_prob | = 0.05 |

conn_delete_prob | = 0.05 |

node_add_prob | = 0.05 |

node_delete_prob | = 0.05 |

activation_default | = sigmoid |

activation_options | = sigmoid |

activation_mutate_rate | = 0.0 |

aggregation_default | = sum |

aggregation_options | = sum |

aggregation_mutate_rate | = 0.0 |

bias_init_mean | = 0.0 |

bias_init_stdev | = 1.0 |

bias_replace_rate | = 0.01 |

bias_mutate_rate | = 0.7 |

bias_mutate_power | = 0.005 |

bias_max_value | = 30.0 |

bias_min_value | = −30.0 |

response_init_mean | = 1.0 |

response_init_stdev | = 0.0 |

response_replace_rate | = 0.0 |

response_mutate_rate | = 0.0 |

response_mutate_power | = 0.0 |

response_max_value | = 30.0 |

response_min_value | = −30.0 |

weight_max_value | = 30 |

weight_min_value | = −30 |

weight_init_mean | = 0.0 |

weight_init_stdev | = 1.0 |

weight_mutate_rate | = 0.8 |

weight_replace_rate | = 0.01 |

weight_mutate_power | = 0.005 |

enabled_default | = True |

enabled_mutate_rate | = 0.01 |

[DefaultSpeciesSet] | |

compatibility_threshold = 3.0 | |

[DefaultStagnation] | |

species_fitness_func = max | |

max_stagnation = 40 | |

species_elitism = 2 | |

[DefaultReproduction] | |

elitism = 3 | |

survival_threshold = 0.2 |

## References

- Alkinani, H.H.; Al-Hameedi, A.T.; Dunn-Norman, S.; Flori, R.E.; Alsaba, M.T.; Amer, A.S. Applications of Artificial Neural Networks in the Petroleum Industry: A Review. In Proceedings of the SPE Middle East Oil and Gas Show and Conference, Manama, Bahrain, 18–21 March 2019; OnePetro: Manama, Bahrain, 2019. [Google Scholar] [CrossRef]
- Gonzalez-Fernandez, I.; Iglesias-Otero, M.A.; Esteki, M.; Moldes, O.A.; Mejuto, J.C.; Simal-Gandara, J. A critical review on the use of artificial neural networks in olive oil production, characterization and authentication. Crit. Rev. Food Sci. Nutr.
**2019**, 59, 1913–1926. [Google Scholar] [CrossRef] [PubMed] - Khoo, Y.; Lu, J.; Ying, L. Solving parametric PDE problems with artificial neural networks. Eur. J. Appl. Math.
**2021**, 32, 421–435. [Google Scholar] [CrossRef] - Mohandes, S.R.; Zhang, X.; Mahdiyar, A. A comprehensive review on the application of artificial neural networks in building energy analysis. Neurocomputing
**2019**, 340, 55–75. [Google Scholar] [CrossRef] - Travassos, X.L.; Avila, S.L.; Ida, N. Artificial Neural Networks and Machine Learning techniques applied to Ground Penetrating Radar: A review. Appl. Comput. Inform.
**2020**, 17, 296–308. [Google Scholar] [CrossRef] - Yeganeh, A.; Shadman, A. Monitoring linear profiles using Artificial Neural Networks with run rules. Expert Syst. Appl.
**2021**, 168, 114237. [Google Scholar] [CrossRef] - Alanis, A.Y.; Arana-Daniel, N.; López-Franco, C. (Eds.) Artificial Neural Networks for Engineering Applications; Elsevier: St. Louis, MO, USA, 2019. [Google Scholar]
- Pancioni, L.; Schwenker, F.; Trentin, E. Artificial Neural Networks in Pattern Recognition; Springer International Publishing: Cham, Switzerland, 2018; Volume 11081. [Google Scholar] [CrossRef][Green Version]
- Adhan, S.; Pintavirooj, C. Thai sign language recognition by using geometric invariant feature and ANN classification. In Proceedings of the 2016 9th Biomedical Engineering International Conference (BMEiCON), Laung Prabang, Laos, 7–9 December 2016; pp. 1–4. [Google Scholar] [CrossRef]
- Trentin, E.; Gori, M. A survey of hybrid ANN/HMM models for automatic speech recognition. Neurocomputing
**2001**, 37, 91–126. [Google Scholar] [CrossRef][Green Version] - Paul, P.S.; Varadarajan, A.S. ANN assisted sensor fusion model to predict tool wear during hard turning with minimal fluid application. Int. J. Mach. Mach. Mater.
**2013**, 13, 398. [Google Scholar] [CrossRef] - Gourdeau, D.; Archambault, L. Discriminative neural network for hero selection in professional Heroes of the Storm and DOTA 2. IEEE Trans. Games
**2020**, 1. [Google Scholar] [CrossRef] - Zhang, H.; Luo, Y.; Liu, D. Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans. Neural Netw.
**2009**, 20, 1490–1503. [Google Scholar] [CrossRef] [PubMed] - Stanley, K.O. Efficient Evolution of Neural Networks through Complexification. Ph.D. Dissertation, The University of Texas at Austin, Austin, TX, USA, 2004. [Google Scholar]
- Caceres, C.; Rosario, J.M.; Amaya, D. Approach of Kinematic Control for a Nonholonomic Wheeled Robot using Artificial Neural Networks and Genetic Algorithms. In Proceedings of the 2017 International Conference and Workshop on Bioinspired Intelligence (IWOBI), Funchal, Portugal, 10–12 July 2017; pp. 1–6. [Google Scholar] [CrossRef]
- Pommer, C.; Sinapius, M. Proof of Concept for Pultrusion Control by Cure Monitoring Using Resonant Ultrasound Spectroscopy. J. Compos. Sci.
**2020**, 4, 115. [Google Scholar] [CrossRef] - Pommer, C.; Sinapius, M. A Novel Approach to Monitoring the Curing of Epoxy in Closed Tools by Use of Ultrasonic Spectroscopy. Sensors
**2017**, 18, 96. [Google Scholar] [CrossRef] [PubMed][Green Version] - Pommer, C. Geregelter Pultrusionsprozess mit In-Situ-Aushärtungsüberwachung. Ph.D. Dissertation, DLR, Deutsches Zentrum für Luft- und Raumfahrt, Köln, Germany, 2019. [Google Scholar]
- Gupta, J.N.; Sexton, R.S. Comparing backpropagation with a genetic algorithm for neural network training. Omega
**1999**, 27, 679–684. [Google Scholar] [CrossRef] - Such, F.P.; Madhavan, V.; Conti, E.; Lehman, J.; Stanley, K.O.; Clune, J. Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning. arXiv
**2017**, arXiv:cs.NE/1712.06567. [Google Scholar] - Awolusi, T.; Oke, O.; Akinkurolere, O.; Sojobi, A.; Aluko, O. Performance comparison of neural network training algorithms in the modeling properties of steel fiber reinforced concrete. Heliyon
**2019**, 5, e01115. [Google Scholar] [CrossRef] [PubMed][Green Version] - McIntyre, A.; Kallada, M.; Miguel, C.G.; da Silva, C.F. Neat-Python. Available online: https://github.com/CodeReclaimers/neat-python (accessed on 12 May 2021).

**Figure 4.**Mean square error between FE model and the ANN during a 16 min initial period and a series of random speed changes between 0.005 m/min and 0.2 m/min every 5 min.

**Figure 6.**Structure of the best genome found; Line thickness represents weight strength; ${V}_{OUT}$, 1, 2, 3 are different Neurons; Inactive Links and some inactive Inputs omitted.

**Figure 7.**Best run after 270 Generations with curing $\alpha $ in orange, the target curing black in the top figure and Speed set by the ANN in the figure at the bottom.

**Figure 8.**Comparison between the results received by the ANN model and the FE-Model for the NEAT-control algorithm.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Pommer, C.; Sinapius, M.; Brysch, M.; Al Natsheh, N. A NEAT Based Two Stage Neural Network Approach to Generate a Control Algorithm for a Pultrusion System. *AI* **2021**, *2*, 355-365.
https://doi.org/10.3390/ai2030022

**AMA Style**

Pommer C, Sinapius M, Brysch M, Al Natsheh N. A NEAT Based Two Stage Neural Network Approach to Generate a Control Algorithm for a Pultrusion System. *AI*. 2021; 2(3):355-365.
https://doi.org/10.3390/ai2030022

**Chicago/Turabian Style**

Pommer, Christian, Michael Sinapius, Marco Brysch, and Naser Al Natsheh. 2021. "A NEAT Based Two Stage Neural Network Approach to Generate a Control Algorithm for a Pultrusion System" *AI* 2, no. 3: 355-365.
https://doi.org/10.3390/ai2030022