Topological Optimization of Artiﬁcial Neural Networks to Estimate Mechanical Properties in Metal Forming Using Machine Learning

: The ability of a metal to be subjected to forming processes depends mainly on its plastic behavior and, thus, the mechanical properties belonging to this region of the stress–strain curve. Forming techniques are among the most widespread metalworking procedures in manufacturing, and aluminum alloys are of great interest in ﬁelds as diverse as the aerospace sector or the food industry. A precise characterization of the mechanical properties is crucial to estimate the forming capability of equipment, but also for a robust numerical modeling of metal forming processes. Characterizing a material is a very relevant task in which large amounts of resources are invested, and this paper studies how to optimize a multilayer neural network to be able to make, through machine learning, precise and accurate predictions about the mechanical properties of wrought aluminum alloys. This study focuses on the determination of the ultimate tensile strength, closely related to the strain hardening of a material; more precisely, a methodology is developed that, by randomly partitioning the input dataset, performs training and prediction cycles that allow estimating the average performance of each fully-connected topology. In this way, trends are found in the behavior of the networks, and it is established that, for networks with at least 150 perceptrons in their hidden layers, the average predictive error stabilizes below 4%. Beyond this point, no really signiﬁcant improvements are found, although there is an increase in computational requirements.


Introduction
Aluminum alloys are among the most widely used materials in the industry, and, although their use is still far from being as widespread as that of steel, they have many advantages that make them a very interesting material whose use is growing regularly [1]. There is a huge number of aluminum alloys, but few of them are typically used in the industrial field [2], sometimes because it is difficult to find new solutions and, sometimes, because they are special materials with optimized properties to fulfill their requirements, according to their application [3]. Aluminum alloys are manufactured by different techniques [4]. Among its many properties of industrial interest, it can be noted that it is a material with a high formability, so it is especially suitable for metal forming processes. Moreover, a precise characterization of the mechanical properties is crucial to estimate the forming capability of equipment but also for a robust numerical modeling of metal forming processes. Among all the mechanical properties, the ultimate tensile strength (UTS) plays a key role in the definition of the onset of the plastic instability by tensile tension [3]. This mechanical property is closely related to the strain hardening of the metal and, therefore, to its forming capacity under metal forming processes. In uniaxial tension, the plastic deformation is limited by the value of UTS since, once it is exceeded, the onset of the plastic instability takes place, and from then on, the behavior of the material is almost unpredictable until fracture [5]. Therefore, it is well known that aluminum alloys present a high ductility [6] because they can withstand a lot of plastic deformation before fracture and, so, they are widely used in metal forming operations [7]; such metals can be manufactured by metal sheet operations and bulk forming processes, such as forging, rolling, extrusion and/or drawing [2]. Ductile materials are able to absorb a great amount of energy before failing, otherwise known as toughness [8].
The design and production of some industrial components rely on the knowledge of the mechanical properties obtained and tensile testing [9]. The UTS may serve to determine the beginning of the plastic instability and provides an insight into the initiation of the fracture or the necking [10]. It also can be used as an input to estimate the forming force in the conventional forming procedures such as stretch-bending or deep-drawing [10].
Metal forming is frequently employed to manufacture components. The microstructure and the mechanical properties of these parts are modified by these processes [2]. For example, the increase in the ultimate stress and hardness observed during the A-6063 extrusion is attributed to the grain size reduction and the temperature increase [11].
In addition to UTS, the strain hardening exponent, the yield strength (YS), the processinduced residual stresses, and the hardness are also important mechanical characteristics [1,3,10]. These properties offer an idea of the in-service behavior of the formed part. Furthermore, their correlations help to understand the response of the component [12], i.e., knowing the difference between UTS and YS can help the designer to predict how much additional stress a component can withstand before failure [5] because YS defines the onset of plastic deformation, and UTS defines the onset of plastic instability.
Knowing the expected behavior of the materials used in industrial designs is critical; however, obtaining these data frequently requires accessing large amounts of resources, which are normally not accessible [1]. Many tests are required to obtain relevant information, which entail having enough time, personnel and facilities be available [13]. Characterizing a metal comprises many tests that require non-trivial quantities of resources [8].
Although it is a relatively new technology and not as widespread in materials science and manufacturing as in other areas [13], artificial intelligence (AI) and machine learning (ML) techniques have been successfully used to make predictions about the metallurgical properties of some materials [1,3,[14][15][16].
Over recent years, AI and ML have received much attention in the field of materials modeling, due to their outstanding ability to analyze a huge amount of data and expose correlations between complex interrelated properties [17]. ML is, perhaps, the most relevant branch of AI, and is the science of making computers learn and act like humans without being explicitly programmed [18,19]. It is often used to discover hidden patterns in complex systems through a training process in which a great amount of noisy data is furnished as input [20,21]. ML can be classified into supervised learning (the machine learns from labeled data) and unsupervised learning (the machine finds patterns in the data without any external help) [20,22,23].
Among the most challenging topics in this field is the search for the best representation of input variables in ML models, which is commonly called feature engineering and comprises a set of activities, such as feature extraction, feature construction and feature selection [23,24]. Feature engineering research is crucial to the application of the ML.
Within current materials science, the scale and speed of data acquisition, the accuracy of the data and the volatility of the data are additional challenges for researchers [3]. It raises the question of how to use and analyze these data in a useful way that supports the decisions of developers and designers [18,23]. Material data tend to be wide in scope and, often, shallow in depth. Here, depth should be understood as the number of observations of the state of a system. The lack of observations is due not only to the cost and difficulty of acquiring data (especially through experimentation), but also to the nature of the data itself. However, fully employing the data is a key part of advanced design systems [23,25].
In recent years, an incipient trend in materials science research is the combination of existing experimental and numerical modeling methodologies with AI techniques [26][27][28]. In general, materials science advances thanks to accumulated experience and already established rules [29,30]. New advances in numerical modeling facilitate the methodical acquisition of large amounts of data, while complicating analysis, hypothesis formulation and pattern prediction. The rise of AI techniques makes up for this deficiency to a great extent [31].
Multilayer artificial neural networks (ANN) can be considered the most remarkable methodology of those that are included into the field of AI because they have demonstrated their capabilities in almost all branches of knowledge and because they are currently receiving a lot of attention from investigators [32]. A multilayer network is able to learn a function by training on a labeled dataset that can be used to perform regressions [33]. ANN are made up of perceptrons (neurons) that regroup forming layers (clusters of neurons) that communicate with each other (in general, perceptrons do not communicate with their own layer companions) [13]. For a fully connected multilayer neural network, the time complexity of the backpropagation training is given by Equation (1). So, it is highly recommended to minimize the number of hidden nodes to reduce the training time [1].
where n is the size of the training dataset, m is the number of features, o is the number of output perceptrons, N is the number of iterations and k is the number of hidden layers (each of them containing h i nodes).
The main objective of this work is to develop a working methodology that allows optimizing the topology of a multilayer neural network in such a way that it is capable of making predictions about the UTS of wrought aluminum alloys [34], maximizing the precision and accuracy of the estimation and minimizing the computational resources [20]. Although this paper only takes into account the UTS, this same approach could be applied to other properties that have already been mentioned, such as the YS or the elongation at break (A).

Methodology
This work is developed following a three-stage scheme, and the data generated in each of them are used as input for the subsequent one [13]. This workflow guarantees that the data that reach each phase are correctly prepared and processed, and are ready to be employed. Therefore, the information resulting from the entire process is a consequence of the initial dataset. Figure 1 schematically shows the three stages that compose the methodology of this work: in the first stage, an initial input dataset is created; in the second stage, the optimization process is carried out through training-prediction cycles; and in the third phase, all available information is analyzed.

Input Dataset Preparation
All initial data on the properties of the materials were obtained from Matmatch GmbH (Munich, Germany) [35]. It is an online library that contains freely accessible specification sheets about material properties [13]. These include a large number of aluminum alloys with very heterogeneous information. The volume of data initially obtained comprises more than 5000 materials and more than 350 properties [1,3].
After obtaining these data, each record must be read and interpreted in an automated way. Each specification sheet contains much more information than is used during this study, so it is necessary to discard the irrelevant data [35]. The following considerations are taken into account to select the records that are found to be useful: • Only datasheets that contain the value of the ultimate tensile stress (UTS) at 20 • C are considered. • Only alloys whose chemical composition is defined at more than 95% are taken into account [13] (note that some datasheets are poorly defined). • Only 9 chemical elements are considered to define the chemical composition of the alloys [29]: Al, Zn, Cu, Si, Fe, Mn, Mg, Ti and Cr. The mass contribution of all other chemical elements is regrouped as "Other". • Only wrought alloys are considered in this study. • Only the specification sheets that include the temper of the alloy are considered. This study only considers the following tempers: F (as fabricated), O (annealed), H (strain hardening) and T (thermally treated) [7].
After taking all these considerations into account, only 2671 materials (the obviated records do not meet the aforementioned conditions), 11 input properties and the UTS are considered. These alloys constitute the initial dataset on which the entire study is developed. One of these properties is categorical (temper) and must be mapped as an integer, while the other 11 are numerical (UTS and chemical composition) and must be normalized to avoid bias [20,22]. Normalization is carried out using Equation (2).
where x i is each of the non-normalized input values, x i is the related normalized value [0 . . . 1], x min is the minimum value for that parameter and x MAX is the maximum value.

Network Optimization by Training-Prediction
The ANN topology denotes the way in which perceptrons are associated and is an essential characteristic in the performance of the network [19]. Layers are shapeless in the sense that all of its nodes are equally relevant, connected the same way, and lack differentiators [22]. Only the initialization step and the following training make its importance change [25].
In this study, the neural network is defined as a fully connected multilayer feedforward topology [33], which comprises an input layer, two hidden layers and an output layer. In this topology, all the perceptrons in each layer are only connected to all ones in the next layer so that the information only flows in one direction, from the input layer to the output layer [22].
The neural network receives an input vector of 11 elements (chemical composition and temper) and returns a prediction about the value of the expected UTS [3]. Therefore, the input layer is made up of 11 nodes, and the output layer only contains a single node. Additionally, the network topology contains two hidden layers whose number of nodes is to be optimized [19]. Figure 2 shows a schematic representation of the network in which the hidden layers are represented as squares. Different topologies are tested to carry out the network optimization: the number of nodes in both hidden layer changes in increments of 10 nodes (from 10 to 200). In this way, 400 topologies are obtained with a number of nodes that varies between 32 and 412. For each of these networks, 10 independent training and prediction iterations are carried out [1,3].
Each iteration consists of four phases: • Dataset shuffling and splitting to create a training subset (80% of records, so 2137) and a testing subset (remaining 20%, so 534). • ANN training, using the data contained in the training subset. • Prediction of the properties for the records in the testing subset. • Results and data storage for further analysis.
The training is configured as follow [1]: • Calculation of the learning rate for each parameter using adaptive moment estimation (ADAM) with β 1 = 0.9, β 2 = 0.999 (algorithm parameters), η = 0.001 (step size) and = 10 −8 (stability factor) [36]. • Early stopping after 20 iterations without significant changes. • Training stops when a training error of less than 0.1 is reached. • Maximum of 100,000 training epochs to avoid infinite loops.

Data Analysis
The optimization process generates a large amount of data that must be processed to generate information that allows to draw conclusions about the network performance [3,13]. For each of the considered 400 topologies, the predictive performance of the 10 iterations is calculated and stored.
With the information obtained about the tests on these topologies, it is possible to build a performance map of 20 × 20 cells; each of these cells represents a topology described by the number of nodes in the two hidden layers. In each position of this data structure, it is possible to store statistical information that allows making comparisons between the different topologies.
Network topologies with the lowest average predictive error (highest accuracy), lowest standard deviation of predictive error (highest precision), and shortest training time (lowest resource usage) are preferred [20]. Table 1 contains some statistical metrics about the information contained in the input dataset. It is interesting to highlight the wide range of values associated with the UTS.   As expected, more complex topologies tend to have higher precision than those with fewer perceptrons and, in fact, the lowest error is related to a network with 160 and 200 nodes in its two hidden layers (2.88%). Moreover, the highest error rate (95.28%) occurs for the simplest network of those that are considered (10 and 10 nodes). More detailed information can be found on Table A1. Figure 3 graphically shows the average predictive error (values above 20% are trimmed to avoid scale-related issues). A region with a very low precision (error greater than 10%) can be seen for topologies with less than 150 perceptrons, while for more complex networks, the error remains lower (less than 10%). It is interesting to note that, as can be seen in the three-dimensional figure, the transition between both regions is quite abrupt, and a step (yellow zone) is formed. This transition zone (yellow) separates an almost flat area from a steep one. This transition can be interpreted as a frontier before which the predictions cannot be trusted because the error is excessive and the model is very unstable (small changes in the network produce large differences). Figure 4 shows the value of the mean predictive error as a function of the number of nodes in the hidden layers. It can be seen that the error asymptotically tends to a value close to 2%. Note that the number of nodes in the hidden layers is the sum of the number of perceptrons in both layers.

UTS [MPa] Al [%] Zn [%] Cu [%] Si [%] Fe [%] Mn [%] Mg [%] Ti [%] Cr [%] Other [%]
It is interesting to highlight that, for neural networks with more than 300 nodes, the average predictive error remains, in all cases, approximately stable at around 4%. It is a very interesting result since it establishes a boundary beyond which no significant improvement can be seen, although there is an increase in computational requirements. In view of these results, it is clear that complex topologies should be privileged over simpler ones; however, it should also be considered that, as the number of nodes increases, achieving significant improvements becomes very expensive in computational terms and, in fact, it is found that a more complex topology does not always guarantee better accuracy.
On the other hand, it is not only necessary to take into account the accuracy (related to the average predictive error) of the results, but also the precision (related to the predictive error standard deviation).
The standard deviation of the error gives an idea of the repeatability of the estimates, and, together with the average error, allow to identify the confidence range in which a prediction is. In a similar way to what happened in the case of the average error, the more complex networks are more precise (the minimum is reached for a network with 180 and 160 perceptrons, respectively), whereas the simpler ones obtain a higher standard deviation value (the maximum is reached for a network with 30 and 10 nodes, respectively). More detailed information can be found on Table A2. Figure 5 graphically shows the standard deviation of the predictive error (values above 10% are trimmed to avoid scale-related issues). The distribution of values is much more irregular than in the case of the average error. However, three areas can be seen: for networks with less than 150 perceptrons, the standard deviation is high (mostly above 10%); for topologies with between 150 and 250 nodes, a very irregular transition zone is produced with scattered high values; and, for the more complex networks (more than 250 nodes), the values are mostly kept below 5%. As can be seen by comparing Figures 3 and 5, although the standard deviation is distributed in a much more irregular way, the trends of both statistical metrics are similar. For networks whose hidden layers contain 150 or more perceptrons, the accuracy and precision stabilize, and there are hardly any significant differences in the performance of these topologies.

Conclusions and Future Work
This paper studies how to optimize the topology of a multilayer artificial neural network to carry out predictions about mechanical properties of aluminum alloys, such as UTS, using machine learning. It is a contribution of great industrial interest since it allows exploring how to obtain sufficiently precise estimates with minimal computational cost and, therefore, using fewer resources. Therefore, the main conclusions of this work are presented as follows: • An artificial neural network with two hidden layers can predict the UTS of wrought aluminum alloys by taking its chemical composition and temper as input. The accuracy of this prediction stabilizes below 4% and even reaches 2.88% in this study. • The predictive ability of an ANN with two hidden layers to estimate the UTS of aluminum alloys is stabilized for topologies that include 150 or more perceptrons in both hidden layers. The precision and accuracy of these networks do not show significant differences that allow us to affirm that one topology is really better than the others. • A multilayer ANN can be used as a tool to, through machine learning, make predictions about the mechanical behavior of a piece of aluminum alloy subjected to forming processes. In theory, these networks can learn to approximate any nonlinear function if the input data set is large enough and has enough perceptrons [37].
This study presents a methodology that allows optimizing the topology of a neural network whose task is to make predictions about the UTS using techniques based on machine learning. In the same way, it would be possible to use this same approach with other properties and even with other materials.
Since this scheme of work is shown to work adequately, a similar method could be applied to test other more complex network architectures. There are a multitude of architectures related to machine learning that allow different problems to be addressed [33]. Acknowledgments: We extend our acknowledgments to the Research Group of the UNED "Industrial Production and Manufacturing Engineering (IPME)" and the Industrial Research Group "Advanced Failure Prognosis (AFP) for Engineering Applications". We also thank Matmatch GmbH for freely supplying all the material data employed to accomplish this study.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations and symbols are used in this manuscript:  Table A1 shows the average predictive error after testing each of the 400 topologies through 10 independent training-prediction iterations. Table A1. Average predictive error (%) for each topology as a function of the amount of nodes in the hidden layers. Hidden Layer  10  20  30  40  50  60  70  80  90  100  110  120  130  140  150  160  170  180  190 Table A2 shows the standard deviation of the predictive error for each of the topologies that are tested.