Geometry and Distortion Prediction of Multiple Layers for Wire Arc Additive Manufacturing with Artificial Neural Networks

Wire arc additive manufacturing (WAAM) is a direct energy deposition (DED) process with high deposition rates, but deformation and distortion can occur due to the high energy input and resulting strains. Despite great efforts, the prediction of distortion and resulting geometry in additive manufacturing processes using WAAM remains challenging. In this work, an artificial neural network (ANN) is established to predict welding distortion and geometric accuracy for multilayer WAAM structures. For demonstration purposes, the ANN creation process is presented on a smaller scale for multilayer beads on plate welds on a thin substrate sheet. Multiple concepts for the creation of ANNs and the handling of outliers are developed, implemented, and compared. Good results have been achieved by applying an enhanced ANN using deformation and geometry from the previously deposited layer. With further adaptions to this method, a prediction of additive welded structures, geometries, and shapes in defined segments is conceivable, which would enable a multitude of applications for ANNs in the WAAM-Process, especially for applications closer to industrial use cases. It would be feasible to use them as preparatory measures for multi-segmented structures as well as an application during the welding process to continuously adapt parameters for a higher resulting component quality.


Introduction
Meeting the growing demand for fast product development, multi-variant product designs, and the ability to manufacture complex geometry components, additive manufacturing (AM) processes have gained increasing interest in recent years. Simultaneously, a variety of different AM technologies emerged following the characteristic layer-by-layer building approach [1,2]. For the manufacturing of metal components, these technologies can be mainly divided into powder bed systems, powder-feed systems, and wire-feed systems [1]. Compared to powder-based processes, wire-feed AM processes are generally characterized by higher deposition rates, larger building space, and high material utilization, whereas powder-based processes exhibit superior geometrical accuracy and resolution [3]. Wire and arc additive manufacturing (WAAM) is a wire-based AM approach that uses conventional arc welding technology for the layer-by-layer deposition of material [4]. Thereby, an electric arc between the base metal and a consumable or nonconsumable electrode is used to create a weld pool and for melting of the wire feedstock. A predefined path of the welding torch is further applied for the part generation.
Based on the high deposition rates and low geometrical restrains, WAAM is particularly suitable for the near-net shape production of large components as well as for incremental manufacturing (IM) [5][6][7]. IM is a production concept where base parts are produced in high volume and then incrementally finalized with different processes such as additive manufacturing and machining [8]. For an efficient manufacturing process, techniques with high deposition rates such as WAAM are considered advantageous for this manufacturing concept. For the combination of WAAM and IM, the concept consists of finalizing a large number of prefabricated metal parts with build-up welding and thus resulting in the required structures and geometry. However, one of the major challenges during WAAM is the occurrence of welding distortion due to a comparatively high heat input and the related inhomogeneous plastic strains [9], leading to the formation of residual stress as well as distortion of the component.
Distortion affects, next to the final part geometry, also the manufacturing process itself. Component warping may lead to unstable process behavior or even process abortion. Hence, the distortion needs to be considered either during offline process planning or insitu by adaptive manufacturing strategies. Since the manufacturing process is complex and distortion is influenced by many parameters such as energy input, temperature regimes, part geometry, inner and outer restraint, and material parameters, an enhanced prediction method is needed.
In order to account for these influencing factors in predictions of the AM process results, several approaches have been implemented. Simulations with thermal finite elements (FE) and other thermal modeling such as thermo-elastic-plastic transient models [10,11], GAMMA-simulation [12] as well as fuzzy logic [13] have been used for welding processes. Due to the difficulty and the often very limited available computational times required for accurate simulations for welding processes, the usage of artificial neural networks (ANN) has been applied to these multi-criteria problems to a much larger extent, often in combination with simulations [11][12][13], although the use cases differ.
Several researchers examine the influence of specific parameters such as welding gases [14,15] with ANNs, often with a focus on the ANN-based prediction of the resulting welding bead geometries. These are mostly based on the usage of welding parameters such as layer count, welding speed and current as well as voltage as inputs with bead width and height as outputs. Variations in the usage of these parameters can be found. Kim  combined the ANN with a support vector machine in order to provide a high prediction quality with multiple inputs in spite of fewer data [16][17][18][19][20][21].
Other approaches include looking at specific operations for welding processes, such as the joining of metal plates with a focus on the resulting distortion [22] or optimization of welding parameters for special purposes such as thin-walled aluminum structures [23]. Tian et al. and Tafarroj et al. examined the possible resulting plate geometries as well as distortion for the substrate for single beads [24,25]. Thus, according to prior research, the prediction of the resulting bead height and width, as well as distortion, is a viable application for ANN. However, the applicability of the current approaches towards different experimental setups or different real-life implementations within a reasonable timeframe is questionable. Distortion is often affected by the welded geometries and related factors such as wall thicknesses which requires geometry-specific adjustments to the ANNs and the machine-learning algorithms. This is not efficient for small to medium scale process chains such as IM, where these welded additive processes must be predictable in order to efficiently design the process timelines. In particular, predictions for possible occurring geometry deficiencies such as deviations in component heights lead to better planning, or with countermeasures taken during the process, to a decrease of manual intervention and an overall increase of efficiency. The current state of the art shows that prediction is generally possible and examines potential training processes and data acquisition solutions. However, the next steps towards larger-scale applicability for different welded structures and components with few predefined parameters or pre-programmed FEM-models require further research.
The strategy of this research is a combination of analyzing welding and geometry parameters with the objective to predict the distortion and build-up of multiple beads for usage in incremental manufacturing on thin plates. It focuses on the application of ANNs without additional software implementations for straightforward transferability. This research mainly distinguishes itself from the previous approaches through its expandability. With the proposed enhanced ANN, the proof-of-concept for a continuous prediction during the welding process is made, which could be applied to larger structures in further research. This continuous prediction can be achieved through a recursive prediction of defined building blocks, which serve as a simplification of defined geometries and their characteristics, such as wall thicknesses. As such, this research is a preliminary feasibility study for the usage and recursive prediction with ANNs exemplary for the combination of single layers. The resulting proposed enhanced ANN predicts welding structure geometry and their distortion in a recursive manner, which leads to a great increase in accuracy without large amounts of additional welding experiments. This enables accurate prediction with reduced time and resource investment and shows good prospects for an application with defined geometries as building blocks.

Materials and Methods
The procedure of this study is visualized in Figure 1. As a necessity for further examination, the parameters for the experiments and the generation of datasets are described at the start of Section 2.1. Following this, in Section 2.2, the different concepts for ANN as well as their data requirements, such as in-and outputs, are developed. With the framework conditions for experiments and required data being set, the used data and the handling of possible outliers are assessed in Section 3.1. Finally, the resulting concepts for ANNs with their different possibilities for used data are analyzed and compared in Section 3.2.

Generation of Datasets
In order to generate a data set for analyzing and predicting multipass welding distortion, welding experiments were performed using a Fronius CMT Advanced 4000 R welding power source (Fronius International GmbH, Austria), with the welding torch being guided by a KUKA KR-22 6-axis robotic system (KUKA Deutschland GmbH, Germany). This multipass welding is typical for usage in IM and forms the basis for larger additive welding structures. Single and multipass welding procedures were carried out on 1.0330 mild steel substrates (200 × 52 × 2 mm). A weld seam length of 150 mm was deposited using a 1.5130 solid wire electrode with a diameter of 1.2 mm. The chemical compositions of the used materials are given in Table 1. All welding experiments were performed using a conventional short arc process and an Argon (99.996%) shielding gas flow of 14 L/min. The contact-tip to workpiece distance was set to 10 mm. For the welding of pre-heated samples, the samples were tempered on a heating plate. Prior to welding, the temperature was verified on the sample surface using type-k thermocouples. To describe the influence of the main process parameter's pre-heating temperature, energy input, and number of layers on the resulting distortion of the baseplate, full fractional parameter variations were conducted according to Table 2. Therefore, there are 48 cases for three different amounts of layers resulting in 144 data points as a potential input for the machine-learning algorithm. During welding and subsequent cooling sequence, the specimens were positioned using a floating bearing arrangement without clamping, allowing free deformation (see Figure 2a). This arrangement was chosen to solely consider the welding distortion under idealized conditions without influences of the clamping conditions. Three stops were used for repeatable positioning of the specimens on the welding table. Only minor deviations of the position in the x-y plane were detected due to the distortion of the samples. Deviations in the z-plane resulted in a change of the contact-tip to workpiece distance along the predetermined weld path. Based on the measurements of welding voltage and current, these showed no significant influences on the deposition process. For the determination of the energy input per layer, welding voltage and current were measured using a weld process data scanner (HKS Prozesstechnik GmbH, Germany) and integrated over the welding time. Welding distortion has three primary components in x, y, and z directions. This study is focused on the main distortion in the y-z-plane as distortions in x-z-plane were found to be considerably low. The distortion in the y-z-plane ( Figure 2b) was measured after cooling the sample using a dial gauge. For the measurement procedure, the sample was positioned on a measurement table with the curvature from distortion facing upwards. Further, the distortion was determined as the highest point (minus sample thickness) at four points along the x-direction using a dial gauge. Further, the resulting weld bead geometries respective build-up height and width were measured in the x-z-planes with a laser scanner, as depicted in Figure 3. Measurements were done on both sides, on the top side, referred to as upper weld bead in this study, as well as on the underside of the plate, called lower weld bead or weld root, in the following paragraphs, respectively.

Development of Artificial Neural Networks (ANN)
An ANN is a machine learning application for building correlation models and consists of a user-defined number of processing elements called neurons. These are partially or fully connected with weighted links and arranged in layers, starting with an input layer leading to one or more hidden layers and ending in an output layer. The weights can be adjusted using different approaches in order to build or train a model for a correlation between inputs and outputs. These training processes are influenced by a high degree of randomization, for example, in which order the data is used to train and is therefore often difficult to reproduce. For that reason, there will not be a best neural network for a use case in most instances. Instead, parameters for ANNs are often configured, so they merely tend to result in good and useful predictions while still depending on randomization. In order to enable an assessment of the quality of the constructed ANN, a division of the given data sets into training, testing, or validation data sets is most often performed. A small part of the data set is excluded from training, which then can be used to test or validate the ANN in order to assess the generalization ability as well as potential overfitting of the network. More information on the structure and development of ANNs can be found in several publications (e.g., [26,27]).
Before starting to build and train these ANNs, concepts for the implementation of the in-and output values are developed. The first type of architecture for an ANN is to use the actual welding parameters as the input. These parameters can be directly influenced by the welder and are displayed in the first five columns of Table 3. The corresponding outputs are the five columns on the right, which are the resulting geometry as well as the distortion. Geometry, in this case, indicates the height and width of both the lower and upper bead. These standard ANNs are only applicable for undistorted "blank" substrates, as it has no way of being informed about a previous distortion with this input data. As a second type of ANN architecture, an enhanced procedure is chosen. This can also be described as a recursive concept, where the geometries and distortion from previous welds on the plate are used as an additional input. This means, that for continuous prediction, the ANN is able to partially reuse previous outputs as inputs. For this approach, there is always a given pre-distortion, which is used in combination with the welding parameters to predict the distortion after the next welded segment, which can consist of one or multiple beads. These two types of ANN are visualized in Figure 4 and serve as a basis for the further examination in this paper. For this work, an approach for ANNs with two hidden layers and an early stopping mechanism against overfitting with a maximum epoch count of 500 is used. For these implementations, ANN parameters such as batch size, layer count, and amount of neurons per layer or moment were set to initial values and then varied by a multidimensional grid search before setting the final parameters for all ANNs. Batch size 1, ReLU activation for the first two layers and linear activation for the output layer were selected this way. After this process of optimizing hyperparameters, the ADAM-optimizer with 150 neurons per hidden layer is applied with a dense sequential implementation in the python-package KERAS version 2.3.1 was chosen. Further details can be found in the Supplementary Materials.

Results and Discussion
The results are organized into separate parts. First, the generated datasets were examined and possible outliers identified. Afterwards, the creation of ANNs and options for dealing with these outliers were discussed. Finally, the resulting ANNs were analyzed for their accuracy and the created correlations for the input parameters and compared to existing research.

Analysis of the Used Data Sets
Examining the data from the welding experiments, two specific cases regarding the general characteristics of the resulting deformation could be distinguished. During welding with low and medium energy input, the main shrinkage distortion in y-plane predominantly occurred at the top surface of the substrate, leading to positive distortion values in y-z-plane. At high energy inputs, the distortion values in y-z-plane were found to show comparatively low, and in particular cases, negative values. This may be explained by an increase in welding depth. Figure 5 compares the weld seam cross-section of a sample welded with low energy input; (a) shows comparatively low weld penetration depth with no visible formation of the weld root. In contrast, when welding with high energy input (b) a distinct formation of the weld root is visible. Depending on the size of the weld root, this leads to a significantly different shape of the specimen in y-z-plane, causing a change in the direction of main distortion. The quality of these generated data sets is decisive for the validity of the generated ANN and is examined in the following paragraphs. As an overview of the created data, four examples for the different slopes of the 48 data points are illustrated in Figure 6. There is no data for the bead count three as the experiments measured one, two, and four layers.
Different exemplary progressions are shown, with the slope of the distortion staying positive in most cases. The noticeable exception is the lowest progression shown in light blue, where high energy input causes full penetration of the weld bed and results in negative distortion into the direction of the weld root. Five of these outlying experiments with a negative slope appeared in total, mostly at low welding speeds with a high wire feed.
The data points showing a reduction of distortion seem inconsistent with the rest of the data and could have a negative impact on the accuracy of the ANNs. To filter out these outliers, different strategies were implemented using an unedited dataset as a point of reference: As a first filter strategy, the changes in distortion between the welding steps (see Figure 6) were examined and the datasets making up the worst 5% of data were removed before training and validating the ANNs. In contrast, in another strategy, the ANNs were trained with unedited data, removed the data with the greatest deviation from the fit of the created model with an implemented algorithm automatically and re-trained with the remaining data points. A third strategy was to remove all the datasets with the highest wire feed, as these appeared to be not suitable for applications in WAAM. For the standard ANN, a dataset using only the data for the maximum build-up of four layers was also used in order to model a possible improvement for the prediction of higher layer counts. The training and test split of the data was randomized for every new construction, with 80% of data used for training and 20% for testing. This construction of the ANNs was repeated ten times for each approach to choose the respective best networks in order to achieve a valid comparison. To assess the importance of the inputs and outputs for the ANNs, different combinations have also been tested. In total, three types of settings were evaluated for standard and enhanced ANNs, which are described as removal of data, as well as the combination of inand outputs in Figure 7 (left). As an example, for the enhanced ANNs, one type of input with two types of outputs was implemented with four strategies for removal of outliers for a total of 8 examined enhanced ANN-structures. The results from these settings have been plotted in Figure 8 for the absolute test and training errors. Generally, the ANNs tend to be better with larger inputs, as seen in the graph in the top right. The most striking result is an accumulation of low errors for the blue-colored highlighting of the enhanced ANNs in the graph on the bottom right. In order to achieve an exemplary comprehension of the comparison of the applicability of these approaches, an ANN from the enhanced, as well as standard category, is selected and compared in the following paragraphs.  Table A1 in the Appendix A. The data shows good results marked in blue with averages of the absolute values for errors being as low as 2.5% for training and 3.0% for test data (~0.3 mm), some average fittings with accuracy between 10% and 30% (~up to 1.3 mm) as well as some outliers.

Results of the implementations of Figures 7 and 8 are displayed in
Below, the results from the enhanced and standard ANNs are visualized and compared. For better comparability of the following diagrams, exactly the same experiments as datasets, as well as the same ANN-modeling concepts, were used for the enhanced and standard ANNs. For a better analysis of the behavior, an enhanced ANN with very good error values for unedited data and the standard ANN with the same unedited dataset for the conventional inputs of welding parameters as well as the amount of beads are shown in Figure 9. The average absolute error for enhanced is 0.32 mm and 1.325 mm for standard. The approach to predict distortion based on the additional input of resulting geometry does result in a lower average absolute error of 0.84 mm, but this is still significantly worse than the enhanced approach. Additionally, a distortion prediction based on the resulting geometry of the welded structure is not feasible for a real use case, so it is not examined further. In these diagrams, the x-axis shows the measured distortion, and the y-axis the predicted distortion. All test and training data were plotted. The target line indicates the values for a perfectly accurate prediction. In both diagrams, the prediction dots were scattered very closely along this line, which proves an accurate prediction model for the created ANNs. In comparison to each other, the enhanced ANN scatters less, which is also supported by the minimum and maximum errors of (4 mm/−7 mm) for the enhanced ANN as well as (6 mm/−9 mm) for the standard ANN. Groups of large errors cannot be clearly assigned to an area of real distortion. Hence, the ANN should work well for all examined distortion values. The prediction also does not result in data points scattered more on one side of the target line, which means that there is no imbalance, such as a shifted prediction towards high distortions.

Discussion of Parameters for ANNs and Comparison with Other Works
Generally, the data shows better results for ANN with more data-inputs from the enhanced model. This was expected from a data-quantity-dependent process such as ANN, where correlation models often vastly improve as the quantity of data increases.
The objective of the calibration of the chosen parameters was to achieve good results while avoiding overfitting. As mentioned previously, the extent of overfitting can be judged by the difference in test and training errors. According to columns one to four of Table A1 in the Appendix A, the differences confirm a good fit with mostly no observable overfitting. For some implementations, the test error is even smaller than the training error, which is caused by the randomization of the split of the data into the two datasets. This reinforces the good fit of the implemented ANNs.
Even if data points are removed according to the implemented strategies, no significant improvement of the predictability can be observed. The ANNs are able to tolerate these outliers and continue to deliver usable results. This is caused by the consistently occurring penetration of the weld bead, although the root formation is mostly smaller and less distinctive than these outliers. Thus, the more extreme data points, where the weld root formation was significantly higher, do not have such a detrimental impact on the data sets as a whole. Having some of these data points in the training data even helps predict the real distortion and penetration welding to some degree, as it shifts the data points as a whole towards the formation of weld roots for certain welding parameters, which reflects reality.
Looking at similar published research, the created ANNs error rate of up to 3% for test data also proves to be comparable. Tian et al. describe the errors for ANNs with a somewhat similar method of measuring distortions at 8-9% as relatively low [24]. Tafarroj et al. achieved an error rate of 2.4-5.3% with a prediction method for a different method of distortion measurement for the joining of steel plates [25]. Casalino et al. shows mean errors for test data in a range from 1-14% depending on the output data with experimental outliers removed by hand and a combined approach with FEM [18]. In Summary, good to very good results have been achieved for a prediction model without large datasets or assistance of a simulation model. The focus on transferability of the presented methodology has no negative effect on the accuracy compared to existing research.

Analysis of ANN and Created Correlations
Below, the correlation of predicted distortion with the input parameters for the enhanced ANNs are examined for a single selected ANN. In analogy to the plot in Figure 9b, the enhanced ANN with the lowest absolute test error was chosen. For this ANN, the standardized residual of the prediction errors over the predicted distortion is shown in Figure 10.   Figure 10. Standardized residual over predicted distortion for an enhanced ANN with unedited data of output distortion (see Figure 9b). Additional list of parameters for the five outliers (Marked as A-E) given.

E -1 ------� �
Even though there are some outliers surpassing an upper and lower limit of three times the standard deviation, this plot visualizes the good fit of this enhanced approach, as over 95% of datapoints are within these boundaries. The standardized residual is even lower for most points, with the average of the absolute standardized residual being 0.485. The five outliers are listed at the bottom of the figure. The low amount of outliers compared to the total amount of 144 datapoints only allows very limited statements about their nature. The only obvious common ground is a current welded layer count of one, which suggests that the ANN is mostly challenged by the prediction of early layers. As noted in Section 3.2, these outliers appear in a broad area where the bulk of the predicted data points are also located. In summary, Figure 10 confirms low prediction errors even for predictions of high and low distortions and an overall ability to predict all trained ranges of parameter values for this preliminary application of an enhanced approach.
For further analysis, the complete dataset is predicted with the enhanced ANN from Figures 9b and 10 and compared to the results from the experiments. This makes it possible to analyze the influence of different input parameters on the predicted distortion. Analyzing the correlations for the non-continuous datasets such as wire feed is difficult, as the points are mostly stacked on top of each other and are difficult to interpret. Therefore, Figure 11 shows the plots for the predicted distortion of the next layer over the continuous dataset energy input as well as the enhanced inputs for the geometries and the previously welded current distortion. Figure 11. Visualization of the influence of inputs on the accuracy of distortion prediction with an enhanced ANN with unedited data from the input of all welding parameters, current geometry and distortion and output distortion. Plotted is the predicted next distortion over (a-d) current upper and lower bead geometry, (e) energy input and (f) current distortion.
The plots b and d with the weld bead widths as inputs show no correlation. Plot a with the current upper weld bead height shows two seemingly separate columns. The reason for this is the current upper height being either a result of welding a single layer or two layers, as these are the two possible input bead counts for the enhanced ANN. Plot f for the predicted next distortion over the current distortion shows a distribution that appears to be a linear correlation. As a result, the created ANN seem to predict higher distortions for already pre-distorted objects, which is the expected behavior and reinforces the good fit of the created ANN. Plots c for the lower weld bead height and e for the energy input illustrate the effect of large amounts of thermal energy causing full-penetration welding as well as reduced distortion. In plot c, for current bead heights above 0.7 mm, no predicted distortion above 10 mm was achieved. This is a reasonable correlation trained by the ANN, when considering the increased bending moment required for large material depositions on the underside of the plate. For very high energy inputs above 40 kJ, the predicted distortion also tends to be lower; this models the effects of full-penetration welding, as described in Section 3.1.
In the diagrams of Figure 11, the error of the predicted distortion to the actual distortion is shown by the color of the dots. No particular areas that tend to result in large errors were identified. Merely the areas with low current lower weld bead height can be described as the area with all of the larger errors. However, in the same area, a very large amount of predictions with a low error are also existent, so the relative amount of large errors remains low. In summary, this concept for an enhanced ANN promises to deliver very good results even for the edges of available data.

Conclusions
In this study, a method for an enhanced ANN-based prediction of distortion for WAAM structures was presented and compared to standard methods. One hundred fortyfour welding experiments with one, two, and four layers as well as different welding parameters were carried out. Several implementations for the data pre-processing were proposed and tested.
In comparison to other publications, very good results have been achieved with the enhanced ANN. The resulting ANNs showed great predictability even for extreme values in the datasets. Analysis of the inputs and predicted distortion showed correlations suggesting a good fit of the created ANN. Depending on the utilized neural networks, robust systems with low error percentages of up to 3% were achieved. The proposed enhanced method proved to result in better predictability than previous approaches, even though the amount of welding experiments was the same as for standard ANNs.

Potentials and Limitations of the Current Study
The presented ANN is very suitable for usage in additive manufacturing on thin substrates, which has a lot of potential in the applications of incremental manufacturing, for example, for finishing stamped metal sheets with small additive manufactured functional structures. A proper application is used in path planning and minimization of process time for continuous welding processes, as the ANN can continuously update predictions for the required amount of remaining layers and thus welding time. With further utilization of ANNs, an implementation to calculate optimal welding parameters for the desired result with minimal distortion is also possible. However, an extrapolation for more complex components, such as multidimensional, multi-layer structures with complex shape and contour parameters based on these datasets requires further adjustments and training data. With the presented experiments, the groundwork was laid for the usage of enhanced ANNs for wire arc additive manufactured parts and the prediction of their geometries and distortions. Especially for larger components and welded structures that are no longer part of the Incremental Manufacturing process, distortion of the substrate metal plate is often negligible. This is either caused by the usage of very thick plates that possess inherently high stiffness or through the added stiffness of the structure created by the added layers. Disruptive effects still occur in this context, which results in shapes and geometries that can differ significantly from their intended results. This is caused by factors that are difficult to measure or control, such as the heat dissipation in the structure and varying layer height.
Possible approaches for an adapted and improved, enhanced prediction is the division of components into smaller building blocks of a defined size and layer count. Thereby, the remaining welding work can be approximated after the creation of each of these building blocks. This reduces the need for manual intervention for the welding of near-net shape components and makes a compromise between excellent prediction, which could be achieved by very time-consuming measurements after each welded layer, and realistic processes applicable to industrial setups.  Data Availability Statement: Data supporting the reported results, including code and graphs, can be found at the following link: https://lnk.tu-bs.de/52f8kz (accessed on 19 May 2021). Table A1. Results for implemented ANNs color-coded according to their error: blue for low errors, red for high errors-absolute error in mm. Shown are the mean values of the absolute error for the predicted distortion for test and training data. Further information for every neural network can be found in the Supplementary Materials.