Bead Geometry Prediction in Laser-Wire Additive Manufacturing Process Using Machine Learning: Case of Study

: In Laser Wire Additive Manufacturing (LWAM), the ﬁnal geometry is produced using the layer-by-layer deposition (beads principle). To achieve good geometrical accuracy in the ﬁnal product, proper implementation of the bead geometry is essential. For this reason, the paper focuses on this process and proposes a layer geometry (width and height) prediction model to improve deposition accuracy. More speciﬁcally, a machine learning regression algorithm is applied on several experimental data to predict the bead geometry across layers. Furthermore, a neural network-based approach was used to study the inﬂuence of different deposition parameters, namely laser power, wire-feed rate and travel speed on bead geometry. To validate the effectiveness of the proposed approach, a test split validation strategy was applied to train and validate the machine learning models. The results show a particular evolutionary trend and conﬁrm that the process parameters have a direct inﬂuence on the bead geometry, and so, too, on the ﬁnal part. Several deposition parameters have been found to obtain an accurate prediction model with low errors and good layer deposition. Finally, this study indicates that the machine learning approach can efﬁciently be used to predict the bead geometry and could help later in designing a proper controller in the LWAM process.


Introduction
Additive manufacturing (AM) is the process of depositing materials layer-by-layer to produce a part, as against subtractive manufacturing, which starts from a block of raw material, removing small parts to obtain the final product. The drawbacks of the subtractive method are its material waste and geometric limitations in creating complex geometries [1]. Laser Wire Additive Manufacturing (LWAM), also referred to as Laser Wire-Feed Metal Additive Manufacturing (LWMAM), is a variant of the Direct Energy Deposition (DED) method. DED creates parts by melting the material into a molten metal pool [2]. LWAM uses a wire-metal feed-stock and a laser source to create the molten pool, as shown in Figure 1. It usually requires a shroud of shielding gas to protect the deposition from contamination. Metal wires offer an environmentally friendly process by being cleaner, more cost-competitive, cheaper, and more abundant than metal powders. In addition to being identical to the wire used in welding applications, a standardized and qualified product can be obtained. Additionally, the wire-feed approach, in comparison with the arc-welding approachm affords a larger build volume at high productivity [3,4]. There is a growing interest in the aeronautical sector to find methods to create complex large parts that lead to the improvement of the buy-to-fly ratio [5]. Therefore, it becomes essential to improve process planning and deposition strategies to get an optimal combination of parameters involved in the LWAM process. One of them is layer geometry. When it is well defined, layer height and width improve the part's geometric accuracy and the surface quality of the product [6]. Figure 2 illustrates the flow path of process planning for LWAM. The importance of the prediction model for the whole LWAM process should be noted; this model can influence, for example, the method by which process parameters are chosen and optimized, improving the surface quality of the final product. Furthermore, the tool's path can be adjusted based on the obtained model and can be utilized when generating the robot code. Thus, if a predictive model can be implemented properly, it will lead to products of higher quality and productivity increases [7]. Methodologies for solving the issue of predicting layer geometry during the deposition process are divided into two approaches. The first approach uses numerical or mathematical models, wherein the layer deposition is predicted from experimental pro-cesses. The second approach uses sensing devices to develop control algorithms that regulate process parameters, such as laser power, the wire feed rate, travel speed and others during the printing process. The feedback control approach is commonly used in industry and employs sensors such as pyrometers or camera vision systems to monitor the melt-pool characteristics and temperature of the deposition [8][9][10]. Much research [11][12][13][14][15][16] has been reported on the use of the control approach to monitor the deposition parameters to minimize observed fluctuation and keep a controlled deposition environment. However, such an approach is not always feasible, due to process uncertainties and variations occurring during the deposition process that augment the system's complexity. Also, sensing devices face some delay in acquiring information from the system and further delay is observed when transferring data between devices and analyzing the information before the control action. Another constraint occurs in the melt pool when laser reflection and conducted heat add complexity to the information acquisition [17]. A literature survey has confirmed this, (e.g., [6,7]) since the majority of validation using this approach is performed for simply shaped structures, focusing on only one parameter.
On the other hand, the mathematical and numerical approach relies on the use of generic equations or simulations to study different deposition scenarios and see the influence of the layer geometry on the final produced part [18][19][20][21][22][23]. To improve the surface quality of the deposition Zhu et al. [24] investigated the influence of varying the laser's power and defocusing distance on deposition. The laser power was decreased at each layer to avoid energy accumulation and reduce the pool size change. Additionally, the same layer thickness was used. This approach yields differences in the top layer when changing defocusing distances due to fluctuating parameters and system error. This is due to the height difference observed between layers, which propagate to the other layers, becoming increasingly pronounced if the difference is not compensated for, hence, the necessity of self-regulating deposition height. As layer thickness needs to be adjusted to satisfy the process's growth, Donadello et al. [25] proposed a reconstruction of the spatial map of the height to see the variation of the layer thickness and found a self-regulating mechanism therein; after an initial transient period showing faster deposition growth in the first layers, deposition rate exhibits a decreasing trend until converging to steady-state, interpreted as an equilibrium condition, is achieved. In this approach, although there exist disturbances from system sensitivity and error, the self-regulating mechanism occurring during the deposition process is obvious. For this reason, the problem of height prediction should be solved in advance of regulating those process parameters occupying a primary level in the 3D printing process.
Machine learning (ML) is applied in many domains, today. Many industrial sectors face new research challenges and complex problems yet to be addressed. The advantage of ML is demonstrated in areas where underlying physical models are difficult to express. The neural network (NN) algorithm is the most used ML method due to its advanced algorithm structure [26]. NNs can be used as supervised ML or unsupervised learning problems for complex problems with highly nonlinear relationships. The architecture of an NN consists of the input layer, a hidden layer, and an output layer [27]. The layers are composed of nodes or neurons. In Khatir et al. [28], an improved artificial neural network (ANN) using arithmetic optimization algorithm (IANN-AOA) for damage quantification problems is proposed. Free vibration and damage identification have been investigated on functionally graded material (FGM) plates, while, in [29], damaged laminated composite structures are identified using an ANN. In this approach, the network is trained using fast convergence speed of gradient descent (GD) techniques and the global search capacity of evolutionary algorithms EAs is used to quickly guarantee the best possible solution. Finally, Zenzen et al. [30] implemented a modified transmissibility damage indicator using local frequency response ratio (LFCR) and ANN to detect the location and size of damage in composite structures while reducing collected data for fast prediction and high accuracy.
Xiong et al. [31] used second-order regression with a neural network for predicting bead geometry in wire and arc additive manufacturing (WAAM) to study the relationship between the process variables and the final bead geometry. While Nagesh and Datta [32] used the back propagation (BP) neural network model for estimating bead geometry for tungsten inert gas (TIG) welding process. Xia et al. [7] developed a roughness measuring method for arc additive manufacturing using a laser sensor-based approach combined with different machine learning models. On the other hand, Milhomme et al. [33] studied first-order parameters to identify correlations between input parameters and geometrical outputs of Ti-6Al-4V beads manufactured with the laser metal powder deposition process. Finally, Karmuhilan and Kumarsood [34] discussed the use of an artificial neural network (ANN) to predict the bead parameters based on given process parameters for the WAAM system.
From the above literature review, one can conclude that the machine learning approach and, more particularly, NNs, have not been applied to LWAM bead geometry prediction considering deposition parameters that influence both the bead width and the height of the produced part. This is due to the complexity of layer deposition stability and the high feed rate of the LWAM process in comparison with other techniques, such as WAAM [36]. Therefore, this paper aims to provide a machine learning approach using a regression algorithm to predict the layer geometry in the LWAM process. Different experiments are performed to deposit sample products and, with the developed prediction model, the height, width, and ratios for upcoming layers are proposed. Furthermore, the influence of the deposition parameters (laser power, wire feed rate and travel speed) on the final produced geometry is discussed based on a neural network prediction model. This case study could provide inspiration and guidance for improving surface roughness in LWAM using a proper feedback control system.
For proper implementation of the prediction algorithm, we employ a technique used by Lim et al. [37] for collecting the data. The method uses scripting languages to propose a method of slicing layers and generating printing paths. The use of the parametric method is adopted to generate printing paths using Rhino's Grasshopper environment (see Figure 2). In Lim et al.'s approach, the prospect of flexibility in model design provides more opportunities for the designer to decide on the size and shape of the layer (flat or curved), path patterns, and other parameters affecting the surface's geometrical quality and printing time. However, this proposal could be improved by taking into consideration the height variation mentioned in Lim et al.'s work and, especially, by finding a way to avoid many of the issues found with G-Code (see e.g., [38]), as well as communicating directly to the printing machine in terms of AM using an industrial robot.
The paper is structured as follows, Section 2 introduces the global approach of the methodology, while Section 3 provides a detailed theoretical background for the proposed prediction model. Section 4 presents the experimental results and discussion. The paper finishes in Section 5 by a conclusion and a proposal of future work.

Methodology
In this paper, the bead geometry measurements for different layers have been obtained using Grasshopper software. Grasshopper uses generative algorithms and parametric modeling that differentiates it from other CAD software, which uses polygon mesh-based representations to model an object. More specifically, Rhino's Grasshopper was used to create the physical part specifications (bead size, layer height, pattern, etc.). Then generative algorithm was implemented and converted to robot programs using the HAL framework. This approach was chosen because the programming methodology interface is suitable for mathematical representations without passing through the G-Code step [39]. This is done using the following steps: • Parametric surface creation: In this step, mathematical tools are used to create a 2D model surface. Alternatively, a CAD part could be imported directly and changed to a parametric CAD part using Grasshoppers tools.
• Pattern creation: In this step, patterns (zigzag, spiral tool path, etc.) are designed and added to the 2D surface. The zigzag pattern is advised for its simplicity of implementation, robustness, and surface roughness [40]. • Layer-to-layer deposition strategy: After depositing the first layer, a suitable travel strategy is required to reach the next layer, and successively. The travel strategy depends on some deposition parameters, such as the laser power, wire feed rate, and robot travel speed. Therefore, the travel strategy should be carefully chosen as it can influence the cooling of the previous layer as well as the bonding between layers. For more details see e.g., [41,42]. • Bead geometry measurements: Finally, a laser scanner was used to obtain the measurements data of the bead geometry during the LWAM process. The laser scanner was mounted on the welding robot's arm to directly measure the bead geometry for each layer deposition. A measurement for a single bead geometry and layer deposition is shown in Figure 3. After having several bead's geometry measurements and dimensions, a prediction function F(x), based on a regression algorithm, is used to predict the bead geometry for the current layer and the successive layers. The prediction function F(x) can then be fed to Grasshopper with two more inputs, the product design specification and the bead dimensions. Next, the designed parametric object is loaded to the HAL framework and converted to a robot program, along with the motion settings and the robot calibration. Finally, a simulated 3D printing program of the final product is ready to be transferred to the real robot to begin manufacturing 3D parts, as shown in Figure 2.
The prediction algorithm, used in this work, includes linear regression and linearization steps for non-linear trend (power decay function). The power decaying function is used to represent the self-regulation of the layer geometry. The prediction model takes as input the training examples (x i , i = 1 to m, m = final layer geometry dimension) and the output is the model prediction that relates the bead geometry changes over layer depositions.
Finally, a neural network technique was used to correlate the bead geometry measurements with the first-order deposition parameters: laser power, wire feed rate, and travel speed. In this way, the prediction algorithm method (discussed in Section 3.1) was used first with the fixed deposition parameters, and then a neural network (discussed in Section 3.2) was used with varied deposition parameters to study their influence on bead geometry.

Prediction Model
To predict the bead geometry for a given sample, a supervised-learning algorithm is employed. Supervised learning is efficient for problems for which it is possible to find a closed-form solution between the inputs and outputs. The geometry prediction is taken as a regression problem in which a mathematical function is derived from a training set.
In this work, a power decay function is used to represent the prediction function and solve the self-regulation mechanism occurring in the deposition process. Furthermore, bead height, width and width-height ratio prediction models are obtained for different deposition parameters. However, since the same algorithm and technique is used for all prediction models, in this section height prediction equations will be discussed and other prediction models equations are omitted for brevity.

Bead Height Prediction Model
The prediction algorithm begins by creating a dataset to store training examples (x i , y i ), i = 1 to m, m = final layer. x i is the input variable (number of layers) known as input features, and y i is the output variable (layer height) that needs to be predicted, also called the target variable.
The output of the regression is a prediction function F(x). The function relates the evolution of the bead height over layers. The algorithm is implemented in Python and is composed of the following steps: First, a mapping function is created to represent a decreasing concave up function trend: a * x b + c; a and c > 0; b < 0 ; a is the vertical stretch of the function; b is the rate of decay of the function and c the horizontal asymptote of the function. The mapping function displays the bead height evolution over layers. Then, the training set is plotted and the data linearized with a natural logarithm. After that, a linear regression algorithm is applied to fit the least-squares line to the natural logarithm plot (linearized model). Finally, the power decay function parameters are derived with which to compose the layer prediction function. The process of the bead height prediction is shown in Figure 4.
where {θ j ; j = 0, 1} are unknown variables. Then, a cost function J(θ) is defined as follows: where {x i , y i ; i = 1, . . . , m}. After that, θ is chosen so that the cost function J(θ) is minimized. A search algorithm called gradient descent is applied.
Note that this update is performed simultaneously for all values of j = 0, . . . , n, and n is the number of unknown variables. In this work, the learning rate (α) technique is used; such that when the learning rate is decreased to zero, the regression algorithm converges to the global minimum rather than oscillating around the minimum. Tuning experiments will help to detect a reasonable learning-rate value. The partial derivative of J(θ) gives the least-mean-squares (LMS) update, which is also known as the Widrow-Hoff learning rule, and it is defined as follows: Finally, a number of iteration and learning rates were selected to converge to the best values of gradient descents. If the prediction output is not good, either the learning rate and the number of iterations are modified or the mapping function parameters are changed. The prediction process is summarized in the flowchart shown in Figure 5. In this work, the number of iterations ranges from 1500 to 2000, with a 0.03 learning rate.

Neural Network Prediction
In this paper, the neural network technique is used to study the correlations between the dispositional first-order parameters (laser power, wire feed rate and travel speed) and bead geometrical measurements (height and width) in the LWAM process. The Levenberg-Marquardt algorithm (LM) is used to train the neural network and solve the optimization problem [43].
The LM algorithm is designed to work specifically with loss functions that take the form of a sum of squared errors. It works without computing the exact Hessian matrix. Instead, it works with the gradient vector and the Jacobian matrix [44]. For the paper self-consistency, the LM algorithm computation is briefly discussed in this subsection.
Let us consider the following loss function which takes the form of a sum of squared errors, where m is the number of training samples. Then let's define the Jacobian matrix of the loss function with respect to the parameters, where m is the number of samples in the data set and n is the number of parameters in the neural network. Now, we can compute the gradient vector of the loss function as, where e is the vector of all error terms. Then, we can approximate the Hessian matrix with the following expression, where λ is a damping factor that ensures the positiveness of the Hessian and I is the identity matrix. Finally, The next expression defines the parameters improvement process with the Levenberg-Marquardt algorithm, for i = 0, 1, . . . , m, were w is the parameters vector. The parameter λ is initialized to be large so that the first updates are small steps in the gradient descent direction. If any iteration results in a fail, then λ is increased by some factor. As we will see in the following, this relatively simple method gave satisfying results for finding the relations between the input parameters and bead geometries.

Experimental Results and Discussion
Experimental test campaigns were carried out on the laser platform at Institut de Soudure. Tests were run by a robotized laser wire-feed system. An ABB 7-axis polyarticulated robot was used to provide the kinematics of the process. A fibre laser source, IPG Photonics, of 10 kW was used as the heat source. The system also used a CoaxPrinter laser processing head to deliver the laser beam and the filler wire to the processing zone. Figure 6 shows the actual LWAM system setup.
To perform the experiments, Grasshopper was used to create the physical part specifications (bead size, layer height, pattern, etc.). A generative algorithm was implemented and converted to robot programs using the HAL framework. In the experiment, a hollow wall of 150 mm × 50 mm was deposited and the complete deposition path was performed in generative algorithms.

Bead Geometry Prediction
To test the prediction algorithm proposed in Section 3.1, a hollow wall of dimension 150 mm × 50 mm with 15 layers was deposited using the following process parameters: laser power = 1.4 kW, wire-feed rate = 1.4 m/min, and travel speed of robot = 0.45 m/min. These initial process parameters were chosen to achieve a stable, uniform, and homogeneous final part, as shown in Figure 7.   Bead height (mm)

Prediction of Bead-Height
Traning sample Validation sample Predicted bead-height curve  Bead width (mm)

Prediction of Bead-Width
Traning sample Validation sample Predicted bead-width curve Figure 9. Result of the bead width prediction.
It is worth noting that the plot of the training set in Figure 8 shows a power decay trend in height across layers. This observed trend is the so-called self-regulation mechanism of the layer height, and it has been discussed by Simone Donadello et al. in [25].
In Figure 10, the bead-width-over-bead-height ratio across layers is plotted. From this experimental result a logarithmic increase can be observed for each layer. In the observation, the width-to-height ratio increases significantly during the first layers before reaching a threshold around layer 10. Layer number Bead width over height ratio

Prediction of Bead width over height
Traning sample Validation sample Predicted curve Figure 10. Bead width over height prediction.
Interestingly, the prediction ratio, shown in Figure 10, indicates that the bead width's increasing ratio is more significant compared with bead height change over layers. The physical phenomenon observed, here, is due to the conduction of heat resulting from the laser-matter interaction. When the wire is irradiated by the laser beam, the energy absorbed by the wire leads to creating molten pools and its deposition on the substrate. As the beads are added layer by layer, the heat source "moves away" from the substrate. The consequence is a decrease in the volume of material allowing the transfer of heat. This leads to an increase in the flux density, which inevitably leads to local heating, a modification of the local properties of the metal, and the wetting of the deposit, among other things. The spread of layers becomes important with the increase of bead widths and the decrease of the bead height. It is observed that this phenomenon stabilizes after a number of deposited layers, as shown Figure 11.

First-Order Deposition Parameters
To investigate the effectiveness of predictive models using neural network (discussed in Section 3.2), several experiments with three deposition parameters were carried out. As mentioned before, these parameters are laser power (P), wire feed rate (F), and travel speed (V), as presented in Table 1. The choices for each of these five sets of parameters result from a test campaign for the development of the process with a wire of 1.2 mm diameter in Inconel ® 718 [45]. These five sets of parameters frame the stability domain of the process, i.e., they allow stable deposition with different bead morphologies, which is the parameter of interest for our study. Figure 12 shows different bead geometry samples for each parameter variation in the experimental design and for each sample, 10 deposition layers were measured (70% training data, 15% validation data and 15% as testing data).  In this study, the Levenberg-Marquardt algorithm (LM) was used to study the influence of the deposition parameters (P, F, V) on bead geometries (height and width). The main parameters for LM are the number of hidden layers and nodes. To obtain a good predictive performance a single hidden layer was chosen, with a size of 10 nodes (for more details see e.g., [7]). Additionally, other parameters were also determined by trial and error using 100 iterations for each sample. Figure 13 graphically illustrates a correlation between the measured height and the width predicted by the LM approach for the different deposition parameters. As the figure shows, the points are scattered around the fit line (represents measured bead geometry), which indicates the correlation between the model output values and the actual values. The value of R 2 reflects the correlation between the model prediction (y p ) and the corresponding target (y e ). The value of R 2 ranges from zero to one, and if R 2 is equal to one, it means a perfect correlation, and it can be computed as follows: where n is the number of the experimental data.
MAPE is the representation of MAE as a percentage.   To evaluate the accuracy of the prediction model obtained for each sample, we also computed RMSE, MAE, MAPE. The prediction accuracy of the regression model was judged excellent for RMSE, MAE and MAPE, being close to 0.
RMSE is computed to measure the standard deviation of residuals.
MAE is the average of the absolute difference between y e and y p values in the data-set.
MAPE is the representation of MAE as a percentage. Table 2 compares the bead geometry prediction performance of different samples for the LWAM process. The results from Table 2 shows that the best test trial reported is sample 5, as it shows good values of prediction accuracy measurements both for bead width and height. This good result was obtained due to the smooth thickness transition across layers and the good distribution of the layer thicknesses by using a high wire-feed rate with low power and travel speed compared with the other measured samples.
On the other hand, the worst performance reported is Sample 2, which shows the worst values of prediction accuracy measurements. It is noted that Sample 2 has the highest RMSE, MAE and MAPE; however, R 2 was good. This is due to the fact that R 2 gives an estimate of the strength of the relationship between y e and the prediction curve F(x), the value reported for this test does not provide a formal explanation of the fitting. Another explanation of the value obtained obtains from the distribution of the y e in sample 2 that shows a similar trend with the prediction curve, which does not fit the y e values. These prediction results indicated that having a high laser power and travel speed with a low wire-feed rate will lead to bad results on the final bead geometry.
Finally, these results are summarized in Table 3, which illustrates the influence of deposition parameters on both the prediction model and the bead geometry's accuracy. It is worth mentioning that the designation "Acceptable bead prediction" in Table 3 takes the trend of the set of parameters with the best correlation coefficients (RMSE, R 2 , MAE, MAPE). While the designation "Good layer deposition" refers to those experimental tests that showed process stability (no stuck wire, stable metal vapour plume, etc.) for a low laser power/travel speed coupling and a high wire-feed speed.

Conclusions
A bead geometry prediction model, in the LWAM process, was developed using a machine learning regression algorithm and neural network. The proposed algorithm predicted bead height, width and the ratio therebetween, and the results confirmed the profile's trend toward stable deposition. The prediction model is flexible and can be adjusted to follow the trends of various layer-deposition behaviours.
The experimental results revealed that the developed machine learning model is capable of predicting the bead geometry of LWAM-deposited layers with acceptable error rates (i.e., 2∼4%). Several data samples have been also collected by a laser scanner to study the influence of different deposition parameters on the bead geometry. The prediction results of low power (P) and travel speed (V) with a high wire feed rate (F) showed better layer deposition and model prediction, as compared with other collected samples.
In summary, the following conclusions are drawn from this study: • regression algorithms can be used for prediction tasks related to the LWAM process, including for supervision tasks; • there exists a power decay in correlating process parameters to bead thickness. • The experimental results showed an increase of width-to-height ratio as a function of layer progression due to the heat conducted from the laser-matter interaction. • Satisfying prediction models were obtained and could be used to predict the geometry of beads as part of LWAM control programming for horizontal and vertical robot movement. • In this case, acceptable bead prediction with good layer deposition was obtained when P, V, and F were 1.4 kW, 1.4 m/min, and 0.45 m/min, respectively.
In the future, the prediction model will be used to develop a robust feedback control system that is able to adjust process parameters automatically for improving the surface quality and geometric accuracy in the LWAM process.