In this work, a data acquisition and open-loop control system was developed. The next sections describe this system and the analysis methods to measure the relevant data and estimate the depth and width of the weld bead.
2.1. Data Acquisition and Open-Loop Control System for GMAW Welding Process
This experimental system was developed to collect values of selected variables and stimulate the GMAW process, allowing an open-loop control and data storage in a computer.
The variables measured by the system are welding voltage, welding current, infrared thermography and welding torch (or workpiece) position. The system sends several set points to the welding power source such as the set point of welding voltage and the set point of the wire feed speed and controls the welding speed. Despite this, the control is in open loop because the system does not use the measurements to change the welding parameters and follows a sequence previously defined by the system operator.
The data acquisition and open-loop control system has six main components: the welding power source, the welding table, the thermographic camera, the laser scanner, the data acquisition and control interface and the computer where data acquisition and stimulus sequence design software are executed [
40]. The components, the data flow and a schematic side view are shown in
Figure 1.
The
welding power source used is the Inversal 450 [
41]. The communication algorithm for remote control of the welding power source and data acquisition was developed, based on the RS232 protocol [
42]. A state machine, implemented in the control interface, defines the operation parameters and obtains the arc variables measurements (welding voltage and welding current) and status of the welding power source. The operation sequence for controlling the welding power source and the data acquisition software was also developed [
40]. A state machine is used to continuously repeat this sequence in the main loop of the interface.
The electro-mechanical system of the
flat welding table is composed of a linear axis with 5 mm of linear movement by revolution and a stepper motor of 1.8 grades by step. The system is used to move the piece, keeping the welding torch fixed. It was developed by students of the Automation and Control Group (GRACO) of UnB. It has a stepper motor control circuit [
43] with signals for modifying the stepper time (speed) and the direction. Other signals show the status of the system and protect against overload [
44]. The structure supports 15 kg of load and up to 15 mm/s of speed. The support structure for the thermographic camera was developed in this work and is shown in
Figure 1b.
The
data acquisition and control interface was developed to synchronize the movement of the piece with the parameters of the welding power source operation and to obtain the arc variables measurements in real-time [
40].
Figure 2 shows the simplicity and small size of the interface (since its greater complexity is in the firmware) as well as the low cost of its components.
A microcontroller PIC18F2550 from Microchip Technology [
45] (Arizona, USA, imported to Brazil by Ichip Tecnologia Ltda—ME) controls the operation. It first receives from the computer the start and end points of the weld, and the sequence of values to be sent to the welding source in each position of the piece. Then, after starting, the interface has an autonomous and independent operation and sends the measurements obtained in each position in real-time to the computer. The acquisition program (synchronized with the interface) obtains the thermographic data regardless of the clock or priorities of the operating system. The USB communication frames between the interface and the computer were created, based on ASCII characters. This allows the control of the welding power source, the welding table, the interface and the operation sequence of the process.
The
thermographic camera ThermoVision A40 [
46] from FLIR Systems (Oregón USA, imported to Brazil by FLIR Systems Brazil) is used to obtain the temperature values of the weld molten puddle. It employs a semiconductor sensor of focal plane array uncooled microbolometer technology; has a temperature range between −40 °C and 2000 °C; a spectral range between 7.5 and 13 μm; a sampling frequency of 120 Hz (120 frames per second) and a maximum resolution of 16-bit monochrome and 8-bit colour. The infrared data is obtained from a Firewire interface [
47] of the computer in a matrix format, with the temperature of each pixel of the image. The camera is fixed at 50 mm from the molten weld pool at a 45° angle from the horizontal plane of the welding table (see
Figure 1b,c).
The emissivity of the molten pool is not constant, so the temperature measurements using this method should be considered approximate. The accurate measuring of the absolute values of temperatures is not considered significant for the purposes of this work.
The
data acquisition and stimulus sequence design software developed using the libraries provided by the thermographic camera manufacturer [
48]. The software Thermo Data Welding (TDW) developed in this work collects and stores the data of the welding process and operation of the whole system in text files. The data are collected using the communication links with the data acquisition interface and the thermographic camera (see
Figure 1a).
Figure 3 shows the primary form of TDW.
The TDW is divided in several modules oriented to specific functions such as: thermographic camera configuration and verification, welding power source configuration (defines inductance, pre-gas and post-gas times, gap wire-arc time, wire diameter, contact tip to work distance, type and thickness of the material, composition and flow rate of the shielding gas), adjustment of the initial position of the piece in the welding table, creation the welding sequence (defines start and end positions, stimulus that will be sent to the welding power source and the sampling period). Three files are created in each experiment (see
Figure 3 bottom) that store the system configuration, the stimulus sequence and the measurements collected.
2.3. Laser Scanner and Macrographic Analysis to Obtain the Geometric Profile of the Weld Bead
The laser scanner and the macrographic analysis algorithm are a hardware and software development to obtain the profile of the weld bead after the completion of the welding process. The laser scanner gets the three-dimensional external geometry and the macrographic analysis algorithm obtains the weld bead depth profile.
The
laser scanner (or profilometer) operation is based on a triangulation technique and image processing algorithms developed specifically for this system. It draws a line on the weld bead with a laser and a low-resolution camera (webcam of 600 × 400 pixels, 0.24 MP) with USB communication captures the image. The camera provides only the laser line image because an optical filter, adjusted to the laser emission frequency of the laser, blocks other light sources.
Figure 5 show this process.
To operate the scanner, after the welding finishes the workpiece remains fixed on the welding table. Then, the control interface moves into the scanning position. A software developed using Matlab/GUI acquires the image from the camera and applies the image processing algorithm. The algorithm filters the image and completes the missing data, then rotates the image to find the baseline corresponding to the base metal surface and calculates the reinforcement and width of the bead.
Figure 5b shows the image obtained by the camera for the current position in red and the calculated profile in white.
The system saves pixel resolution, position information, reinforcement, width and profile of the weld bead for each position in a text file. These data are useful for three-dimensional reconstruction and statistical analysis of the weld bead geometry. The system can scan a weld bead up to approximately 20 mm wide and 10 mm high. The calibration is done with an automatic procedure using pieces with known dimensions. A resolution of 0.035 mm of width and 0.05 mm of height has been verified with an error of less than 5% of the full scale. This is an acceptable low-cost solution for research purposes despite its slow performance and low image resolution.
The
macrographic analysis algorithm was developed using Matlab scripts and needs an image of the welding workpiece. The workpiece is cut in a longitudinal direction, that is, in the direction of the torch movement (see
Figure 6a,b). If a misalignment of the cut tool with the weld bead is detected, it may be necessary to perform more than one cut-analysis operation. In these cases, the maximum value of each position is used.
The cut piece will be polished and etched using Nital solution (usually 2.5%) to clearly show the weld bead penetration, as can be seen in
Figure 6b. An image processing algorithm analyses the image of this side of the piece, then corrects misalignment and filters the image for the border detection procedure. The base-line (shown in red in
Figure 6c) is detected. The thickness and length of the piece are known and are used to calculate the scale coefficients. The weld bead penetration limit zone is detected (shown in yellow in
Figure 6c) and the difference between it and the base-line returns as the weld bead depth profile. The profile data is filtered, and the missing values are filled with a linear interpolation method.
In the traditional transversal cut, is not possible to obtain enough information to make a dynamic model because of the difficulty to obtain a continuous data set of penetration. This macrographic analysis algorithm allows us to obtain a continuous data set of penetration to a dynamic model.
2.4. Sensor Fusion to Estimate the Depth and Width of the Weld Bead
A cooperative, data-in/data-out and centralized sensor fusion [
18] algorithm was developed based on a Multilayer Perceptron (MLP) artificial neural network to estimate two characteristics of the weld bead geometry. The inputs of the neural network add previous measurements and estimates values to capture the process dynamics.
The MLP is composed of an input layer, one or more hidden layers and an output layer. This work uses only one hidden layer and a single output as shown in
Figure 7.
For a better training performance, all inputs have to be normalized in the interval [−1, 1] by Equation (1). In this equation xi = [x1, x2, … xn] are the n inputs and x’i = [x’1, x’2, … x’n] represent the normalized values.
In mathematical terms, Equation (2) represents the input of a neuron in the hidden layer where wi,j is the weight of neuron j associated with the input i, and bj is the bias of neuron j. Equation (3) expresses the output of the hidden layer where f is the tansig activation function presented in Equation (4).
Equation (5) represents the output layer of the neural network where y’j and vj are respectively the output and weight associated with neuron j of the hidden layer, and bout is the output bias.
Finally, the output y’out has to be restored to the original interval (inverse normalization), which is done through Equation (6).
The same MLP structure is used to estimate both the weld bead depth (
) and the weld bead width (
) by changing the input parameters of the welding current (
i) to the welding voltage (
u). A selection signal is used to change the parameters (weights), the inputs and the output of the network. A controller evaluates the two algorithms in sequence, updating the weights of the neural network for each case.
Figure 8 shows the block diagram of the estimator.
The artificial neural network has eight neurons in the input layer, twelve in the hidden layers and one in the output layer. The size of the hidden layer was selected by balancing the computational cost (neuron quantity) and the estimation error by testing different configurations.
The input variables are the thermographic peak (
Tp), the thermographic base (
Tb), the thermographic area (
Ta), the thermographic volume (
Tv), the thermographic width (
Tw), the measurements of welding current or voltage (
iu) in the actual (
nT) and previous sample (
nT-T) and the previous estimated value [
]. The symbol
T is the sample time and
is the sample number. The activate function is the hyperbolic tangent sigmoid transfer function [
49]. The network training should be done with experimental measurements of the parameters of input and output, and by using the backpropagation algorithm.
The Selection signal allows switching between the estimation of the weld bead depth or the weld bead width. To estimate the weld bead depth, this algorithm uses the features of the infrared image and measurements of the welding current as indicators of the energy delivered (welding energy) and the last value of the weld bead depth. To estimate the weld bead width, the infrared features, the welding voltage and the last value of the weld bead width are used.
For the training and validation of the neural network, the input values of the thermographic and arc variables are obtained from the data acquisition system (see
Section 2.1) and infrared feature extraction algorithm (see
Section 2.2). The real values of the weld bead depth are obtained from the macrographic analysis. The real values of the weld bead width are obtained from the laser scanner (see
Section 2.3).
These dynamic estimator’s models are valid only for the current process conditions and must be updated if these change, such as electrode diameter and material, base material, gas type and flow rate, welding current or welding voltage range, among others.
Several programs were developed for data processing and data plotting. All the curves, the statistical analysis, the image processing and the neural network training and testing were made using Matlab scripts.
We must emphasize the importance of simplifying the estimating algorithm due to real-time requirements. The use of the proposed parameters and not the complete infrared image can help reduce the resources used and increase the speed of the estimation process. The same structure for the estimation of two parameters in sequential mode minimizes the hardware requirements but increases the processing time. If less processing time is required and the hardware resource is not restricted, two equal neural networks can be used. The implementation and synthesis of the neural network can be done with the methodology and tools proposed in [
50,
51]. Ayala et al., show a floating-point implementation in hardware for a neural network architecture that guarantees low latency.
2.5. FPGA Implementation of Weld Bead Depth and Width Estimators
The MLP implementation in hardware is based on the model presented in
Section 2.4. The overall architecture consists of multiple neurons in parallel, which are synchronized and controlled by a Finite State Machine (FSM) as shown in
Figure 9. All operations are carried out using customized variable width floating-point arithmetic and trigonometric libraries based on the IEEE 754 standard. These units are described in [
52,
53] and provide much better precision and a larger dynamic range suited for small and large real numbers compared to fixed-point or simple integer arithmetic.
The top-level neural network architecture begins its operation with the activation of the start signal, at this point, the FSM sends a start_neuron command to each neuron which receives the network inputs and its respective weights (a) and bias (b) The neuron then outputs its value (y) and activates a ready_neuron signal. The FSM then activates a start_mult signal which begins multiplying each neuron output by its respective weight (v). After receiving the ready_mult signal, the FSM executes its final states which sums all neuron outputs multiplied by their weights with the output bias (bout). Finally, the ready_nn signal is set and the network output (yout) becomes available.
The neuron module has a similar architecture and is depicted in
Figure 10. Again, an FSM receives a
start signal and begins the first stage of multiplying each input with its respective weight and accumulating the results. To simplify the architecture, the bias value (
bj) is inserted in the same pipeline and multiplied by 1. After that, the resulting value enters the activation function as described in Equation (4). The exponential operation uses a CORDIC module described in [
53] while the rest of the arithmetic operators use the same floating-point units described in [
52].
To optimize FPGA resource usage, all sequential operations inside the neuron use shared floating-point units which reduces them to a total of two adders, one multiplier and one divider. Likewise, the multipliers used in the top-level architecture are also shared with the multipliers inside each neuron avoiding resource duplication.
The final step to the MLP hardware implementation was to address the input and output normalization steps described in Equations (1) and (6). To avoid adding more logic to the MLP architecture in hardware, a transform operation was applied to each weight and bias. To calculate the transforms, we began by substituting Equation (1) into Equation (2) and obtaining the Equation (7).
From Equation (10) it is clear that the neuron weights and bias can be recalculated to include the normalization as expressed in Equations (11) and (12).
An equivalent transform was calculated for the output inverse normalization where Equation (5) is substituted into Equation (6).
Once again, from Equation (14) it is possible to isolate the expressions that transform the weights and bias used to compute the inverse normalization.
With the use of these transforms, there is no need to add any logic to the MLP implemented in hardware.