Axiomatic Design of a Test Artifact for PBF-LM Machine Capability Monitoring †

Bed


Introduction
Additive Manufacturing (AM) is an innovative production process that is highly suitable for many applications.AM is mainly based on the concept of creating matter just where it is needed, as opposed to subtractive technologies [1][2][3][4][5].That is possible by subdividing the CAD model into two-dimensional slices and building the parts by a layerby-layer building strategy.Starting from this basic concept, many different technologies involving many types of material (polymers, metals, biomaterials, ceramics, etc.) have been developed in the last few decades [6].
Metal AM technologies, among all the others, raised much attention due to their potential impact on the industrial environment.In fact, AM technology can produce parts with more enhanced properties [7] than traditionally manufactured parts, with all the advantages coming from adopting an additive production technology rather than a subtractive one.Based on this consideration, metal AM presented many possibilities, ranging from enhancing part performance to insourcing strategic parts.Among all the metal AM technologies [5], within the industrial environment, one in particular has been heavily adopted: Powder Bed Fusion Laser Melting (PBF-LM or L-PBF) [7,8].PBF-LM is a powder bed technology, which means that parts are built within a metal powder bed.The powder bed is created by spreading very thin powder layers (typically from 20 µm to 100 µm).After the layer is spread, a highly focused laser melts the metal powder, realizing the part slice.PBF-LM technology has many fields of application (biomedical, aerospace, energy, etc.), thanks to the very wide variety of materials processable: lightweight alloys (aluminum and titanium alloys), high-temperature materials (nickel-based and cobalt-based alloys), stainless steels, precious metals, etc.However, introducing PBF-LM technology as a standard manufacturing technique brings new challenges and issues.One of the most important aspects is quality assurance [9], along with design for additive manufacturing [10,11], process efficiency, and control [12,13].
From a quality assurance point of view, PBF-LM fabricated parts have to be acceptable in terms of material printability and geometry compliance with tolerances.In particular, with regard to the first aspect mentioned, it is necessary to ensure a defect-free part, through the identification of material printability map boundaries.For this purpose, usually, the definition of material printability is performed through the analysis of Melt Pool shape printing several single and multi-tracks on a powder bed [14][15][16][17][18].In order to have a fully dense and defect-free material, the Melt Pool formation has to be fully governed by conduction, avoiding the presence of unproper melting regimes such as Keyhole, Lack of Fusion, or Balling [19][20][21].This analysis could be carried out through Melt Pool monitoring in situ or ex situ assessment analyzing printed specimens [22][23][24][25].Conversely, as soon as additively manufactured parts start to be produced in an industrial line, thanks to the technology applications spreading [26][27][28][29][30][31][32][33][34][35], the importance of PBF-LM machines' capability determination and their monitoring through time is constantly increasing [36,37].
To monitor and verify the system's performance, many authors and PBF-LM providers propose using samples or functional mock-ups.Samples are typically identified as test artifacts [38][39][40][41][42][43][44][45][46][47][48][49][50][51][52][53].In most cases, the design of test artifacts is characterized by a high level of complexity and a large number of geometric features, which allow the evaluation of many different capabilities.Focusing on the use of test artifacts to check machine performances in an additive manufacturing production line, results robustness and a fast test time are more important than the possibility of evaluating, at the same time, many different capabilities.
The idea of this paper is to use Axiomatic Design (AD) methodology to address the issue of machine capability assessment in an industrial production line.AD is a methodology developed by Suh in the 1990s [54] and it helps with the development of products and systems that respond to functional requirements.
For AD, the best designs had two things in common (Suh's design axioms).Maintain the independence of the Functional Requirements or FRs, (Axiom one) and minimize the information content (Axiom two).When these axioms are applied the resulting design solutions are optimal for a given set of FRs and candidate Design Parameters (DPs) [70,72].The independence of FRs is assured when the design matrix that connects FRs and DPs is diagonal (called "uncoupled design").
In this paper, the AD approach is used to analyze a well-known test artifact proposed by the National Institute of Standards and Technology (NIST) [73] in order to determine its criticalities if applied to the needs of industrial additive manufacturing production lines (Section 2).Based on the previous analysis, a new concept of a test artifact is proposed (Section 3), physically produced and tested (Section 4), evaluating the benefits of the new design.

Analysis of the NIST Test Artifact
The section describes the main characteristics of the test artifact proposed by NIST (Section 2.1) and analyzes, from an AD point of view, its ability to monitor and verify the system's performance in the context of an industrial production line (Section 2.2).

Description of Characteristics and Main Features
Among the various test artifacts proposed in the literature usable for the monitoring and qualification of AM processes, one of the most relevant is that proposed by NIST [73].The NIST artifact, represented in Figure 1a, assesses many aspects related to the AM process, so it is a sample that can be fruitfully used to compare different AM systems or additive manufacturing technologies.The NIST artifact has been used as a reference in this paper's development.In this paper, the AD approach is used to analyze a well-known test artifact proposed by the National Institute of Standards and Technology (NIST) [73] in order to determine its criticalities if applied to the needs of industrial additive manufacturing production lines (Section 2).Based on the previous analysis, a new concept of a test artifact is proposed (Section 3), physically produced and tested (Section 4), evaluating the benefits of the new design.

Analysis of the NIST Test Artifact
The section describes the main characteristics of the test artifact proposed by NIST (Section 2.1) and analyzes, from an AD point of view, its ability to monitor and verify the system's performance in the context of an industrial production line (Section 2.2).

Description of Characteristics and Main Features
Among the various test artifacts proposed in the literature usable for the monitoring and qualification of AM processes, one of the most relevant is that proposed by NIST [73].The NIST artifact, represented in Figure 1a, assesses many aspects related to the AM process, so it is a sample that can be fruitfully used to compare different AM systems or additive manufacturing technologies.The NIST artifact has been used as a reference in this paper's development.The considered specimen is characterized by many features, such as central cylinders, staircases, ramps, pins, holes, line features, lateral features, top surfaces, massive material, etc.The features are grouped in a single geometry characterized by lateral dimensions of 100 × 100 mm 2 (see [73] for further detail).Typically, the specimen shall be printed in the center of the machine building area and shall be geometrically inspected (Figure 1b).
The drawback of this test artifact is represented by its purpose: it has been designed, in fact, mainly to compare different AM technologies rather than to compare systems that nominally should have the same characteristics and performances.Furthermore, realizing this test artifact with PBF-LM technology features is prone to being characterized by a non-diagonal design matrix.The considered specimen is characterized by many features, such as central cylinders, staircases, ramps, pins, holes, line features, lateral features, top surfaces, massive material, etc.The features are grouped in a single geometry characterized by lateral dimensions of 100 × 100 mm 2 (see [73] for further detail).Typically, the specimen shall be printed in the center of the machine building area and shall be geometrically inspected (Figure 1b).
The drawback of this test artifact is represented by its purpose: it has been designed, in fact, mainly to compare different AM technologies rather than to compare systems that nominally should have the same characteristics and performances.Furthermore, realizing this test artifact with PBF-LM technology features is prone to being characterized by a non-diagonal design matrix.

AD Analysis of NIST Test Artifact
The NIST test artifact, used in this paper as a reference point, has been evaluated using the AD approach.The aim is to assess the quality of the design choices in relation to the typical objectives of an industrial production line based on AM and, in particular, the PBF-LM process.
The analysis begins with our interpretation of the Customer Needs (CNs) that have been used by NIST when they develop their well-known test artifact.Therefore, the inputs behind the reference test artifact design are considered the following: − CN1.Assess overall machine capabilities through the realization of a test artifact to compare multiple printed materials using multiple AM technologies and test the limitations of the system (regardless of the specific additive manufacturing technology); − CN2.The test artifact shall have as many features as possible to maximize the infor- mation content per system inspected; − CN3.Test artifact features shall be feasible for all the possible additive manufac- turing commercial systems available (without any constraints related to additive manufacturing technology).
The Functional Requirements (FRs) were identified based on the CNs and are listed in Table 1.Using these FRs, we identified the corresponding Design Parameters (DPs) for the NIST test artifact (Table 2).According to the AD approach, the design matrix has been compiled to identify the relationship between FRs and DPs (Figure 2).From the analysis, some criticalities related to the use of this design as an artifact for the PBF-LM system performance check were identified.In particular, the design matrix shows that this artifact is suboptimal from an AD standpoint.In detail, analyzing the accordance of the design with the AD first axiom (i.e., "Maintain the independence of FRs" [54]), it is possible to observe that: • The matrix is neither diagonal nor square, so the design is coupled; • The use of certain features (such as bulk material) makes features not independent.
The superposition of effects means that information retrievable by their analysis is potentially affected by this design choice since all the features are grouped into a single part.This is confirmation that the use of a single part as a test artifact potentially introduces mutual influences among features.In fact, the analysis of every single feature is meaningless, as they are mutually influenced.Considering the application addressed in this paper, this choice could be a very critical element.
The superposition of effects means that information retrievable by their analysis is potentially affected by this design choice since all the features are grouped into a single part.This is confirmation that the use of a single part as a test artifact potentially introduces mutual influences among features.In fact, the analysis of every single feature is meaningless, as they are mutually influenced.Considering the application addressed in this paper, this choice could be a very critical element.
In addition, this effect becomes very relevant when considering additive manufacturing technologies that produce metal parts, such as nickel-based or cobalt-based, widely used in PBF-LM applications.In fact, as is well known, PBF-LM technology is characterized by very high residual stresses due to the printing process itself.Therefore, to determine specific machine capabilities, it is desirable to remove any possible mutual influence of features to be inspected.That allows for more representative data that can eventually be correlated with information about the process.
To summarize, the use of this artifact design for the performance check of the PBF-LM system in an AM production line is non-optimal.The proposition of a different test artifact design carried out in this paper starts with these preliminary considerations.

CNs Review
The AD analysis carried out on the NIST test artifact shows that the CNs used to drive the design phase do not appropriately match the requirements coming from the context considered within this work (i.e., the industrial AM production line).Therefore, CNs have been reviewed and redefined according to the scenario considered within the work.
The new CNs are:  In addition, this effect becomes very relevant when considering additive manufacturing technologies that produce metal parts, such as nickel-based or cobalt-based, widely used in PBF-LM applications.In fact, as is well known, PBF-LM technology is characterized by very high residual stresses due to the printing process itself.Therefore, to determine specific machine capabilities, it is desirable to remove any possible mutual influence of features to be inspected.That allows for more representative data that can eventually be correlated with information about the process.
To summarize, the use of this artifact design for the performance check of the PBF-LM system in an AM production line is non-optimal.The proposition of a different test artifact design carried out in this paper starts with these preliminary considerations.

CNs Review
The AD analysis carried out on the NIST test artifact shows that the CNs used to drive the design phase do not appropriately match the requirements coming from the context considered within this work (i.e., the industrial AM production line).Therefore, CNs have been reviewed and redefined according to the scenario considered within the work.
The new CNs are: Focusing on the scenario considered, it is evident that the new CNs and those related to the reference design differ significantly (except partially for CN1).The overall goal of the test artifact is still to assess machine performance.However, since the new scenario specifically considers PBF-LM technology, more needs, such as the machine time-occupancy and the check cost, should be considered.

FRs Definition and Redesign Using AD
The list of CNs defined in Section 3.1 is used to identify a new set of FRs.Starting from their definition, we have considered the following assumptions [54]:

•
CN6 shall not be considered an FR.It shall be considered a non-Functional Requirement (nFR) instead; • CN2 and CN3 lead to FRs that will not satisfy the Independence Axiom because of their very nature.For this reason, in the following AD analysis, we considered speed as an actual FR and cost as a Selection Criteria (SC); • CN5 represents more of a constraint than an actual FR.Therefore, it will be considered as such.
Additionally, considering the progressive development of PBF-LM technology and the quest for higher productivity, the number of lasers will be considered an additional constraint.This work will consider single laser machines; however, the study can easily be extended to multi-laser machines, which are rapidly growing in the industrial scenario.With this consideration, the highest-level CN is still represented by CN1, the machine capability assessment for a periodic performance check.Accordingly, the first design choice (that is DP1) consists of the adoption of an inspection strategy, which eventually means the realization of a test artifact.This DP comes from the choice of not changing how machine performances are checked.Then, as long as the test artifact is representative of the machine's performance, its realization is the design choice adopted.
Then, based on the classical AD Zig-Zagging process, once this first DP has been identified, lower-level FRs are defined as follows:

•
FR1. Produce the test artifact in less than 8 h.

•
FR2. Analyze the test artifact in less than 12 h.• FR3.Deliver results with measure accuracy below 10 microns in the whole working space.
These FRs come from the CNs analysis and their interpretation, as shown above.Then lower level FRs and DPs are defined using the Zig-Zagging process iteratively.
FRs and DPs lists are highlighted in Tables 3 and 4. The design matrix in Figure 3 shows that the design of the proposed test artifact is uncoupled (the first AD axiom is verified).

Resulting Artifact Design and Discussion
The geometrical model of the new test artifact obtained using the AD approach is represented in Figure 4.The test artifact consists simply of an array of pins printed above the building platform (i.e., the gray parallelepiped with four counterbores to allow for bolting it into the printing machine).The main novelty represented by the design is the use of distinct components and the absence of printed material below the main features.The design matrix in Figure 3 shows that the design of the proposed test artifact is uncoupled (the first AD axiom is verified).represented in Figure 4.The test artifact consists simply of an array of pins printed above the building platform (i.e., the gray parallelepiped with four counterbores to allow for bolting it into the printing machine).The main novelty represented by the design is the use of distinct components and the absence of printed material below the main features.This section will go through the reason behind it in better detail.The use of the AD strategy for this problem has resulted in a comprehensive redefinition of the overall approach and optimization of an existing procedure.In particular, the AD approach led us to analyze the specific needs related to the monitoring and verification of the system's performance in the context of an industrial production line, build on them in the design process, and achieve an improved and optimized design.
Optimization of the test artifact design has brought significant advantages: the artifact building time has been reduced by 90% compared to the original one (consider approximately 10 5 mm 3 of printed materials for the NIST artifact [73] and about 10 4 mm 3 for the redesigned one), and the inspection time has been halved (from about 16 h to 8 h).
The compromise reached with the new test artifact must consider that the lower number of features measurable, with regard to the reference NIST artifact, leads to a reduction in the amount of information obtainable.This loss, however, does not affect the effectiveness of the measures achievable using the test artifact for the features still present.
It is important to consider that any modification of the laser due to an internal fluctuation of the PBF-LM system is detected automatically by the machine, as well as other major failures (i.e., laser source malfunctioning, excessive oxygen concentration, etc.).Conversely, the realization of the pins is fundamental because it makes it possible to verify the dimensional performance of the systems that are otherwise unverifiable.
The owner of the PBF-LM production line needs to have the same dimensional performance among all the available systems.The basic assumption is that PBF-LM is characterized by poor accuracy performance, mainly due to its physical nature.In fact, the powder melting process has the inherent disadvantage of having a mantle of powder not melted, limiting accuracy to the powder grain size itself (typically in the range of 5-65 µm).
In addition, two other factors influence the nominal dimension of a PBF-LM printed component:

•
Material shrinkages during the solidification, which creates residual stress.• Beam offset compensation.
The first effect can be simulated and compensated or directly compensated through a trial-and-error process.Conversely, the modification of beam offset compensation is allowed in commercial systems to change the laser trajectory, compensating the beam spot radius (Figure 5) [73].Beam offset compensation is a global correction applied to every printing position on the building platform.This correction has to be performed and This section will go through the reason behind it in better detail.The use of the AD strategy for this problem has resulted in a comprehensive redefinition of the overall approach and optimization of an existing procedure.In particular, the AD approach led us to analyze the specific needs related to the monitoring and verification of the system's performance in the context of an industrial production line, build on them in the design process, and achieve an improved and optimized design.
Optimization of the test artifact design has brought significant advantages: the artifact building time has been reduced by 90% compared to the original one (consider approximately 10 5 mm 3 of printed materials for the NIST artifact [73] and about 10 4 mm 3 for the redesigned one), and the inspection time has been halved (from about 16 h to 8 h).
The compromise reached with the new test artifact must consider that the lower number of features measurable, with regard to the reference NIST artifact, leads to a reduction in the amount of information obtainable.This loss, however, does not affect the effectiveness of the measures achievable using the test artifact for the features still present.
It is important to consider that any modification of the laser due to an internal fluctuation of the PBF-LM system is detected automatically by the machine, as well as other major failures (i.e., laser source malfunctioning, excessive oxygen concentration, etc.).Conversely, the realization of the pins is fundamental because it makes it possible to verify the dimensional performance of the systems that are otherwise unverifiable.
The owner of the PBF-LM production line needs to have the same dimensional performance among all the available systems.The basic assumption is that PBF-LM is characterized by poor accuracy performance, mainly due to its physical nature.In fact, the powder melting process has the inherent disadvantage of having a mantle of powder not melted, limiting accuracy to the powder grain size itself (typically in the range of 5-65 µm).
In addition, two other factors influence the nominal dimension of a PBF-LM printed component:

•
Material shrinkages during the solidification, which creates residual stress.
The first effect can be simulated and compensated or directly compensated through a trial-and-error process.Conversely, the modification of beam offset compensation is allowed in commercial systems to change the laser trajectory, compensating the beam spot radius (Figure 5) [73].Beam offset compensation is a global correction applied to every printing position on the building platform.This correction has to be performed and repeated each time the machine type and process parameters change, considering the industrial production line.
Once the CAD model is compensated, it is crucial to have machines with the same dimensional accuracy.
The beam offset compensation varies between systems, so correcting it on each machine is critical to achieving the same dimensional performance.This is why printing pin features in the test artifact should be preferred to the use of other geometrical features.Once the CAD model is compensated, it is crucial to have machines with the same dimensional accuracy.
The beam offset compensation varies between systems, so correcting it on each machine is critical to achieving the same dimensional performance.This is why printing pin features in the test artifact should be preferred to the use of other geometrical features.
The other important thing to monitor and check is whether the system can point the laser where it is required (i.e., the accuracy of the machine in laser pointing).The laser is highly focused in the PBF-LM system, so direct measurement results are difficult and costly.An indirect method of obtaining this information is to measure the position of the axles of the pins, thus indirectly assessing the accuracy of the machine.
Hence, analyzing the pins alone makes it possible to check whether a PBF-LM system works well.The dimensions of these pins, considering these factors, shall be as follows: • Small enough to make shrinkage effects negligible in their diameter.• Big enough to avoid being damaged by the re-coater during the printing processing.
In the proposed design the diameter dimension (4 mm) has been aligned with the pin present in the NIST artifact considering that it is verified in [73] how it allows the above two problems to be avoided.
It is important to note that the proposed artifact minimizes the superposition of effects, leaving key system features independent.This means that there is no additional effect as a result of other variables in the measure of each feature.

Experimental Validation
After the design analysis, which highlighted the necessity to remove the superposition of effects to get a test artifact able to catch differences between PBF-LM commercial systems, an experimental campaign was carried out to validate the obtained results.
The campaign has been structured by collecting data retrieved by analyzing the reference test artifact produced by the PBF-LM system EOS Eosint M-280 (EOS GmbH, Krailling, Germany), provided with one ytterbium fiber laser characterized by a beam wavelength of 1070 nm, a minimum spot size of 80 µm, and a maximum laser power of 400 W. Furthermore, during all the tests, the building platform temperature was kept constant at 80 °C and an argon gas flow was continuously pumped inside the building The other important thing to monitor and check is whether the system can point the laser where it is required (i.e., the accuracy of the machine in laser pointing).The laser is highly focused in the PBF-LM system, so direct measurement results are difficult and costly.An indirect method of obtaining this information is to measure the position of the axles of the pins, thus indirectly assessing the accuracy of the machine.
Hence, analyzing the pins alone makes it possible to check whether a PBF-LM system works well.The dimensions of these pins, considering these factors, shall be as follows:

•
Small enough to make shrinkage effects negligible in their diameter.

•
Big enough to avoid being damaged by the re-coater during the printing processing.
In the proposed design the diameter dimension (4 mm) has been aligned with the pin present in the NIST artifact considering that it is verified in [73] how it allows the above two problems to be avoided.
It is important to note that the proposed artifact minimizes the superposition of effects, leaving key system features independent.This means that there is no additional effect as a result of other variables in the measure of each feature.

Experimental Validation
After the design analysis, which highlighted the necessity to remove the superposition of effects to get a test artifact able to catch differences between PBF-LM commercial systems, an experimental campaign was carried out to validate the obtained results.
The campaign has been structured by collecting data retrieved by analyzing the reference test artifact produced by the PBF-LM system EOS Eosint M-280 (EOS GmbH, Krailling, Germany), provided with one ytterbium fiber laser characterized by a beam wavelength of 1070 nm, a minimum spot size of 80 µm, and a maximum laser power of 400 W. Furthermore, during all the tests, the building platform temperature was kept constant at 80 • C and an argon gas flow was continuously pumped inside the building chamber to maintain oxygen content under 100 ppm.All the experimental campaigns are carried out on test artifacts made of Inconel 718 superalloy (for detailed chemical and mechanical properties, refer to [13]).In the following, the test artifacts considered for this work were built three consecutive times with the same machine, same process parameters (listed in Table 5), same metal alloy, and same time span, and accounting for a layer thickness of 40 µm.As previously stated, the machine considered within this work is a commercial PBF-LM system fitting a 250 × 250 mm 2 building platform and a single laser.Test artifacts have been geometrically inspected, measuring the artifact still attached to the building platform, with the Coordinate Measurement Machine Carl Zeiss Contura G2 (Carl Zeiss AG, Oberkochen, Germany).The measuring system has a declared accuracy of 2 µm.The overall estimated repeatability error, which results from the measuring machine accuracy and its interaction with pin surface roughness, has been experimentally evaluated under 4 µm.
The data obtained by the geometrical inspections are pin diameters and pin positions.The pin inspection has been structured, defining five planes parallel to the building platform.On each plane, 12 points are acquired along the circumference obtained by the pin and plane intersection.The number of planes and points has been selected after a preliminary analysis to ensure a reliable measure of the circularity and cylindricity of the pins.Figure 6 represents how the plane is chosen and how the pin is measured on each of these.In particular, planes are chosen to be parallel to the building platform (i.e., with the normal parallel to the machine Z axis).The number of planes is a trade-off value between a fast measure (few planes) and a high representativity (many planes).On each plane, the intersection with the pin nominally creates a circumference (the crosshatched circumference in Figure 6a); measurement points are taken touching the pin with a radial movement towards the pin center.The 12 points collected are angularly outdistanced by a constant angle of 30 • (Figure 6b).ness of 40 µm.As previously stated, the machine considered within this work is a commercial P LM system fitting a 250 × 250 mm 2 building platform and a single laser.Test artifacts h been geometrically inspected, measuring the artifact still attached to the building form, with the Coordinate Measurement Machine Carl Zeiss Contura G2 (Carl Zeiss Oberkochen, Germany).The measuring system has a declared accuracy of 2 µm.The o all estimated repeatability error, which results from the measuring machine accuracy its interaction with pin surface roughness, has been experimentally evaluated under 4 The data obtained by the geometrical inspections are pin diameters and pin positi The pin inspection has been structured, defining five planes parallel to the building form.On each plane, 12 points are acquired along the circumference obtained by the and plane intersection.The number of planes and points has been selected after a pre inary analysis to ensure a reliable measure of the circularity and cylindricity of the p Figure 6 represents how the plane is chosen and how the pin is measured on each of th In particular, planes are chosen to be parallel to the building platform (i.e., with the mal parallel to the machine Z axis).The number of planes is a trade-off value betwe fast measure (few planes) and a high representativity (many planes).On each plane intersection with the pin nominally creates a circumference (the crosshatched circum ence in Figure 6a); measurement points are taken touching the pin with a radial movem towards the pin center.The 12 points collected are angularly outdistanced by a cons angle of 30° (Figure 6b).As stated in Section 3.2, the choice to define these parameters only comes from main aim of the proposed new test artifact, i.e., to identify with as much reliabilit possible, the few parameters that are vital to be managed within the context consid in this work.In the following, 22 pins will be considered.This value can, howeve As stated in Section 3.2, the choice to define these parameters only comes from the main aim of the proposed new test artifact, i.e., to identify with as much reliability as possible, the few parameters that are vital to be managed within the context considered in this work.In the following, 22 pins will be considered.This value can, however, be increased to enhance the spatial resolution of the data.Tables 6 and 7 and Figure 7 contain the difference between the average diameter measured on each pin and the nominal one (4 mm).increased to enhance the spatial resolution of the data.Tables 6 and 7 and Figure 7 contain the difference between the average diameter measured on each pin and the nominal one (4 mm).increased to enhance the spatial resolution of the data.Tables 6 and 7 and Figure 7 contain the difference between the average diameter measured on each pin and the nominal one (4 mm).Analyzing the diameters in Figure 7, it is possible to outline several considerations.The first is that the average pin diameter is more constant in the new design with regard to the reference design.Partially, this could be a direct consequence of the higher number of pins considered in the new sample (the higher the sample number, the lower the sensitivity to potential outliers).However, this consideration contradicts the lower (and even, in this case, more constant) standard deviation value.In all its forms, the standard deviation represents the sample scattering.Therefore, the higher the standard deviation value, the higher the scattering within the sample.Thus, as a first conclusion, the new test artifact design appears to be more able to catch the actual average pin diameters than the reference NIST artifact.The actual average diameter definition's importance has already been explained in Section 3.
The pin position, conversely, has been defined as a point resulting from the intersection between the pin axis and the building platform's top surface.This choice comes from the plane definition's repeatability, thanks to the building platform construction accuracy and the perpendicularity between pin and plane.With this procedure, the pin position is measured.It is important to highlight that the authors considered, within this work, the coordinate system origin in the center of the building platform.Tables 8 and 9 list pin position deviation values, assessed as the difference between the actual position and the nominal one.These data have been obtained using the same test artifact geometrical inspections used to assess the average pin diameter.Analyzing the diameters in Figure 7, it is possible to outline several considerations.The first is that the average pin diameter is more constant in the new design with regard to the reference design.Partially, this could be a direct consequence of the higher number of pins considered in the new sample (the higher the sample number, the lower the sensitivity to potential outliers).However, this consideration contradicts the lower (and even, in this case, more constant) standard deviation value.In all its forms, the standard deviation represents the sample scattering.Therefore, the higher the standard deviation value, the higher the scattering within the sample.Thus, as a first conclusion, the new test artifact design appears to be more able to catch the actual average pin diameters than the reference NIST artifact.The actual average diameter definition's importance has already been explained in Section 3.
The pin position, conversely, has been defined as a point resulting from the intersection between the pin axis and the building platform's top surface.This choice comes from the plane definition's repeatability, thanks to the building platform construction accuracy and the perpendicularity between pin and plane.With this procedure, the pin position is measured.It is important to highlight that the authors considered, within this work, the coordinate system origin in the center of the building platform.Tables 8 and 9 list pin position deviation values, assessed as the difference between the actual position and the nominal one.These data have been obtained using the same test artifact geometrical inspections used to assess the average pin diameter.
Furthermore, it is possible to observe the data listed in the tables graphically in Figures 8 and 9 (referred to as the X axis) and Figures 10 and 11 (referred to as the Y axis).These plots have the scope to illustrate whether and how the pin positions printed are modified with respect to nominal ones.The scaling factor is a material-dependent parameter used to compensate for the thermal shrinkages that occur after the powder melts when the material solidifies.The scaling factor is evaluated as the regression angular coefficient of the average deviation from the nominal diameter measured on different pins as a function of the nominal pin position.Tables 10 and 11 list the regression coefficients and the coefficient of determination (adjusted R-squared) evaluated on data obtained by test artifact geometrical inspection.By plot comparison, it is evident how much the test artifact design influences the result retrievable by its analysis.Data from the reference design shows that printing the pin features above a base of massive materials substantially reduces the difference between the real and nominal positions.As a drawback, however, it is possible to note that data is heavily scattered from the same machine and using the same alloy.Regressions are also highly scattered.Linear regressions have been used to fit the data since the shrinkage compensation is supposed to be linear [7].By analyzing the data in Tables 10 and 11, it is possible to estimate the variability in the regression coefficient.In the reference design, the coefficients are so scattered that the variability range of the coefficient (0.0017 for X and 0.0012 for Y) is bigger than every single coefficient.In the proposed design, on the contrary, values are more similar among the tests carried out (variability range of the coefficient 0.0001 for X and 0.0002 for Y).Therefore, it is possible to calculate a percentage of variation of the coefficient, which is 7.9% for the X-axis and 9.4% for what concerns the Y-axis in the proposed design.The regression value in the building platform center (coordinate system origin) is not essential, as it depends on the relative alignment between the plate and the scanning system, which is not guaranteed in the system used in this work.This is why relative alignment between the scanning system and building plate is not important in building applications since once the job is started, it is self-aligned.This is important in repair applications, where the alignment between parts on the platform and the laser is crucial for the repair to succeed.Analyzing the plot retrieved by the analysis of the new artifact, it is evident how the data are less affected by noise, are more regular, and are more repeatable (Figure 7).In addition, the new test artifact design increased the measurement system's accuracy, where the reference system is represented by the test artifact itself and its measure.
In the NIST reference design, some noise is introduced by the bulk base of the artifact printed below the features.This is due to the fact that in PBF-LM applications, the exposure of such a large area without any interruption potentially leads to the production of material characterized by strong internal stresses, which unavoidably introduce deformations in the part.These deformations can be so intense, depending on the process parameters and materials used, that they can cause test artifact distortions and consequently can affect the measurement accuracy.All of these are emphasized by the test artifact geometrical inspection strategy, which imposes measuring the part while still attached to the building platform and without any heat treatment (i.e., stress relief).
Conversely, the proposed test artifact is not affected by this issue, making the measurement of appropriate axis scaling possible without any significant shrinkage deformation along the X and Y axes due to the artifact material itself.In fact, by analyzing the plots obtained by the new test artifact design, it is possible to identify the difference between the X and Y axes through the identification of the specific regression coefficient (Tables 10 and 11).
Furthermore, an effective advantage derived from the application of the proposed test artifact for machine capability evaluation is related to the high replicability of the industrial part's printing conditions through the distribution of features to be measured, such as pins, all over the building platform.

Conclusions
Within this work, we defined the problem of how to check PBF-LM machine behaviors in a hypothetical additive production line.After the choice to adopt an ex situ strategy, we started by considering a well-known NIST test artifact from the AD perspective.The AD evaluation confirmed that the NIST artifact is not optimal in the scenario of machine capability assessment in an industrial production line.The redefinition of CNs for the considered scenario has been used to develop a different approach to test artifact design.The analysis shows that the AD approach to redesigning the test artifact produced an excellent result compared to the reference design.
The main characteristics of the proposed test artifact are: • An uncoupled design matrix was obtained through better identification of the parameters used to tune the process.• The artifact building time has been reduced by 90% compared to the NIST one (FR1), and the inspection time has changed from 16 h to 8 h (FR2).• Allows the reliable identification of the beam offset and the scaling factor (FR3).
• Assess the performance all over the building platform (FR3.4).
Further experimental campaigns could be carried out by investigating the robustness of the proposed approach by testing different types of superalloys and multilaser machines, which are structurally more complex in their setting operations.

Figure 1 .
Figure 1.NIST test artifact [73,74]: (a) geometry and main features, (b) position of the artifact on the building platform.

Figure 1 .
Figure 1.NIST test artifact [73,74]: (a) geometry and main features, (b) position of the artifact on the building platform.

Figure 2 .
Figure 2. FRs-DPs matrix of test artifact proposed by NIST.

Figure 2 .
Figure 2. FRs-DPs matrix of test artifact proposed by NIST.

Figure 4 .
Figure 4. CAD model of the redesigned test artifact.

Figure 4 .
Figure 4. CAD model of the redesigned test artifact.
12,  x FOR PEER REVIEW 9 of 20 repeated each time the machine type and process parameters change, considering the industrial production line.

Figure 6 .
Figure 6.Measure on each pin: (a) a-e planes on which circumferences are identified, (b) how circumference of each pin is measured on each plane.

Figure 6 .
Figure 6.Measure on each pin: (a) a-e planes on which circumferences are identified, (b) how the circumference of each pin is measured on each plane.

Figure 7 .
Figure 7. Deviations from the nominal diameter: (a) NIST reference test artifact, (b) proposed new test artifact.

Figure 7 .
Figure 7. Deviations from the nominal diameter: (a) NIST reference test artifact, (b) proposed new test artifact.

Machines 2024 ,
12,  x FOR PEER REVIEW 14 of 20 and the coefficient of determination (adjusted R-squared) evaluated on data obtained by test artifact geometrical inspection.

Table 1 .
FRs of NIST test artifact.

Table 2 .
DPs of NIST test artifact.
Evaluate machine capability assessment for a periodic performance check;

Table 3 .
FRs of proposed test artifact.

Table 4 .
DPs of proposed test artifact.
* CMM: Coordinate Measuring Machine considered the industry standard.

Table 4 .
DPs of proposed test artifact.
* CMM: Coordinate Measuring Machine considered the industry standard.

Table 5 .
Process parameters set used during the test.

Table 5 .
Process parameters set used during the test.

Table 6 .
Pin average deviation from nominal diameter measured on the NIST test artifact (Confidence Interval ±0.004 mm).

Table 6 .
Pin average deviation from nominal diameter measured on the NIST test artifact (Confidence Interval ±0.004 mm).

Table 7 .
Pin average deviation from nominal diameter measured on new test artifact (Confidence Interval ±0.0007 mm).

Table 7 .
Pin average deviation from nominal diameter measured on new test artifact (Confidence Interval ±0.0007 mm).

Table 6 .
Pin average deviation from nominal diameter measured on the NIST test artifact (Confidence Interval ±0.004 mm).

Table 7 .
Pin average deviation from nominal diameter measured on new test artifact (Confidence Interval ±0.0007 mm).

Table 8 .
Pins axial deviation with regard to nominal position-NIST artifact.

Table 8 .
Pins axial deviation with regard to nominal position-NIST artifact.

Table 9 .
Pins axial deviation with regard to nominal position-proposed design.

Table 10 .
X direction regression models analysis.

Table 10 .
X direction regression models analysis.

Table 10 .
X direction regression models analysis.

Table 11 .
Y direction regression models analysis.