Q-Learning Algorithm for Predicting Mechanical Properties of Inconel 718 Processed by Selective Laser Melting

Abstract

This study introduces a model-free reinforcement learning framework based on Q-Learning (QLA) for the multi-objective optimization of Selective Laser Melting (SLM) process parameters for Inconel 718. To efficiently handle the limited experimental dataset, a tabular Q-Learning approach was implemented, in which each parameter combination was treated as a discrete state and every possible transition as an action. Four key process variables laser power (P), scan speed (S), layer thickness (T), and hatch spacing (H) were optimized for two output responses: relative density (RD) and Vickers hardness (VH). The Q-Learning agent iteratively explored various parameter combinations, observed the resulting material properties, and continuously updated its policy to converge toward optimal conditions. The optimal parameter set identified by the framework was P = 270 W, S = 800 mm/s, H = 0.1 mm, and T = 0.08 mm. Despite relying on only 16 experimental trials, the model achieved exceptionally low prediction errors of 0.0503% for RD and 0.0857% for VH, demonstrating substantial reductions in both experimental effort and material consumption. The results confirm that reinforcement learning can autonomously and effectively identify optimal SLM parameter settings, highlighting its strong potential to enhance precision, efficiency, and overall quality in the additive manufacturing of metallic components.

1. Introduction

Machine learning (ML) is a subfield of artificial intelligence that is concerned with training systems to improve themselves. It is useful in various domains, including computer vision, prediction, information retrieval, and the optimization of AM design, processes, and manufacturing [1]. Advanced machine learning algorithms can help optimize process parameters and prevent powder distribution errors during additive manufacturing [2]. Machine learning (ML) techniques provide a more adaptable and cost-efficient method for predicting mechanical properties based on processing and material properties [3].

Traditional methods, such as the central composite design (CCD) in the response surface technique, were used to investigate the effects of four critical SLM process parameters, namely laser power (P), scan speed (V), hatch spacing (HS), and layer thickness (LT), on the SLM-processed Inconel 718. On a comprehensive dataset of 30 experimental samples, the RD model achieved a prediction accuracy with a prediction error of 29.86% [4]. GA (genetic algorithm), GA-BPNN (Backpropagation Neural Network), and AGA-BPNN (adaptive GA BPNN) models on SLM-ed IN718 and RD data. The models consider critical process parameters, including laser power, scan speed, hatch spacing, layer thickness, and RD data points. For 25 experimental samples used in a single-objective optimization framework, the RD error percentages were 26.5% with the GA model, 24.7% with the GA-BPNN model, and 20.1% with the AGA-BPNN model [5]. An evolutionary approach (EVoNN) was used to optimize parameters such as laser power (W), scanning velocity (mm/s), and hatch spacing (mm) based on relative density. The investigation was carried out using IN718 SLM, with 16 experiments resulting in an RD error rate of 6.85% [6]. Response Surface Methodology (RSM), specifically the Box–Behnken design, was used to optimize parameters such as laser power, layer thickness, and scanning speed on the hardness of the IN718 SLM. The investigation focused on a single target and involved 15 experimental trials, resulting in a hardness error rate of 10.94% [7]. Taguchi and super ranking concept techniques, along with process parameters such as laser power (LP), scan speed (SS), and hatch distance (HD), were used to establish a single optimal condition for Inconel 625 using selective laser melting (SLM). The study included nine experimental trials with a hardness error rate of 22.01% [8]. Artificial Neural Networks (ANN) with a focus on process parameters such as Laser Power (LP), Scan Speed (SS), and Hatch Spacing (HS), specifically in the context of Inconel 718 with Selective Laser Melting (SLM). This study included 20 experiments with a hardness error rate of 3.26% [9]. Overall, these methods require a large amount of data, which yields significant prediction errors, and rely heavily on regression models or complex neural networks, which are not always suitable for real-time or small data applications. This limits their ability to reduce costs, save manufacturing time, reduce material waste, and ensure high quality products. They highlighted the utilization of optimization algorithms. These constraints can be solved by applying Q-learning, a subset of the Reinforcement Learning algorithm [10].

This study describes a model-free reinforcement learning approach, specifically Q-learning, for optimizing the SLM parameter values. Reinforcement Learning (RL) algorithms have gained popularity in recent years owing to their ability to increase decision-making efficiency, optimization, and autonomous systems. A review of the current literature reveals that its application for improving the mechanical properties and microstructures in additive manufacturing remains unexplored. Unlike traditional approaches, the proposed Q-learning algorithm (QLA) does not require a regression equation and performs well on smaller datasets. It also learns optimal parameter combinations through state-action-based iterative improvements, making it more flexible and efficient. A model-free, off-policy reinforcement learning approach, specifically a subset of Q-learning, for the optimization of process parameters in metal additive manufacturing. The QLA architecture distinctly represents the interactions among the agent, action space, state space, and environment [11]. Deep Q-Learning was applied to challenges with a continuous action domain. This is a model-free method based on deterministic policy gradients. This algorithm performs well with continuous actions, network designs and hyperparameters [12].

The objective of the current study was to investigate and optimize the influence of the process parameters on the mechanical properties and microstructures. Previous studies have examined the selective laser melting (SLM) method for metal additive manufacturing (AM) [13]. SLM uses laser powder bed fusion (LPBF) [14] to fabricate complex structures [15]. This technology has been widely used for manufacturing titanium, aluminum, stainless, and nickel-based products. Inconel 718 (IN718) is one of the most studied nickel materials employed in this method owing to its wide range of applications in aircrafts, gas turbines, and turbocharger rotors [16,17]. Over 50 process parameters influence the product quality of components made using SLM [18]. The relationship between the processing parameters and the resulting microstructural and mechanical properties of Inconel 718 produced by Selective Laser Melting (SLM) has been widely studied. SLM provides major advantages over conventional fabrication methods by enabling rapid solidification, refined microstructures, and precise control of the molten pool. These conditions lead to ultrafine cellular dendrites, reduced segregation, and improved strengthening. Therefore, optimizing the SLM process parameters is essential to fully exploit its capability to tailor the microstructure and enhance the mechanical performance [19,20]. Design of experiments methods can be applied to determine crucial parameters [21]. The Taguchi design of experiments is useful for selecting significant parameters [22]. Optimized SLM processing parameters have been investigated for their effects on mechanical properties [23]. Based on the literature survey, four important parameters, laser power (P), scan speed (S), hatch spacing (H), and layer thickness (T), were shortlisted, which cover critical aspects such as stability, enhancing densification and bonding, uniform melt pool overlap enhancing microstructural uniformity, fewer imperfections such as porosity, distribution of thermal energy, rate of solidification, finer layers, and strong interparticle bonding. Figure 1 shows an overview of the workflow used in this study.

Figure 1. Flowchart of the current study.

The relative density and Vickers hardness were selected because they serve as comprehensive, sensitive, and experimentally feasible indicators of the component quality in SLM. They reflect both process-induced porosity and mechanical resistance, making them ideal for the performance evaluation and optimization of over 50 influential SLM process parameters. These process parameters were varied to determine the relative density and Vickers hardness. RD and VH are suitable for machine learning and reinforcement learning-based optimization owing to their consistent trends with process parameters, quantifiable nature, and clear objective functions for multi-objective optimization.

2. Materials and Methods

2.1. Inconel 718

Grade 718 is a representative nickel-based superalloy that incorporates niobium and is among the most frequently employed materials for high-temperature applications owing to its precipitation-strengthening properties [24]. The chemical composition of the IN718 alloy [25] is presented in Table 1. Niobium provides precipitation strengthening through the formation of the γ″ (Ni₃Nb) phase, which significantly enhances the high-temperature strength, creep resistance, and microstructural stability.

Table 1. Chemical composition of the IN718 alloy (in wt.%).

Ni	Fe	Cr	Nb	Al	Ti	Mo	Mn	Si	C	O
54.04	17.8	18.18	5.3	0.49	1.12	3.02	0.02	0.18	0.04	0.015

Energy-dispersive X-ray (EDX) analysis was performed on the Inconel 718 samples to determine their elemental composition, as shown in Figure 2. EDX mapping revealed a uniform distribution of Ni, Cr, Fe, and Nb, indicating good metallurgical bonding and homogeneity.

Figure 2. EDX analysis of Inconel 718.

The Samples were fabricated using an EOS M290 metal 3D printing machine (EOS GmbH, Krailling, Germany) [26] from the IN718 powder material, which was selected for testing. Rectangular sample dimensions were printed using this machine, and each printed sample measured 30 mm × 20 mm × 6 mm. Sixteen (16) test specimens were extracted from the SLM-fabricated Inconel 718 blocks using a Wire electrical discharge machine (Electronica HITECH Job Master D-Zire, Electronica HiTech Machine Tools Pvt. Ltd., Pune, India). This process ensured high-dimensional accuracy without mechanical deformation. Each cube-shaped sample (10 mm × 10 mm × 6 mm) was fabricated using an L16 experimental design.

2.2. Taguchi Design of Experiment and Sample(s) Extraction

The number of experiments to be conducted with the selected orthogonal array is based on the total number of parameters involved and the levels at which they can be varied. Based on an exhaustive literature review, an L16 orthogonal array was created using the Taguchi design of experiments, which included four parameters—laser power (P), scan speed (S), hatch spacing (H), and layer thickness (T)—at four different levels owing to their associated properties. Laser power (P) optimization is critical for achieving a stable and homogenous melt pool, better layer bonding, and fewer flaws such as porosity. The scan speed (S) determines the distribution of heat energy and the rate of solidification in molten pools. An optimized scan speed encourages full fusion and the formation of strong interparticle bonds. Hatch spacing (H), or the distance between parallel laser trajectories, determines the bonding and homogeneity of the melt pool region. The layer thickness (T) determines the depth of each powder layer before melting and solidification in SLM. Because all the process parameters can be modified at four levels, the resulting orthogonal array is L16. The lower and upper bounds used for the parameters were laser power (P) of 250–310 W, Scan speed (S) of 800–1100 mm/s, hatch spacing of 0.09–0.12 mm, and layer thickness of 0.04–0.1 mm. Table 2 presents the selected parameters, their levels, and category values.

Table 2. SLM process parameters, their levels, and category values.

Parameter	Symbol	Units	Levels
			1	2	3	4
Laser Power	P	W	250	270	290	310
Scan Speed	S	mm/s	800	900	1000	1100
Hatch Spacing	H	mm	0.09	0.1	0.11	0.12
Layer Thickness	T	mm	0.04	0.06	0.08	0.1

2.3. Performance Measures

The relative density (RD) and Vickers hardness (VH) were also investigated. The Relative density (RD) was determined using a density measurement kit, and the Vickers hardness (VH) was determined using the Vickers hardness test on the SLM IN718 sample(s).

Calculation of Relative Density (RD) and Vickers Hardness (VH)

Relative density (RD), also known as specific gravity, is defined as the ratio of a material’s density to that of a reference substance, typically water at 4 °C for solids and liquids. In this study, RD was determined using the ME-DNY-43 density measurement kit, which is based on Archimedes’ principle, thereby facilitating an accurate assessment of the porosity of SLM-fabricated Inconel 718 specimens.

The Vickers hardness test is a precise method for evaluating material hardness. In this study, the VH of the fabricated samples was measured using an ECONOMET VH-1MDX hardness tester according to the ASTM E384 standards [27]. A diamond-shaped indenter with a square base was pressed into the polished surface of the sample using a load of 500 g (0.5 kgf) and a dwell time of 15 s.

This machine calculates the density of the sample based on the difference in weight measurements using Equation (1) [28,29].

$${\mathsf{\rho }}_{\mathrm{density}}=\left({\mathsf{\rho }}_{\mathrm{fluid}}-{\mathsf{\rho }}_{\mathrm{air}}\right)·{\displaystyle \frac{{\mathrm{mass}}_{\mathrm{air}}}{{\mathrm{mass}}_{\mathrm{air}}-{\mathrm{mass}}_{\mathrm{fluid}}}}+{\mathsf{\rho }}_{\mathrm{air}}$$

(1)

where

• ρ_density is the density of the sample (g/cm³ or kg/m³);
• ρ_fluid is the density of the immersion fluid (typically distilled water, ~0.998 g/cm³ at 20 °C);
• ρ_air is the density of air (typically ~0.0012 g/cm³ at room temperature and pressure);
• mass_air is the mass of the sample measured in air (g);
• mass_fluid is the mass of the sample measured in fluid (apparent mass) (g).

The RD of each sample was measured three times by immersing it in water, with individual readings provided by the machine, referred to as trials 1, 2, and 3, as shown in Figure 3a. For each sample, the average of these values was calculated and used to determine the actual value (i.e., the experimental RD). This procedure was repeated for all 16 samples representing the pre-test RD measurements.

Figure 3. Bar chart for Inconel 718: (a) relative density and (b) Vickers hardness.

This machine calculates the Vickers hardness number (VHN) using Equation (2) [30], depending on the diagonal length of the indentation:

VHN = 1.854.4 × F/d²

(2)

where

• VHN is the Vickers hardness number;
• F = Applied load (in kilograms-force, kgf);
• d = Average diagonal length of the indentation (in millimeters, mm);
• 1.854 = Geometrical constant for Vickers indenter shape.

Each sample was sequentially rough-polished using emery sheets with grit sizes of 200, 400, 600, 800, 1000, 1200, and 1500, followed by final polishing using a disc polishing machine. A Vickers hardness tester was then used to take three measurements at different locations on each sample, referred to as Trials 1, 2, and 3 (Figure 3b). The average of these three measurements was taken as the experimental Vickers hardness (VH) value for each sample, and the same procedure was applied to all 16 samples to obtain the post-test VH measurements. Figure 4 illustrates the procedure used to measure the Vickers hardness indentation geometry and the specific locations of the indentation points. For each indent, the diagonal lengths d1 and d2 were measured, as shown in Figure 4a. Hardness measurements were performed at three equally spaced positions along the vertical axis of the sample surface: the upper region (P1), centre region (P2), and lower region (P3), as shown in Figure 4b. For each sample, the VH value was calculated as the mean of the three regional measurements (P1–P3). In total, 48 hardness impressions were performed (3 indents × 16 samples).

Figure 4. Vickers indentations: (a) measurements of the diagonal lengths d1 and d2 for each indent and (b) locations of the three hardness test points (P1, P2, and P3) on the sample surface.

2.4. Microstructural Analysis

2.4.1. Optical Microscope

Figure 5a,b show optical micrographs of block 1 along the build-up plane (BUP) and long transverse (LT) directions. On the plane orthogonal to the build direction (BUD), the structure resembles a collection of partially sintered particles. Microstructural analysis of the built plane indicated the presence of two hatched layers in the cross-section. During the printing of Block 1, the angle between the first and second layers was set to 67°, and the LT section micrograph revealed clear melt pool boundaries.

Figure 5. Optical micrographs of the IN718 specimen (Block 1): (a) along the build plane, and (b) along the plane orthogonal to the long transverse direction.

Block 4 exhibited low-quality physical and mechanical characteristics. This behavior may result from a diminished energy density. The LT section micrograph in Figure 6 confirms the presence of significant porosity, which leads to reduced mechanical performance.

Figure 6. Optical micrographs along the short traverse of (a) block 1 showing low porosity and (b) block 4 showing high porosity.

2.4.2. SEM Images of Inconel 718

The SEM images of the build-plane sections of the additively manufactured IN718 blocks are shown in Figure 7. The internal microstructural features of the melt pools are distinct in the SEM micrographs but are not visible in the optical images owing to resolution limitations. The cellular features within the solidified melt pools exhibited a range of orientations. The microstructural image indicates that the cellular structures extended beyond the edges of the melt pool. This result affirms the existence of internal microstructural features within the melt pools [31]. Cellular morphology develops as a consequence of local compositional changes and constitutional supercooling [32].

Figure 7. SEM micrographs of all printed IN718 blocks along the build plane. Block-1–Block-16 corresponds to experiments 1–16.

2.5. Effect of SLM Process Parameters

The Relative density (RD) and Vickers hardness (VH) of parts produced using selective laser melting (SLM) are primarily dependent on the melt pool characteristics, which are mainly controlled by an integrated factor of laser power (P), scan speed (S), hatch spacing (H), and layer thickness (T), called volumetric energy density (VEd). There are three main mechanisms for the formation of porosity: lack of fusion, entrapped gas, and keyhole formation [33,34,35]. VEd governs the extent of powder melting and bonding. Low VEd results in insufficient melting, whereas a very high VEd causes overmelting and gas entrapment. The optimal energy density (VEd) ensures the best combination of relative density (RD) and Vickers hardness (VH) with minimal porosity and cracking, as shown in Table 3. Furthermore, VEd is inversely proportional to the scan speed, layer thickness, and hatch spacing; hence, VEd decreases with an increase in these parameters’ values. As VEd is proportional to P, RD and VH follow a trend similar to that of VEd [36,37]. The relative density decreased with increases in S, H, and T, which follows the relationship discussed previously.

Table 3. L₁₆ (4⁴) orthogonal array-based experimental plan and Q-learning results.

Sample Exp. No	P* (W)	S* (mm/s)	H* (mm)	T* (mm)	Energy Density (J/mm3)	Exp. RD (%)	Exp. VH (HV)	Pred. RD (%)	Pred. VH (HV)	%RD Error	%VH Error	Reward (R)
1	250	800	0.09	0.04	86.81	99.84	344.1	99.85	412.5	0.01	19.8779	7.02
2	250	900	0.1	0.06	46.3	99.72	354.7	99.85	412.5	0.1304	16.2955	−3.27
3	250	1000	0.11	0.08	28.41	98.93	338.3	99.85	412.5	0.93	21.9332	−13.10
4	250	1100	0.12	0.1	18.94	99.68	341.3	99.29	329.6	0.3913	3.4281	−1.28
5	270	800	0.1	0.08	42.19	99.43	350.2	99.38	349.9	0.0503	0.0857	23.65
6	270	900	0.09	0.1	33.33	98.57	353.2	98.85	367.4	0.2841	4.0204	−0.58
7	270	1000	0.12	0.04	56.25	99.97	357.9	99.38	349.9	0.5902	2.2353	−1.20
8	270	1100	0.11	0.06	37.07	99.38	349.9	99.72	354.7	0.3421	1.3718	3.29
9	290	800	0.11	0.1	32.95	99.31	308.5	99.84	344.1	0.5337	11.5397	−7.89
10	290	900	0.12	0.08	33.57	99.19	313.3	99.84	344.1	0.6553	9.8308	−8.09
11	290	1000	0.09	0.06	53.7	99.93	352.8	98.57	353.2	1.361	0.1134	8.12
12	290	1100	0.1	0.04	65.91	99.85	412.5	99.72	354.7	0.1302	14.0121	−2.61
13	310	800	0.12	0.06	53.82	99.29	329.6	99.85	412.5	0.564	25.1517	−11.52
14	310	900	0.11	0.04	78.22	99.92	335.4	99.84	344.1	0.0801	2.5939	6.82
15	310	1000	0.1	0.1	31	98.85	367.4	99.85	412.5	1.0116	12.2754	−10.94
16	310	1100	0.09	0.08	39.14	99.55	332.4	99.43	350.2	0.1205	5.355	1.90

The controlled process parameters (P*, S*, H*, and T*) indicate the level of the process parameters. The parameter values at different levels are listed in Table 2. The energy density is calculated using the formula Energy density (VE_d) = P∕SHT.

The Relative density (RD) increased with increasing laser power. At low power, insufficient energy causes incomplete melting and a lack of fusion porosity, thereby lowering the RD. At excessively high powers, overheating and vaporization lead to keyhole porosity and the formation of microcracks. Within the optimal range, a higher laser power improved the powder fusion, reduced the porosity, and stabilized the melt pool, thereby enhancing the RD. However, excessive power causes thermal instability and defect formation, thereby reducing part quality [38]. At high speeds, the reduced energy input causes shallow melt pools, a lack of fusion, and unmelted particles, lowering the RD. At low speeds, excessive energy leads to overheating, vaporization, and keyhole pores. Thus, an optimum scan speed ensures efficient melting and minimizes defects [39,40,41]. At low hatch spacings, excessive overlap leads to residual stress, cracking, and balling. At high hatch spacings, the reduced track overlap causes unmelted gaps and a lack of fusion, lowering the RD. Optimal hatch spacing ensures proper overlap, thereby minimizing porosity [4,42]. At a high layer thickness, insufficient energy causes incomplete melting and interlayer voids, thereby reducing the RD. At low layer thicknesses, better melting improves the density but may cause overheating and longer build times. An optimal layer thickness ensures full fusion and stable density [43]. The Vickers hardness (VH) increased with increasing laser power, which can be attributed to the reduction in porosity at higher power levels. The decrease in VH with increasing scan speed, hatch spacing, and layer thickness followed a similar trend to that of a lower relative density (RD). Higher values of these parameters reduce the volumetric energy density (VE_d), leading to increased porosity and, consequently, lower hardness [44,45,46,47].

3. Results and Discussion

3.1. Process Optimization

Optimizing process parameters helps obtain the best results compared to other methods. It avoids trial-and-error testing with actual materials to derive optimal parameters, as it is an expensive process [48]. A certain performance measurement can be attained if the optimal combination of process parameters is well defined [11]. Traditional optimization algorithms use analytical/mathematical approaches and are best suited for continuous, convex, and differentiable problems, whereas non-traditional optimization algorithms use heuristic/metaheuristic/AI-based approaches for complex, nonlinear, non-differentiable, discrete, or combinatorial problems, providing additional benefits. Thus, a model-free Reinforcement Learning (RL) methodology, specifically Q-learning, was chosen, which was transformed into an effective nontraditional optimization solution in the realm of metal AM.

3.2. Q-Learning Algorithm

The Q-learning algorithm is one of the most widely used and representative reinforcement learning approaches and is known for its off-policy learning strategy [49]. This algorithm simulates the learning behavior of an agent interacting with the environment through trial and error. The actions taken by the agent, along with their outcomes, were assessed and encoded as Q-values, which informed future decision-making processes.

The Q-learning mechanism is mathematically formulated to address complex decision-making and optimization challenges in reinforcement learning. The agent’s behavior is a balance between two critical phases, exploration and exploitation, which are used to discover and refine the optimal policy. The iterative update of action value pairs based on environmental feedback forms the core of the Q-learning process, serving as the basis for optimal search strategies in reinforcement-based algorithms.

It focuses on determining the optimal actions and mapping situations to actions, wherein an agent selects actions that maximize the reward in a given state. As illustrated in Figure 8, the Q-learning framework begins with the agent observing the current state (S_t), which consists of the selected SLM process parameters, such as the laser power (P), scan speed (S), hatch distance (H), and layer thickness (T). Based on this state, the agent executes an action (A_t) by either exploring new parameter combinations or exploiting the previously learned optimal settings. The environment, representing the SLM process or experimental setup, responds by transitioning to a new parameter state (S_t+1) and generating a reward (R_t) calculated from the experimental outcomes of the relative density (RD) and Vickers hardness (VH). Subsequently, the agent updates the Q value using the Bellman equation to improve its decision-making policy. Through multiple iterations, the agent learns to maximize the cumulative reward by selecting the most effective parameter combination. Once the maximum reward value is achieved, it signifies the optimal process condition that best approximates the real physical problem, thus ensuring data-efficient and adaptive optimization of the SLM process.

Figure 8. Proposed Q-learning process.

3.3. Q-Learning Algorithm Parameters

The process begins by initializing the Q-table with zero values for all the state–action combinations. All possible input parameter combinations define the state space, and all possible parameter values constitute the action space. Next, the Q-learning hyperparameters were configured as follows. These include the learning rate (alpha), which is 0.1, the discount factor (gamma), which is 0.9, and the number of episodes (iterations), which is 50, as presented in Table 4. At the beginning of each episode, the initial state is determined by randomly selecting input parameter combinations. The Q-table is used for prediction by choosing the input-parameter combination of the state that maximizes the Q-value, thereby producing optimal results with minimal error.

Table 4. Q-learning algorithm conditions and parameter constraints.

Sl. No	Parameter	Value/Description/Reason Behind the Chosen Value
1	State Space (S)	16 states (each row in the dataset representing a unique combination of 4 parameters)
2	Action Space (A)	16 actions (each action represents choosing any one of the 16 possible states)
3	Q-value Q (s, a)	16×16(Q-table initialized with zeros; updated iteratively using Q-learning formula)
4	Reward (R)	Logarithmic Cumulative value of errors: −(log (%RDerror) + log (%VH error))
5	Learning Rate (α)	0.1(Determines how much new information overrides old Q-values)
6	Discount Factor (γ)	0.9 (long-term reward consideration)
7	Number of Episodes (Iterations)	50 (Number of training episodes for agent to learn optimal policy)

The algorithm iteratively updates the Q-function based on the method proposed by [50].

$${\mathrm{Q}}^{\mathrm{new} }\left({\mathrm{s}}_{\mathrm{t}}, {\mathrm{a}}_{\mathrm{t}}\right)\leftarrow \mathrm{Q}\left({\mathrm{s}}_{\mathrm{t}},{\mathrm{a}}_{\mathrm{t}}\right)+\alpha \left[\mathrm{r}+\gamma ·{\mathrm{m}\mathrm{a}\mathrm{x}}_{\mathrm{a}} \mathrm{Q}\left({\mathrm{s}}_{\mathrm{t}+1}, \mathrm{a}\right)-\mathrm{Q}\left({\mathrm{s}}_{\mathrm{t}},{\mathrm{a}}_{\mathrm{t}}\right)\right]$$

(3)

where

• Q(s_t, a_t) is the current Q-value for taking action a_t in state s_t;
• Q^new(s_t, a_t) is the updated Q-value after learning from new experience;
• α (alpha) is the Learning rate (0 < α ≤ 1): Determines how much new information overrides old;
• r is the Reward received after taking action a_t in state s_t;
• γ (gamma) is the Discount factor (0 < γ ≤ 1): Determines importance of future rewards;
• s_t is the current state at time step t;
• a_t is the action taken at time step t;
• s_t+1 is the next state after action a_t;
• max_a Q (s_t+1, a) is the maximum estimated future Q-value from the next state s_t+1.

Q-Table

The Q-table is a fundamental element of the Q-learning algorithm. It contains the predicted cumulative reward for all possible actions in every state. In this study, the Q-table is a 16 × 16 matrix as shown in Table 3, where each state represents a unique experimental combination of four input process parameters: Laser Power (P), Scan Speed (S), Hatch Spacing (H), and Layer Thickness (T). Each action corresponds to a transition to another state, thereby enabling the agent to explore or exploit different parameter settings. The Q-values were initially set to zero. During the learning process, they are iteratively updated using the Bellman Equation, as shown in Equation (3).

In the implemented Q-learning framework, the reward function is defined to effectively translate the prediction accuracy into a quantitative reward as mentioned in Table 4.

This formulation ensures that lower combined percentage errors in relative density (RD) and Vickers hardness (VH) correspond to higher rewards, thereby guiding the agent toward optimal decision-making. For example, in Sample Experiment No. 3, the calculated reward value was −13.10, which matched the value presented in Table 4. The highest reward value of 23.65, observed for sample experiment No. 5, reflects the most accurate prediction performance. Within this learning framework, the agent iteratively selects actions that represent combinations of the process parameters that minimize the prediction error.

This demonstrates that the lowest total error has the maximum reward value, as shown in Figure 9 (heatmap with diagonal values). Figure 10 presents the Q-value convergence over 50 episodes fitted with an exponential trendline, which highlights the learning trajectory.

Figure 9. Q-table heatmap with diagonal Q values.

Figure 10. Convergence plot of rewards or Q-values.

3.4. Calculation of Error Percentage

The percentage prediction error was calculated using Equation (4) [51] and is presented in Table 4.

$$\mathrm{Percentage} \mathrm{prediction} \mathrm{error} ={\displaystyle \frac{ \mathrm{experimental} \mathrm{value} - \mathrm{predicted} \mathrm{value} }{ \mathrm{experimental} \mathrm{value} }}\times 100$$

(4)

• Experimental Value: This is the actual experimental or observed value.
• Predicted Value: This is the value estimated by a Q-learning algorithm.
• Percentage prediction Error: This indicates the deviation of the prediction from the actual measurement, expressed as a percentage of the experimental value.

The error percentage quantifies how close the model is to real-world results for properties such as relative density (RD) or Vickers hardness (VH).

3.5. Analysis of Results and Evaluation of Optimized Value

As discussed in the preceding section, the performance measurements were relative density (RD) and Vickers hardness (VH). Table 3 shows the results of the 16 studies and the variability in both the performance metrics. The measured relative density ranged from 98.57% to 99.97%. VH values ranging from 308.5 VH at the minimum to 412.5 VH at the maximum. The maximum relative density was observed in Experiment 7, and the maximum Vickers hardness was observed in Experiment 12. The scatter diagrams in Figure 11a demonstrate the trends in the relative density, whereas Figure 11b illustrates the variation in the Vickers hardness among the tested samples. The correlation between the actual and predicted results was very close, as shown in Figure 11a,b.

Figure 11. Trends of testing samples for (a) relative density and (b) Vickers hardness.

3.5.1. Optimization Value Selection Criteria

The identification of the optimized values in Table 3 is based on the experiment having low prediction errors for both RD and VH (%RD and% VH errors). This criterion was used to evaluate each row, as shown in Table 5.

Table 5. Percentage of error evaluation and reasons for each row.

Sample Exp. no	Exp. RD (%)	Exp. VH (HV)	%RD Error	%VH Error	Selected
1	99.84	344.1	0.01	19.8779	No Reason: %VH Error is High
2	99.72	354.7	0.1304	16.2955	No Reason: %VH Error is High
3	98.93	338.3	0.93	21.9332	No Reason: %VH Error is High
4	99.68	341.3	0.3913	3.4281	No Reason: %VH Error is High
5	99.43	350.2	0.0503	0.0857	YES (%RD and %VH both are lowest)
6	98.57	353.2	0.2841	4.0204	No Reason: %VH Error is High
7	99.97	357.9	0.5902	2.2353	No Reason: %VH Error is High
8	99.38	349.9	0.3421	1.3718	No Reason: %VH Error is High
9	99.31	308.5	0.5337	11.5397	No Reason: %VH Error is High
10	99.19	313.3	0.6553	9.8308	No Reason: %VH Error is High
11	99.93	352.8	1.361	0.1134	No Reason: %RD Error is High
12	99.85	412.5	0.1302	14.0121	No Reason: %VH Error is High
13	99.29	329.6	0.564	25.1517	No Reason: %VH Error is High
14	99.92	335.4	0.0801	2.5939	No Reason: %VH Error is High
15	98.85	367.4	1.0116	12.2754	No Reason: %VH Error is High
16	99.55	332.4	0.1205	5.355	No Reason: %VH Error is High

3.5.2. Analysis of Relative Density (RD) Values Derived from Experiment vs. Q-Learning

Based on the experimental values, the minimum relative density was recorded in Experiment 6, and the maximum was observed in Experiment 7. The highest percentage of error was recorded in Experiment 11, whereas the lowest percentage of RD error occurred in Experiment 1. Based on the Q-learning optimization framework, Experiment 5 was selected as the optimal case for RD, as it yielded the lowest prediction error (0.0503%).

3.5.3. Analysis of Vickers Hardness (VH) Values Derived from Experiment vs. Q-Learning

Based on the experimental values, the minimum Vickers hardness was reported in Experiment 9, whereas the maximum was observed in Experiment 12. The highest percentage of VH error was recorded in Experiment 13, whereas the lowest percentage of VH error was recorded in Experiment 5. According to the Q-learning optimization results, Experiment 5 also produced the lowest VH prediction error (0.0857%), confirming it as the optimal parameter combination for the hardness. Experimental numbers 1–4, 6–16 were not considered because the range of values was not as per the optimized selection criteria mentioned in Section 3.5.1.

Finally, Experiment 5 was considered a predicted parameter combination for optimization, as shown in Table 5.

3.6. Comparison of Classical Method Results and Proposed Work Results

The comparative results of the classical methods and the proposed Q-learning approach are summarized in Table 6. A comparative analysis of the literature indicates that Cuiyuan et al. reported a relative density error of 29.86%, Jing et al. observed 26.5%, and Tiwari et al. documented 6.85%. For the Vickers hardness, Hardik reported an error of 10.94%, whereas Bharath reported an error of 3.26%. The Q-learning optimization approach achieved significantly lower error values, with only 0.0503% for the relative density and 0.0857% for the Vickers hardness, outperforming the RSM, GA, ANN, and EvoNN, as shown in Table 6.

Table 6. Analysis of the percentage of error across different methods.

Sl. No	Method	Input Parameters	No. of Experiments	No. of Objectives	Error%	Author, Year
1	RSM	P, S, H, T	30	2 (RD, SR)	RD = 29.86%	Cuiyuan Lu et al. (2022) [4]
2	GA	P, S, H, T	25	1 (RD)	RD = 26.5%	Jing Shi et al. (2022) [5]
3	EvoNN + cRVEA	P, S, H	16	3 (RD, SR, SEC)	RD = 6.85%	Tiwari et al. (2023) [6]
4	RSM (Box–Behnken)	P, S, T	15	1 (VH)	VH = 10.94%	Hardik Nadiyadi et al. (2021) [7]
5	ANN	P, S, H	20	2 (VH, HT)	VH = 3.26%	Bharath et al. (2021) [9]
6	Q-learning	P, S, H, T	16	2 (RD, VH)	RD = 0.0503%, VH = 0.0857%	Proposed work

These results demonstrate that by employing a Q-learning algorithm based on four input parameters (P, S, H, T) and 16 experimental trials, the prediction accuracy for the multi-objective outputs (relative density and Vickers hardness) was notably improved. The proposed Q-learning algorithm (QLA) significantly reduces data requirements while achieving high predictive accuracy compared with conventional additive manufacturing optimization techniques.

4. Conclusions

This study investigated the influence of four critical SLM parameters—laser power, scan speed, hatch spacing, and layer thickness—on the relative density (RD) and Vickers hardness (VH) of Inconel 718 specimens fabricated via selective laser melting. Although RD and VH are commonly studied properties in additive manufacturing, they were chosen here as representative indicators of both densification quality and mechanical performance, providing a robust basis for their optimization. The novelty of this study lies in the application of a model-free reinforcement learning framework (Q-learning) for the process parameter optimization. Unlike regression-based approaches and conventional machine learning methods (ANN, GA, PSO, RSM), which require large datasets and still yield relatively high error rates, the Q-learning algorithm efficiently optimizes RD and VH using only 16 experimental trials designed using a Taguchi L16 array. The algorithm successfully identified the optimal process parameters (P = 270 W, S = 800 mm/s, H = 0.1 mm, T = 0.08 mm), achieving ultra-low prediction errors of 0.0503% for RD and 0.0857% for VH, respectively.

These findings demonstrate that reinforcement learning can serve as a powerful and data-efficient alternative to traditional optimization techniques in additive manufacturing. Beyond Inconel 718, the proposed framework can be extended to optimize additional responses (e.g., surface roughness, residual stress, and fatigue strength) and applied to other alloys (Ti-6Al-4V and AlSi10Mg) and AM processes, such as Directed Energy Deposition (DED) and Electron Beam Melting (EBM). Thus, this study contributes to the advancement of the integration of artificial intelligence and reinforcement learning into metal additive manufacturing, offering a new pathway for intelligent process optimization. However, the limitations include the fact that this research used a smaller number of datasets and no independent validation. In future, there is scope for researchers to use a large number of datasets with independent validation in similar works.