# Application of Machine Learning Techniques to Predict the Mechanical Properties of Polyamide 2200 (PA12) in Additive Manufacturing

## Abstract

**:**

## 1. Introduction

- Estimation of mechanical properties of AM-manufactured parts without prior knowledge about the material.
- Understanding how the mechanical properties depend on the part positioning in the build chamber.
- Compare performance of Linear regression models and machine learning proposed models, and choose the best models for the prediction of mechanical properties.
- Discuss which of the investigated features are the most significant and can be used to predict the mechanical properties, and how the mechanical properties can be controlled and managed based on the obtained results.

## 2. Materials and Methods

#### 2.1. Build Layout

#### 2.2. Conditioning of Specimens and Tensile Testing

#### 2.3. Application of Machine Learning Techniques to Predict Mechanical Properties without Prior Knowledge about Material Properties (PA)

#### Short Introduction to the Machine Learning Techniques Used

**Decision Tree Regressor**is a recursive algorithm that splits data into smaller subsets (separate classes) in order to form a tree, and it is important to choose the correct metrics for the best data split and to determine when a tree node should become a terminal.

**Gradient Boost Regressor**is an ensemble of decision trees. Instead of building one tree, this method predicts the desired outcome based on the additive regression model that uses decision trees as a weak learner [39]. Sequential fitting of a parameterized function (base learner) to current “pseudo”-residuals is done at each iteration by optimizing regression loss (e.g., least squares, absolute error) [40]. Friedman et al. [40] describe “pseudo”-residuals as minimization of the gradient of the loss function with respect to values of the regression model at each training data point for the current step.

**AdaBoost Regressor**(short for Adaptive Boost) is also an ensemble machine learning method. It works similarly to the Gradient Boost regressor, and the only difference is in the way weak learners are created at each iteration. Thus, AdaBoost changes the sample distribution at each iteration by changing the weights of each feature (the ones with the biggest error will have the highest weights).

## 3. Results and Discussion

#### 3.1. Description of the Collected Data

#### 3.2. Prediction of Tensile Modulus and Comparison of Models’ Performances

#### 3.3. Prediction of Nomial Stress and Comparison of Models’ Performances

#### 3.4. Prediction of Elongation at Break and Comparison of Models’ Performances

#### 3.5. Feature Importance for Prediction of Tensile Modulus

#### 3.6. Feature Importance for Prediction of Nominal Stress

^{2}, while the values range for volume is even smaller (1029.925–1035.427 mm

^{3}). However, the value range for a surface feature for all specimens (without separation on orientations) has a significant difference (e.g., 1381.555–1441.187 mm

^{2}) compared with volume values (1029.925–1035.427 mm

^{3}).

#### 3.7. Feature Importance for Prediction of Elongation at Break

## 4. Conclusions

## Funding

## Conflicts of Interest

## Abbreviations

DOE | Desifn of Experiment |

MSE | Mean Square Error |

GBR | Gradient Boosting Regressor |

ABR | AdaBoost Regressor |

DTR | Decision Tree Regressor |

## References

- Coy, J.A.; Kuball, C.M.; Roppenecker, D.B.; Lueth, T.C. Flexural modulus of lasersintered pa 2200. In Proceedings of the ASME 2013 International Mechanical Engineering Congress and Exposition, San Diego, CA, USA, 15–21 November 2013; American Society of Mechanical Engineers: New York, NY, USA, 2013; p. V02AT02A011. [Google Scholar]
- Dong, L.; Barth, N.; Correia, J.; Ahzi, S. Modeling and numerical simulation of selective laser sintering. In Proceedings of the 2016 17th International Conference on Thermal, Mechanical and Multi-Physics Simulation and Experiments in Microelectronics and Microsystems (EuroSimE), Montpellier, France, 18–20 April 2016; pp. 1–4. [Google Scholar]
- Rüsenberg, S.; Josupeit, S.; Schmid, H.J. A Method to Characterize the Quality of a Polymer Laser Sintering Process. Adv. Mech. Eng.
**2014**, 6, 185374. [Google Scholar] [CrossRef] - Kumaresan, T.; Gandhinathan, R.; Ramu, M.; Ananthasubramanian, M.; Pradheepa, K.B. Design, analysis and fabrication of polyamide/hydroxyapatite porous structured scaffold using selective laser sintering method for bio-medical applications. J. Mech. Sci. Technol.
**2016**, 30, 5305–5312. [Google Scholar] [CrossRef] - Ganci, M.; Zhu, W.; Buffa, G.; Fratini, L.; Bo, S.; Yan, C. A macroscale FEM-based approach for selective laser sintering of thermoplastics. Int. J. Adv. Manuf. Technol.
**2017**, 91, 3169–3180. [Google Scholar] [CrossRef] - Wang, R.J.; Wang, L.; Zhao, L.; Liu, Z. Influence of process parameters on part shrinkage in SLS. Int. J. Adv. Manuf. Technol.
**2007**, 33, 498–504. [Google Scholar] [CrossRef] - Wang, R.J.; Li, J.; Wang, F.; Li, X.; Wu, Q. ANN model for the prediction of density in selective laser sintering. Int. J. Manuf. Res.
**2009**, 4, 362–373. [Google Scholar] [CrossRef] - Mielicki, C.; Gronhoff, B.; Wortberg, J. Effects of laser sintering processing time and temperature on changes in polyamide 12 powder particle size, shape and distribution. AIP Conf. Proc.
**2014**, 1593, 728–731. [Google Scholar] - Baturynska, I.; Semeniuta, O.; Martinsen, K. Optimization of process parameters for powder bed fusion additive manufacturing by combination of machine learning and finite element method: A conceptual framework. Procedia CIRP
**2018**, 67C, 227–232. [Google Scholar] [CrossRef] - Brajlih, T.; Valentan, B.; Balic, J.; Drstvensek, I. Speed and accuracy evaluation of additive manufacturing machines. Rapid Prototyp. J.
**2011**, 17, 64–75. [Google Scholar] [CrossRef] - Baturynska, I. Statistical analysis of dimensional accuracy in additive manufacturing considering STL model properties. Int. J. Adv. Manuf. Technol.
**2018**, 97, 2835–2849. [Google Scholar] [CrossRef] - Caulfield, B.; McHugh, P.; Lohfeld, S. Dependence of mechanical properties of polyamide components on build parameters in the SLS process. J. Mater. Process. Technol.
**2007**, 182, 477–488. [Google Scholar] [CrossRef] - Hur, S.M.; Choi, K.H.; Lee, S.H.; Chang, P.K. Determination of fabricating orientation and packing in SLS process. J. Mater. Process. Technol.
**2001**, 112, 236–243. [Google Scholar] [CrossRef] - Lee, P.H.; Chung, H.; Lee, S.W.; Yoo, J.; Ko, J. Dimensional Accuracy in Additive Manufacturing Processes. In Proceedings of the ASME 2014 International Manufacturing Science and Engineering Conference collocated with the JSME 2014 International Conference on Materials and Processing and the 42nd North American Manufacturing Research Conference, Detroit, MI, USA, 9–13 June 2014; p. V001T04A045. [Google Scholar]
- Savalani, M.; Hao, L.; Dickens, P.; Zhang, Y.; Tanner, K.; Harris, R. The effects and interactions of fabrication parameters on the properties of selective laser sintered hydroxyapatite polyamide composite biomaterials. Rapid Prototyp. J.
**2012**, 18, 16–27. [Google Scholar] [CrossRef] - Wegner, A.; Witt, G. Understanding the decisive thermal processes in laser sintering of polyamide 12. AIP Conf. Proc.
**2015**, 1664, 160004. [Google Scholar] - Yang, H.J.; Hwang, P.J.; Lee, S.H. A study on shrinkage compensation of the SLS process by using the Taguchi method. Int. J. Mach. Tools Manuf.
**2002**, 42, 1203–1212. [Google Scholar] [CrossRef] - Drummer, D.; Wudy, K.; Drexler, M. Modelling of the aging behavior of polyamide 12 powder during laser melting process. AIP Conf Proc.
**2015**, 1664, 160007. [Google Scholar] - Schmid, M.; Levy, G. Quality management and estimation of quality costs for additive manufacturing with SLS. In Proceedings of the Fraunhofer Direct Digital Manufacturing Conference, Berlin, Germany, 14–15 March 2012. [Google Scholar]
- Goodridge, R.; Ziegelmeier, S. Powder bed fusion of polymers. In Laser Additive Manufacturing; Elsevier: Amsterdam, The Netherlands, 2017; pp. 181–204. [Google Scholar]
- Mjali, K.V.; Botes, A. The influence of the concept of ?line energy? on the mechanical properties of laser formed commercially pure grade 2 titanium alloy plates. Procedia Manuf.
**2018**, 26, 267–275. [Google Scholar] [CrossRef] - Flores Ituarte, I.; Wiikinkoski, O.; Jansson, A. Additive Manufacturing of Polypropylene: A Screening Design of Experiment Using Laser-Based Powder Bed Fusion. Polymers
**2018**, 10, 1293. [Google Scholar] [CrossRef] - Gümüs, S.; Lackner, J.M.; Polat, Ş.; Kraschitzer, W.; Hanning, H.; Bayram, A.; Kaya, M.; Çallı, M.; Alkan, A. Failure behavior of PA12 based SLS lattice structure with macro-porosity. MATEC Web Conf.
**2018**, 188, 03007. [Google Scholar] [CrossRef] - Taufik, M.; Jain, P.K. Role of build orientation in layered manufacturing: A review. Int. J. Manuf. Technol. Manag.
**2013**, 27, 47–73. [Google Scholar] [CrossRef] - Majewski, C.; Hopkinson, N. Effect of section thickness and build orientation on tensile properties and material characteristics of laser sintered nylon-12 parts. Rapid Prototyp. J.
**2011**, 17, 176–180. [Google Scholar] [CrossRef] - Ituarte, I.F.; Coatanea, E.; Salmi, M.; Tuomi, J.; Partanen, J. Additive manufacturing in production: A study case applying technical requirements. Phys. Procedia
**2015**, 78, 357–366. [Google Scholar] [CrossRef] - Munguía, J.; Ciurana, J.; Riba, C. Neural-network-based model for build-time estimation in selective laser sintering. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf.
**2009**, 223, 995–1003. [Google Scholar] [CrossRef] - Ituarte, I.F.; Huotilainen, E.; Wiikinkoski, O.; Tuomi, J. Experiments with machine vision for polymer flowability analysis in powder bed fusion. In Proceedings of the 3rd International Conference on Progress in Additive Manufacturing (Pro-AM 2018), Singapore, 14–17 May 2018. [Google Scholar]
- Kotsiantis, S.B. Decision trees: A recent overview. Artif. Intell. Rev.
**2013**, 39, 261–283. [Google Scholar] [CrossRef] - Sebastiani, F. Machine learning in automated text categorization. ACM Comput. Surv. (CSUR)
**2002**, 34, 1–47. [Google Scholar] [CrossRef] - Huang, G.B.; Zhu, Q.Y.; Siew, C.K. Extreme learning machine: Theory and applications. Neurocomputing
**2006**, 70, 489–501. [Google Scholar] [CrossRef] - Martinsen, K.; Downey, J.; Baturynska, I. Human-machine interface for artificial neural network based machine tool process monitoring. Procedia CIRP
**2016**, 41, 933–938. [Google Scholar] [CrossRef] - Li, L.; Ota, K.; Dong, M. Deep Learning for Smart Industry: Efficient Manufacture Inspection System with Fog Computing. IEEE Trans. Ind. Inform.
**2018**, 14, 4665–4673. [Google Scholar] [CrossRef] - Semeniuta, O.; Dransfeld, S.; Falkman, P. Vision-based robotic system for picking and inspection of small automotive components. In Proceedings of the 2016 IEEE International Conference on Automation Science and Engineering (CASE), Fort Worth, TX, USA, 21–25 August 2016; pp. 549–554. [Google Scholar]
- Ogorodnyk, O.; Lyngstad, O.V.; Larsen, M.; Wang, K.; Martinsen, K. Application of machine learning methods for prediction of parts quality in thermoplastics injection molding. In Proceedings of the International Workshop of Advanced Manufacturing and Automation, Changzhou, China, 20–21 September 2018; pp. 237–244. [Google Scholar]
- Standard Terminology for Additive Manufacturing: Coordinate Systems and Test Methodologies; ISO/ASTM International Standard: Geneva, Switzerland, 2013.
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res.
**2011**, 12, 2825–2830. [Google Scholar] - Aluja-Banet, T.; Nafria, E. Stability and scalability in decision trees. Comput. Stat.
**2003**, 18, 505–520. [Google Scholar] [CrossRef] - Ye, J.; Chow, J.H.; Chen, J.; Zheng, Z. Stochastic gradient boosted distributed decision trees. In Proceedings of the 18th ACM conference on Information and Knowledge Management, Hong Kong, China, 2–9 November 2009; pp. 2061–2064. [Google Scholar]
- Friedman, J.H. Stochastic gradient boosting. Comput. Stat. Data Anal.
**2002**, 38, 367–378. [Google Scholar] [CrossRef] - Schmid, M.; Amado, A.; Wegener, K. Polymer powders for selective laser sintering (SLS). AIP Conf. Proc.
**2015**, 1664, 160009. [Google Scholar] - Zhang, Z.; Joshi, S. An improved slicing algorithm with efficient contour construction using STL files. Int. J. Adv. Manuf. Technol.
**2015**, 80, 1347–1362. [Google Scholar] [CrossRef] - Minetto, R.; Volpato, N.; Stolfi, J.; Gregori, R.M.; Da Silva, M.V. An optimal algorithm for 3D triangle mesh slicing. Comput.-Aided Des.
**2017**, 92, 1–10. [Google Scholar] [CrossRef]

**Figure 3.**Schematic visualization of parts’ orientations and dimensional features (where t—thickness, w—width and L—length).

**Figure 4.**Distribution of tensile modulus for different orientations based on kernel distribution estimation (The straight line (1650 MPa) corresponds to the value from the EOS Balanced datasheet).

**Figure 5.**Distribution of nominal stress for different orientations based on kernel distribution estimation. The straight solid (48 MPa for the XYZ orientation) and dashed lines (42 MPa for the ZYX orientation) correspond to the values from the EOS Balanced datasheet.

**Figure 6.**Distribution of elongation at break for different orientations based on kernel distribution estimation. The straight solid (18% for the XYZ orientation) and dashed lines (4% for the ZYX orientation) correspond to the values from the EOS Balanced datasheet.

**Figure 7.**Feature relative importance based on the specimen orientations (

**a**) Tensile Modulus—ZYX orientation. (

**b**) Tensile Modulus—All orientations.

**Figure 8.**Visualization of Tensile Modulus for ZYX orientation (

**a**) As a function of central coordinates in the X- and Y-axes in mm. (

**b**) As a function of central coordinates in the X- and Z-axes in mm.

**Figure 9.**Feature relative importance based on the specimen orientations (

**a**) Nominal stress—ZYX orientation. (

**b**) Nominal Stress—Angle orientation.

**Figure 11.**Comparisson of the correlations between Nominal Stress and coordinates in the X-axis (for all data) (

**a**) Nominal Stress vs minimal coordinate in the X-axis. (

**b**) Nominal Stress vs central coordinate in the x axis.

**Figure 12.**Relative importance of the features based on the specimen orientations (

**a**) Elongation at break—Angle orientation. (

**b**) Elongation at break—All orientations.

Parameters | Value |
---|---|

Virgin/aged PA2200 powder ratio,% | 50/50 |

Layer thickness, mm | 120 |

EOS P395 system settings | Balance |

AM system warm up time, min | 120 |

AM system cooling down time, min | 240 |

Working chamber temperature, °C | 180.5 |

Removal chamber temperature, °C | 130.0 |

Statistical Characteristics | XYZ | XZY | ZYX | Angle |
---|---|---|---|---|

std | 31.912 | 35.776 | 95.763 | 46.63 |

mean | 1066.308 | 1051.951 | 958.25 | 1013.545 |

25% | 1046.308 | 1035.374 | 908.038 | 983.455 |

50% | 1067.77 | 1055.196 | 980.038 | 1010.291 |

75% | 1088.461 | 1074.403 | 1032.381 | 1043.599 |

max | 1148.078 | 1112.222 | 1090.35 | 1118.547 |

min | 968.483 | 933.376 | 648.079 | 907.983 |

Statistical Characteristics | XYZ | XZY | ZYX | Angle |
---|---|---|---|---|

std | 0.665 | 0.604 | 5.101 | 3.334 |

mean | 37.341 | 35.429 | 22.031 | 30.773 |

25% | 36.918 | 35.019 | 18.303 | 28.478 |

50% | 37.476 | 35.491 | 22.19 | 29.91 |

75% | 37.832 | 35.809 | 25.826 | 31.383 |

max | 39.186 | 36.576 | 32.241 | 37.744 |

min | 35.519 | 34.132 | 10.09 | 26.219 |

Statistical Characteristics | XYZ | XZY | ZYX | Angle |
---|---|---|---|---|

std | 0.727 | 0.679 | 1.096 | 2.311 |

mean | 13.383 | 13.265 | 3.499 | 7.079 |

25% | 13.158 | 12.875 | 2.719 | 5.594 |

50% | 13.582 | 13.497 | 3.26 | 6.343 |

75% | 13.893 | 13.663 | 4.017 | 7.262 |

max | 14.353 | 14.287 | 7.451 | 12.537 |

min | 11.18 | 10.618 | 1.917 | 4.316 |

**Table 5.**Prediction of Tensile Modulus with the help of machine learning techniques. Linear means linear regression models, GBR stands for Gradient Boost Regressor, DTR for Decision Tree Regressor, and ABR for AdaBoost Regressor.

# Data Points | Orientation | Linear | GBR | DTR | ABR | ||||
---|---|---|---|---|---|---|---|---|---|

R^{2} | MSE | R^{2} | MSE | R^{2} | MSE | R^{2} | MSE | ||

110 | XYZ | 0.129 | 602.945 | 0.272 | 503.781 | −0.76 | 1219.019 | 0.404 | 412.905 |

40 | XZY | −0.927 | 1421.147 | −2.297 | 2431.052 | −2.313 | 2442.847 | −2.102 | 2286.854 |

142 | ZYX | 0.48 | 4151.161 | 0.809 | 1524.994 | 0.764 | 1886.726 | 0.801 | 1586.234 |

74 | Angle | 0.03 | 1193.589 | −0.159 | 1426.052 | −0.469 | 1808.483 | 0.264 | 905.402 |

325 | All | 0.528 | 3876.678 | 0.888 | 916.721 | 0.819 | 1488.903 | 0.879 | 995.159 |

**Table 6.**Prediction of Nominal Stress with the help of machine learning techniques. Linear means linear regression models, GBR stands for Gradient Boost Regressor, DTR for Decision Tree Regressor, and ABR for AdaBoost Regressor.

# Data Points | Orientation | Linear | GBR | DTR | ABR | ||||
---|---|---|---|---|---|---|---|---|---|

R^{2} | MSE | R^{2} | MSE | R^{2} | MSE | R^{2} | MSE | ||

110 | XYZ | 0.393 | 2.093 | 0.197 | 2.768 | 0.173 | 2.852 | 0.372 | 2.165 |

40 | XZY | −0.645 | 0.827 | −0.013 | 0.509 | −0.043 | 0.525 | 0.50 | 0.251 |

142 | ZYX | 0.252 | 15.983 | 0.902 | 2.102 | 0.893 | 2.293 | 0.843 | 3.356 |

74 | Angle | 0.867 | 0.527 | 0.855 | 0.578 | 0.839 | 0.642 | 0.906 | 0.373 |

325 | All | 0.763 | 10.193 | 0.964 | 1.537 | 0.963 | 1.609 | 0.937 | 2.689 |

**Table 7.**Prediction of Elongation at break with a help of machine learning techniques. Linear means linear regression models, GBR stands for Gradient Boost Regressor, DTR—Decision Tree Regressor, and ABR—AdaBoost Regressor.

# Data Points | Orientation | Linear | GBR | DTR | ABR | ||||
---|---|---|---|---|---|---|---|---|---|

R^{2} | MSE | R^{2} | MSE | R^{2} | MSE | R^{2} | MSE | ||

110 | XYZ | 0.261 | 0.0907 | 0.0146 | 0.121 | −0.168 | 0.143 | 0.165 | 0.103 |

40 | XZY | 0.476 | 0.667 | 0.46 | 0.681 | 0.453 | 0.696 | 0.486 | 0.655 |

142 | ZYX | 0.269 | 0.963 | 0.67 | 0.434 | 0.587 | 0.545 | 0.638 | 0.476 |

74 | Angle | 0.889 | 0.500 | 0.795 | 0.927 | 0.749 | 1.129 | 0.903 | 0.439 |

325 | All | 0.965 | 0.739 | 0.987 | 0.284 | 0.985 | 0.326 | 0.971 | 0.616 |

© 2019 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Baturynska, I. Application of Machine Learning Techniques to Predict the Mechanical Properties of Polyamide 2200 (PA12) in Additive Manufacturing. *Appl. Sci.* **2019**, *9*, 1060.
https://doi.org/10.3390/app9061060

**AMA Style**

Baturynska I. Application of Machine Learning Techniques to Predict the Mechanical Properties of Polyamide 2200 (PA12) in Additive Manufacturing. *Applied Sciences*. 2019; 9(6):1060.
https://doi.org/10.3390/app9061060

**Chicago/Turabian Style**

Baturynska, Ivanna. 2019. "Application of Machine Learning Techniques to Predict the Mechanical Properties of Polyamide 2200 (PA12) in Additive Manufacturing" *Applied Sciences* 9, no. 6: 1060.
https://doi.org/10.3390/app9061060