Classifying Thermal Degradation of Polylactic Acid by Using Machine Learning Algorithms Trained on Fourier Transform Infrared Spectroscopy Data

Abstract

Polylactic acid (PLA) is the most common polymeric material in the 3D printing industry but degrades under harsh environmental conditions such as under exposure to sunlight, high-temperatures, water, soil, and bacteria. An understanding of degradation phenomena of PLA materials is critical to manufacturing robust products by using 3D printing technologies. The objective of this study is to evaluate four machine learning algorithms to classify the degree of thermal degradation of heat-treated PLA materials based on Fourier transform infrared spectroscopy (FTIR) data. In this study, 3D printed PLA specimens were subjected to high-temperatures for extended periods of time to simulate thermal degradation and subsequently examined by using two types of FTIR spectrometers: desktop and portable spectrometers. Classifiers created by multi-class logistic regression and multi-class neural networks were appropriate prediction models for these datasets.

1. Introduction

Polymer materials are widely used in a variety of industries such as construction, aerospace, electronics, automotive, additive manufacturing, and others. In the automotive industry, the number of parts composed of polymer materials has increased as polymeric materials exhibit advantages in minimal corrosion, design freedom, and weight reduction versus conventional materials [1]. The low weight of polymeric materials also provides greater fuel efficiencies, making these materials crucial in the automotive industry. Polymer materials have, moreover, been commonly used as inks for 3D printing in the additive manufacturing industry, in which more than 50% of parts are currently made with polymers [2].

As opposed to metal, polymers degrade under harsh environmental conditions such as exposure to sunlight, high-temperatures, water, soil, and bacteria [3]. Degradation changes various properties of the polymers and is classified into five categories: thermal degradation, photo degradation, chemical degradation, biological degradation, and mechanical degradation. Thermal degradation is caused by prolonged exposure to high temperatures and is a typical route for degradation across all polymer products. Thermal degradation is further classified into two cases, thermal deposition without oxygen and thermal oxidation with oxygen. Photo degradation occurs upon exposure to any wavelength of ultraviolet light. Polymer materials chemically degrade when exposed to rain, sunlight, high-temperatures, and bacteria due to oxidation and hydrolysis of the polymer. Biological degradation is caused by environmental bacteria and only occurs when the polymers are biodegradable. Mechanical degradation refers to the fatigue and fracture of polymeric materials, which is a similar degradation mechanism to that of metals. Many researchers have studied the various routes by which polymers degrade by using several different experimental methods, which are listed in Table 1.

Table 1. Testing methods for Thermal-, Photo-, Chemical-, and Biological-degradation.

Degradation	Polymer	Testing Methods	References
Thermal degradation	Acrylonitrile-butadiene-styrene (ABS)	Mechanical test FTIR Chemiluminescence	[4]
	Polyamides	TGA IR Mechanical test	[5]
	EPDM, Nitrile rubbers, PMMA, PAN, PA, PVC, PS	FTIR	[6]
	Poly(methyl methacrylate) (PMMA)	TA-FTIR TG	[7]
	Polyacrylonitrile	TGA FTIR Molecular weight Viscosity	[8]
Photodegradation	Polyamide 6,6 (PA66)	DSC WAXD FTIR Mechanical test SEM	[9]
	Syndiotactic polypropylene (sPP)	FTIR DSC Mechanical test SEM	[10]
	Poly(vinyl chloride)	TGA Chromatographic analysis UV-spectrophotometry IR	[11]
Chemical degradation	Polyesters	FTIR Viscosity	[12]
	Polyamide 6	Viscosity Rheological measurements	[13]
	Poly(vinyl chloride) (PVC)	FTIR SEM NMR	[14]
Biological degradation	Polyethylene	TGA DSC FTIR	[15]
	Polylactic Acid (PLA)	DSC X-ray diffraction FTIR Weight loss determination	[16]
	Low-density polyethylene (LDPE)	FTIR Molecular weight SEM	[17]

Abbreviations: FTIR, Fourier transform infrared; TGA, Thermogravimetric analysis; IR, Infrared Spectroscopy; TA-FTIR, Thermal analysis-Fourier transform infrared spectroscopy; TG, Thermogravimetry; DSC, Differential scanning calorimetry; WAXD. Wide-angle X-ray diffraction; SEM, Scanning electron microscopy.

The mechanical test in the table means a general terminology to characterize the mechanical properties such as elastic modulus, yield strength, and so on. Among these methods, Fourier-transform infrared spectroscopy (FTIR) is a representative technique to analyze polymer degradation and involves obtaining an infrared spectrum of the polymer. Researchers should perform preprocessing and feature selection to extract meaningful information from FTIR spectra, because spectra can include artifacts due to the acquisition environment and operation of the instrument [18]. Recently researchers employed deep learning techniques to analyze vibrational spectra including those obtained by FTIR spectroscopy [19]. Deep learning techniques offer possibilities to reduce human errors in preprocessing and feature selection in FTIR spectra that would enable the extraction of accurate and robust information. Yang et al. reported 20 published studies in deep learning analyses of vibrational spectral analysis [19]. However, FTIR spectra is hard to be interpreted so that an alternate methodology is necessary. Zhang studied the thermal degradation of 3D printed polylactic acid (PLA) and Acrylonitrile-butadiene-styrene (ABS) materials by using FTIR and analyzed the resulting spectra by using artificial neural networks (ANNs) [20,21]. Choi et al. reported that the mechanical properties of one 3D printed PLA material changed with the degree of thermal degradation [22]. Zhang had used ANNs with Tensorflow and had not applied the preprocessing of normalization [21]. This study classified degraded the 3D printed PLA material by using four machine learning algorithms that included multi-class decision forest, multi-class decision jungle, multi-class logistic regression, and ANNs to analyze the FTIR datasets. The ANNs was used with the normalization.

2. Methodology

2.1. A 3D Printed Material

The PLA material were fabricated by a 3D printer (DP201, Shindoh) with processing parameters set to 0.2 mm, 100%, 200 °C, and 60 °C for layer thickness, infill density, nozzle temperature, and bed temperature, respectively. The morphology of the material follows ASTM D638 Type IV designs. Figure 1 shows the 3D printed PLA specimens inside a chamber used for high-temperature storage tests. Choi et al. reported that the mechanical properties of the type IV tensile specimens changed with the degree of thermal degradation [22]. The study only focuses on the classification issue.

Figure 1. Pictures of 3D PLA printed specimens (ASTM D638 Type IV) in a storage chamber.

2.2. High-Temperature Storage Testing for PLA Thermal Degradation

High-temperature storage tests were performed to assess the thermal degradation of PLA specimen. The storage chamber (SH-662, ESPEC, Osaka, Japan) used for this test had two control parameters of time and temperature that allowed two varieties of degradation tests, temperature-controlled storage tests and time-controlled storage tests. Temperature-controlled storage tests had a fixed incubation time of 24 h and one of eight different temperatures that included 20 $℃$, 40 $℃$, 60 $℃$, 80 $℃$, 100 $℃$, 120 $℃$, 140 $℃,$ and 160 $℃$. Time-controlled storage tests used a fixed temperature of 160 $℃$ and one of eight different incubation times that included 0 h, 3 h, 6 h, 9 h, 12 h, 15 h, 18 h, 21 h, and 24 h. A diagram that represents all of the thermal degradation testing conditions is shown in Figure 2. FTIR spectra were obtained from PLA specimens subjected to these tests by using two types of FTIR spectrometers: desktop and portable spectrometers. For all tests, relative humidity was set to be 0%.

Figure 2. A diagram that illustrates all testing conditions used for PLA Thermal degradation. The X-axis shows the temperature at which specimens were stored, while the Y-axis shows the time at which the specimens were incubated.

2.3. Data Generated by Fourier-Transform Infrared Spectroscopy Spectrometers

In this study, two FTIR spectrometers were used to obtain the spectra of degraded PLA specimens: a Nicolet iS10 (ThermoFisher Scientific) which is a desktop FTIR spectrometer and a 4300 Handheld FTIR (Agilent) that is portable. Table 2 shows the specifications for the spectra obtained by each FTIR spectrometer. One hundred FTIR spectra were recorded at each random measurement locations. Atmospheric noise in the spectra has been corrected by software.

Table 2. Specifications of data generated by two Fourier-Transform Infrared (FTIR) spectrometers.

Spectrometers	Desktop (Nicolet iS10)	Portable (4300 Handheld FTIR)
Resolution	0.48 cm−1	3.73 cm−1
Range of wavenumber	650 cm−1–4000 cm−1	1000 cm−1–4000 cm−1
Number of data points	6948	806
Number of repetitions	100	100

Four training data sets were used to evaluate the FTIR data and are categorized in Table 3.

Table 3. Definition of four training data sets.

Number of Training Data Sets	Description
1st training data set	Data generated by the desktop FTIR spectrometer under temperature-controlled storage tests
2nd training data set	Data generated by the desktop FTIR spectrometer under the time-controlled storage tests
3rd training data set	Big data generated by both desktop and portable FTIR spectrometers under the temperature-controlled storage tests
4th training data set	Big data generated by both desktop and portable FTIR spectrometers under time-controlled storage tests

Figure 3 shows FTIR spectra in each of the four training data sets. Among the one hundred repeated FTIR experiments, spectra from two of the repetitions are randomly selected.

Figure 3. Representative FTIR spectra from the four training data sets used to train the machine learning algorithms: (a) 1st training data set; (b) 2nd training data set; (c) 3rd training data set; (d) 4th training data set.

2.4. Machine Learning Algorithms

Four machine learning algorithms provided by Microsoft Azure Machine Learning Studio (MLS) were used to classify the degree of thermal degradation of PLA specimens. This analysis is categorized as a multi-class classification problem and allows the application of methods of multi-class decision forest, multi-class decision jungle, multi-class logistic regression, and multi-class neural network. The decision forest and decision jungle algorithms are ensemble learning methods for classification and rely on building multiple decision trees and voting on the most popular output class. The decision jungle algorithm is a recent version of the decision forest algorithm, while the logistic regression algorithm is a popular statistical method to predict the probability of an outcome for classification tasks. The neural network algorithm is known as artificial neural networks and which described briefly in a previous study [20]. Each of the four algorithms used here has hyperparameters, as shown in Table 4, which require tuning for higher accuracy in the classification. A min-max normalizer was used for artificial neural networks.

Table 4. Hyperparameters to be optimized for the four machine learning algorithms.

Machine Learning Algorithm	Hyperparameters
Multi-class Decision Forest	Number of decision trees Maximum depth of the decision trees Number of random splits per node Minimum number of samples per leaf node
Multi-class Decision Jungle	Number of decision DAGs Maximum depth of decision DAGs Maximum width of the decision DAGs Number of optimization steps per decision DAG layer
Multi-class Logistic Regression	Optimization tolerance L1 regularization weight L2 regularization weight Memory size for L-BFGS
Multi-class Neural Network with Min-Max normalizer	Number of hidden nodes Learning rate Number of learning iterations Initial learning weights diameter

MLS provides a drag-and-drop graphical user interface that allows machine learning models to be created as depicted in Figure 4.

Figure 4. Flow diagram of machine learning models for multi-class classification.

2.5. Confusion Matrix for Multi-Class Classification

Confusion matrices were used to evaluate the prediction models generated by four machine learning algorithms. The confusion matrix is a tabular method to evaluate the performance of the prediction models, as shown in Figure 5. Confusion matrices include four key terminologies, which are true positive (TP), true negative (TN), false positive (FP), and false negative (FN).

Figure 5. Confusion Matrices; (a) binary classification and (b) Multi-class classification.

For binary classification, the four terminologies are defined as follows:

TP is the number of predictions that the prediction model correctly classifies the positive class as positive.

TN is the number of predictions that the prediction model correctly classifies the negative class as negative.

FP is the number of predictions that the prediction model incorrectly classifies the negative class as positive.

FN is the number of predictions that the prediction model incorrectly classifies the positive class as negative.

By comparison, four terminologies are used for the multi-class classification and are defined as shown in Figure 5b. Based on the four terminologies, the overall accuracy of the prediction model is defined as follows

$$Accuracy=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$$

(1)

3. Results and Discussion

3.1. Degradation of PLA Specimens

Thermal degradation of PLA specimens was induced by subjecting specimens to different two temperature storage conditions, in which either temperature or time was held constant while the other varied. Figure 6 shows images of specimens subjected to a temperature-controlled storage test in which specimens were incubated at different temperatures for 24 h. Specimens stored at 160 $℃$ became yellow in color, yet color variation in the other specimens could not be discerned by visual inspection.

Figure 6. Images of PLA specimens degraded by using the temperature-controlled storage test in which specimens were incubated at various temperatures for 24 h.

Figure 7 shows the PLA specimens that were degraded in a time-controlled storage test, in which temperature was constant but incubation time varied. Yellow coloration in PLA specimens increased after 18 h, yet visual inspection revealed little to no coloration could be observed prior to 18 h of incubation.

Figure 7. Images of PLA specimens degraded under the time-controlled storage test, in which temperature was held constant at 160 $℃$ and incubation time was varied.

3.2. Validation for Four Machine Learning Algorithms

Four machine learning algorithms were evaluated by using a confusion matrix and overall accuracy, which are shown in Table 5 for all four machine learning algorithms and all training data sets.

Table 5. Overall accuracies of the four machine learning algorithms.

Training Data Set	Multiclass Decision Forest	Multiclass Decision Jungle	Multiclass Logistic Regression	Multi-Class Neural Network
1st data set	0.6875	0.6938	0.8875	0.8688
2nd data set	0.3813	0.3375	0.4750	0.4063
3rd data set	0.6069	0.5597	0.6604	0.6824
4th data set	0.6972	0.6872	0.8882	0.8101

Analyses of the overall accuracies reveal that multi-class logistic regression and multi-class neural networks have superior accuracies versus other methods. Confusion matrices for the multi-class logic regression and multi-class neural network methods for all training datasets are shown in Figure 8, Figure 9, Figure 10 and Figure 11.

Figure 8. Confusion matrices for two machine learning algorithms trained on the 1st data set.

Figure 9. Confusion matrices for two machine learning algorithms trained on the 2nd training data set.

Figure 10. Confusion matrices for two machine learning algorithms trained on the 3rd data set.

Figure 11. Confusion matrices for two machine learning algorithms trained on the 4th data set.

Figure 8 illustrates that the prediction models based on the multi-class logistic regression and the multi-class neural network appropriately classify the degree of thermal degradation. The true positives have relatively high probabilities in both algorithms. For the multi-class logistic regression, the probabilities of the predicted class are 89.5%, 5.3%, and 5.3% at 100 $℃$, 120 $℃$, and 140 $℃,$ respectively, when the true class is 100 $℃$. For the multi-class neural network, the predicted probabilities are 94.7% and 5.3% at 100 $℃$ and 140 $℃$, respectively, when the true class is 100 $℃$. Both algorithms show a high predictive capacity.

The confusion tables in Figure 9 reveal poor predictive capacities for both models trained using the second dataset. All four machine learning algorithms trained on the second dataset could not generate accurate predictive models.

The size of the data in both the third and fourth datasets were reduced from 6949 to 806. For these datasets, the FTIR spectra were obtained by using both the desktop and the portable spectrometers. The confusion tables in Figure 10 show that the predictive models based on the third and fourth datasets properly classify the degree of thermal degradation. True positives are relatively higher than the others of FP and FN. These results indicate that more data is apparently needed to improve the accuracy of the prediction models for both machine learning algorithms trained on the third and fourth datasets.

The confusion tables shown in Figure 11 reveal that models trained on the fourth dataset more accurately predict time-controlled storage testing behaviors than models trained on the third dataset (Figure 10). Reducing the size of the data in the FTIR spectra may improve the accuracy of the prediction models. The results show that data from two different types of spectrometers can be combined to predict the degree of thermal degradation. Additional FTIR spectral data are necessary to improve the accuracies of the models.

4. Conclusions

Here, PLA specimens were thermally degraded by either a temperature-controlled storage test or a time-controlled storage test, in which the temperature or incubation time were held constant while the other was varied, respectively. FTIR spectra acquired from these temperature-treated specimens by using both desktop and portable spectrometers, and four machine learning algorithms were employed to classify the degree of thermal degradation in the degraded PLA specimens. Among these models, multi-class logistic regression and multi-class neural network algorithms were shown to accurately predict degradation behaviors, as evaluated by confusion matrices and overall accuracy. Training from the first data set generated by the desktop FTIR spectrometer under temperature-controlled storage tests resulted in overall accuracies of 88% and 87%, respectively, for the multi-class logistic regression and multi-class neural networks. Training by using the fourth data set resulted in similar overall accuracies of 89% and 81% for the respective algorithms. The results also show that FTIR spectra obtained from two different spectrometers can be combined and used to train one predictive model, which ultimately accurately classified the degree of thermal degradation of 3D printed PLA specimens. The overall accuracy of the prediction models could be improved by using additional FTIR spectra to retrain the models. Moreover, other training strategies such as hyper parameter tuning, other normalization methods, and applying partial spectral ranges could similarly enhance the predictive capacity of models and are the subject of future studies.