Classifying Thermal Degradation of Polylactic Acid by Using Machine Learning Algorithms Trained on Fourier Transform Infrared Spectroscopy Data

: Polylactic acid (PLA) is the most common polymeric material in the 3D printing industry but degrades under harsh environmental conditions such as under exposure to sunlight, high-temperatures, water, soil, and bacteria. An understanding of degradation phenomena of PLA materials is critical to manufacturing robust products by using 3D printing technologies. The objective of this study is to evaluate four machine learning algorithms to classify the degree of thermal degradation of heat-treated PLA materials based on Fourier transform infrared spectroscopy (FTIR) data. In this study, 3D printed PLA specimens were subjected to high-temperatures for extended periods of time to simulate thermal degradation and subsequently examined by using two types of FTIR spectrometers: desktop and portable spectrometers. Classiﬁers created by multi-class logistic regression and multi-class neural networks were appropriate prediction models for these datasets. to classify the degree of thermal degradation in the degraded PLA specimens. Among these models, multi-class logistic regression and multi-class neural network algorithms were shown to accurately predict degradation behaviors, as evaluated by confusion matrices and overall accuracy. Training from the ﬁrst data set generated by the desktop FTIR spectrometer under temperature-controlled storage tests resulted in overall accuracies of 88% and 87%, respectively, for the multi-class logistic regression and multi-class neural networks. Training by using the fourth data set resulted in similar overall accuracies of 89% and 81% for the respective algorithms. The results also show that FTIR spectra obtained from two di ﬀ erent spectrometers can be combined and used to train one predictive model, which ultimately accurately classiﬁed the degree of thermal degradation of 3D printed PLA specimens. The overall accuracy of the prediction models could be improved by using additional FTIR spectra to retrain the models. Moreover, other training strategies such as hyper parameter tuning, other normalization methods, and applying partial spectral ranges could similarly enhance the predictive capacity of models and are the subject of future studies.


Introduction
Polymer materials are widely used in a variety of industries such as construction, aerospace, electronics, automotive, additive manufacturing, and others. In the automotive industry, the number of parts composed of polymer materials has increased as polymeric materials exhibit advantages in minimal corrosion, design freedom, and weight reduction versus conventional materials [1]. The low weight of polymeric materials also provides greater fuel efficiencies, making these materials crucial in the automotive industry. Polymer materials have, moreover, been commonly used as inks for 3D printing in the additive manufacturing industry, in which more than 50% of parts are currently made with polymers [2].
As opposed to metal, polymers degrade under harsh environmental conditions such as exposure to sunlight, high-temperatures, water, soil, and bacteria [3]. Degradation changes various properties of the polymers and is classified into five categories: thermal degradation, photo degradation, chemical degradation, biological degradation, and mechanical degradation. Thermal degradation is caused by prolonged exposure to high temperatures and is a typical route for degradation across all polymer products. Thermal degradation is further classified into two cases, thermal deposition without oxygen and thermal oxidation with oxygen. Photo degradation occurs upon exposure to any wavelength of ultraviolet light. Polymer materials chemically degrade when exposed to rain, sunlight, high-temperatures, and bacteria due to oxidation and hydrolysis of the polymer. Biological degradation is caused by environmental bacteria and only occurs when the polymers are biodegradable. Mechanical degradation refers to the fatigue and fracture of polymeric materials, which is a similar degradation mechanism to that of metals. Many researchers have studied the various routes by which polymers degrade by using several different experimental methods, which are listed in Table 1. Table 1. Testing methods for Thermal-, Photo-, Chemical-, and Biological-degradation.
The mechanical test in the table means a general terminology to characterize the mechanical properties such as elastic modulus, yield strength, and so on. Among these methods, Fourier-transform infrared spectroscopy (FTIR) is a representative technique to analyze polymer degradation and involves obtaining an infrared spectrum of the polymer. Researchers should perform preprocessing and feature selection to extract meaningful information from FTIR spectra, because spectra can include artifacts due to the acquisition environment and operation of the instrument [18]. Recently researchers employed deep learning techniques to analyze vibrational spectra including those obtained by FTIR spectroscopy [19]. Deep learning techniques offer possibilities to reduce human errors in preprocessing and feature selection in FTIR spectra that would enable the extraction of accurate and robust information. Yang et al. reported 20 published studies in deep learning analyses of vibrational spectral analysis [19]. However, FTIR spectra is hard to be interpreted so that an alternate methodology is necessary. Zhang studied the thermal degradation of 3D printed polylactic acid (PLA) and Acrylonitrile-butadiene-styrene (ABS) materials by using FTIR and analyzed the resulting spectra by using artificial neural networks (ANNs) [20,21]. Choi et al. reported that the mechanical properties of one 3D printed PLA material changed with the degree of thermal degradation [22]. Zhang had used ANNs with Tensorflow and had not applied the preprocessing of normalization [21]. This study classified degraded the 3D printed PLA material by using four machine learning algorithms that included multi-class decision forest, multi-class decision jungle, multi-class logistic regression, and ANNs to analyze the FTIR datasets. The ANNs was used with the normalization.

A 3D Printed Material
The PLA material were fabricated by a 3D printer (DP201, Shindoh) with processing parameters set to 0.2 mm, 100%, 200 • C, and 60 • C for layer thickness, infill density, nozzle temperature, and bed temperature, respectively. The morphology of the material follows ASTM D638 Type IV designs. Figure 1 shows the 3D printed PLA specimens inside a chamber used for high-temperature storage tests. Choi et al. reported that the mechanical properties of the type IV tensile specimens changed with the degree of thermal degradation [22]. The study only focuses on the classification issue.
properties such as elastic modulus, yield strength, and so on. Among these methods, Fouriertransform infrared spectroscopy (FTIR) is a representative technique to analyze polymer degradation and involves obtaining an infrared spectrum of the polymer. Researchers should perform preprocessing and feature selection to extract meaningful information from FTIR spectra, because spectra can include artifacts due to the acquisition environment and operation of the instrument [18]. Recently researchers employed deep learning techniques to analyze vibrational spectra including those obtained by FTIR spectroscopy [19]. Deep learning techniques offer possibilities to reduce human errors in preprocessing and feature selection in FTIR spectra that would enable the extraction of accurate and robust information. Yang et al. reported 20 published studies in deep learning analyses of vibrational spectral analysis [19]. However, FTIR spectra is hard to be interpreted so that an alternate methodology is necessary. Zhang studied the thermal degradation of 3D printed polylactic acid (PLA) and Acrylonitrile-butadiene-styrene (ABS) materials by using FTIR and analyzed the resulting spectra by using artificial neural networks (ANNs) [20,21]. Choi et al. reported that the mechanical properties of one 3D printed PLA material changed with the degree of thermal degradation [22]. Zhang had used ANNs with Tensorflow and had not applied the preprocessing of normalization [21]. This study classified degraded the 3D printed PLA material by using four machine learning algorithms that included multi-class decision forest, multi-class decision jungle, multi-class logistic regression, and ANNs to analyze the FTIR datasets. The ANNs was used with the normalization.

A 3D Printed Material
The PLA material were fabricated by a 3D printer (DP201, Shindoh) with processing parameters set to 0.2 mm, 100%, 200 ºC, and 60 ºC for layer thickness, infill density, nozzle temperature, and bed temperature, respectively. The morphology of the material follows ASTM D638 Type IV designs. Figure 1 shows the 3D printed PLA specimens inside a chamber used for high-temperature storage tests. Choi et al. reported that the mechanical properties of the type IV tensile specimens changed with the degree of thermal degradation [22]. The study only focuses on the classification issue.

High-Temperature Storage Testing for PLA Thermal Degradation
High-temperature storage tests were performed to assess the thermal degradation of PLA specimen. The storage chamber (SH-662, ESPEC, Osaka, Japan) used for this test had two control parameters of time and temperature that allowed two varieties of degradation tests, temperature-

High-Temperature Storage Testing for PLA Thermal Degradation
High-temperature storage tests were performed to assess the thermal degradation of PLA specimen. The storage chamber (SH-662, ESPEC, Osaka, Japan) used for this test had two control parameters of time and temperature that allowed two varieties of degradation tests, temperature-controlled storage tests and time-controlled storage tests. Temperature-controlled storage tests had a fixed incubation time of 24 h and one of eight different temperatures that included 20°C, 40°C, 60°C, 80°C, 100°C, 120°C, 140°C, and 160°C. Time-controlled storage tests used a fixed temperature of 160°C and one of eight different incubation times that included 0 h, 3 h, 6 h, 9 h, 12 h, 15 h, 18 h, 21 h, and 24 h. A diagram that represents all of the thermal degradation testing conditions is shown in Figure 2. FTIR spectra were obtained from PLA specimens subjected to these tests by using two types of FTIR spectrometers: desktop and portable spectrometers. For all tests, relative humidity was set to be 0%. ℃, 80 ℃, 100 ℃, 120 ℃, 140 ℃, and 160 ℃. Time-controlled storage tests used a fixed temperature of 160 ℃ and one of eight different incubation times that included 0 h, 3 h, 6 h, 9 h, 12 h, 15 h, 18 h, 21 h, and 24 h. A diagram that represents all of the thermal degradation testing conditions is shown in Figure 2. FTIR spectra were obtained from PLA specimens subjected to these tests by using two types of FTIR spectrometers: desktop and portable spectrometers. For all tests, relative humidity was set to be 0%.

Figure 2.
A diagram that illustrates all testing conditions used for PLA Thermal degradation. The Xaxis shows the temperature at which specimens were stored, while the Y-axis shows the time at which the specimens were incubated.

Data Generated by Fourier-Transform Infrared Spectroscopy Spectrometers
In this study, two FTIR spectrometers were used to obtain the spectra of degraded PLA specimens: a Nicolet iS10 (ThermoFisher Scientific) which is a desktop FTIR spectrometer and a 4300 Handheld FTIR (Agilent) that is portable. Table 2 shows the specifications for the spectra obtained by each FTIR spectrometer. One hundred FTIR spectra were recorded at each random measurement locations. Atmospheric noise in the spectra has been corrected by software. Four training data sets were used to evaluate the FTIR data and are categorized in Table 3. Table 3. Definition of four training data sets.

Number of Training Data Sets
Description Figure 2. A diagram that illustrates all testing conditions used for PLA Thermal degradation. The X-axis shows the temperature at which specimens were stored, while the Y-axis shows the time at which the specimens were incubated.

Data Generated by Fourier-Transform Infrared Spectroscopy Spectrometers
In this study, two FTIR spectrometers were used to obtain the spectra of degraded PLA specimens: a Nicolet iS10 (ThermoFisher Scientific) which is a desktop FTIR spectrometer and a 4300 Handheld FTIR (Agilent) that is portable. Table 2 shows the specifications for the spectra obtained by each FTIR spectrometer. One hundred FTIR spectra were recorded at each random measurement locations. Atmospheric noise in the spectra has been corrected by software. Four training data sets were used to evaluate the FTIR data and are categorized in Table 3.

Machine Learning Algorithms
Four machine learning algorithms provided by Microsoft Azure Machine Learning Studio (MLS) were used to classify the degree of thermal degradation of PLA specimens. This analysis is categorized as a multi-class classification problem and allows the application of methods of multiclass decision forest, multi-class decision jungle, multi-class logistic regression, and multi-class neural network. The decision forest and decision jungle algorithms are ensemble learning methods for classification and rely on building multiple decision trees and voting on the most popular output class. The decision jungle algorithm is a recent version of the decision forest algorithm, while the logistic regression algorithm is a popular statistical method to predict the probability of an outcome for classification tasks. The neural network algorithm is known as artificial neural networks and which described briefly in a previous study [20]. Each of the four algorithms used here has

Machine Learning Algorithms
Four machine learning algorithms provided by Microsoft Azure Machine Learning Studio (MLS) were used to classify the degree of thermal degradation of PLA specimens. This analysis is categorized as a multi-class classification problem and allows the application of methods of multi-class decision forest, multi-class decision jungle, multi-class logistic regression, and multi-class neural network. The decision forest and decision jungle algorithms are ensemble learning methods for classification and rely on building multiple decision trees and voting on the most popular output class. The decision jungle algorithm is a recent version of the decision forest algorithm, while the logistic regression algorithm is a popular statistical method to predict the probability of an outcome for classification tasks. The neural network algorithm is known as artificial neural networks and which described briefly in a previous study [20]. Each of the four algorithms used here has hyperparameters, as shown in Table 4, which require tuning for higher accuracy in the classification. A min-max normalizer was used for artificial neural networks. MLS provides a drag-and-drop graphical user interface that allows machine learning models to be created as depicted in Figure 4.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 7 of 13 hyperparameters, as shown in Table 4, which require tuning for higher accuracy in the classification. A min-max normalizer was used for artificial neural networks. MLS provides a drag-and-drop graphical user interface that allows machine learning models to be created as depicted in Figure 4.

Confusion Matrix for Multi-Class Classification
Confusion matrices were used to evaluate the prediction models generated by four machine learning algorithms. The confusion matrix is a tabular method to evaluate the performance of the prediction models, as shown in Figure 5. Confusion matrices include four key terminologies, which are true positive (TP), true negative (TN), false positive (FP), and false negative (FN).

Confusion Matrix for Multi-Class Classification
Confusion matrices were used to evaluate the prediction models generated by four machine learning algorithms. The confusion matrix is a tabular method to evaluate the performance of the prediction models, as shown in Figure 5 For binary classification, the four terminologies are defined as follows: TP is the number of predictions that the prediction model correctly classifies the positive class as positive.
TN is the number of predictions that the prediction model correctly classifies the negative class as negative.
FP is the number of predictions that the prediction model incorrectly classifies the negative class as positive.
FN is the number of predictions that the prediction model incorrectly classifies the positive class as negative.
By comparison, four terminologies are used for the multi-class classification and are defined as shown in Figure 5b. Based on the four terminologies, the overall accuracy of the prediction model is defined as follows TP TN TP TN FP FN (1)

Degradation of PLA Specimens
Thermal degradation of PLA specimens was induced by subjecting specimens to different two temperature storage conditions, in which either temperature or time was held constant while the other varied. Figure 6 shows images of specimens subjected to a temperature-controlled storage test in which specimens were incubated at different temperatures for 24 h. Specimens stored at 160 ℃ became yellow in color, yet color variation in the other specimens could not be discerned by visual inspection. For binary classification, the four terminologies are defined as follows: TP is the number of predictions that the prediction model correctly classifies the positive class as positive.
TN is the number of predictions that the prediction model correctly classifies the negative class as negative.
FP is the number of predictions that the prediction model incorrectly classifies the negative class as positive.
FN is the number of predictions that the prediction model incorrectly classifies the positive class as negative.
By comparison, four terminologies are used for the multi-class classification and are defined as shown in Figure 5b. Based on the four terminologies, the overall accuracy of the prediction model is defined as follows Accuracy = TP + TN TP + TN + FP + FN (1)

Degradation of PLA Specimens
Thermal degradation of PLA specimens was induced by subjecting specimens to different two temperature storage conditions, in which either temperature or time was held constant while the other varied. Figure 6 shows images of specimens subjected to a temperature-controlled storage test in which specimens were incubated at different temperatures for 24 h. Specimens stored at 160°C became yellow in color, yet color variation in the other specimens could not be discerned by visual inspection. Figure 7 shows the PLA specimens that were degraded in a time-controlled storage test, in which temperature was constant but incubation time varied. Yellow coloration in PLA specimens increased after 18 h, yet visual inspection revealed little to no coloration could be observed prior to 18 h of incubation.

Validation for Four Machine Learning Algorithms
Four machine learning algorithms were evaluated by using a confusion matrix and overall accuracy, which are shown in Table 5 for all four machine learning algorithms and all training data sets. Appl. Sci. 2020, 10, x FOR PEER REVIEW 9 of 13 Figure 6. Images of PLA specimens degraded by using the temperature-controlled storage test in which specimens were incubated at various temperatures for 24 h. Figure 7. shows the PLA specimens that were degraded in a time-controlled storage test, in which temperature was constant but incubation time varied. Yellow coloration in PLA specimens increased after 18 h, yet visual inspection revealed little to no coloration could be observed prior to 18 h of incubation.

Validation for Four Machine Learning Algorithms
Four machine learning algorithms were evaluated by using a confusion matrix and overall accuracy, which are shown in Table 5 for all four machine learning algorithms and all training data sets. Analyses of the overall accuracies reveal that multi-class logistic regression and multi-class neural networks have superior accuracies versus other methods. Confusion matrices for the multi-

Validation for Four Machine Learning Algorithms
Four machine learning algorithms were evaluated by using a confusion matrix and overall accuracy, which are shown in Table 5 for all four machine learning algorithms and all training data sets. Analyses of the overall accuracies reveal that multi-class logistic regression and multi-class neural networks have superior accuracies versus other methods. Confusion matrices for the multi-  Analyses of the overall accuracies reveal that multi-class logistic regression and multi-class neural networks have superior accuracies versus other methods. Confusion matrices for the multi-class logic regression and multi-class neural network methods for all training datasets are shown in Figures 8-11. Figure 8 illustrates that the prediction models based on the multi-class logistic regression and the multi-class neural network appropriately classify the degree of thermal degradation. The true positives have relatively high probabilities in both algorithms. For the multi-class logistic regression, the probabilities of the predicted class are 89.5%, 5.3%, and 5.3% at 100°C, 120°C, and 140°C, respectively, when the true class is 100°C. For the multi-class neural network, the predicted probabilities are 94.7% and 5.3% at 100°C and 140°C, respectively, when the true class is 100°C. Both algorithms show a high predictive capacity.   Figure 8 illustrates that the prediction models based on the multi-class logistic regression and the multi-class neural network appropriately classify the degree of thermal degradation. The true positives have relatively high probabilities in both algorithms. For the multi-class logistic regression, the probabilities of the predicted class are 89.5%, 5.3%, and 5.3% at 100 ℃, 120 ℃, and 140 ℃, respectively, when the true class is 100 ℃ . For the multi-class neural network, the predicted probabilities are 94.7% and 5.3% at 100 ℃ and 140 ℃, respectively, when the true class is 100 ℃. Both algorithms show a high predictive capacity.    Figure 8 illustrates that the prediction models based on the multi-class logistic regression and the multi-class neural network appropriately classify the degree of thermal degradation. The true positives have relatively high probabilities in both algorithms. For the multi-class logistic regression, the probabilities of the predicted class are 89.5%, 5.3%, and 5.3% at 100 ℃, 120 ℃, and 140 ℃, respectively, when the true class is 100 ℃ . For the multi-class neural network, the predicted probabilities are 94.7% and 5.3% at 100 ℃ and 140 ℃, respectively, when the true class is 100 ℃. Both algorithms show a high predictive capacity.  The confusion tables in Figure 9 reveal poor predictive capacities for both models trained using the second dataset. All four machine learning algorithms trained on the second dataset could not generate accurate predictive models.
The size of the data in both the third and fourth datasets were reduced from 6949 to 806. For these datasets, the FTIR spectra were obtained by using both the desktop and the portable spectrometers. The confusion tables in Figure 10 show that the predictive models based on the third and fourth datasets properly classify the degree of thermal degradation. True positives are relatively higher than the others of FP and FN. These results indicate that more data is apparently needed to improve the accuracy of the prediction models for both machine learning algorithms trained on the third and fourth datasets. The confusion tables in Figure 9 reveal poor predictive capacities for both models trained using the second dataset. All four machine learning algorithms trained on the second dataset could not generate accurate predictive models. The size of the data in both the third and fourth datasets were reduced from 6949 to 806. For these datasets, the FTIR spectra were obtained by using both the desktop and the portable spectrometers. The confusion tables in Figure 10 show that the predictive models based on the third and fourth datasets properly classify the degree of thermal degradation. True positives are relatively higher than the others of FP and FN. These results indicate that more data is apparently needed to improve the accuracy of the prediction models for both machine learning algorithms trained on the third and fourth datasets.   The size of the data in both the third and fourth datasets were reduced from 6949 to 806. For these datasets, the FTIR spectra were obtained by using both the desktop and the portable spectrometers. The confusion tables in Figure 10 show that the predictive models based on the third and fourth datasets properly classify the degree of thermal degradation. True positives are relatively higher than the others of FP and FN. These results indicate that more data is apparently needed to improve the accuracy of the prediction models for both machine learning algorithms trained on the third and fourth datasets.  The confusion tables shown in Figure 11 reveal that models trained on the fourth dataset more accurately predict time-controlled storage testing behaviors than models trained on the third dataset ( Figure 10). Reducing the size of the data in the FTIR spectra may improve the accuracy of the prediction models. The results show that data from two different types of spectrometers can be combined to predict the degree of thermal degradation. Additional FTIR spectral data are necessary to improve the accuracies of the models.

Conclusions
Here, PLA specimens were thermally degraded by either a temperature-controlled storage test or a time-controlled storage test, in which the temperature or incubation time were held constant while the other was varied, respectively. FTIR spectra acquired from these temperature-treated specimens by using both desktop and portable spectrometers, and four machine learning algorithms were employed to classify the degree of thermal degradation in the degraded PLA specimens. Among these models, multi-class logistic regression and multi-class neural network algorithms were shown to accurately predict degradation behaviors, as evaluated by confusion matrices and overall accuracy. Training from the first data set generated by the desktop FTIR spectrometer under temperature-controlled storage tests resulted in overall accuracies of 88% and 87%, respectively, for the multi-class logistic regression and multi-class neural networks. Training by using the fourth data set resulted in similar overall accuracies of 89% and 81% for the respective algorithms. The results also show that FTIR spectra obtained from two different spectrometers can be combined and used to train one predictive model, which ultimately accurately classified the degree of thermal degradation of 3D printed PLA specimens. The overall accuracy of the prediction models could be improved by using additional FTIR spectra to retrain the models. Moreover, other training strategies such as hyper parameter tuning, other normalization methods, and applying partial spectral ranges could similarly enhance the predictive capacity of models and are the subject of future studies.