Applying Machine Learning to Nanoindentation Data of (Nano-) Enhanced Composites

Carbon fiber reinforced polymers (CFRPs) are continuously gaining attention in aerospace and space applications, and especially their multi-scale reinforcement with nanoadditives. Carbon nanotubes (CNTs), graphene, carbon nanofibers (CNFs), and their functionalized forms are often incorporated into interactive systems to engage specific changes in the environment of application to a smart response. Structural integrity of these nanoscale reinforced composites is assessed with advanced characterization techniques, with the most prominent being nanoindentation testing. Nanoindentation is a well-established technique, which enables quantitative mapping of nanomechanical properties with the μm surficial and nm indentation resolution scale and high precision characterization. This feature enables the characterization of the interface in a statistical and quantitative manner and the correlation of (nano-) reinforcement to interface properties of CFRPs. Identification of reinforcement is performed with k-Nearest Neighbors and Support Vector Machine classification algorithms. Expertise is necessary to describe the physical problem and create representative training/testing datasets. Development of open source Machine Learning algorithms can have an influential impact on uniformity of nanometry data creation and management. The statistical character of nanoindentation is a key factor to supply information on heterogeneity of multiscale reinforced composites. Both the identification of (nano-) reinforcement and quality assessment of composites are provided by involving artificial intelligence.


Introduction
The carbon fiber reinforced composites (CFRP) market is progressively growing [1,2]. The increasing needs of CFRPs for construction, automotive, and (aero-) space fields raise the necessity of improving interfacial properties [3]. Techniques with the potential of industrial application such as "green" electroplating technology, thermal spraying, and laser cladding introduce a nanometer's thin film on carbon fibers (CFs) surface. The subsequent incorporation of functional groups and novel sizing formulations improves the interphase and consequently, the CFRP mechanical properties [4][5][6]. Moreover, reinforcement with nanomaterials, such as CNTs growth on CFs is reported to effectively deal with the brittle fracture mechanism of the epoxy-based CFRPs similarly to aforementioned processes [3,7,8]. These processes are commonly adopted to increase mechanical interlocking unidirectionally in order to fulfil maximum potential of composites according to the law of mixture [3,9]. In this scope, powerful and high-speed characterization through nanoindentation is pre-treated to facilitate enhanced bonding in the interface prior to manufacturing. Functionalization techniques contain Low Pressure Plasma (LPP) modification, Active Screen Plasma (ASP) modification, electropolymerization with methacrylic acid, and Chemical Vapor Deposition of carbon nanotubes, while unmodified CFs were used for the reference specimen preparation [20][21][22]. This study aims to classify CFRPs with different modifications on CF's surface. For instance, since LPP and ASP functionalize CFs by developing oxygen species on the surface, both belong to the same ontology and the mechanism of reinforcement is thus considered as one class. The reinforcement mechanism by monomer electrografting and carbon nanotubes growth was independent and belong to separate classes of reinforcement. The data have undergone preprocessing via k-means clustering [13] to identify the phases of CFRPs and retrieve the information derived by interface, reinforcement cluster, and CFs. Epoxy matrix properties tend to mislead classification statistical metrics by increasing false positive predictions due to high similarity of nanomechanical properties, since epoxy properties are not affected by interfacial reinforcement. A total of 649 indentations and data from four different classes (pristine, with oxygen functionalities, monomer sizing, growth of CNTs) were used in this work to construct classification models. This is a useful concept since it is not necessary to test all components of the dataset, but only those that are representative. This can be done in practice by grouping a set, for instance, by similarity in order to handle diversity [23]. Similarly, chemical structure is used as a descriptor to define the class of reinforcement by grouping oxidation functionalities, sizing formulations with monomers, and grown carbon nanostructures on CF's surface.

Nanoindentation Testing
In order to determine the nanomechanical properties of CFRP specimens, Hysitron TriboLab ® Nanomechanical Test Instrument (Minneapolis, MN, USA) was used. The instrument capabilities enable loading from 1 × 10 −3 mN to 30 mN and with a high load and displacement resolution of 1 nN and 0.04 nm, respectively. All measurements have been performed by keeping constant the maximum indentation depth 200 nm, in accordance to ISO 14577-1 [24]. A standard three-sided pyramidal Berkovich tip was used, with an average curvature radius of 100 nm [13]. In all measurements, a total of 64 indents were performed by applying a square 8 × 8 grid, with spacing of 5 µm to avoid any indentation-to-indentation interaction [6,25]. The measurement environment was characterized by 45% humidity content, and 23 • C ambient temperature. In order to operate under closed loop control, the feedback control option was selected. A smooth surface was obtained by wet polishing using SiC grinding papers. The used granulometry involved grit papers of 400, 1000, 1200, 2000 and 4000 for 10 min each, by Struers LaboPol-2 grinding, lapping and polishing machine.

Computational Details
The KNN [26] and SVC [27] algorithms were used to perform classification of nanoindentation data. In order to deal with imbalanced original training sets, undersampling and SMOTE algorithms [18] were involved to balance classification populations. The caret and e1017 packages were used to implement the KNN, SVM, and tuning (accessed date: 1 August 2019) in R 3.6.1. Synthpop, tidyverse, sampling, and partykit (accessed date: 16 August 2019) in R 3.6.1 were used to implement the undersampling and SMOTE computation. The final SVM models were constructed and validated using the same caret package. All statistical calculations were performed using R 3.6.1 on 64-bit Windows 10 Home (Intel ® Core™ i5-8250U CPU @ 1.60 GHz, 1801 Mhz, 4 Cores, 8 Logical Processors and 8.00 GB RAM). Computational time did not exceed a few minutes, except from the polynomial SVC kernel, which required 40 min in order to be tuned to optimum prediction accuracy.

Statistical Metrics
In order to evaluate the performance of the prediction model, commonly adopted statistical metrics were measured. The use of Accuracy, Precision, Recall, F1 was performed in each case [28][29][30], after tuning each model to optimum Accuracy after parameterization.
True positives (TP) represent correct classifications of reinforcement mechanisms (positive sample), true negatives (TN) symbolize correct classifications of negative samples, false positives (FP) signify the incorrect classifications of negative predictions into positive, and false negatives (FN) denote positive samples that were incorrectly identified as negative samples. Recall is the proportion of positive samples that were correctly classified. Precision is the proportion of positive samples out of the sum of positive observations. F1 score is used to evaluate the model ability to classify (best value: 1).

Nanoindentation Mapping and Data Clustering
CFRP specimens were tested with nanoindentation and the methodology to perform quantitative mapping of E r is described. As derived by load-displacement curves, which were fitted with the Oliver-Pharr model [31], the E r was measured for each nanoindentation event. A total of 6 attributes generated by nanoindentation testing raw data were used to perform data clustering and establish unsupervised connections of these features to the identity of the CFRP phase. Parameters of contact depth (h c in nm), maximum force (P max in µN), E r (in GPa), nanohardness (H in GPa), final depth after fully unloaded (h f in nm), and m is determined by a least square fitting procedure.
The optimum number of clusters was calculated according to using the elbow method [32]. The optimum number was 4 ( Figure 1a), but the physical problem suggested that clusters can be more (Table 1). In this case, the data were clustered in four groups using R Studio, namely epoxy matrix, CF interface, CF, and the "reinforcement" cluster ( Figure 1b, Table 1).
True positives (TP) represent correct classifications of reinforcement mechanisms (positive sample), true negatives (TN) symbolize correct classifications of negative samples, false positives (FP) signify the incorrect classifications of negative predictions into positive, and false negatives (FN) denote positive samples that were incorrectly identified as negative samples. Recall is the proportion of positive samples that were correctly classified. Precision is the proportion of positive samples out of the sum of positive observations. F1 score is used to evaluate the model ability to classify (best value: 1).

Nanoindentation Mapping and Data Clustering
CFRP specimens were tested with nanoindentation and the methodology to perform quantitative mapping of Er is described. As derived by load-displacement curves, which were fitted with the Oliver-Pharr model [31], the Er was measured for each nanoindentation event. A total of 6 attributes generated by nanoindentation testing raw data were used to perform data clustering and establish unsupervised connections of these features to the identity of the CFRP phase. Parameters of contact depth (hc in nm), maximum force (Pmax in μN), Er (in GPa), nanohardness (H in GPa), final depth after fully unloaded (hf in nm), and m is determined by a least square fitting procedure.
The optimum number of clusters was calculated according to using the elbow method [32]. The optimum number was 4 ( Figure 1a), but the physical problem suggested that clusters can be more (Table 1). In this case, the data were clustered in four groups using R Studio, namely epoxy matrix, CF interface, CF, and the "reinforcement" cluster (Figure 1b, Table 1).
(a)   Information obtained by Table 1 was used to correlate colors, increment from blue to red to increment of Er as depicted in Figure 2.  Information obtained by Table 1 was used to correlate colors, increment from blue to red to increment of E r as depicted in Figure 2.  Information obtained by Table 1 was used to correlate colors, increment from blue to red to increment of Er as depicted in Figure 2. By the observation of the colormap, it is possible to make assumptions over the composite performance upon a different reinforcement mechanism. A robust mechanical response is obtained in the case of PMAA and carbon nanotubes compared to the pristine specimen, while another positive feature is the gradual continuity of interface, which allows uniform load transfer. Additionally, composite attributes attained the properties of CFs in a high extend in the case of carbon nanotube reinforcement (Figure 2d), as determined by k-means microstructure clustering. Low pressure plasma treatment demonstrated slightly deteriorated reduced elastic modulus overall, compared to the pristine specimen.

Descriptor Preprocessing
Epoxy matrix is not correlated to the mechanism of reinforcement, and these data do not contribute to identification of the reinforcement by surface modification of CFs. As a result, it was reasonable to remove these data using R language. After performing k-means clustering, the epoxy matrix nanoindentation events with a mean value of Er of about 7 GPa were not included by using library "dplyr" in order to obtain a proper descriptor of reinforcement and enhance classification model performance.

Split of Training and Test Sets
A leave-some-out (LSO) method was used to create a random partition split into a 70% training set used for model construction, and a 30% test set for holdout validation on observations. In detail, datasets of untreated CFRPs were used as reference, while CFRPs with CFs surface modification were included. Surface modification contained the introduction of oxygen species on the fiber surface, the attachment of monomers (sizing formulations), and the growth of CNTs, which are commonly adapted in literature [3][4][5][6][7][8]. Seed of random numbers was called to achieve a statistically unbiased estimation. Then, the statistical metrics were calculated and reported for each case. By the observation of the colormap, it is possible to make assumptions over the composite performance upon a different reinforcement mechanism. A robust mechanical response is obtained in the case of PMAA and carbon nanotubes compared to the pristine specimen, while another positive feature is the gradual continuity of interface, which allows uniform load transfer. Additionally, composite attributes attained the properties of CFs in a high extend in the case of carbon nanotube reinforcement (Figure 2d), as determined by k-means microstructure clustering. Low pressure plasma treatment demonstrated slightly deteriorated reduced elastic modulus overall, compared to the pristine specimen.

Descriptor Preprocessing
Epoxy matrix is not correlated to the mechanism of reinforcement, and these data do not contribute to identification of the reinforcement by surface modification of CFs. As a result, it was reasonable to remove these data using R language. After performing k-means clustering, the epoxy matrix nanoindentation events with a mean value of E r of about 7 GPa were not included by using library "dplyr" in order to obtain a proper descriptor of reinforcement and enhance classification model performance.

Split of Training and Test Sets
A leave-some-out (LSO) method was used to create a random partition split into a 70% training set used for model construction, and a 30% test set for holdout validation on observations. In detail, datasets of untreated CFRPs were used as reference, while CFRPs with CFs surface modification were included. Surface modification contained the introduction of oxygen species on the fiber surface, the attachment of monomers (sizing formulations), and the growth of CNTs, which are commonly adapted in literature [3][4][5][6][7][8]. Seed of random numbers was called to achieve a statistically unbiased estimation. Then, the statistical metrics were calculated and reported for each case.

Results and Statistical Performance of Model Grid Tuning, Undersampling and SMOTE Setup
A KNN classification model was initially used, as it is reported to be efficient in small datasets due to the use of Euclidean distance to construct the model [44]. However, as demonstrated in Table 2, the KNN model does not function well in regard to the imbalanced dataset. In this direction, SVM classification was employed in order to improve the metrics (Equations (1)-(4)) and resolve the issue of insufficient prediction efficiency of CF modified with PMAA and with oxygen species. Four different kernels were used in the analysis, namely sigmoidal, linear, radial, and polynomial kernel. The last was comparatively more descriptive. In all kernel cases studies, the model was optimized by grid tuning. The model input parameters values are, in the case of gamma, were tuned in-between 0.01, 0.05, 0.1, 0.2, 0.5, 1, 2, 4, 6, 8, and 10; and cost 1, 10, 50, 100, 200, 400, 1000, and 2000 in a total of 88 interactions. Polynomial kernel was tuned in two steps: firstly, optimal degree between 3, 4, 5, 6, 7, and Coef0 between 0.01, 0.1, 0.5, 1, 2, 4, 6, 8, 10, and 12 in a total of 50 interactions were selected. In the next step, gamma was tuned at values 0.01, 0.1, 0.5, 1, 4, while cost was ranging between 1, 10, 50, 100, 400, 1000, 2000 in a total of 30 interactions. The flow chart is summarized in Figure 3.

Results and Statistical Performance of Model Grid Tuning, Undersampling and SMOTE Setup
A KNN classification model was initially used, as it is reported to be efficient in small datasets due to the use of Euclidean distance to construct the model [44]. However, as demonstrated in Table  2, the KNN model does not function well in regard to the imbalanced dataset. In this direction, SVM classification was employed in order to improve the metrics (Equations (1)-(4)) and resolve the issue of insufficient prediction efficiency of CF modified with PMAA and with oxygen species. Four different kernels were used in the analysis, namely sigmoidal, linear, radial, and polynomial kernel. The last was comparatively more descriptive. In all kernel cases studies, the model was optimized by grid tuning. The model input parameters values are, in the case of gamma, were tuned in-between 0.01, 0.05, 0.1, 0.2, 0.5, 1, 2, 4, 6, 8, and 10; and cost 1, 10, 50, 100, 200, 400, 1000, and 2000 in a total of 88 interactions. Polynomial kernel was tuned in two steps: firstly, optimal degree between 3, 4, 5, 6, 7, and Coef0 between 0.01, 0.1, 0.5, 1, 2, 4, 6, 8, 10, and 12 in a total of 50 interactions were selected. In the next step, gamma was tuned at values 0.01, 0.1, 0.5, 1, 4, while cost was ranging between 1, 10, 50, 100, 400, 1000, 2000 in a total of 30 interactions. The flow chart is summarized in Figure 3.  Optimized SVM model with polynomial kernel was obtained at (degree, Coef0, gamma, cost): (3, 0.5, 0.01, 100) and results are presented in Table 3. The F1 values of CF_pmaa and oxygen_species are still considered low. Regarding the fact that this result is affected by the imbalanced dataset, undersampling methodology was used. The selected data population was random and representative of each class. The model was tuned at (degree, Coef0, gamma, cost): (4, 2, 0.1, 50) using a polynomial kernel.
The reduction of the data population to 144 observations lead to inability of this model to classify the mechanism of reinforcement in most cases (Table 4). This may be attributed to the undersampling risk of losing important information contained in the majority class [15], especially in small datasets. Consequently, it would be ideal if oversampling is performed and each class population reaches the number of majority class observations. However, lack of data in this case identifies SMOTE (selective oversampling of minority classes) as a more appropriate strategy to handle the imbalanced dataset. Synthetic data were produced by using package "synthpop." The results are reproducible and representative for each class (Figure 4), in order to create the new training dataset. The model was tuned at (degree, Coef0, gamma, cost): (6, 0.5, 0.1, 1). Afterwards, an original test set was used to evaluate the model functionality.   As it is evidenced by Table 5, SMOTE strategy was more appropriate to deal with the imbalanced dataset and achieve better metrics of SVM model. F1 score is improved regarding the overall performance and could enable the detection of an unknown dataset.

Model Validation
For each class of reinforcement, a representative sample was obtained in order to validate the classification model. In Table 6, it evidenced that the maximum counts recognize the type of reinforcement in unknown specimens and could be used as a criterion for the effective application of the trained polynomial kernel SVM model.

Conclusions
In this work, an effective methodology is presented for constructing the E r mapping of a tested CFRP by using k-means algorithm. The quantitative map and cluster centers are correlated to literature reports, while five more nanoindentation data categories were clustered, which can support nanohardness, maximum load mapping, etc. The most promising prediction of SVM was improved by tackling imbalance issues by using typical undersampling and SMOTE strategies. Undersampling did not work well for the specific dataset, contrary to SMOTE through which an accuracy of 64.1% was reached. Additionally, F1 score, which is generated by the contribution of recall and precision metrics, ranged between 42.1% and 73.3%. As a validation, for the functionality of the classification model, random datasets were tested in order to monitor classification ability. In all cases, the maximum number of classified nanoindentation events leads to the correct detection of mechanism of reinforcement, and consequently, F1 scores were identified as sufficient for the case of classification in the nanoindentation case.
The effect of modification of carbon fibers surface was examined; reinforcement of the interface was correlated to three ontological subclasses of reinforcement type class, namely the growth of fibrous nanotubes (high aspect ratio), the addition of oxygen functional groups on the graphitic crystal lattice of carbon fibers, and the electrodeposition of thin films using acrylic monomers. In this case, the structure-property relations are established using Machine Learning models in order to describe efficiently relevant cases of an interface reinforcement type class.