Deep Convolutional Neural Network applied on copper surface coatings based on polyvinyl alcohol and silver nanoparticles

In order to design effective protective coatings against corrosion, the polyvinyl alcohol (PVA) as compound and composite with silver nanoparticles (nAg/PVA) were electrodeposited on copper surface employing electrochemical techniques such as linear potentiometry and cyclic voltammetry. A new paradigm was used to distinguish the features of coatings, i.e. a Deep Convolutional Neural Network (CNN) was implemented to automatically and hierarchically extract the discriminative characteristics from the information given by optical microscopy images. The main arguments that invoke a CNN implementation in the surface science of materials are the following: artificial intelligence techniques can be successfully applied to learn differences between surface coatings; based on their popularity for image processing, CNN can model images related to the problem of coatings; deep learning is able to extract the features that are distinguishable between material surfaces. To provide an overview of the copper surface, CNN was applied on microscope slides (CNN@microscopy) and inherently learnt distinctive characteristics for each class of surface morphology. The material surface morphology controlled by CNN without the interference of the human factor was successfully conducted, in our study, to extract the similarities/differences between unprotected and protected surfaces to establish the PVA and nAg/PVA performance to retard the copper corrosion.


Introduction
The corrosion consequences are revealed by the short lifetime of metals/alloys due to their degradation as well as the release of corrosion products causing environmental pollution.The effects can be delayed by assembling protective layers at the metal-environment interface; one of the easiest and effective ways involves the adsorption of corrosion inhibitors on the metallic surfaces [1][2][3][4][5][6][7][8][9][10][11].The analysis of metal/alloy corrosion, under different inhibitors, has been investigated solely for steel by the earlier popular models of natural computation of artificial neural networks (ANN), however only on numerical data, e.g.quantum chemical calculations [12] and electrochemical impedance spectroscopy curves [13].
Deep learning (DL) has recently emerged within computer science both as a hot topic and a top performing paradigm for practical scenarios, supported by large amounts of data and high power of computation.One of its most exciting and promising areas of application is represented by image interpretation.Among the imaging tasks, analysis of microscopy slides is one essential subject of DL.In the last couple of years, there have been a substantial number of literature entries on the topic, for the different fields where microscopy plays a critical role.The DL techniques typically tailored for such imaging data are mainly the convolutional neural networks (CNN) with their essential automatic feature extraction capability, which makes them even more valuable for an artificial learning independent of the human expert.
Several DL models have been recently proposed for optical microscopy data and they are subsequently reviewed after the real-world domain from which the microscopy images were drawn.A field receiving special attention is medicine.For example, microscopic histopathological slides have been lately scanned by DL for supporting cancer diagnosis of different types, e.g.located at breast level [14,15] or at the colon region [16,17].Regions expressing molecular biomarkers of immunohistochemistry are detected also by a DL network in [18].An overview on recent DL models for landmark detection, segmentation, diagnosis and prognosis from microscopy biomedical images is conducted in [19].Biology is clearly another equally important domain where microscopy and prediction are extremely valuable.DL for bioinformatics has targeted the digital reconstruction of 3D neurons by CNN segmentation of neuronal microscopic images [20], the segmentation and classification of populations of cells [21], the characterization of subvisible particles in protein formulations in pharmaceutical drugs [22] and microbial parasite examination by DL segmentation [23].Turning to physical sciences, physical and chemical information from atomically resolved scanning transmission electron microscopic images has also been extracted with the help of deep learning [24].
The ANN (shallow) predecessors have been likewise proposed for the investigation of microscopy images in materials science [25].But the feature complexity of real-world images and the need for their automatic representation make DL a more performing candidate for the task.
The current work therefore puts forward a CNN model for the examination of the effect of uninhibited and inhibited hydrochloric acid solution on the copper surface.The polymeric films based on poly(vinyl) alcohol (PVA) and poly(vinyl) alcohol with silver nanoparticles (nAg/PVA) were discussed according to optical microscopy.By microscopy a limited optical image is designed at a fixed plane providing information related to a narrow area.To extend the optical image information to the whole surface, a CNN architecture was employed to accurately distinguish between copper standard state and copper surface changes after linear potentiometry (LP) and cyclic voltammetry (CV) in: (i) 0.1 mol L -1 HCl blank solution; (ii) 0.1 mol L -1 HCl solution containing 0.1% PVA; (iii) 0.1 mol L -1 HCl solution containing 0.1% PVA and nAg.

Modeling of coatings
The linear potentiometry (LP) and cyclic voltammetry (CV), applying ten cycles, were carried out, with the potential scan rate of 1.0 mV s -1 and 100 mV s -1 , respectively, in the potential range of -1.0 V and 1.0 V, for copper specimens immersed in 0.1 mol L -1 HCl solution without and with PVA and PVA/nAg, stated as follows: (1) electrodeposited coating on standard copper surface through LP in: (1a) 0.1 mol L -1 HCl; (1b) 0.1 mol L -1 HCl solution containing PVA; (1c) 0.1 mol L -1 HCl solution containing nAg/PVA prepared as in the previous study was described [26]; (2) electrodeposited coating on standard copper surface through CV under the same conditions mentioned above.The materials, methodology and devices were described in another study [11,26].

Convolutional Neural Networks for Copper Slide Interpretation
The present study appoints a CNN for the analysis of copper slides.CNN are able to perform both feature extraction and classification through their unique architecture.First of all, convolutional layers gradually learn feature representations from low-level to high-level structures in an automatic manner from the training images.Subsequently, the pattern of input-output mapping is discovered in the secondary classification step.
The architecture proposed for the problem at hand is presented in Fig. 1.The designed flow and parameters had been experimentally chosen such as to counteract the danger of overfitting induced by the small sample size (453 examples), i.e. overspecialization on the training samples and inability to generalize on new instances.The input samples (volumes) of 150 x 150 x 3 (denoting image width and height in pixels and number of color channels) enter through two convolutional layers, the first with kernel size (KS) 7 and kernel depth (KD) 8, while the second with KS = 5 and KD = 16.Every formed filter KS x KS is convolved with the image and the resulting KD activation maps are stacked in a volume for the next layer [28].Each convolution is followed by Rectified Linear Unit (ReLU) for nonlinearity and Max-pooling (window size 2 x 2) for downsampling the representation.Subsequently, the information passes through two fully connected layers.Dropout is employed between all layers (with rate 0.25 after the convolutional ones and 0.9 after the first fully connected one) and cuts a subset of activations from the preceding layer.The last fully connected layer outputs for every copper slide under examination the scores (probabilities) for each possible labeling state.

Characterization of coatings by optical microscopy
The optical microscopy images presented in Fig. 2 show the copper surface morphology before (Fig. 2a) and after LP and CV in 0.1 mol L -1 HCl solution (Fig. 2b) and in 0.1 mol L -1 HCl solution containing PVA and nAg/PVA as corrosion inhibitors (Figs.2c-d).
The standard sample reveals a characteristic morphology of copper surface before corrosion.(Fig. 2a).As shown in Fig. 2b, after LP and CV, an irregular coating can be observed on the copper surface with corrosion spots randomly spread over the surface, especially after CV.In the presence of PVA the copper surface morphology is completely changed and the feature of another surface layer is obvious, after both electrochemical measurements (Fig. 2c-LP, CV), but more nuanced after CV (Fig. 2c-CV).The addition of nAg/PVA inhibitor in 0.1 mol L -1 HCl solution leads to the occurrence of a smoother layer adsorbed on the copper surface after LP, without the silver nanoparticles highlighted (Fig. 2d-LP).After CV, the surface morphology is completely changed due to the nAg connection to the PVA macromolecular chain, the surface layer feature being revealed by the presence of silver nanoparticles deposits unevenly distributed over the surface (Fig. 2d-CV).
Microscopy images show the morphology on a very narrow surface area and the comments traditionally target carefully selected images according to predicted or expected changes and appearance.As images in Fig. 2b-d show (first row), after LP the optical microscopy images reveal a confusing surface morphology for the layers obtained in acidic solutions without and with inhibitors.
Thus, it is necessary to compare in an objective manner a multitude of microscopic slides obtained for copper behavior in HCl solution, uninhibited and inhibited with PVA and nAg/PVA, during LP.In order to achieve the purpose, the CNN method compares the microscope slides of the metal surfaces unmodified and modified under various conditions to measure how well the different categories are distinguished.

Experimental Setup
For each classification sub-problem in turn, the data was randomly split into 67% samples for training and the rest for testing, maintaining the proportion between the classes.The CNN learns the specificities from the images of each class from the training set and then applies the knowledge to distinguish the previously unseen images from the test set.
The total number of samples is 453 and the distribution over the 7 classes is relatively balanced: from a minimum of 12% for the CVnAg/PVA up to a maximum of 17% for all slides of the class LPuninhib.As some classes of images appear in several of the classification sub-tasks mentioned above, the same split is maintained for each sub-problem.All images are resized to 150x150 pixels within the current experiment, due to computer memory limitations triggered by the use of CNN.
Since the data set has a reduced size as compared to the usual applications of CNN, several measures are taken with the purpose of increasing the number of samples and thus reducing overfitting.The main concern comes from the reduced size of the training set, as the model is not exposed to sufficient slides and therefore is not able to generalize well.To overcome this drawback, data augmentation is utilized in order to generate new images that suffer various random transformations.Accordingly, the CNN does not meet the same image twice.Data augmentation in the current experiment includes shearing transformation, zoom range and horizontal flip.For the same purpose of limiting overfitting, dropout follows each convolutional layer and is also added just before the final dense layer.In addition, a limited number of convolutional layers ( 2) is considered and L1 and L2 weight regularization (0.01) is utilized, forcing the weights to take smaller values in order to simplify the model.The number of epochs for learning the weights is empirically set to 50.An epoch has passed when all the data make a forward and backward pass through the network.In order to increase training speed, the data is split into batches.A batch size of 16 samples is considered.

CNN results and visualization
The CNN generally proves to be very efficient in distinguishing between classes, reaching perfect classification in four out of the six problems enumerated in subsection 2.1.
The model completely discerns the electrodeposited coating assembled by CV by that reached by LP.It also differentiates between the CV electrodeposited coating and the metal in its standard form.Also, the two inhibitors are recognized when the coating is induced by CV.
The results for the case studies StandardLP and LPPVA ─ LPnAg/PVA, where the model has misclassified samples, are illustrated in Table 1, with deeper per class analysis in Table 2 for the former.

Discussion
The obtained classification accuracies for all 6 subsets of data illustrate that there is a high resemblance between LPPVA and LPnAg/PVA.Two LPPVA samples are misclassified for LPnAg/PVA within the StandardLP task, as observed in the confusion matrix from Table 1 and further illustrated in Fig. 3.The same misclassifications lead to a reduced sensitivity rate in how these samples are confused by the model, they can be identified in Fig. 4, third row, as samples 5 and 15.As observed, in both situations, the probabilities of assigning the samples to the reference class, i.e.LPPVA, are relatively high, of 0.39 and 0.23, respectively.The interpretation is that the two samples have characteristics that are common to both categories and the distinction between them is difficult.This observation is sustained by the fact seen in the same third plot from Fig. 4 that, for other samples, even if the decision was the correct one, the probabilities of assigning the samples to LPnAg/PVA is still very high: this can be especially seen for slides 1-6, 13, 14, but also for 8, 9, 11 or 12.
There is a clear demarcation between the slides of the standard copper and the rest of the categories.The observation is underlined in the first row of Table 2, but it can be also noticed in the first plot of Fig. 4, where the probabilities of assigning the samples to the actual class are equal to 1 in all situations.As concerns the LPuninhib samples, they are usually very distinct as opposed to the other classes, with only two exceptions, samples 4 and 6 in the second plot from Fig. 4, when they are found similar to LPPVA.In one of the two cases (sample 6, also illustrated in the first image from Fig. 3) the decision even favors the latter.LPnAg/PVA samples are in most of the cases well detected, with minor resemblances to LPPVA and one sample (7 in the fourth plot from Fig. 4) even confused to the LPPVA.
The specificity (Table 2) illustrates that most of the times when the samples are misclassified, they are put in either LPPVA or LPnAg/PVA, and less often in LPuninhib.Precision, F1 and balanced accuracy also underline that the categories that are more difficult to distinguish are LPPVA and LPnAg/PVA.
An interesting aspect is that LPPVA is well distinguished in the LPPVAnAg/PVA binary classification task and the only sample that is misclassified is the last slide from Fig. 3. Clearly, when having four classes, the task of the CNN is more complex and this leads to confusion in interpreting the LPPVA slides, while for binary classification they are easier to separate.

Important data derived from CNN for materials science regarding the corrosion problem
a. Type of corrosion can be determined by comparative examination of the slides acquired for the standard metal/alloy and those of the sample exposed to corrosive media without inhibitors.Thus, in our study, the slides of the surface fractions attained for the corroded sample in the acid environment without inhibitors were not confused with those of the reference sample, indicating that the surface was totally affected and generalized corrosion was induced on the copper surface.b.Surface coverage degree or the coated surface fraction with the protective layer formed by the adsorption of some organic compounds may be estimated by comparison between the slides obtained for copper surface modified in inhibited acid media with those acquired in uninhibited acid environment.Particularly, under the conditions mentioned in our study, in the presence of PVA, some similar slides to those of the uninhibited sample were found, showing that there are unprotected surface fractions.CNN applied on microscopy slides identifies minor similarities between uninhibited and inhibited samples, meaning that the surface coverage degree reaches a high level.c.CNN can extract the peculiarities of surface layers formed by the adsorption of distinct systems.As shown in our study, in most of the cases, LPnAg/PVA samples were well delimited; minor similarities with LPPVA which were detected suggest that the silver nanoparticles were not evenly distributed in the PVA film.d.CNN accurately distinguishes differences or similarities between the coatings presenting the same surface chemistry but assembled by different methods.

Conclusions
The optical microscopy images designed a different morphology of copper surfaces after CV and LP, but PVA and nAg/PVA coatings obtained during linear potentiometry are easily confused.
The problem has been resolved using CNN method applied on microscopic slides showing that LPPVA was well distinguished in the LPPVA/nAg binary classification task in proportion of 97.5%.
The study therefore showed that the novel popular deep learning computational paradigm can be successfully used to complement the standard means for investigating metal/alloy corrosion inhibition processes by objective multiple slide examination and additional estimation of affected/protected surface area.

Figure 1 .
Figure 1.The architecture of the CNN for inspecting copper slides.Feature representation is achieved by the flow of convolutional layers with ReLU nonlinearity and Max-Pooling.Classification is performed by the two fully-connected layers and leads to scores for each of the copper states.Dropout is used in-between layers to further prevent overfitting.

PreprintsFigure 2 .
Figure 2. The optical microscopy images of copper surface (x80): a -before corrosion; b, c, d -after LP and CV (ten cycles) in 0.1 mol L -1 HCl solution and 0.1 mol L -1 HCl solution containing PVA and nAg/PVA as corrosion inhibitors.

Fig. 3 Figure 3 .
Fig. 3 shows the samples that are misclassified by the model.Fig. 4 provides more insight about the decision of the model regarding the belonging to a specific class in the form of probabilities for each test sample for the StandardLP task.

Figure 4 .
Figure 4. Classification probabilities for every sample from the test set, when considering each class in turn.From top to bottom, the classes represented are Standard copper, LPuninhib, LPPVA and LPnAg/PVA.

Table 1 .
Prediction accuracy and confusion matrix for the more complex classification tasks.

Table 2 .
Various statistical measures for the performance obtained by each class for StandardLP task.