Multiphase Identification Through Automatic Classification from Large-Scale Nanoindentation Mapping Compared to an EBSD-Machine Learning Approach

Abstract

Characterising and quantifying complex multiphase steels is a challenging and time-consuming process, which is often open to subjectivity when based on image analysis of optical metallographic or SEM images. The properties of multiphase steels are highly sensitive to their individual phase properties and fractions, necessitating the development of robust characterisation tools. This paper presents a method for classifying nanoindentation maps into proportional fractions of up to five distinct microstructural regions in dual-phase and complex-phase steels. The phases/regions considered are ferrite, ferrite containing mobile dislocations, bainite, tempered martensite, and untempered martensite. A range of microstructures with varying fractions of phases were evaluated using both SEM/EBSD and nanoindentation. A machine learning (ML) approach applied to EBSD data showed good consistency in characterising a two-phase system. However, as the microstructural system complexity increased, variations were observed between different analysts and the sensitivity to the ML training data increased when four phases were present (reaching up to ~11% difference in the ferrite phase fraction determined). The proposed nanoindentation mapping technique does not show operator sensitivity and enables the quantification of additional microstructural features, such as identifying and quantifying ferrite regions with a high density of mobile dislocations and the degree of martensite tempering.

1. Introduction

Dual-phase (DP) and complex-phase (CP) steels are advanced high-strength materials engineered with a specific microstructure composed primarily of ferrite and martensite/bainite phases. These microstructures are achieved through controlled heat treatments and alloying additions to achieve high strength and good ductility. Commercially produced DP and CP steels often consist of a complex mixture of ferrite, upper and lower bainite, autotempered martensite, untempered martensite, and occasionally very small amounts of retained austenite. The retained austenite is often in the form of very thin films, whilst the ferrite and the secondary hard phase grain/phase size have sizes of around 4–10 μm [1,2,3]. The type, amount, and distribution of the different phases affects the strength, ductility, and formability of the final product [2,4].

Historically, etching and optical or electron microscopy micrographs have been the primary methods used for defining the “second phase” fraction in DP and CP steels, where the “second phase” is the total non-ferrite fraction [5]. Colour tint etching methods have been developed that can distinguishing between bainite and martensite in specific steels [6,7]. However, while these methods can visually distinguish phases, they often lack the precision required for more complex microstructural characterisations. Electron backscatter diffraction (EBSD) has emerged as a valuable tool for obtaining crystallographic data, offering greater insight into microstructural features. This technique allows for improved phase discrimination by capturing detailed crystallographic information. Nevertheless, EBSD cannot easily distinguish between BCC phases, such as ferrite, bainite, and martensite, due to their similar crystallographic structures [8]. To enhance phase identification, additional information such as crystallographic orientation, crystal defects, or pattern quality indices extracted from EBSD data has been utilised. Manual methods based on misorientation distributions or pattern quality indices have been developed to differentiate phases [9,10,11], but these require significant operator expertise and are inherently subjective.

Recent advancements in machine learning (ML) have enabled faster, more accurate, and less subjective phase classification in steel [8,12]. For example, Breumier et al. [12] applied ML algorithms to EBSD data, allowing automatic segmentation of ferrite, bainite, and martensite. It was reported that this approach demonstrated high accuracy and reduced subjectivity compared to conventional methods, but the labelling process for identifying different phases remained a challenge. To address this, aggregating labels from multiple experts was recommended to further decrease the subjectivity. For more detailed phase identification, transmission electron microscopy (TEM) provides finer resolution, for example, for carbide morphologies to identify tempered martensite compared to lower amounts of bainite, and the presence of thin film-retained austenite. However, TEM has limitations as it only offers localised measurements, posing challenges in quantifying bulk volume fractions of multi-component steels, which can be heterogenous over larger (many microns) scale, e.g., due to as-cast segregation.

As the different phases in DP and CP steels have different mechanical properties, hardness (or tensile) testing can be used to show the macroscopic difference in strength/ductility. Combining micro/nano hardness testing with electron microscopy has been used for DP steels to characterise the microstructure by Zhang et al. [13], where the sample was lightly etched after hardness testing and the indents were correlated with the specific phases present. This method enables the different phases to be identified using two methods. Once imaging and hardness is correlated using machine learning, large-area imaging allows a prediction of hardness to be mapped over much larger areas [14]. However, the initial correlation is time-consuming, and as samples need to be etched and each indent examined in the SEM and assigned to a phase, there is still some operator subjectivity when determining which phase an indent is sampling. With all hardness techniques, indent size and spacing selection are important. Consideration is required of the feature size, to ensure indents predominantly sample the feature only, and feature distribution, to determine indent spacing to ensure a representative volume is sampled, to minimise error. It is generally accepted that the features being detected should be >2.5× larger than the indention; if they are any smaller than this then interfaces/boundaries play a significant role on the hardness value [15]. There is also the trade-off between indentation size relative to the phase size and the difficulty in locating the ident after etching, with some nanoindentation methods using indents < 100 nm in size. Therefore, there is a need to identify alternate approaches that minimise operator subjectivity and can characterise the complex DP and CP microstructures.

2. Experimental and Materials

This study discusses the characterisation of DP and CP microstructures using EBSD imaging coupled with machine learning (EBSD-ML) phase identification used in the commercial AZtec 3.1 software and compares that to nanoindentation mapping coupled with a segmentation approach to identify separate feature distributions, considering accuracy and any subjectivity in the approaches.

For this study, cold-rolled steel with composition Fe-0.135C-1.85Mn-0.55Cr-0.25Si (wt%) was used. Samples of 1.2 × 10 × 160 mm were guillotined and heat-treated using a Gleeble HDS V40 thermo-mechanical simulator (Dynamic Systems Inc., Poestenkill, NY, USA), with typical continuous annealing profiles used to generate DP and CP microstructures. A typical profile for the heat treatment can be seen in Figure 1, where stage I is the heating stage and predominantly allows recrystallisation to take place prior to full or partial transformation to austenite. Stage II can be at an intercritical or supercritical temperature; in this work, both have been used to generate different microstructures. During the initial slower cooling in Stage III, ferrite is formed, giving the final ferrite fraction (either all formed during this cooling stage or additional to that retained due to the intercritical heat treatment). The faster cooling in Stage III results in the formation of a second phase (bainite or martensite). Stage IV is the overage stage and results in further second phase formation and tempering of any martensite that is present. The cooling during Stage V can result in any remaining austenite forming untempered martensite. The composition selected does not result in any retained austenite being present.

Identification of the microstructural phases present was carried out using SEM/EBSD analysis in a JEOL JSM-7800F (Tokyo, Japan) with an Oxford Instruments Symmetry S2 detector (Abingdon, UK) using samples prepared to a 0.05 µm finish (the same preparation as the nanoindentation samples). Microhardness indents were used to create a 500 × 500 µm region on each sample to ensure both SEM, and later, nanoindentation testing was carried out in the same location. For each sample, a single area was then characterised. This classification of the phases from the EBSD images was carried out using a specific post-processing routine in AZtecCrystal 3.1 software [16]. A combination of band contrast, band slope and pattern quality was employed for distinguishing the phases. This method requires phases to be initially manually identified by an operator based on the mentioned indices from the EBSD maps before automatic recognition via the ML method. This approach reduces ongoing subjectivity compared to full user manual identification; however, the initial training of the data is dependent on the quality of the initial decisions. The initial identification is more challenging when characterising a steel that contains ferrite, upper/lower bainite, as well as martensite, where selecting “representative” regions for each of these phases for training is difficult and time consuming, particularly if added complexities of spatial heterogeneity due to segregation during casting needs to be considered to ensure sufficient representative areas are characterised.

The nanohardness classification method uses a NanoTest Xtreme (from Micro Materials Ltd., Wrexham, UK) where 1000 indents, separated by 10 µm distance, in a grid of 100 by 10, were performed for each microstructure condition using samples polished to a 0.05 µm finish using a vibratory polisher (VibroMet 2—Buehler Ltd., Lake Bluff, IL, USA).The spacing was selected to be larger than the microstructural band spacing in this material caused by micro-segregation (6 µm) and ensuring no interaction between neighbouring indents [17]. A rolling analysis of the data showed the average nanohardness value to be stable after 500 data points, suggesting that 1000 is sufficient to represent the area being assessed. This number and distribution of indents also ensured a similar area of microstructure was mapped both by SEM/EBSD and nanoindentation. The nanohardness measurement was conducted in force-controlled mode applying a force of 1 mN. The typical pyramidal indentation sizes were around 50–120 nm in diagonal length and 20–45 nm deep. It has been reported that the area of influence of each indent equates to approximately ×10–15 the indent depth [17], therefore in this instance this is 0.3–0.6 µm.

The nanohardness data was processed using an optimisation classification conducted in Matlab 2023a. A maximum of 5 distinct microstructural regions were allowed in the classification index, representing: ferrite; ferrite boundaries and ferrite containing mobile dislocations; bainite; tempered martensite; and untempered martensite. A separate classification for the grain boundary/mobile dislocation containing ferrite region was used as literature has highlighted that the presence of geometrically necessary dislocations (GNDs) close to the interface between ferrite and the second phase can locally increase the flow stress of the material [14] and affect the macroscopic mechanical properties by interface voiding [18]. These GNDs have been reported up to 1 µm from a boundary or interface [19]. For this study, bainite was classified as a single phase, although typically it can be categorised as upper, lower, or granular bainite; however, as the hardness between these phases is a continuum it is difficult to differentiate between them [20].

The nano-Hv classification process involved an iterative optimisation loop, where the 5 different classifications were characterised based on normal distributions, each characterised and optimised to an independent mean (µ) and standard deviation (σ). To find the fitted values for µ_1–5, σ_1–5, and the relative area fractions (Af_1–5) of these distributions, a root mean square (RMS) method was employed against the probability density function (PDF) of the experimental data. This systematically varied each of these parameters whilst also reducing the number of phases should <5 distributions be detected. Convergence/average error in all cases fell below 5%. A normal distribution was used curve fitting for this study, as the majority of the variation in the hardness within a phase can be attributed to local variations in composition, with the below average hardness values falling in solute-poor regions and the high hardness from the interdendritic solute-rich regions. A lognormal distribution was trialled, but resulted in a lower quality of fit (RMS).

3. Results and Discussion

3.1. Case Study 1: Tempered Dual-Phase Steel

In the first case study, the EBSD-ML method was applied to characterise a DP steel with simple ferrite plus martensite microstructures (approximately 65% ferrite obtained by intercritical heating), where the martensite was in the as-quenched (Stage III involved quenching to room temperature and avoiding Stage IV) or tempered condition (320 °C for 20 min treatment applied after quenching, which is below the martensite finish temperature, Mf, for this material). The band contrast and phase-classified maps of the untempered microstructure are shown in Figure 2a and Figure 2b, respectively. To assess the user subjectivity of the initial classification, training was carried out by three skilled metallurgists knowledgeable on steel microstructures. All researchers assessed the same image and therefore this study assesses the variability due to subjectivity rather than the accuracy of measuring the “true” area fraction. The error between the classification of the different individuals is shown by the value in brackets in Figure 2f. As these were experienced users, the repeatability for each individual was <0.5% for three repeat assessments of the same image.

The classification in the untempered condition was based on a binary phase classification (ferrite and martensite). The fraction of phases is given in Figure 2f, which shows 65.1% of ferrite in the untempered condition. For each sample, three images were used for the phase fraction measurement. The difference in the fraction of the phases measured by different researchers was less than 1%, indicating the low subjectivity of the ML method for the untempered condition. For the classification of phases after tempering, two different approaches were used. The initial approach employed the same binary classification database developed for the untempered microstructure, which revealed an increased fraction of ferrite of 67.5% compared to the 65.1% in the untempered condition (Figure 2f). As the fraction of the ferrite phase should not change due to tempering, this marginally higher fraction is accompanied by a higher error in measurement, suggesting classification was more challenging in this microstructure. Figure 2c shows the band contrast map of the DP microstructure after tempering. Significant variation in the band contrast map can be observed in the martensite regions, indicating that different amounts of tempering occurred within the different martensite areas. The differences in tempering response within martensite can be attributed to localised variations in the chemical composition of elements, particularly carbon [21]. Consequently, areas of martensite with reduced band contrast (and band slope) were misclassified as ferrite, as shown in Figure 2d (shown by a white arrow). This misclassification suggests reduced accuracy when using a binary classification for tempered DP steels where subtle changes in microstructure may not be distinctly recognised. Furthermore, a significant difference in the ferrite phase fraction (7.3%) was measured by the different users showing the high subjectivity of this approach for the tempered condition.

The second classification approach introduced a third category to account for regions of intermediate contrast observed post-tempering. The resultant classified image, displayed in Figure 2e, highlights the regions of intermediate contrast in green, identifying this as tempered martensite. According to this trinary classification, the microstructure consists of approximately 24.1% tempered martensite, 13.7% untempered martensite, and 62.2% ferrite. Hence, for the tempered condition, the binary classification appears to overestimate the ferrite content, possibly misidentifying some more heavily tempered martensite as ferrite, whereas the trinary classification appears to underestimate ferrite by classifying some ferrite regions as tempered martensite. The untempered martensite in this sample likely occurred in areas of reduced tempering; however, it has similar band contrast characteristics to that of the untempered martensite in the untempered sample. Such discrepancies highlight the challenges of applying the EBSD-ML method for phase identification in complex microstructures and emphasise the need for careful calibration and validation of classification algorithms to improve the accuracy of the method. Regarding the subjectivity of the second approach (trinary classification), while a smaller difference in phase fraction (maximum difference of 4.3%) is observed for the different operators, compared to the binary classification, it is still significant.

Figure 3 shows the nanoindentation classification for the untempered and tempered microstructures seen in Figure 2. In the as-quenched condition (Figure 3a), the nanoindentation hardness distribution is dominated by the peak representing ferrite boundaries and ferrite with mobile dislocations from the martensitic transformation at 2.1 GPa. As the ferrite phase containing mobile dislocations can extend up to 1 µm from each ferrite/martensite interface [19] then in a microstructure with approximately 35% martensite present as small regions (an approximate typical martensite island size of 20 µm), the ferrite with mobile dislocations can constitute a considerable portion of the overall ferrite phase. A second large peak occurs in the region of 7.5 GPa, correlated with untempered martensite. Minor peaks corresponding to bainite and autotempered martensite (which will form on cooling) are also present. A current limitation of the EBSD-ML method is that only two or three phases can be segmented due to the similarity between phase characteristics (for example, lower bainite and tempered martensite cannot be readily distinguished and granular/upper bainite is often mis-indexed as ferrite). After tempering (Figure 3b), it is evident that two major changes occur. Firstly, the untempered martensite hardness drops to 5.8 GPa, showing a significant tempering affect. Secondly, the ferrite with a mobile dislocations peak has shifted to a lower hardness value, suggesting recovery has taken place, reducing the dislocation density and redistributing remaining dislocations throughout the ferrite more uniformly. This has also increased the hardness of the “ferrite” peak seen in Figure 3a, which would have been the initially soft ferrite regions in the centre of ferrite grains, thereby resulting in only one ferrite peak. The bainite peak in both cases remains relatively unaffected, as expected, as short tempering times has little effect on bainite hardness [22]. The quantified phase fractions, determined based on the relative area under the peaks, are given in Figure 2c.

Comparing the quantification via EBSD-ML (Figure 2f) and via nanohardness (Figure 2c), there are a couple of significant differences. Firstly, as the untempered sample was then tempered, the ferrite fraction between these two samples should be consistent. The EBSD-ML method showed a greater variability between the samples (5.3% based on the average values of the different users in the trinary classification) compared to the nanohardness method where only 1% difference was seen for ferrite fraction (summed ferrite and ferrite + dislocations) between the tempered and untempered conditions. This reduction in variability suggests that hardness gives a better indication of phase type. Secondly, the nanoindentation method did not identify untempered martensite after tempering. Whilst the rate of tempering can vary due to compositional inhomogeneity, some tempering is expected for all martensite regions in this sample. This is shown in the nanohardness results where the hardness of the martensite drops from 7.5 to 5.8 GPa, but there is still a single distribution for martensite. However, SEM/EBSD-ML still shows the presence of untempered martensite when using a ternary classification. This shows that the user is relied upon to define the correct number of phases before the analysis.

3.2. Case Study 2: Complex-Phase Steel

The second case study examines the variability/subjectivity induced when assessing a complex phase material (i.e., >3 phases). To generate a CP microstructure, the Zone III cooling rate (Figure 1) was set to 30 °C/s and Zone IV used an overage at 325 °C for 7 min. This heat treatment produced a complex microstructure with ferrite, bainite, tempered martensite, and untempered martensite. To assess the variability/subjectivity in characterisation of these complex systems, two users trained the EBSD-ML to assess the same image. Figure 4 shows the EBSD images with user-identified phases and the quantified phase percentages. It can be seen that there are significant differences in the amount of ferrite and tempered martensite quantified.

For this case study, the subjectivity in phase identification increased with phase complexity. For a simple two-phase system, the variability between users was <1%, which increased to 4% in a three-phase system. In the four-phase system, this increased to 11%. The results suggest that the EBSD-ML method can offer repeatable characterisation in binary systems where the two phases are clearly distinct, as in the case of ferrite and martensite, but obtaining non-subjective consistent quantification is challenging for three- and four-phase systems.

Figure 5 presents the nanoindentation map for the sample from case study 2, displaying peaks associated with five classifications: ferrite, ferrite boundaries/mobile dislocations, bainite, tempered martensite, and untempered martensite. The method for producing these peaks in Matlab 2023a is highly repeatable and the same optimised distribution was obtained after running the optimisation 10 times on the same data. As there are no user inputs, this provides an independent method for characterisation of the different phases present and therefore consistency in measurement. Some discrepancies in phase percentages determined can be seen between the SEM/EBSD-ML and nanoindentation method, which mainly arises from the ferrite quantification and the significant amount of ferrite boundaries/ferrite with dislocations identified by nanohardness. As band contrast and band slope are used by SEM/EBSD to define the phase, distinguishing between the bainitic ferrite and ferrite-containing mobile dislocations is difficult, as both can show a similar response due to the internal misorientation from dislocations. Therefore, the different characterisation methods give different absolute values, but it has been found that the nanoindentation approach provides greater consistency in measurements.

Although the method introduced for quantifying phase fractions was implemented on a DP steel grade, it was also evaluated under different conditions, including those involving complex phase microstructures. This demonstrates that the proposed approach is applicable to a broad range of steel grades with complex microstructures comprising mixtures of ferrite, bainite, martensite, and even retained austenite. The only essential requirement for the application of this method is that the constituent phases exhibit distinguishable hardness values.

Care must be taken to select an appropriate indent size relative to the phase size in the microstructure. In microstructures with fine features, a smaller indent size is necessary to ensure that indents are confined within individual phases.

Nanohardness distributions provide valuable insights into both the phase fractions and the mechanical strength of individual phases. However, the current method does not account for the spatial distribution of phases, which limits its ability to predict the overall mechanical strength of the material. Since phase distribution can significantly influence the bulk mechanical properties, this remains a notable limitation.

To advance the method, reducing the spacing between indents in the nanohardness grid could enable mapping of the phase distribution within the microstructure. By associating each indent with a specific phase based on its hardness value, it becomes feasible to generate a spatially resolved phase map. Implementing this refinement would allow for the estimation of the overall strength of steel grades using nanohardness measurements.

4. Conclusions

In this study, the use of nanoindentation classification for phase identification in DP and CP steels, using Matlab 2023a optimisation to determine individual phase nanohardness distributions, has been compared to EBSD-ML phase identification. The EBSD-ML approach showed subjectivity to users, due to the requirement for initial manual phase identification for the ML training, giving variability in the measured phase percentages, whereas the nanoindentation method demonstrated less subjectivity. The EBSD-ML method showed variations between individual assessments of around 1% when a two-phase ferrite and martensite microstructure was assessed, with the discrepancy increasing to 11% when evaluating complex microstructures with four phases (ferrite, bainite, tempered martensite, and untempered martensite). Such discrepancies highlight the challenges of applying EBSD-based phase identification in complex microstructures and emphasise the need for careful calibration and validation of classification algorithms to improve the accuracy and reliability of the method. Nanoindentation was able to distinguish an additional region in the microstructures—that of ferrite grain boundaries/ferrite with mobile dislocations adjacent to the phase boundaries—as well as readily distinguishing bainite and tempered martensite which was more challenging with EBSD-ML using band contrast and band slope. It is proposed that nanoindentation provides a method that can yield results with low subjectivity and operator dependence.