Identifying Minerals from Image Using Out-of-Distribution Artificial Intelligence-Based Model

: Deep learning has increasingly been used to identify minerals. However, deep learning can only be used to identify minerals within the distribution of the training set, while any mineral outside the spectrum of the training set is inevitably categorized erroneously within a predetermined class from the training set. To solve this problem, this study introduces the approach that combines a One-Class Support Vector Machine (OCSVM) with the ResNet architecture for out-of-distribution mineral detection. Initially, ResNet undergoes training using a training set comprising well-defined minerals. Subsequently, the first two layers obtained from the trained ResNet are employed to extract the discriminative features of the mineral under consideration. These extracted mineral features then become the input for OCSVM. When OCSVM discerns the mineral in the training set’s distribution, it triggers the subsequent layers within the trained ResNet, facilitating the accurate classification of the mineral into one of the predefined categories encompassing the known minerals. In the event that OCSVM identifies a mineral outside of the training set’s distribution, it is categorized as an unclassified or ‘unknown’ mineral. Empirical results substantiate the method’s capability to identify out-of-distribution minerals while concurrently maintaining a commendably high accuracy rate for the classification of the 36 in-distribution minerals.


Introduction
Rocks serve as foundational constituents of the Earth and record the evolutionary narrative of our planet.They play a pivotal role within the multidisciplinary realm of Earth sciences.As rocks are composed of a variety of minerals, the accurate identification of these minerals is of supreme importance [1,2].Traditional mineral identification techniques primarily rely on visual observation of physical properties like shape, color, and texture, but their precision is dependent upon the expertise of the observer [1,2].Alternatively, methods such as chemical analysis, X-ray diffraction analysis, differential thermal analysis, and polarizing microscope analysis offer enhanced accuracy in mineral identification.However, these methods are expensive, take a long time to execute, and can cause sample damage [3][4][5][6][7].In contrast to these resource-intensive approaches, the acquisition of mineral images is a convenient, efficient, and cost-effective avenue for analysis.Consequently, an increasing body of research has begun to pivot towards mineral identification using image-based techniques.In particular, numerous studies have harnessed the potential of deep learning to identify minerals from images, yielding commendable results [3][4][5][6][7].However, it is crucial to underscore the intrinsic limitation of traditional deep learning methodologies, as they can exclusively identify minerals within the scope of the training dataset's distribution.Any mineral falling beyond the confines of this distribution is erroneously categorized into one of the predefined classes from the training datasets-an evident and undesirable misclassification.This limitation is exacerbated by the extensive diversity comprising more than 6000 known mineral categories worldwide [1], rendering it impractical to encompass all of them within the training datasets.Minerals that fall outside the scope of the training datasets necessitate distinct methods for isolation and identification, such as manual or instrumental techniques.
Current strategies addressing the inherent limitation of conventional deep learning models, specifically their ability to exclusively recognize in-distribution (ID) categories from the training set, encompass an array of techniques, including out-of-distribution (OOD) detection [8], uncertainty estimation [9], semi-supervised learning [10], and generative models [11].Notably, OOD detection methodologies have emerged as particularly reliable, providing accurate predictions for samples outside the training set distribution, and solely necessitating in-distribution data for training [12][13][14][15][16].
Exemplifying the efficacy of OOD detection, Jiang et al. [17] adeptly employed this technique to discern between known and unknown instances of plant diseases.Similarly, Saadati et al. [18] conducted OOD detection to support the robustness of insect classification models.Furthermore, the utility of OOD detection extends beyond these domains, showcasing notable promise in areas of medical image diagnosis [19], network security [20], and quality control [21].In light of these compelling precedents, it becomes evident that the isolation and identification of out-of-distribution minerals require specific attention.The main contributions of this paper are as follows:

Datasets
In this study, we collect a comprehensive dataset of 183,688 mineral images, encompassing 36 distinct categories of common minerals, as detailed in Table 1.These are the same images as those presented in the work of Zeng et al. [6] and Wu et al. [3] and were sourced from a reputable repository of mineral data, Mindat.org[22].Notably, the dataset is divided into training, validation, and testing subsets, each allocated in a ratio of 8:1:1, respectively.Four samples of fluorite are shown in Figure 1.In addition to the in-distribution dataset, a separate collection of 18,368 mineral images is acquired.These images correspond to 15 categories of minerals, as shown in Table 2, and were acquired from the same authoritative source (Mindat.org).This auxiliary dataset, representative of out-of-distribution minerals, has been assembled to assess the model's ability to recognize and distinguish mineral types beyond the scope of the training set.Some of the images of the out-of-distribution minerals are shown in Figure 2.

Methodology
The methodology employed for mineral identification is illustrated in Figure 3.To discern minerals that fall outside the established set of 36 known minerals, One-Class Support Vector Machines (OCSVM) is leveraged for out-of-distribution (OOD) detection.Similar to the techniques outlined in previous works [23][24][25], the process is initiated by ob-

Methodology
The methodology employed for mineral identification is illustrated in Figure 3.To discern minerals that fall outside the established set of 36 known minerals, One-Class Support Vector Machines (OCSVM) is leveraged for out-of-distribution (OOD) detection.Similar to the techniques outlined in previous works [23][24][25], the process is initiated by obtaining a feature extraction from the mineral image, with the intent of refining and augmenting the efficacy of OCSVM [23][24][25].Crucially, a Deep Neural Network (DNN) is integrated into our model for the extraction of mineral-specific features.The DNN is trained on the training set, which comprises the 36 recognized mineral categories shown in Table 1.Subsequently, OCSVM is deployed, with the mineral features derived from the initial layers of the DNN serving as input.This pivotal step serves to ascertain whether the mineral in question pertains to the in-distribution category of the 36 known minerals or falls into the out-of-distribution category.Upon OCSVM's determination that the mineral is classified as out of distribution, the model promptly halts and apprises the user that the input image represents an unknown mineral.In contrast, when OCSVM identifies the mineral as in-distribution, the model seamlessly proceeds to deploy the remaining layers of the DNN to apprise the user of the specific known mineral category to which the input image belongs.

Methodology
The methodology employed for mineral identification is illustrated in Figure 3.To discern minerals that fall outside the established set of 36 known minerals, One-Class Support Vector Machines (OCSVM) is leveraged for out-of-distribution (OOD) detection.Similar to the techniques outlined in previous works [23][24][25], the process is initiated by obtaining a feature extraction from the mineral image, with the intent of refining and augmenting the efficacy of OCSVM [23][24][25].Crucially, a Deep Neural Network (DNN) is integrated into our model for the extraction of mineral-specific features.The DNN is trained on the training set, which comprises the 36 recognized mineral categories shown in Table 1.Subsequently, OCSVM is deployed, with the mineral features derived from the initial layers of the DNN serving as input.This pivotal step serves to ascertain whether the mineral in question pertains to the in-distribution category of the 36 known minerals or falls into the out-of-distribution category.Upon OCSVM's determination that the mineral is classified as out of distribution, the model promptly halts and apprises the user that the input image represents an unknown mineral.In contrast, when OCSVM identifies the mineral as in-distribution, the model seamlessly proceeds to deploy the remaining layers of the DNN to apprise the user of the specific known mineral category to which the input image belongs.

Mineral Feature Extraction
The mineral feature extraction process capitalizes on the remarkable image classification capabilities of ResNet, a convolutional neural network architecture with a proven track record [26].Assuming the feature extracted by ResNet is written as  ∈  (where W is the width, H is the height, and C is the number of channels of the feature extracted), to enhance the performance of OCSVM in the context of OOD detection, a pivotal dimensionality reduction step was introduced.This process is illustrated by Formula

Mineral Feature Extraction
The mineral feature extraction process capitalizes on the remarkable image classification capabilities of ResNet, a convolutional neural network architecture with a proven track record [26].Assuming the feature extracted by ResNet is written as f ∈ R W×H×C (where W is the width, H is the height, and C is the number of channels of the feature extracted), to enhance the performance of OCSVM in the context of OOD detection, a pivotal dimensionality reduction step was introduced.This process is illustrated by Formula (1), which involves the concatenation of individual channel values, q k to create a more concise representation of f .Each q k corresponds to the value derived from the kth channel within the mineral feature map, as illustrated in Formula (2).This dimensionality reduction facilitates the OOD detection process and bolsters the overall performance of the model.q = (q 1 , q 2 , . . . ,q c ), Minerals 2024, 14, 627 5 of 10

OOD Detection by OCSVM
To ascertain whether an input image pertains to the in-distribution category of the 36 known minerals, the mineral-specific features x extracted from the DNN, are provided as input to the OCSVM.These features undergo a crucial transformation, being mapped to a higher-dimensional space, as outlined in Formula (3).
The classification outcome for the input image hinges on the result of Formula (3): if this result surpasses zero, the image is identified as an in-distribution mineral; conversely, if the result is less than or equal to zero, the image is categorized as an out-of-distribution mineral.In Formula (3), sgn designates the sign function, x i corresponds to the features derived from the ith known mineral training data.K(x i , x) represents the Radial Basis Function (RBF), as expounded in Formula (4), responsible for the transformation of the known mineral training data into a higher-dimensional space with the objective of maximizing the separation between these training data points and the origin within that space.The parameters α i and ρ are determined through the training process using the known mineral training datasets.
In Formula (4), the parameter denoted as σ represents the bandwidth, a pivotal factor governing the behavior of the Radial Basis Function (RBF).The significance of σ in this context is notably profound, as its magnitude inherently influences the classification process.Specifically, a larger value of σ tilts the balance toward categorizing a greater number of indistribution samples as out-of-distribution, while conversely, a smaller σ biases the model toward classifying a greater proportion of out-of-distribution samples as in-distribution.In alignment with prior research and in accordance with established convention, the present study maintains σ at the value 1/|x|.It is essential to underscore that |x| in this context designates the feature dimension.

Experimental Results and Analysis
The model's implementation is facilitated through the utilization of the Python programming language, executed in a Linux environment, while drawing upon the robust framework provided by Keras, Tensorflow, and Sklearn.In pursuit of optimal efficiency during the DNN training process, a GPU (Graphics Processing Unit) is employed.The precise specifications of the experimental configuration are comprehensively detailed in Table 3 for reference.

Evaluation Metrics
The evaluation of the model's performance hinges on two key metrics: OOD Detection Accuracy and Mineral Identification Accuracy.These metrics serve as crucial indicators of the model's proficiency in its respective tasks.OOD Detection Accuracy, a binary classification metric, assesses the model's effectiveness by distinguishing whether a mineral is in distribution or out of distribution.This metric includes three essential components: ID Accuracy, OOD Accuracy, and Overall Accuracy, which are calculated as shown in Formulas ( 5)- (7).ID Accuracy gauges the ratio of correctly identified in-distribution minerals to the total known mineral test datasets.Conversely, OOD Accuracy quantifies the ratio of the correctly identified out-of-distribution minerals to the overall count within the unknown mineral datasets.Notably, the Overall Accuracy mirrors the average of ID Accuracy and OOD Accuracy, given that the known and unknown mineral test data are maintained at equal proportions in this study.Mineral Identification Accuracy, a metric applicable to multi-class classification, evaluates the model's capacity to correctly identify minerals within their respective categories.This metric, which is similar to OOD Detection Accuracy, contains the trio of ID Accuracy, OOD Accuracy, and Overall Accuracy, but focuses on the performance of the model in identifying the concrete categories of in-distribution and outof-distribution minerals.These rigorous and multifaceted metrics offer a comprehensive assessment of the model's performance in distinguishing between mineral categories and detecting minerals that deviate from the established training datasets.

ID Accuracy =
correctly identified in − distribution minerals total known mineral test dataset OOD Accuracy = correctly identified out − of − distribution minerals total unknown mineral dataset ( Overall Accuracy = correctly identified minerals total mineral dataset (7)

Mineral Features Selection
As expounded in Section 3, the mineral features are extracted by the well-trained ResNet prior to OCSVM detection.In the case of ResNet50, a total of 49 mineral features can be derived from this process.To ascertain the optimal mineral features for OCSVM OOD detection, each of the 49 sets of features is independently subjected to OCSVM analysis, yielding 49 distinct accuracy values.This analysis is graphically presented in Figure 4, showcasing the Overall Accuracy of OOD detection associated with each mineral feature extracted from the 49 layers of ResNet.Upon careful examination of Figure 4, it becomes evident that the mineral features extracted from the second layer of ResNet50 emerge as the most promising, attaining a remarkable Overall Accuracy of 82.1%.Consequently, the features derived from the second layer of ResNet50 are chosen as the prime candidates for OCSVM-based OOD detec- Upon careful examination of Figure 4, it becomes evident that the mineral features extracted from the second layer of ResNet50 emerge as the most promising, attaining a remarkable Overall Accuracy of 82.1%.Consequently, the features derived from the second layer of ResNet50 are chosen as the prime candidates for OCSVM-based OOD detection, given their demonstrably robust performance.

Performance
Table 4 presents a comprehensive overview of the OOD Detection Accuracy and Mineral Identification Accuracy, offering profound insights into the model's performance.Notably, this analysis reveals that the model excels in its ability to correctly identify 82.1% of the test minerals as either known or unknown categories, with 96.4% accuracy achieved in discerning in-distribution test minerals as known categories.Moreover, 67.8% of the out-ofdistribution test minerals are adeptly classified as unknown categories, substantiating the model's competence in addressing the challenge of minerals that deviate from the training set.As highlighted in the Introduction section, contemporary mineral image identification methods are often constrained to categorize minerals within the bounds of the training set's distribution, leading to erroneous identification of out-of-distribution minerals.In this context, the model distinguishes itself by achieving 67.8% accuracy in classifying out-ofdistribution minerals as unknown categories.This OOD Accuracy is lower than that of other applications listed in references [17][18][19][20][21] because minerals in the same category may have different colors and textures, as shown in Figure 1, while different categories of minerals may also have the same colors and textures [6].This makes mineral identification more challenging, resulting in similarly lower ID Accuracy than other applications.The model attains a commendable 74.1% accuracy in identifying in-distribution minerals through the utilization of the state-of-the-art convolutional neural network, ResNet.The performance of each of the 36 known mineral categories is presented in Figure 5, affording a granular understanding of the model's accuracy across distinct mineral types.As shown in Figure 5, anglesite exhibits the lowest accuracy compared to the other minerals.On one hand, this could be because anglesite may be colorless, white, gray, yellow, or pale shades of blue or green, and may present as well-formed crystals, nodular, granular, or massive aggregates.On the other hand, there are only 1797 anglesite images in the dataset.The diversity of colors and forms and the limited number of images for learning makes it difficult for deep learning models to extract anglesite features.Hence, anglesite had the lowest accuracy.Other minerals shown in Figure 5 exhibit the lower accuracy compared to others.This could also be because of their diversity in colors and forms, and the limited number of images for learning.Additionally, a comparative analysis with other related studies is performed and the results are shown in Table 5.Compared to the study of Zeng et al. [6], which employed the same dataset of 36 known minerals, the model exhibits marginally lower ID Accuracy, but it has good OOD detection ability.Notably, this model surpasses other related studies in OOD detection, highlighting its proficiency in mineral identification tasks beyond the training set's confines.Additionally, a comparative analysis with other related studies is performed and the results are shown in Table 5.Compared to the study of Zeng et al. [6], which employed the same dataset of 36 known minerals, the model exhibits marginally lower ID Accuracy, but it has good OOD detection ability.Notably, this model surpasses other related studies in OOD detection, highlighting its proficiency in mineral identification tasks beyond the training set's confines.This study 36 74.1

Conclusions
A novel model designed to excel in identifying out-of-distribution minerals, harnessing the combined capabilities of OCSVM and the ResNet50 network is introduced.OCSVM plays a pivotal role in classifying mineral features extracted through ResNet50, endowing the model with the capacity to detect both out-of-distribution and in-distribution minerals.In comparison to traditional methods reliant on labor-intensive and time-

Conclusions
A novel model designed to excel in identifying out-of-distribution minerals, harnessing the combined capabilities of OCSVM and the ResNet50 network is introduced.OCSVM plays a pivotal role in classifying mineral features extracted through ResNet50, endowing the model with the capacity to detect both out-of-distribution and in-distribution minerals.In comparison to traditional methods reliant on labor-intensive and time-consuming experimental mineral species determination, this approach emerges as a more practical and cost-effective alternative.Additionally, when compared to other conventional deep learning methodologies, the model exhibits the unique capability to differentiate out-ofdistribution minerals, addressing a critical limitation in the field of mineral identification.Further expanding the in-distribution datasets would enhance the model's performance and its broader applicability in the field of mineral identification.

( 1 )
OOD detection is introduced in the mineral identification deep learning model to solve the problem of current methods incorrectly identifying the mineral outside of the distribution of the training set as one of the categories in the training set.(2) One-Class Support Vector Machine (OCSVM) is combined with ResNet for mineral identification to identify minerals outside the distribution of the training set as unknown, rather than assigning it one of the mineral categories in the training set, as is the case in other traditional deep learning models.(3) Comprehensive experiments are performed to show that deep learning combined with OOD detection can identify the minerals outside the distribution of the training set as unknown, while maintaining a high in-distribution accuracy.

Figure 3 .
Figure 3.The architecture of the model proposed in the paper.

Figure 3 .
Figure 3.The architecture of the model proposed in the paper.

Figure 4 .
Figure 4. Overall Accuracy of OOD detection associated with each mineral feature extracted from layers 1 to 49 of ResNet50.

Figure 4 .
Figure 4. Overall Accuracy of OOD detection associated with each mineral feature extracted from layers 1 to 49 of ResNet50.

Figure 5 .
Figure 5. Accuracy of the 36 known category minerals.

Table 1 .
Mineral names and number of samples in the in-distribution/known category datasets.

Table 1 .
Mineral names and number of samples in the in-distribution/known category datasets.

Table 2 .
Mineral names and number of samples in the out-of-distribution/unknown category datasets.#No.

Table 2 .
Mineral names and number of samples in the out-of-distribution/unknown category datasets.#

No. Mineral Quantities #No. Mineral Quantities
Minerals 2023, 13, x FOR PEER REVIEW 4 of 10

Table 4 .
Accuracy of our mineral identification model combining OCSVM and ResNet50.

Table 4 .
Accuracy of our mineral identification model combining OCSVM and ResNet50.
Figure 5. Accuracy of the 36 known category minerals.

Table 5 .
Comparisons of mineral identification ID Accuracy with other studies.

Table 5 .
Comparisons of mineral identification ID Accuracy with other studies.