Enhancing Oracle Bone Character Category Discovery via Character Component Distillation and Self-Merged Pseudo-Label

Abstract

Oraclebone characters (OBCs) are crucial for understanding ancient Chinese history, but existing recognition methods only recognize known categories in labeled data, neglecting novel categories in unlabeled data. This work introduces a novel approach to discovering new OBC categories in unlabeled data through generalized category discovery. We address the challenges posed by OBCs’ instinctive characteristics, such as misleading contrastive views from random cropping, sub-optimal learned representation, and insufficient supervision for unlabeled data. Our method features a symmetrical structure enhanced by character component distillation and self-merged pseudo-label. We utilize random geometric transforms to create symmetrical contrastive views to avoid misleading views. Then, the proposed character component distillation procedure optimizes symmetrical shared character components for better transferable representation. Finally, we construct a self-merged pseudo-label from the model and a symmetrical teacher model to provide stable and robust supervision for unlabeled data. Extensive experiments validate the superiority of our method in recognizing ’All’ and ’Novel’ OBC categories, providing an effective tool to aid OBC researchers.

1. Introduction

Oracle bone characters (OBCs) record divination and historical events of the Shang dynasty, holding the key to unlocking the mysteries of ancient Chinese history. Recognizing novel OBC categories is essential for understanding and deciphering the ancient information conveyed by these ancient scripts. Due to the scarcity and complexity of the original materials of OBCs, the study of OBCs has been dominated by authoritative experts in the past. Recently, advancements in deep learning [1,2] and digitization have enabled researchers to harness deep learning techniques for advanced OBC research, such as font translation [3,4], rejoining [5], and retrieval [6,7]. Even multi-modal large language models have also been successfully applied in generating visual guides for people to better understand OBCs by training with corresponding textual contextualization [8]. As an important underlying part, the recognition of OBCs is vital for these advanced applications and fundamental in practical OBC research. About 4500 OBC categories have been confirmed from unearthed oracle bones, of which only about 1000 categories have been recognized. Hence, recognizing these novel categories is of great significance.

For OBC recognition, supervised classification [9,10,11,12] methods and unsupervised domain adaptation methods [13,14] have been successfully applied to recognize authentic OBC images. But the key issue of these methods is the limited recognition capability, where the category set is fixed to the labeled set for training. Therefore, these methods can only recognize known OBC categories. Meanwhile, annotating massive OBC images requires domain knowledge and is time-consuming in real practice. Considering the lack of labeled data and the high annotation cost of OBCs, data augmentation methods have also been explored [15,16,17]. However, these methods only recognize OBC categories occurring in labeled data and assume consistent category sets between labeled and unlabeled data, where unlabeled data may contain novel OBC categories that do not occur in labeled categories. Therefore, it is urgent to develop a method for recognizing OBCs that can leverage unlabeled data and recognize new categories from unlabeled data to meet these practical needs in OBC research. We summarize these objectives as OBC category discovery, as depicted in Figure 1. The goal of OBC category discovery is to train a model with labeled and unlabeled data simultaneously, where the labeled data only contains known OBC categories while the unlabeled data contains novel OBC categories. After training, the model is required to recognize both known categories and novel categories that occur in labeled and unlabeled data.

Figure 1. An illustration of OBC category discovery. Given labeled OBC images only contain known categories and unlabeled images contain known and unknown categories; both known and novel categories need to be recognized. The Chinese characters at the bottom right of each OBC image are the corresponding deciphered Chinese characters.

Recently, the generalized category discovery (GCD) problem first proposed by [18] provides a solution to recognize novel categories from unlabeled data. Choi et al. [18] utilize a pre-trained self-supervised model fine-tuned by contrastive learning and realizes this objective via non-parametric K-means clustering [19], followed and improved by several research papers [20,21,22,23,24]. These methods pave the way for OBC category discovery. However, the characteristics of OBCs lead to several limitations when applying GCD methods developed for common objects to OBCs: (1) Random cropping used in contrastive learning may generate misleading views since OBCs are characterized by high inter-class similarity and extensive noise, especially when data is scarce. Considering the case of a complex OBC being cropped into two distinct views that coincidentally match two simple OBC categories, either instance-level or category-level contrastive learning is not reasonable since these views represent two distinct categories different from the category of the original image. (2) Image-level contrastive learning or distillation is sub-optimal for neglecting the learning of shared components by different OBC categories. The valuable prior knowledge that characters are made up of some shared components could be the key to transferable representations, considering that rarely used Chinese characters can be guessed from their radicals. (3) The supervision for labeled and unlabeled data is asymmetrical, which is insufficient for unlabeled data, especially considering the high inter-class similarity of novel OBC categories to known categories. The supervision for unlabeled data usually contains instance-level contrastive learning or self-distillation, while the supervision for the labeled data contains category-level contrastive learning as well as ground truth optimization.

To address these limitations, we propose an OBC category discovery method enhanced by a character component distillation procedure and a self-merged pseudo-label. Concretely, we employ random geometric transforms instead of random cropping to avoid the misleading effect and overfitting on limited data. The character component distillation procedure optimizes features of character components in symmetrical views extracted by masks generated from pixel clustering procedures. The self-merged pseudo-label is constructed by the model’s prediction and a symmetrical teacher providing stable features for known categories, compensating for the asymmetrical supervision between labeled and unlabeled data. We leverage the robust self-merged pseudo-label to supervise unlabeled data to compensate for the asymmetrical supervision between labeled and unlabeled data. Extensive experiments on three OBC recognition datasets demonstrate the superiority of our method compared with state-of-the-art methods. In summary, our main contributions are as follows:

• We propose an OBC category discovery method with a symmetrical structure that effectively recognizes novel OBC categories from unlabeled data.
• To reinforce the learned representation, we propose a character component distillation procedure that distills character components in symmetrical views by extracting spatial-disjoint components from foreground masks based on K-means clustering.
• To compensate for the asymmetrical supervision, we propose a self-merged pseudo-label based on the predictions of the model and a symmetrical teacher for known and novel categories as stable and robust supervision for unlabeled data.

2. Related Research

2.1. Oracle Bone Character Recognition

Oracle bone character recognition is fundamental for high-level OBC research. Early attempts are primarily based on traditional low-level image features. Guo et al. [25] utilize shape- and sketch-related image features to build a hierarchical representation for OBCs. Meng et al. [26] utilize Hough transform and template matching to recognize OBCs. Noticing the great potential of deep learning, OBC researchers have been engaging in deep convolutional neural networks [9,10,11] and transformers [12] to realize OBC recognition. Recently, more practical problems have been explored in OBC recognition. Some recent research focuses on addressing the inherited class imbalance problem of OBCs by augmentation methods. Yue et al. [15] propose a WGAN-based data augmentation method to generate OBC images similar to the original image. Wang et al. [16] propose a CycleGAN-based data augmentation method to generate OBC images from glyphs. Li et al. [17] consider the feature similarity between similar OBC images and propose a data augmentation method based on adversarial learning and mixup to improve the generalization of tail classes by mixing the similar OBC images. The main deficiency of mainstream supervised recognition methods is that they are limited to the labeled category used for training and are not able to recognize samples from novel OBC categories. Other research [13,14] considers the importance of utilizing clean and standard handwritten OBC fonts provided by experts to improve the recognition performance on noisy authentic OBC images. Unsupervised domain adaption has been adopted to transfer knowledge from handwritten OBC fonts to authentic OBC images, where the model is trained on the labeled source domain (e.g., handwritten OBC fonts) with unlabeled data from the target domain (e.g., authentic OBC images) and required to recognize the images from the target domain. However, these researches are based on supervised learning with the acquisition of all category information or partly unsupervised learning, where the shared category set between the labeled source domain and unlabeled target domain is a prerequisite. R-GNN [27] utilizes recurrent graph neural networks to recognize OBC font by capturing both the local fine-grained details and the global contextual information of oracle bone inscriptions. However, this method focuses on determining the font style to discover the historical period of oracle bone scripts, not recognizing novel OBC categories. Conf-UNet [28] proposes a method for the recognition of OBCs that predicts the corresponding large seal scripts, achieving primary extrapolating capability. However, this method requires a corresponding large seal script database for training and testing, where some OBCs may not have a strict one-to-one mapping with large seal scripts. In contrast, we do not assume a shared category set between labeled data and unlabeled data, and our method is capable of categorizing novel categories in unlabeled data.

In summary, compared with previous research, our method can utilize unlabeled data and recognize novel OBC categories from unlabeled data. Our method achieves better performance than other GCD methods on three OBC recognition datasets. Therefore, our method not only fills the gap in OBC recognition research where recognition has been limited to labeled data but also promotes computer-aided OBC category discovery research.

2.2. Generalized Category Discovery

Generalized category discovery (GCD) aims to categorize known and novel categories occurring in unlabeled data. Transferable representation from known categories to novel categories is the key to the categorization of novel categories. Better clustering utilization, richer supervision, and stronger augmentations are bright directions to transferable and robust representation, either parametric or non-parametric. Non-parametric methods usually tackle this task similarly to deep clustering [29] with deep clustering K-means [19], where better clustering utilization is essential. GCD [18] first addresses this task by supervised contrastive learning [30] and unsupervised contrastive learning [31]. While GCD successfully realized the task through unsupervised contrastive learning, the supervision solely from contrastive learning is insufficient, and the relationship of unlabeled data is ignored. Some recent research adopts community detection algorithms to realize category-level contrastive learning on unlabeled data: DCCL [20] adopts the InfoMap algorithm to find communities in unlabeled images and performs category-level contrastive learning between the assigned communities; Yang et al. [21] adopt the Louvain algorithm for similar purposes to achieve category-level contrastive learning and constrain the consistency between weak and strong augmentations. These researches utilize the underlying relationship between unlabeled data to enhance its representation, but the community detection algorithms can be complex. CMS [24] further realizes and simplifies GCD by KNN and contrastive mean-shift learning. It greatly enhances the representation of unlabeled data by contrastive learning based on simple KNN neighbors. However, misleading views and insufficient supervision remain issues to tackle regarding the application to OBCs. Meanwhile, parametric methods are faster in inference since they realize classification by computing similarities between parametric prototypes and image features, where richer supervision and augmentation are beneficial. SimGCD [22] proposes a parametric method based on a self-distillation framework based on soft pseudo-label and entropy regularization. The self-distillation objective and entropy regularization successfully alleviate the bias of parametric classifiers and enhance the learned representation, along with the fast inference speed brought by parametric classifiers. However, for its application to OBCs, the problems of misleading views and insufficient supervision also need to be tackled. SPT [23] further introduces an alternating learning scheme of spatial augmentation by wrapping learnable parameters. By wrapping image patches by learnable parameters, SPT enhances the model’s ability to focus on local features for better transferable representations. However, this wrapping procedure may split the internal components of OBCs into pieces, leading to sub-optimal transferable representation.

In comparison, our method alleviates the misleading effect caused by random cropping views. Moreover, our method reinforces the learned transferable representation by a character component distillation procedure and provides stable and robust pseudo-labels as sufficient supervision to supervise unlabeled data to enhance the learned representation.

3. Proposed Method

3.1. Preliminary

Given a labeled dataset ${\mathcal{D}}^l=\{x_i^l,y_i^l\}$ where $y_i^l\in {\mathcal{Y}}_{known}$ is the known category set, GCD aims to train a model by an unlabeled dataset ${\mathcal{D}}^u=\{x_i^u,y_i^u\}$ where $y_u^u\in {\mathcal{Y}}_{known}\cup {\mathcal{Y}}_{novel}$ is the unlabeled category set that contains novel categories from ${\mathcal{Y}}_{novel}$. The model is trained with both the labeled data ${\mathcal{D}}^l$ and unlabeled data ${\mathcal{D}}^u$. In the training stage, the ground truth $x_i^u$ of an unlabeled sample $x_i^u$ is not accessible. The model is required to recognize samples from both known and novel categories in the test set ${\mathcal{D}}^{test}=\{x_i^u,y_i^u\}$ where $y_u^u\in {\mathcal{Y}}_{known}\cup {\mathcal{Y}}_{novel}$.

As depicted in Figure 2, we adopt the parametric self-distillation framework proposed by SimGCD [22], where a prototypical classifier R performs classification by calculating the cosine similarity between randomly initialized prototypes $\{r_1,\dots ,r_K\},K=|{\mathcal{Y}}_{known}\cup {\mathcal{Y}}_{novel}|$ and image feature $h_i=f\left(x_i\right)$ extracted by the backbone f. The number of novel classes $|{\mathcal{Y}}_{novel}|$ is known a priori [22]. The base learning objective can be divided into a contrastive learning part (Equations (1) and (2)) and a self-distillation part (Equation (4)). The contrastive learning part consists of an unsupervised part ${\mathcal{L}}_{con}^u$ and a supervised part ${\mathcal{L}}_{con}^l$.

Figure 2. The overview of the proposed method. Random geometric transforms generate symmetrical contrastive views $x_i^0$ and $x_i^1$. The character component distillation procedure distills the symmetrical character components in different views. Meanwhile, the self-merged pseudo-label, merged from the predictions of the model (also denoted as the student) and a symmetrical teacher model, is used as supervision for unlabeled data.

Specifically, given two randomly augmented views $x_i^0$ and $x_i^1$ of the same image $x_i$ in a batch $\mathcal{B}$, the unsupervised part takes $x_i^1$ as the positive sample while other images are negative samples. In contrast, the supervised ${\mathcal{L}}_{sup}$ takes all samples that share the same class label with $x_i$ as positive samples, and samples from other classes are negative samples:

$${\mathcal{L}}_{con}^u=-{\displaystyle \frac{1}{\left|\mathcal{B}\right|}}\sum _{i\in \mathcal{B}}log{\displaystyle \frac{exp(\varphi (x_i^0,x_i^1)/{\tau }_u)}{{\sum }_{j\ne i}exp(\varphi (x_i^0,x_j^1)/{\tau }_u)}},$$

(1)

$${\mathcal{L}}_{con}^l=-{\displaystyle \frac{1}{|{\mathcal{B}}_l|}}\sum _{i\in {\mathcal{B}}_l}{\displaystyle \frac{1}{|{\mathcal{N}}_i|}}\sum _{k\in {\mathcal{N}}_i}log{\displaystyle \frac{exp(\varphi (x_i^0,x_k^1)/{\tau }_c)}{{\sum }_{j\ne n_i}exp(\varphi (x_i^0,x_i^1)/{\tau }_c)}},$$

(2)

where $\varphi (x_i,x_j)$ calculates the cosine similarity between the embedding $g\left(f\left(x_i\right)\right)$ and $g\left(f\left(x_j\right)\right)$ projected by a multi-layer perceptron (MLP) g. ${\tau }_u$ and ${\tau }_c$ are temperature hyper-parameters. ${\mathcal{N}}_i$ is the index set of all samples that share the same label with $x_i$. $|{\mathcal{B}}_l|$ is the number of labeled samples in the batch. For the self-distillation part, soft pseudo-label $q=R(x_i,{\tau }_t)$ generated from one view is used to supervise the prediction $p=R(x_i,{\tau }_s)$ of another view, where ${\tau }_s$ and ${\tau }_t$ are temperature hyper-parameters for calculating softmax probabilities for prediction and soft pseudo-label. The self-distillation part can be formulated as follows:

$$R(x_i,\tau )=\mathrm{softmax}\left(\{\cos (h_i,r_1)/\tau ,\dots ,\cos (h_i,r_K)/\tau \}\right),$$

(3)

$${\mathcal{L}}_{cls}^u=-{\displaystyle \frac{1}{\left|\mathcal{B}\right|}}\sum _{i\in \mathcal{B}}H(p_i,q_i)-ϵ\mathcal{H}\left(\overline{p}\right),{\mathcal{L}}_{cls}^l=-{\displaystyle \frac{1}{|{\mathcal{B}}_l|}}\sum _{i\in {\mathcal{B}}_l}H(p_i,y_i),$$

(4)

where H is the cross-entropy loss and $\mathcal{H}\left(\overline{p}\right)$ is a mean-entropy maximization regularizer [32] with its weight $ϵ$. The total loss of the base parametric self-distillation framework can be formulated as follows:

$${\mathcal{L}}_{base}=(1-\lambda )({\mathcal{L}}_{con}^u+{\mathcal{L}}_{cls}^u)+\lambda ({\mathcal{L}}_{con}^l+{\mathcal{L}}_{cls}^l),$$

(5)

where $\lambda$ is the weight of supervised learning, set to 0.35, following [22].

3.2. Geometric Transforms for Generating Contrastive Views

Learning on contrastive views generated by random augmentation from the same image is the fundamental step in contrastive learning [31,33] to learn better image representation, also adopted in GCD methods [18,22,23,24]. The most common practice in generating contrastive views is random cropping. However, random cropping is potentially harmful to OBCs, which is characterized by high inter-class similarity and extensive noise. Hence, generating cropped OBC images with missing key parts may eventually lead to overfitting.

To alleviate the misleading effect caused by random cropping while keeping the variance of contrastive views, we propose to employ geometric transforms as the alternative, as depicted in Figure 3. Specifically, we only apply random flip on $x_i^0$ while we apply random scaling, random translating, random rotating, and random shearing to generate $x_i^1$ from $x_i^0$. These geometric transforms are reasonable to OBCs based on the insight that handwritten characters can be crooked, but each character still has high similarity with its symmetrical crooked version. In this manner, we can generate symmetrical contrastive views for the same OBC image, keeping its essential structure.

Figure 3. Comparison of random geometric transforms and random cropping.

3.3. Character Component Distillation

After substituting random cropping with random geometric transforms, the symmetrical contrastive views $x_i^0$ and $x_i^1$ are used for contrastive learning. Similar to modern Chinese characters, different OBCs share some inner components, which are critical in the transferring learning process of recognizing characters. Optimizing the consistency between these components in symmetrical views further reinforces the learned representation. Therefore, we propose a character component distillation procedure shown in Figure 4 to enhance transferable representation from known to novel OBC categories by distilling extracted features of symmetrical character components.

Figure 4. The diagram of the proposed character component distillation procedures.

Similar to the concept of radicals in modern Chinese characters, connected components within a single OBC are likely to be shared over various OBC categories. To realize the character component distillation, splitting a single OBC into several connected components and extracting features from connected components is essential, requiring masks of these connected components. Inspired by [34], this goal can be realized through pixel clustering procedures. Firstly, the foreground mask $M_i\sim \{0,1\}$ of $x_i$ can be obtained by K-means clustering [19] on pixel values. The foreground mask $M_i$ divides the OBC image into character and background regions. We further split the foreground mask $M_i$ into several connected region masks by a spatial clustering method [35] based on density. Concretely, neighboring pixels sharing the same pixel value are assigned to the same connected region. After the K-means and spatial clustering, we obtain the connected region masks $M_i=\{M_i^1,\dots ,M_i^c\}$ for $x_i$, where $c_i$ is the number of the connected components. With the obtained connected region masks, we can perform masked average pooling $M_i^j$ to extract features $f_i^j$ for each connected component:

$$f_i^j={\displaystyle \frac{1}{|M_i^j|}}{M}_i^j\odot h_i,$$

(6)

where $h_i$ is the extracted feature of $x_i$, $|M_i^j|$ is the sum of connected region mask $M_i^j$, and ⊙ denotes the element-wise multiplication. In this way, we obtain the connected components’ features $\{{f_i^0}^1,\dots ,{f_i^0}^{c_i}\}$ for $x_i^0$ and $\{{f_i^1}^1,\dots ,{f_i^1}^{c_i}\}$ for $x_i^1$. The core insight in character component distillation is the consistency of connected components in symmetrical views. To this end, features $f_i^j$ for each connected component are projected into an embedding space $e_i^j=\varphi \left(f_i^j\right)$ by a MLP $\varphi$ and then classified by another prototypical classifier T of n random initialized prototypes $\{t_1,\dots ,t_n\}$:

$$T(f_i^j,\tau )=\mathrm{softmax}\left(\{\cos (f_i^j,t_1)/\tau ,\dots ,\cos (f_i^j,t_n)/\tau \}\right),$$

(7)

where n is set to 65536 in all experiments. Finally, the component soft pseudo-label ${q_i}^j=T({e_i^0}^j,{\tau }_t)$ generated from one view is used to supervise the prediction of another view ${p_i}^j=T({e_i^1}^j,{\tau }_s)$. The character component distillation loss can be formulated as follows:

$${\mathcal{L}}_{ccd}=-{\displaystyle \frac{1}{{\sum }_{i\in \mathcal{B}}{c}_i}}\sum _{i\in \mathcal{B}}\sum _{1\le j\le c_i}H({p_i}^j,{q_i}^j).$$

(8)

3.4. Self-Merged Pseudo-Label

By equipping with the proposed character component distillation, the learned representation has been reinforced. Furthermore, we seek to leverage richer supervision compensating for the asymmetrical supervision between labeled and unlabeled data to obtain robust representation. Motivated by the commonly used teacher–student framework in semi-supervised learning [36,37], we propose to use pseudo-labels to supervise the model on unlabeled data. Specifically, a symmetrical teacher backbone $f_t$ of the same structure as the student backbone f is slowly updated by the exponential moving average (EMA): ${\theta }_t={\mu }_{ema}·{\theta }_t+(1-{\mu }_{ema})·{\theta }_s$ of the student backbone’s parameters ${\theta }_s$. After $N_{warm}$ epochs of warming up, the teacher generates stable feature ${h_i^u}^t=f^t\left(x_i^u\right)$, and the teacher’s prediction $R^t(x_i^u,\tau )$ is used to supervise the student on unlabeled data:

$$R^t(x_i^u,\tau )=\mathrm{softmax}\left(\{\cos ({h_i^u}^t,r_1)/\tau ,\dots ,\cos ({h_i^u}^t,r_K)/\tau \}\right).$$

(9)

However, stability derived from slow updating brings the issue of adhering to known categories. Since the representation learning of novel categories is fully based on self-distillation, supervision of novel categories is insufficient at the early stage of training. As a result, the teacher is stable toward known categories but unresponsive to novel categories. In other words, the feature ${h_i^u}^t$ is stable to known categories while the feature $h_i^u$ could be more suitable for novel categories. Based on this insight, we propose to merge the predictions of teacher and student to provide robust pseudo-labels, namely self-merged pseudo-labels. Since the prototypes used for classification $\{r_1,\dots ,r_K\}$ are symmetrically shared by the teacher and student, we can directly merge the distance of $f_i^t$ to prototypes representing known categories $\{r_1,\dots ,r_{|{\mathcal{Y}}_{known}|}\}$ with the distance of $f_i$ to prototypes representing novel categories $\{r_{|{\mathcal{Y}}_{known}|+1},\dots ,r_K\}$. The self-merged pseudo-label $R^m(x_i^u,\tau )$ can be formulated as follows:

$$R^m(x_i^u,\tau )=\mathrm{softmax}\left(\begin{array}{cc}\hfill & \{\cos ({h_i^u}^t,r_1)/\tau ,\dots ,\cos ({h_i^u}^t,r_{|{\mathcal{Y}}_{known}|})/\tau ,\hfill \\ \hfill & \cos (h_i^u,r_{|{\mathcal{Y}}_{known}|+1})/\tau ,\dots ,\cos (h_i^u,r_K)/\tau \}\hfill \end{array}\right).$$

(10)

Then, the cross-entropy loss is calculated between the student’s prediction $R(x_i^u,{\tau }_s)$ and self-merged pseudo-label $R^m(x_i^u,{\tau }_s)$ with the same temperature ${\tau }_s$. For simplicity, we denote $R(x_i^u,{\tau }_s)$ as $R\left(x_i^u\right)$ and $R^m(x_i^u,{\tau }_s)$ as $R^m(x_i^u$). The self-merged pseudo-label loss can be formulated as follows:

$${\mathcal{L}}_{sst}=-{\displaystyle \frac{{\lambda }_{known}}{|{\mathcal{B}}_u^{known}|}}\sum _{i\in {\mathcal{B}}_u^{known}}H(R\left(x_i^u\right),R^m\left(x_i^u\right))-{\displaystyle \frac{{\lambda }_{novel}}{|{\mathcal{B}}_u^{novel}|}}\sum _{i\in {\mathcal{B}}_u^{novel}}H(R\left(x_i^u\right),R^m\left(x_i^u\right)),$$

(11)

where ${\mathcal{B}}_u^{known}$ and ${\mathcal{B}}_u^{novel}$ represent unlabeled images $x_i^u$ predicted as known and novel categories with confidence score over a fixed threshold: $\max \left(R^m\left(x_i\right)\right)\ge 0.5$. ${\lambda }_{known}$ and ${\lambda }_{novel}$ are balancing weights for known and novel categories, considering the different amounts of samples from known and novel categories in unlabeled data. Finally, the total loss can be formulated as follows:

$${\mathcal{L}}_{all}={\mathcal{L}}_{base}+{\lambda }_{ccd}·{\mathcal{L}}_{ccd}+{\lambda }_{sst}·{\mathcal{L}}_{sst},$$

(12)

where ${\lambda }_{ccd}$ and ${\lambda }_{sst}$ are balancing weights for the proposed character component distillation and self-merged pseudo-label.

4. Experiments

4.1. Datasets and Evaluation

We evaluate our method on three OBC recognition datasets: Oracle-241 [13], Oracle-241-hand [13], and OBI125 [15]. Oracle-241 contains 241 OBC categories, where 50,168 images are used for training and 13,806 images are used for testing. Oracle-241-hand shares the same categories as Oracle-241, but the images are high-quality handwritten OBC fonts from experts, containing 10,861 images used for training and 3730 images for testing. OBI125 contains 125 OBC categories, where 3131 images are used for training and 730 images are used for testing. Examples from each dataset are shown in Figure 5. Following GCD [18], half of all categories are sampled as the ‘Known’ categories, and the remaining categories are regarded as the ‘Novel’ categories. For ‘Known’ OBC categories, half of the training images are sampled as labeled data, while all remaining data from both ‘Known’ and ‘Novel’ OBC categories are used as unlabeled data. The data splits are shown in Table 1. For calculating the accuracy metric for classification, we adopt the Hungarian algorithm to find the optimal assignment between the prototype assignment and the ground truth following GCD [18]. The accuracy scores are reported for the ‘All’, ‘Known’, and ‘Novel’ OBC categories.

Figure 5. Example OBC images from each dataset.

Table 1. Data splits used in the experiments.

	Oracle-241	Oracle-241-Hand	OBI125
# Labeled Classes	120	120	62
# Unlabeled Classes	241	241	125
# Labeled Images	12,136	2570	840
# Unlabeled Images	38,032	8291	2291

4.2. Implementation Details

Following SimGCD [22], we adopt an ImageNet pre-trained DINO [38] with the ViT-B/16 [2] as the backbone f. We only finetune the last block of the backbone, and the [CLS] token is used as the image representation. We train our model with an initial learning rate of 0.05 with the SGD optimizer. The batch size is set to 128. For Oracle-241, ${\lambda }_{sst}$ is set to 2, $N_{warm}$ is set to 70, and ${\lambda }_{cdt}$ is set to $0.00005$. For Oracle-241-hand, ${\lambda }_{sst}$ is set to 4, $N_{warm}$ is set to 40, and ${\lambda }_{cdt}$ is set to $0.0001$. For OBI125, ${\lambda }_{sst}$ is set to 2, $N_{warm}$ is set to 10, and ${\lambda }_{cdt}$ is set to $0.0001$. ${\lambda }_{known}$ and ${\lambda }_{novel}$ are both set to 0.5 for Oracle-241 and set to 0.66 and 0.33 for Oracle-241-hand and OBI125. The teacher is EMA-updated with ${\mu }_{ema}$ set to 0.999 after the initial three epochs. The geometric transformation includes scaling between 0.6 and 1.1, translating by −2% to 2%, rotating by −10 to 10 degrees, and shearing by −45 to 45 degrees. We keep the best checkpoint with the highest average of accuracy on known classes and the unsupervised Silhouette score. Other hyper-parameters and training schedules are adopted from SimGCD [22]. We implement all experiments with an NVIDIA GeForce RTX 4090 GPU.

4.3. Comparison with State-of-the-Art Methods

In this section, we compare our method with K-means [19] and several state-of-the-art methods including GCD [18], SimGCD [22], SPT [23], and CMS [24].

Evaluation on OBC recognition datasets. Table 2 shows the results on three OBC recognition datasets. Our method achieves the best performance on All and Novel categories on all evaluated OBC recognition datasets, especially on the OBI125 dataset. These results demonstrate the superiority of our method in the OBC category discovery task. For All categories, our method outperforms CMS by 1.04% on the Oracle-241 and 2.20% on the Oracle-241-hand dataset, and SPT by 14.52% on the OBI125 dataset. For Novel categories, our method outperforms CMS by 3.85% on the Oracle-241 dataset, SimGCD by 2.95% on the Oracle-241-hand dataset, and SPT by 15.43% on the OBI125 dataset. These improvements demonstrate the effectiveness of our method for learning transferable representation of OBCs characterized by high inter-class similarity. Our method also achieves the best performance on Known categories on the OBI125 dataset. In comparison, only applying K-means to the raw feature without training yields poor performance since the backbone has not been fine-tuned for the OBC recognition task. GCD lags behind other SOTA methods since it only considers the basic unsupervised instance-level and supervised category-level contrastive targets. SimGCD achieves better performance for its self-distillation framework and effective entropy regularization; CMS also achieves this for its effective category-level contrastive learning based on KNN neighbors for unlabeled data. However, misleading views and insufficient supervision remain areas for SimGCD and CMS to improve, whereas our method utilizes geometric transforms instead of cropping to avoid misleading views; the proposed character component distillation and the self-merged pseudo-label provide sufficient supervision. SPT shows a strong performance in OBI125 for its spatial prompts wrapping image patches, enabling the model to focus on local features to alleviate overfit on limited data. However, the wrapping process used in SPT splits the internal components of OBC into pieces, leading to sub-optimal transferable representation. In contrast, our method splits spatial-disjoint components for better transferable representation. In summary, the enhanced representation from the character component distillation and the stable and robust supervision from the self-merged pseudo-label are the keys to the superior performance of our method. Notably, our method outperforms other methods by a significant gap on the OBI125 dataset where the number of categories is significant, but the samples are relatively scarce. This situation is practical for OBC research, where available labeled data are scarce, and many categories remain unexplored. Therefore, our method can provide a robust and helpful tool to assist researchers in discovering novel OBC categories from unlabeled data.

Table 2. Results on three OBC recognition datasets.

	Oracle-241			Oracle-241-Hand			OBI125
Method	All	Known	Novel	All	Known	Novel	All	Known	Novel
K-means [19]	9.35	9.51	9.20	41.45	39.33	43.35	21.51	21.88	21.07
GCD [18]	56.88	64.84	53.15	67.87	77.13	63.71	26.98	42.62	17.92
SimGCD [22]	68.53	74.87	62.61	78.55	84.89	72.85	41.78	58.78	21.96
SPT [23]	64.47	78.84	51.05	70.16	79.57	61.69	50.14	67.43	29.97
CMS [24]	70.37	78.04	63.20	78.34	90.49	67.40	39.32	54.45	21.66
Ours	71.41	76.08	67.05	80.54	85.80	75.80	64.66	81.17	45.40

4.4. Ablation Study

Effectiveness of proposed components. To verify the effectiveness of our proposed components, we conduct extensive ablation experiments, and the results are shown in Table 3. Our baseline is the simple SimGCD with random cropping replaced by random geometric transforms. By using the proposed character component distillation, we observe a consistent performance improvement on All and Novel categories over all three datasets. These results substantiate that the proposed distillation procedure of character components is beneficial to learning transferable representations. Meanwhile, training by the proposed self-merged pseudo-label yields a similar boost, validating the effectiveness of the proposed self-merged pseudo-label to provide rich supervision for robust representation. Notably, we observe that directly using the teacher’s prediction for supervision results in poorer performance, since the teacher’s prediction is less accurate than the student’s on novel categories. Finally, by equipping all the proposed components, our method reaches its best performance on All categories, demonstrating the compatibility and effectiveness of our proposed components. Moreover, we find that the self-merged pseudo-label brings more improvement than character component distillation on noisy datasets (Oracle-241 and OBI125) compared with the clean dataset (Oracle-241-hand), indicating the importance of rich supervision on unlabeled authentic OBC data.

Table 3. Ablation study on the components. ‘CCD’ denotes the character component distillation. ‘SST’ denotes the self-merged pseudo-label, where ‘plain’ denotes directly using the teacher’s prediction as pseudo-label.

		Oracle-241			Oracle-241-Hand			OBI125
CDT	SST	All	Known	Novel	All	Known	Novel	All	Known	Novel
✗	✗	66.82	76.32	57.94	80.11	87.55	73.41	55.48	77.61	29.67
✓	✗	67.28	76.30	58.85	80.24	87.15	74.02	56.71	78.37	31.45
✗	✓(plain)	70.72	75.42	66.34	79.17	85.51	73.46	59.18	79.13	35.91
✗	✓	71.15	75.09	67.47	80.48	87.27	74.38	63.42	79.13	45.10
✓	✓	71.41	76.08	67.05	80.54	85.80	75.80	64.66	81.17	45.40

Influence of ${\lambda }_{ccd}$. We further investigate the influence of the hyper-parameter ${\lambda }_{ccd}$ in Table 4. It can be observed that on the clean dataset Oracle-241-hand and small-scale dataset OBI125, a relatively larger ${\lambda }_{ccd}$ (0.0001) yields better performance, while on the noisy and large dataset Oracle-241, a relatively lower ${\lambda }_{ccd}$ (0.00005) is better. We posit that character component distillation plays a pivotal role in scenarios where data is either clean or limited. We suppose this result is because clean OBC data facilitate more straightforward component extraction, while data scarcity stands to gain significantly from the additional distillation process.

Table 4. Ablation study on the weight ${\lambda }_{ccd}$ of character component distillation.

	Oracle-241			Oracle-241-Hand			OBI125
${\mathsf{\lambda }}_{\mathit{ccd}}$	All	Known	Novel	All	Known	Novel	All	Known	Novel
0.000025	66.29	76.06	57.16	79.44	87.66	72.03	56.16	75.06	34.12
0.00005	67.28	76.30	58.85	79.25	87.83	71.52	56.44	76.84	32.64
0.0001	65.86	76.50	55.91	80.24	87.15	74.02	56.71	78.37	31.45
0.0002	65.65	77.20	54.86	78.61	88.74	69.49	56.03	76.84	31.75

Influence of $N_{warm}$. We also investigate the influence of the number of warming-up epochs $N_{warm}$ in Table 5. We select $N_{warm}$ = ‘immediate’, 10, 20, 30, 45, 50, 60, 70 (up to $1/3$ of total training epochs) for the ablation experiments, where ’immediate’ denotes no warming up. It can be observed that a long warming-up is beneficial when OBC data is noisy and sufficient (Oracle-241), while a short warming-up is preferable when data is scarce (OBI125). We hypothesize that data characterized by high noise and substantial volume requires an extended period of warming up for the teacher model to achieve a state of stability and accuracy. On the contrary, scarce data only needs a short warming-up to avoid overfitting.

Table 5. Ablation study on the number of warming-up epochs ${\mathit{N}}_{\mathit{w}\mathit{a}\mathit{r}\mathit{m}}$.

	Oracle-241			Oracle-241-Hand			OBI125
${\mathbf{N}}_{\mathbf{warm}}$	All	Known	Novel	All	Known	Novel	All	Known	Novel
immediate	63.16	74.93	52.17	70.99	83.25	59.96	60.14	75.32	42.43
10	65.88	75.22	57.16	74.05	86.02	63.27	63.42	79.13	45.10
20	69.06	77.13	61.53	78.74	87.27	71.06	61.10	78.12	41.25
30	70.57	76.95	64.61	80.40	85.91	75.45	60.82	79.13	39.47
40	70.33	76.09	64.95	80.48	87.27	74.38	59.45	77.35	38.58
50	70.57	76.23	65.28	79.76	86.08	74.07	58.49	77.35	36.50
60	70.98	76.68	65.65	80.00	86.13	74.48	56.99	76.84	33.83
70	71.15	75.09	67.47	80.40	86.13	75.24	55.89	77.86	30.27

T-SNE Visualization. In addition, we visualized the feature space by T-SNE [39] in Figure 6 and Figure 7 to qualitatively verify the effectiveness of our method in representation learning. Compared with DINO and SimGCD, our method reduces noise and obtains more precise and compact clusters. For the Oracle-241 dataset, which represents massive noisy labeled data, our method obtains clearer clusters with reduced noisy sample points in the middle region. This validates the effectiveness of our method under extensive noise caused by archaeological excavations and rubbing procedures. For the OBI125 dataset, which represents limited labeled data, our method obtains better clusters with a more compact distribution compared with DINO and SimGCD. This demonstrates the effectiveness of our method under scarce labeled data, which meets the circumstances where OBC images from novel categories are insufficient for unsupervised contrastive learning.

Figure 6. T-SNE visualization of the feature space of OBI125.

Figure 7. T-SNE visualization of the feature space of Oracle-241.

4.5. Time Complexity

To analyze the time complexity of our method, we compare the elapsed time of one training epoch on the Oracle-241 dataset. Since our method is based on the parametric self-distillation framework proposed by SimGCD [22], we only consider the extra time complexity of our method. The elapsed time in one training epoch of our model and other models are shown in Table 6. Our method needs more time for the character component distillation and the self-merged pseudo-label. Let the number of labeled and unlabeled samples be $|{\mathcal{D}}^l|$ and $|{\mathcal{D}}^u|$, the number of all categories K, the feature dimension of prototypes d, and the number of character component prototypes n; we first analyze time complexity for one sample. For each sample, the extra time complexity for the character component distillation is $\mathcal{O}\left(nd\right)$ for calculating the distance of the L2-normed feature with character component prototypes. The extra time complexity for the self-merged pseudo-label is $\mathcal{O}\left(Kd\right)$ for searching the nearest prototype by calculating the distance of the L2-normed feature with prototypes. For the whole dataset, the extra time complexity of our method is $\mathcal{O}\left(\right(|{\mathcal{D}}^l|+|{\mathcal{D}}^u\left|\right)·nd+|{\mathcal{D}}^u|·Kd)$. Compared with other methods, although our method needs more time in training, the trade-off of time is reasonable for improved performance.

Table 6. The running time (s) by one epoch of different methods on Oracle-241.

Methods	GCD	SimGCD	SPT	CMS	Ours
Time	101 s	121 s	145 s	235 s	170 s

5. Discussion

5.1. Methodology

Based on the experimental results and analysis, we summarize the key factors to better performance in the OBC category discovery task, which is generally difficult and time-consuming by traditional research methods. Learning transferable and robust representation is essential in recognizing known and novel OBC categories, considering the high inter-class similarity of OBCs. We can effectively reinforce the learned representation by the proposed distillation procedure of symmetrical character components in different contrastive views. Meanwhile, we proposed the self-merged pseudo-label compensating for the asymmetrical supervision between labeled and unlabeled data, which is also critical to providing sufficient supervision for robust representation. The experimental results show that the self-merged pseudo-label can gain more improvement on noisy authentic OBC images. Overall, our method achieves the best recognition performance on ‘All’ and ‘Novel’ categories on three OBC recognition datasets. For the ‘All’ category, our method achieves recognition accuracy of 71.41% (2.88% improvement), 80.54% (1.99% improvement), and 64.66% (22.88% improvement), respectively. For the ’Novel’ category, our method achieves recognition accuracy of 67.05% (4.44% improvement), 75.80% (2.95% improvement), and 45.40% (23.44% improvement), respectively. These significant improvements (especially on the OBI125 dataset) are achieved by the alleviated misleading effect, better transferable representation brought by the proposed character component distillation, and richer supervision from the self-merged pseudo-label. Our method provides an effective and robust tool for OBC researchers to facilitate relevant OBC recognition research by massive unlabeled data, and novel OBC categories within unlabeled data can be effectively recognized.

5.2. The Opted Teacher–Student Framework

The teacher model in our work is an EMA-updated model with the same structure as the student model. We opted for the teacher–student framework for several reasons. Firstly, the teacher–student framework is suitable for learning on unlabeled data with limited labeled data, which has been demonstrated and adopted by the research field of semi-supervised learning. Secondly, in addition to the self-distillation objective in the base parametric self-distillation framework, the teacher–student framework can provide additional supervision to unlabeled data in the form of high-quality pseudo-labels. Meanwhile, other methods to transfer the knowledge from the labeled to unlabeled categories, such as self-labeling, are prone to be biased due to error accumulation, considering the setting of GCD. In comparison, the EMA-updated teacher provides more stable and robust supervision and is more suitable for the OBC category discovery task.

5.3. Limitations and Future Research

There are still some limitations in applying our method in practical OBC research. The primary limitation of our method is the imbalanced nature of novel OBC categories in unlabeled data. The distribution of novel OBC categories could be severely imbalanced in unlabeled data, which will cause the model to be biased toward the major categories. Therefore, using a fixed threshold for all OBC categories to generate pseudo-labels is sub-optimal, where the model is less confident with minor categories than major categories. Another limitation of our method is the essence of character components. In modern Chinese characters and OBCs, radicals are not just spatial-disjoint components, which are also likely to overlap with each other. To address these limitations, dynamic or adaptive thresholds, class reweighting methods, and character radical extraction modules can be utilized to provide a more robust solution.

Moreover, in more practical circumstances, novel OBC categories could be continually labeled by experts, while the amount of labeled data is constant in the GCD setting. In future studies, we will utilize active learning techniques to continually utilize the recognized samples from novel OBC categories to obtain better performance.

6. Conclusions

In this paper, we proposed a novel oracle bone character (OBC) category discovery method based on a parametric self-distillation GCD framework generating symmetrical contrastive views by random geometric transforms. By the proposed character component distillation procedure on symmetrical character components, we reinforce better transferable representation essential to transfer knowledge from known to novel OBC categories. To further compensate for the asymmetrical supervision between labeled and unlabeled data and obtain robust representation, we construct a self-merged pseudo-label by the predictions of the model itself and a symmetrical teacher model to provide supervision for unlabeled data. In addition to alleviating misleading views, compared to other methods, our method can better transfer from known categories to novel categories by the better transferable representation learned by the character component distillation procedure. The richer supervision from the self-merged pseudo-label also enables our method to learn stronger and more robust representations from unlabeled data. Therefore, our method achieves significant improvement and superior performance on three OBC recognition datasets, especially on the OBI125 dataset (64.66% accuracy for ‘All’, 81.17% accuracy for ‘Known’, and 45.40% accuracy for ‘Novel’), where the labeled data is limited, so transferable representation and robust representation are crucial. Experimental results verified that our method achieves the best performance in recognizing ’All’ and ’Novel’ OBC categories on three OBC recognition datasets, providing a helpful tool to assist OBC researchers in recognizing novel OBC categories.