Skip to Content
AlgorithmsAlgorithms
  • Article
  • Open Access

2 September 2026

23 Pages

Beyond Pseudo-Labels: Dual-Level Knowledge Distillation for Enhanced Deep Clustering

,
and
1
Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11451, Saudi Arabia
2
Computer Science Department, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, Al-Kharj 16278, Saudi Arabia
*
Author to whom correspondence should be addressed.

Abstract

This study introduces a novel clustering approach, namely Teacher–Student-based Deep Clustering (TSDC), that relies on intra- and inter-distillation-based feature representations. In fact, TSDC distils knowledge from (i) high- to low-response channels, forcing the latter to mimic the former, and (ii) deeper to shallower layers, prompting the transfer of semantic information to enhance representational consistency. Unlike CNN-based deep clustering that relies on pseudo-labels to improve representations, TSDC introduces a newly formulated objective function to simultaneously minimize losses from clustering and intra- and inter-distillation. The proposed approach was rigorously investigated using benchmark datasets and relevant performance measures. In particular, a linear top classifier protocol was adopted to assess TSDC performance. Notably, TSDC outperformed existing deep clustering frameworks, yielding improved classification accuracy. The empirical findings highlight the efficacy of combining intra- and inter-distillation to enrich feature representations. Notably, the most prominent improvement was observed on CIFAR-100, where classification accuracy rose from 54.77 ± 0.07 to 57.58 ± 1.85.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.