Search Results (133)

Accurate heavy-duty vehicle load estimation is crucial for transportation and environmental regulation, yet current methods lack precision in data accuracy and practicality for field implementation. We propose a Self-Supervised Reconstruction Heterogeneous Graph Convolutional Network (SSR-HGCN) for load estimation using On-Board Diagnostics (OBD) data. The method integrates physics-constrained heterogeneous graph construction based on vehicle speed, acceleration, and engine parameters, leveraging graph neural networks’ information propagation mechanisms and self-supervised learning’s adaptability to low-quality data. The method comprises three modules: (1) a physics-constrained heterogeneous graph structure that, guided by the symmetry (invariance) of physical laws, introduces a structural asymmetry by treating kinematic and dynamic features as distinct node types to enhance model interpretability; (2) a self-supervised reconstruction module that learns robust representations from noisy OBD streams without extensive labeling, improving adaptability to data quality variations; and (3) a multi-layer feature extraction architecture combining graph convolutional networks (GCNs) and graph attention networks (GATs) for hierarchical feature aggregation. On a test set of 800 heavy-duty vehicle trips, SSR-HGCN demonstrated superior performance over key baseline models. Compared with the classical time-series model LSTM, it achieved average improvements of 20.76% in RMSE and 41.23% in MAPE. It also outperformed the standard graph model GraphSAGE, reducing RMSE by 21.98% and MAPE by 7.15%, ultimately achieving < 15% error for over 90% of test samples. This method provides an effective technical solution for heavy-duty vehicle load monitoring, with immediate applications in fleet supervision, overloading detection, and regulatory enforcement for environmental compliance. Full article

The Background/Objectives: The excessive dependence on data annotation, the lack of labeled data, and the substantial expense of data annotation, especially in healthcare, have constrained the efficacy of conventional supervised learning methodologies. Self-supervised learning (SSL) has arisen as a viable option by utilizing unlabeled data via pretext tasks. This paper examines the efficacy of supervised (pseudo-labels) and unsupervised (no pseudo-labels) pretext models in semi-supervised learning (SSL) for the classification of coronary artery disease (CAD) utilizing cardiac MRI data, highlighting performance in scenarios of data scarcity, out-of-distribution (OOD) conditions, and adversarial robustness. Methods: Two datasets, referred to as CAD Cardiac MRI and Ohio State Cardiac MRI Raw Data (OCMR), were utilized to establish three pretext tasks: (i) supervised Gaussian noise addition, (ii) supervised image rotation, and (iii) unsupervised generative reconstruction. These models were evaluated against  Simple Framework for Contrastive Learning (SimCLR), a prevalent unsupervised contrastive learning framework. Performance was assessed under three data reduction scenarios (20%, 50%, 70%), out-of-distribution situations, and adversarial attacks utilizing FGSM and PGD, alongside other significant evaluation criteria. Results: The Gaussian noise-based model attained the highest validation accuracy (up to 99.9%) across all data reduction scenarios and exhibited superiority over adversarial perturbations and all other employed measures. The rotation-based model exhibited considerable susceptibility to attacks and diminished accuracy with reduced data. The generative reconstruction model demonstrated moderate efficacy with minimal performance decline. SimCLR exhibited strong performance under standard conditions but shown inferior robustness relative to the Gaussian noise model. Conclusions: Meticulously crafted self-supervised pretext tasks exhibit potential in cardiac MRI classification, showcasing dependable performance and generalizability despite little data. These initial findings underscore SSL’s capacity to create reliable models for safety-critical healthcare applications and encourage more validation across varied datasets and clinical environments. Full article

Wildfires pose a regularly increasing threat to ecosystems and critical infrastructure. The severity of this threat is steadily increasing. The growing threat necessitates the development of technologies for rapid and accurate early detection. However, the prevailing fire point detection algorithms, including several deep learning models, are generally constrained by the inherent hard threshold limitations in their decision-making logic. As a result, these methods lack adaptability and robustness in complex and dynamic real-world scenarios. To address this challenge, the present paper proposes an innovative two-stage, semi-supervised anomaly detection framework. The framework initially employs a Transformer-based autoencoder, which serves to transform raw fire-free time-series data derived from satellite imagery into a multidimensional deep anomaly feature vector. Self-supervised learning achieves this transformation by incorporating both reconstruction error and latent space distance. In the subsequent stage, a semi-supervised XGBoost classifier, trained using an iterative pseudo-labeling strategy, learns and constructs an adaptive nonlinear decision boundary in this high-dimensional anomaly feature space to achieve the final fire point judgment. In a thorough validation process involving multiple real-world fire cases in Yunnan Province, China, the framework attained an F1 score of 0.88, signifying a performance enhancement exceeding 30% in comparison to conventional deep learning baseline models that employ fixed thresholds. The experimental results demonstrate that by decoupling feature learning from classification decision-making and introducing an adaptive decision mechanism, this framework provides a more robust and scalable new paradigm for constructing next-generation high-precision, high-efficiency wildfire monitoring and early warning systems. Full article

Depth estimation algorithms are widely applied in various fields, including 3D reconstruction, autonomous driving, and industrial robotics. Monocular self-supervised algorithms for depth prediction offer a cost-effective alternative to acquiring depth through hardware devices such as LiDAR. However, current depth prediction networks, predominantly based on conventional encoder–decoder architectures, often encounter two critical limitations: insufficient feature fusion mechanisms during the upsampling phase and constrained receptive fields. These limitations result in the loss of high-frequency details in the predicted depth maps. To overcome these issues, we introduce differential attention operators to enhance global feature representation and refine locally upsampled features within the depth decoder. Furthermore, we equip the decoder with a deformable bin-structured prediction head; this lightweight design enables per-pixel dynamic aggregation of local depth distributions via adaptive receptive field modulation and deformable sampling, enhancing the decoder’s fine-grained detail processing by capturing local geometry and holistic structures. Experimental results on the KITTI and Make3D datasets demonstrate that our proposed method produces more accurate depth maps with finer details compared to existing approaches. Full article

Fault identification in Printed Circuit Board Assembly (PCBA) testing is essential for assuring product quality; nevertheless, conventional methods still have difficulties due to the lack of labeled faulty data and the “black box” nature of advanced models. This study introduces a label-free, interpretable self-supervised framework that uses two pretext tasks: (i) an autoencoder (reconstruction error and two latent features) and (ii) isolation forest (faulty score) to form a four-dimensional representation of each test sequence. A two-component Gaussian Mixture Model is used, and the samples are clustered into normal and fault groups. The decision is explained with cluster mean differences, SHAP (LinearSHAP or LinearExplainer on a logistic-regression surrogate), and a shallow decision tree that generated if–then rules. On real PCBA data, internal indices showed compact and well-separated clusters (Silhouette 0.85, Calinski–Harabasz 50,344.19, Davies–Bouldin 0.39), external metrics were high (ARI 0.72; NMI 0.59; Fowlkes–Mallows 0.98), and the clustered result used as a fault predictor reached 0.98 accuracy, 0.98 precision, and 0.99 recall. Explanations show that the IForest score and reconstruction error drive most decisions, causing simple thresholds that can guide inspection. An ablation without the self-supervised tasks results in degraded clustering quality. The proposed approach offers accurate, label-free fault prediction with transparent reasoning and is suitable for deployment in industrial test lines. Full article

Counting and density estimation of cone cells using adaptive optics (AO) imaging plays an important role in the clinical management of retinal diseases. A novel deep learning approach for the cone counting task with minimal manual labeling of cone cells in AO images is described in this paper. We propose a hybrid multi-task semi-supervised learning (MTSSL) framework that simultaneously trains on unlabeled and labeled data. On the unlabeled images, the model learns structural and relational features by employing two self-supervised pretext tasks—image inpainting (IP) and learning-to-rank (L2R). At the same time, it leverages a small set of labeled examples to supervise a density estimation head for cone counting. By jointly minimizing the image reconstruction loss, the ranking loss, and the supervised density-map loss, our approach harnesses the rich information in unlabeled data to learn feature representations and directly incorporates ground-truth annotations to guide accurate density prediction and counts. Experiments were conducted on a dataset of AO images of 120 subjects captured using a device with a retinal camera (rtx1) with a wide field-of-view. MTSSL gains strengths from hybrid self-supervised pretext tasks of generative and predictive pretraining that aid in learning global and local context required for counting cones. The results show that the proposed MTSSL approach significantly outperforms the individual self-supervised pipelines with an RMSE score improved by a factor of 2 for cone counting. Full article

Bearings are vital to rotating machinery, where undetected faults can cause severe failures. Conventional fault diagnosis methods depend on manual feature engineering and labeled data, struggling with complex industrial conditions. This study introduces an innovative unsupervised framework combining masked self-supervised learning with the Swin Transformer for bearing fault diagnosis. The novel integration leverages masked Auto Encoders to learn robust features from unlabeled vibration signals through reconstruction-based pretraining, while the Swin Transformer’s shifted window attention mechanism enhances efficient capture of fault-related patterns in long-sequence signals. This approach eliminates reliance on labeled data, enabling precise detection of unknown faults. The proposed method achieves 99.53% accuracy on the Paderborn dataset and 100% accuracy on the CWRU dataset significantly, surpassing other unsupervised Auto Encoder-based methods. This method’s innovative design offers high adaptability and substantial potential for predictive maintenance in industrial applications. Full article

Accurate genome binning is essential for resolving microbial community structure and functional potential from metagenomic data. However, existing approaches—primarily reliant on tetranucleotide frequency (TNF) and abundance profiles—often perform sub-optimally in the face of complex community compositions, low-abundance taxa, and long-read sequencing datasets. To address these limitations, we present MBGCCA, a novel metagenomic binning framework that synergistically integrates graph neural networks (GNNs), contrastive learning, and information-theoretic regularization to enhance binning accuracy, robustness, and biological coherence. MBGCCA operates in two stages: (1) multimodal information integration, where TNF and abundance profiles are fused via a deep neural network trained using a multi-view contrastive loss, and (2) self-supervised graph representation learning, which leverages assembly graph topology to refine contig embeddings. The contrastive learning objective follows the InfoMax principle by maximizing mutual information across augmented views and modalities, encouraging the model to extract globally consistent and high-information representations. By aligning perturbed graph views while preserving topological structure, MBGCCA effectively captures both global genomic characteristics and local contig relationships. Comprehensive evaluations using both synthetic and real-world datasets—including wastewater and soil microbiomes—demonstrate that MBGCCA consistently outperforms state-of-the-art binning methods, particularly in challenging scenarios marked by sparse data and high community complexity. These results highlight the value of entropy-aware, topology-preserving learning for advancing metagenomic genome reconstruction. Full article

Monocular fisheye camera distance estimation is a crucial visual perception task for autonomous driving. Due to the practical challenges of acquiring precise depth annotations, existing self-supervised methods usually consist of a monocular distance model and an ego-motion predictor with the goal of minimizing a reconstruction matching loss. However, they suffer from inaccurate distance estimation in low-texture regions, especially road surfaces. In this paper, we introduce a weakly supervised learning strategy that incorporates semantic segmentation, instance segmentation, and optical flow as additional sources of supervision. In addition to the self-supervised reconstruction loss, we introduce a road surface flatness loss, an instance smoothness loss, and an optical flow loss to enhance the accuracy of distance estimation. We evaluate the proposed method on the WoodScape and SynWoodScape datasets, and it outperforms the self-supervised monocular baseline, FisheyeDistanceNet. Full article

Credit card fraud remains a major cause of financial loss around the world. Traditional fraud detection methods that rely on supervised learning often struggle because fraudulent transactions are rare compared to legitimate ones, leading to imbalanced datasets. Additionally, the models must be retrained frequently, as fraud patterns change over time and require new labeled data for retraining. To address these challenges, this paper proposes an ensemble unsupervised learning approach for credit card fraud detection that combines Autoencoders (AEs), Self-Organizing Maps (SOMs), and Restricted Boltzmann Machines (RBMs), integrated with an Adaptive Reconstruction Threshold (ART) mechanism. The ART dynamically adjusts anomaly detection thresholds by leveraging the clustering properties of SOMs, effectively overcoming the limitations of static threshold approaches in machine learning and deep learning models. The proposed models, AE-ASOMs (Autoencoder—Adaptive Self-Organizing Maps) and RBM-ASOMs (Restricted Boltzmann Machines—Adaptive Self-Organizing Maps), were evaluated on the Kaggle Credit Card Fraud Detection and IEEE-CIS datasets. Our AE-ASOM model achieved an accuracy of 0.980 and an F1-score of 0.967, while the RBM-ASOM model achieved an accuracy of 0.975 and an F1-score of 0.955. Compared to models such as One-Class SVM and Isolation Forest, our approach demonstrates higher detection accuracy and significantly reduces false positive rates. In addition to its performance, the model offers considerable computational efficiency with a training time of 200.52 s and memory usage of 3.02 megabytes. Full article

Accurate estimation of the state of health (SOH) of lithium-ion batteries confronts two critical challenges: the extreme scarcity of labeled data in large-scale operational datasets and the mismatch between existing methods (relying on full charging–discharging conditions) and shallow charging–discharging conditions prevalent in real-world scenarios. To address these challenges, this study proposes a self-supervised learning framework for SOH estimation. The framework employs a dual-time-scale collaborative pre-training approach via masked voltage sequence reconstruction and interval capacity prediction tasks, enabling automatic extraction of cross-time-scale aging features from unlabeled data. Innovatively, it integrates domain knowledge into the attention mechanism and incorporates time-varying factors into positional encoding, significantly enhancing the capability to extract battery aging features. The proposed method is validated on two datasets. For the standard dataset, using only 10% labeled data, it achieves an average RMSE of 0.491% for NCA battery estimation and 0.804% for transfer estimation between NCA and NCM. For the shallow-cycle dataset, it achieves an average RMSE of 1.300% with only 2% labeled data. By synergistically leveraging massive unlabeled data and extremely sparse labeled samples (2–10% labeling rate), this framework reduces the labeling burden for battery health monitoring by 90–98%, offering an industrial-grade solution with near-zero labeling dependency. Full article

Objectives: This study investigates whether scan time in the high-resolution magnetic resonance imaging (MRI) of human embryos can be reduced without compromising spatial resolution by applying zero-shot self-supervised learning (ZS-SSL), a deep-learning-based reconstruction method. Methods: Simulations using a numerical phantom were conducted to evaluate spatial resolution across various acceleration factors (AF = 2, 4, 6, and 8) and signal-to-noise ratio (SNR) levels. Resolution was quantified using a blur-based estimation method based on the Sparrow criterion. ZS-SSL was compared to conventional compressed sensing (CS). Experimental imaging of a human embryo at Carnegie stage 21 was performed at a spatial resolution of (30 μm)³ using both retrospective and prospective undersampling at AF = 4 and 8. Results: ZS-SSL preserved spatial resolution more effectively than CS at low SNRs. At AF = 4, image quality was comparable to that of fully sampled data, while noticeable degradation occurred at AF = 8. Experimental validation confirmed these findings, with clear visualization of anatomical structures—such as the accessory nerve—at AF = 4; there was reduced structural clarity at AF = 8. Conclusions: ZS-SSL enables significant scan time reduction in high-resolution MRI of human embryos while maintaining spatial resolution at AF = 4, assuming an SNR above approximately 15. This trade-off between acceleration and image quality is particularly beneficial in studies with limited imaging time or specimen availability. The method facilitates the efficient acquisition of ultra-high-resolution data and supports future efforts to construct detailed developmental atlases. Full article

Creating high-fidelity digital twins (DTs) for Industry 4.0 applications, it is fundamentally reliant on the accurate 3D modeling of physical assets, a task complicated by the inherent imperfections of real-world point cloud data. This paper addresses the challenge of reconstructing accurate, watertight, and topologically sound 3D meshes from sparse, noisy, and incomplete point clouds acquired in complex industrial environments. We introduce a robust two-stage completion-to-reconstruction framework, C2R3D-Net, that systematically tackles this problem. The methodology first employs a pretrained, self-supervised point cloud completion network to infer a dense and structurally coherent geometric representation from degraded inputs. Subsequently, a novel adaptive surface reconstruction network generates the final high-fidelity mesh. This network features a hybrid encoder (FKAConv-LSA-DC), which integrates fixed-kernel and deformable convolutions with local self-attention to robustly capture both coarse geometry and fine details, and a boundary-aware multi-head interpolation decoder, which explicitly models sharp edges and thin structures to preserve geometric fidelity. Comprehensive experiments on the large-scale synthetic ShapeNet benchmark demonstrate state-of-the-art performance across all standard metrics. Crucially, we validate the framework’s strong zero-shot generalization capability by deploying the model—trained exclusively on synthetic data—to reconstruct complex assets from a custom-collected industrial dataset without any additional fine-tuning. The results confirm the method’s suitability as a robust and scalable approach for 3D asset modeling, a critical enabling step for creating high-fidelity DTs in demanding, unseen industrial settings. Full article

Intelligent fault diagnosis for rotary machinery often suffers performance degradation under domain shifts between training and deployment environments. To address this, we propose a robust fault diagnosis framework incorporating three key components: (1) an order-frequency-based preprocessing method to normalize rotational variations, (2) a U-Net variational autoencoder (U-NetVAE) to enhance adaptation through reconstruction learning, and (3) a test-time training (TTT) strategy enabling unsupervised target domain adaptation without access to source data. Since existing works rarely evaluate under true domain shift conditions, we first construct a unified cross-domain benchmark by integrating four public datasets with consistent class and sensor settings. The experimental results show that our method outperforms conventional machine learning and deep learning models in both F1-score and recall across domains. Notably, our approach maintains an F1-score of 0.47 and recall of 0.51 in the target domain, outperforming others under identical conditions. Ablation studies further confirm the contribution of each component to adaptation performance. This study highlights the effectiveness of combining mechanical priors, self-supervised learning, and lightweight adaptation strategies for robust fault diagnosis in the practical domain. Full article

Virulence factors (VFs), produced by pathogens, facilitate pathogenic microorganisms to invade, colonize, and damage the host cells. Accurate VF identification advances pathogenic mechanism understanding and provides novel anti-virulence targets. Existing models primarily utilize protein sequence features while overlooking the systematic protein–protein interaction (PPI) information, despite pathogenesis typically resulting from coordinated protein–protein actions. Moreover, a severe imbalance exists between virulence and non-virulence proteins, which causes existing models trained on balanced datasets by sampling to fail in incorporating proteins’ inherent distributional characteristics, thus restricting generalization to real-world imbalanced data. To address these challenges, we propose a novel Generative and Contrastive self-supervised learning framework for Virulence Factor identification (GC-VF) that transforms VF identification into an imbalanced node classification task on graphs generated from PPI networks. The framework encompasses two core modules: the generative attribute reconstruction module learns attribute space representations via feature reconstruction, capturing intrinsic data patterns and reducing noise; the local contrastive learning module employs node-level contrastive learning to precisely capture local features and contextual information, avoiding global aggregation losses while ensuring node representations truly reflect inherent characteristics. Comprehensive benchmark experiments demonstrate that GC-VF outperforms baseline methods on naturally imbalanced datasets, exhibiting higher accuracy and stability, as well as providing a potential solution for accurate VF identification. Full article