Search Results (421)

Road crack images captured by smartphones suffer from low resolution, uneven illumination, and complex background interference. Mobile devices also have limited resources for real-time high-accuracy segmentation. A two-stage framework combines a high-accuracy dual-path teacher model with a knowledge-distilled lightweight student model. The teacher model integrates a CNN-KAN path for local texture extraction and a VMamba path for global context modeling at linear complexity. A dedicated KAN-based fusion module learns adaptive nonlinear mappings between the two feature streams. On public crack datasets, the teacher model achieves an mIoU of 0.8087 and an mDice of 0.9028. It is then transferred to a self-constructed smartphone dataset built from continuous 30 fps video, where it reaches an mIoU of 0.7084 with strong robustness to illumination and blur. A GAN-based super-resolution strategy further improves the mIoU by 4.01%. A ConvLSTM-based knowledge distillation framework compresses the teacher into a lightweight MobileViT student model. This reduces the parameter count from 57.80 M to 1.57 M and cuts the GPU inference time from 99.56 ms to 1.46 ms, while retaining an mIoU of 0.7078. The deployed student model runs at 15 to 20 frames per second on an Android smartphone. An ablation study confirms that the ConvLSTM-based temporal distillation contributes beyond standard distillation. This framework provides a practical solution for real-time road crack monitoring on smartphones. Full article

Children with autism spectrum disorder (ASD) often exhibit atypical patterns of visual attention allocation and social-cue processing. Eye-tracking scanpath (ETSP) retains information about fixation points, saccade paths and their temporal changes in the form of images, providing an intuitive and computable data representation for analyzing ASD-related visual attention patterns. However, in ASD auxiliary identification studies, the same participant often generates multiple eye-tracking recordings or multiple visual representation samples. If participant independence is not properly considered during model evaluation, the training and test sets may share individualized eye-movement patterns from the same child. In such cases, the model may learn subject-specific characteristics rather than stable and transferable ASD-related visual attention features, leading to an overestimation of its recognition ability on unseen participants. To address this issue, we propose a Global–Local Collaborative Fusion Network (GLCF-Net) under a strict participant-independent splitting protocol. Specifically, the proposed method first maps ETSP images into patch token sequences through a shared Patch Embedding layer. A CNN-based local branch is then used to extract local trajectory morphology, path density, and spatial neighborhood structure, while a ViT-based global branch models cross-region gaze transitions and the overall attention distribution. Finally, a gated adaptive fusion module dynamically integrates local and global information to enhance the representation of stable visual attention features. In the primary repeated stratified five-fold participant-level evaluation, averaging the two out-of-fold probabilities for each participant yielded an Accuracy of 87.0% and a ROC-AUC of 93.7%; the original participant split, retained as a secondary analysis, yielded an Accuracy of 83.52% and a ROC-AUC of 90.27%. Under the reported frozen-backbone configurations, the model also showed a balanced pattern across Accuracy, Recall, and F1-score. These results characterize performance for unseen participants within the same dataset and acquisition conditions. Full article

Hyperspectral image (HSI) classification is central to environmental monitoring, yet real-time deployment of deep learning models on resource-constrained edge platforms remains challenging due to high spectral dimensionality and computational overhead. In this paper, we propose TinyCapsViT, an ultra-lightweight hybrid architecture that integrates convolutional feature extraction, transformer-based attention, and capsule-inspired representation learning for efficient HSI classification under strict TinyML constraints. The model employs a minimal convolutional stem using pointwise and depthwise separable convolutions to capture local spectral-spatial features, followed by a compact tokenization strategy and learnable positional embeddings. A lightweight self-attention module enables global context modeling with reduced computational complexity, while a capsule-inspired refinement block with squash nonlinearity and residual scaling enhances feature discrimination. The proposed architecture contains only 2781 trainable parameters, representing up to a 30× reduction compared with larger architectures such as ResNet and Vision Transformer, and approximately 11× and 9.5× fewer parameters than the CNN and 3D-CNN baselines, respectively. Extensive experiments across benchmark hyperspectral datasets demonstrate that TinyCapsViT achieves up to 99.61% overall accuracy and maintains competitive classification performance despite its substantially reduced model complexity. Although TinyCapsViT does not consistently achieve the highest classification accuracy compared with larger baseline models, it provides a favourable trade-off between classification performance and computational efficiency. These results demonstrate the potential of TinyCapsViT as a practical solution for real-time hyperspectral analysis on resource-constrained edge platforms and UAV-based environmental monitoring applications. Full article

Facial expression recognition technology is vital for security, verification, and personalization, but it faces challenges due to variations in scale, illumination, occlusion, and facial expressions. This paper presents a hybrid architecture that combines Vision Transformers (ViTs) to capture global context with EfficientNet-B3 for multi-scale feature extraction. Unlike simple concatenation, our approach projects the ViT’s [CLS] token and the EfficientNet’s global pooling features into a shared 512-dimensional space before merging, enabling better alignment of global and local features. When tested on the FERPlus dataset, it reaches an accuracy of 94.4 ± 0.3%, surpassing several recent methods, notably existing transformer- and CNN-based methods. Ablation studies show each component’s contribution, with the full model outperforming the no-fusion version by 2.6%. With around 98 million parameters and an inference time of ~23 ms per image, it balances efficiency and high performance, suitable for real-time use on suitable hardware. Evaluation via confusion matrix, t-SNE visualization, and comparisons with recent techniques such as HLA-ViT (90.13%), AU-ViT (90.15%), and CCFER (91.24%) demonstrates its robustness and discriminative feature learning. This work highlights the promise of hybrid deep learning architectures in tackling real-world facial expression recognition challenges. Full article

Reliable multisensor perception is a key requirement for practical waste sorting, yet many low-cost sensor configurations cannot rely on hardware synchronization or carefully controlled acquisition. We present a pilot-scale multisensor waste-sorting testbed that combines an unsynchronized RGB camera with a long-wave infrared (LWIR) camera for object classification on a continuously moving conveyor, and introduce ThermalRGBTrash, a new paired RGB–LWIR video dataset for this task. To enable fusion under asynchronous acquisition, we develop a fully software-based registration pipeline that combines SuperPoint–SuperGlue matching with an adaptive sliding-window strategy designed to recover from long-wave infrared sensor artifacts, including non-uniformity correction events. Across 281,439 matched frame pairs from 19 paired videos, the registration pipeline achieves a mean spatial alignment error of 2.27 pixels and matches 99.98% of attempted frame pairs. We then detect and segment objects with Mask R-CNN, track them across the conveyor, and classify each tracklet using frozen DINOv2 self-supervised Vision Transformer (ViT-L/14) features with a lightweight multilayer perceptron head. RGB and LWIR representations are combined through late fusion. On 550 tracklets under video-disjoint 10-fold cross-validation, the fused pipeline reaches a macro F1 score of 0.924, outperforming RGB alone (0.886) and LWIR alone (0.856). On a mixed-class test set of 351 tracklets reserved exclusively for final evaluation, fusion reaches a macro F1 score of 0.947. The fusion advantage persists across multiple backbone and pretraining choices, while ablation studies support the chosen temporal sampling and pooling design. These results show that accurate RGB–LWIR object classification is achievable without synchronization hardware, and establish ThermalRGBTrash as a benchmark for future work on practical multisensor perception in conveyor-based waste sorting. Full article

Unmanned mining trucks operate in harsh environments such as those in open-pit mines, where online fault diagnosis of critical drivetrain bearings faces severe challenges including slow response, high precision requirements, and strong interference from realistic on-site noise. To address the insufficient generalization capability of existing diagnostic methods in real-world noisy scenarios, this paper proposes a multimodal adversarial transfer learning framework for bearing fault diagnosis in unmanned mining trucks. First, to bridge the domain shift gap between laboratory data and on-site truck data, an augmented multimodal dataset is constructed based on real-vehicle noise grafting. This approach fuses authentic background noise collected from the field with clean laboratory fault signals, thereby simulating graded on-site interference. Second, a deep feature extraction network integrating CNN, ViT, and CBAM attention mechanisms is designed. Building upon this backbone, an adversarial training scheme combined with a hierarchical adaptive fine-tuning strategy is introduced to formulate a domain-adversarial transfer learning model. This model is capable of extracting robust features that are both fault-discriminative and domain-invariant from multimodal signals (vibration and current). Experimental results on the constructed noise-augmented dataset demonstrate that the proposed method maintains high diagnostic accuracy in cross-domain scenarios with strong noise and limited samples, significantly outperforming conventional approaches. This study provides an effective technical pathway for real-time and highly reliable “edge-terminal” fault diagnosis of unmanned mining trucks operating in realistic noisy environments. Full article

Chinese local gazetteer maps contain visual evidence of historical cities and architecture, but their hand-drawn, low-texture, and small-sample characteristics make automatic recognition difficult. This study develops a transfer-learning workflow to classify gazetteer-derived architectural images and examines its usefulness for digital heritage image organization, indexing, and retrieval. A dataset of 685 images covering nine classification labels was constructed from Chinese local gazetteer maps. Five representative models, ResNet-18, MobileNetV3-Small, EfficientNet-B0, DenseNet-121, and ViT-B/16, were trained and compared under the same transfer-learning setting. Their performance was evaluated using accuracy, macro-average F1 score, class-level metrics, confusion matrices, difficult-sample analysis, and Grad-CAM visualization. CNN-based models outperformed the Vision Transformer on this task. EfficientNet-B0 achieved the best results, with a test accuracy of 87.38% and a macro-average F1 score of 86.60%; after disabling class weights, the macro-average F1 score increased to 86.81%. Misclassifications mainly occurred in visually similar or context-dependent category pairs, including gate tower and drum tower, bell tower and drum tower, and buildings inside and outside the city. The results show that compact CNNs can support classification, indexing, and retrieval of hand-drawn architectural images in gazetteer archives, although the workflow should be regarded as an auxiliary tool rather than direct evidence for specific extant buildings. Full article

Medical image analysis has undergone transformative progress with the application of deep learning models. However, existing architectures often struggle to effectively balance local feature extraction with global contextual understanding, which is crucial for complex diagnostic tasks such as Retinopathy of Prematurity (ROP) detection. In this study, we present a pretrained lightweight Conformer model tailored for medical image classification. The model integrates convolutional layers for capturing fine-grained spatial features with transformer blocks that capture long-range dependencies, creating a unified architecture capable of robust representation learning. We evaluate the model across multiple benchmark medical imaging datasets, including ROP, BloodMNIST, RetinalMNIST and other MedMNIST benchmark datasets. With 93.61% accuracy on the ROP dataset and 99.12% accuracy on BloodMNIST, experimental results show competitive classification performance while lowering model complexity to 12.4 million parameters and 3.2 GFLOPs. Experimental results demonstrate that the comparative studies versus CNN-based and transformer-based architectures, such as ResNet50, Swin-Tiny, ConvNeXt-Tiny, Vision Transformer, and MedViT. The findings show that in clinical settings with limited resources, the suggested lightweight Conformer offers a practical and computationally efficient alternative for medical image interpretation. Furthermore, the lightweight design ensures computational efficiency, making it suitable for deployment in resource-constrained healthcare environments. These findings validate the lightweight Conformer model’s potential for scalable, accurate, and real-time medical image classification. Full article

To address the challenges of analyzing vibration signals, overlapping fault characteristic frequency bands, and difficulties in accurately distinguishing similar faults, this study proposes a diagnostic classification scheme for GIS disconnect switch mechanical fault detection. The scheme integrates multi-scale RGB image representation with a Squeeze-and-Excitation Convolutional Neural Network Attention (SE-CNN-Attention) mechanism. First, continuous wavelet transform is employed to decouple the one-dimensional vibration signal into low-, medium-, and high-frequency physical bands, which are then mapped to the red, green, and blue channels to generate a 64 × 64 × 3 RGB time–frequency feature image, enabling three-dimensional encoding of time, frequency, and energy intensity. Based on this approach, a SE-CNN-Attention fusion model is developed. The two-dimensional CNN automatically extracts region-specific features from the time–frequency images. Concurrently, the squeeze-and-excitation (SE) attention mechanism adaptively assigns weights to enhance fault-sensitive frequency bands and suppress noise interference. The model adopts global average pooling instead of traditional fully connected layers, combined with Dropout regularization and a Softmax classifier, to achieve efficient and robust fault classification. Experiments are conducted under four operating conditions with a total of 19,800 sample groups collected. The findings indicate that the proposed approach attains an overall classification accuracy exceeding 94%, outperforming traditional support vector machine (SVM), baseline convolutional neural network (CNN), Vision Transformer (ViT), and ResNet-18 models. The SE-Attention module significantly enhances the model’s focus on key frequency band features while accelerating convergence speed and improving classification stability. This method provides reliable technical support for intelligent maintenance and fault warning of GIS disconnect switch equipment and holds significant engineering application value. Full article

Accurate land cover mapping is a fundamental tool for environmental management and ecosystem monitoring. This study presents a comparative evaluation of four deep learning architectures, namely a Convolutional Neural Network (CNN), UNet++, ConvNeXt and Vision Transformer (ViT), for land cover classification into five primary classes: water, cropland, forest, low/natural vegetation and built-up. The broader Lake Kerkini basin was selected as the primary training and evaluation area. The multispectral input data were generated through Google Earth Engine and consisted of multi-seasonal Sentinel-2 composite mosaics for the 2021 mapping year, covering winter, spring, summer and autumn. To obtain a more reliable performance estimate and mitigate the effects of spatial autocorrelation, a four-fold spatial cross-validation approach was implemented. Under this spatial validation framework, the convolution-based architectures achieved the strongest performance. CNN obtained the highest numerical fold-mean performance, with an overall accuracy of 81.53% and a macro-averaged F1 score (Macro-F1) of 80.09%, followed closely by UNet++ and ConvNeXt. Non-parametric repeated-measures statistical testing indicated a significant overall architecture effect, with CNN, UNet++ and ConvNeXt showing broadly comparable fold-level Macro-F1 performance, while the tested ViT configuration trained from scratch ranked last across all spatial folds. Regional transferability was further evaluated in the nearby independent Lake Doirani region, where the convolutional architectures, particularly CNN and UNet++, showed strong agreement with the WorldCover-derived reference labels without fine-tuning. Finally, feature-importance analysis indicated that specific spectral-seasonal channels, especially the Blue band (B2) in winter and the Short-Wave Infrared band (B12) in summer, were consistently influential in the models’ predictions. Overall, under the tested 2021 Mediterranean case-study conditions, the results highlight the importance of spatially rigorous validation and show that the evaluated convolution-based configurations achieved stronger performance than the tested ViT configuration trained from scratch. Full article

Robotic systems for search-and-rescue operations require robust, non-line-of-sight victim detection in order to locate trapped individuals behind obstacles with high precision. This paper presents a robotic victim-localization system based on a convolutional neural network—vision transformer (CNN-ViT) architecture trained on an open-source radar dataset for through-wall presence detection. In addition to binary presence detection, the proposed approach uses attention information from the transformer layers to estimate the region of the radar signal most relevant to the victim location. The model is deployed on a robotic platform and tested in an additional realistic environment, where classification and distance-estimation outputs are fused into a heatmap-style spatial representation. This enables the system to localize the estimated victim position on the map generated by the robot. To enhance robustness, the system is evaluated using both a leave-one-subject-out (LOSO) protocol on the original open-source radar dataset and additional experimental sessions collected with the robotic platform. On the original dataset, the model achieved victim-detection F1 scores of 82–96% and distance-estimation MAE of 0.25–0.65 m relative to the robot. On newly collected, previously unseen data, it achieved F1 scores of 72–92% and an MAE of 0.16–0.78 m on correctly classified present samples. The complete pipeline was further deployed on a mobile robot in an environment different from the one used to collect the original dataset, illustrating the potential of the proposed system for practical search-and-rescue scenarios. Full article

This paper investigates adversarial robustness and explainability in AI-generated face detection through the Robust and Explainable Detection (RED) framework. RED unifies CNN and Vision Transformer (ViT) backbones with FGSM/PGD adversarial training and Grad-CAM-based interpretability in one reproducible pipeline. Experiments on the Kaggle real/fake face dataset with stratified 70/15/15 splits report accuracy, F1, AUC-ROC, Adversarial Robustness Index (ARI), and Explainability Fidelity (EF). Xception and ResNet-50 reach 97.53% and 96.64% validation accuracy, respectively, while ViT-B/16 attains 69.08%. RED supports forensic and legal applications requiring both detection accuracy and transparent model behavior. Full article

Skin cancer is considered a deadly disease globally, and the timely identification of the disease may save human life. This research presents a CNN–Transformer-based fusion framework for automated multi-class skin lesion classification. This approach combines ResNet50 and Vision Transformer (ViT) to categorize skin lesions using the HAM10000 dataset. To assess their efficacy, a comparison with CNN and ViT models is also carried out. Seven classes of skin cancer are used for training the models, and class weighting is used to correct dataset asymmetry. According to the experimental dataset, the suggested hybrid framework shows improved performance over CNN and ViT, considering the accuracy (0.97) and macro-averaged F1-score (0.95). Furthermore, the efficiency of the suggested model is demonstrated by the fact that it delivers performance that is competitive with several existing approaches. Overall results indicate that hybrid CNN–Transformer architectures present a viable path for automated skin lesion categorization. Grad-CAM++ is integrated to enhance model understanding and promote medical confidence by enabling physicians to view visualizations that show the areas impacting the model’s conclusions. But there are still issues, including poor generalization, computational complexity, and a lack of external validation. Future research will concentrate on enhancing interpretability for practical implementation, integrating clinical information, and evaluating several datasets. Full article

Background/Objectives: Colorectal cancer (CRC) is the second leading cause of cancer-related mortality worldwide, with polyp miss rates of up to 26% reported during colonoscopy and classification accuracy remaining highly operator-dependent. Accurate multi-class polyp subtype classification is clinically critical, as it directly determines treatment decisions: adenomatous polyps require resection, whereas hyperplastic lesions may warrant only surveillance. This study aims to systematically compare five deep learning architectures for five-class gastrointestinal polyp classification and to provide clinically interpretable diagnostic insights through Grad-CAM visualisation. Methods: ResNet50, VGG16, EfficientNet-B3, DenseNet121, and Vision Transformer (ViT-B/16) were evaluated on the Kvasir Dataset V2 (5000 images, five classes) under a unified training and evaluation protocol on common GPU hardware. All models employed ImageNet transfer learning with a redesigned multi-layer classification head. Two optimisation strategies were applied: SGD with cosine annealing for CNN architectures, and AdamW with linear warmup for ViT-B/16. Gradient-weighted Class Activation Mapping (Grad-CAM) was applied to generate spatial attention heatmaps for qualitative clinical interpretation. Results: Under a single 80/10/10 split, ViT-B/16 attained the highest accuracy (97.2%); however, because a single split is sensitive to sampling, the evaluation was strengthened with stratified five-fold cross-validation (mean ± SD). Under cross-validation, EfficientNet-B3 achieved the highest accuracy at 95.90 ± 0.35%, followed closely by ViT-B/16 (95.12 ± 0.72%), then ResNet50 (91.74 ± 0.74%), DenseNet121 (90.32 ± 0.70%), and VGG16 (88.50 ± 1.72%); the small standard deviations indicate that all models, including ViT-B/16, were stable across folds. Pairwise McNemar tests with Holm correction found that every difference was statistically significant (p < 0.05), including the EfficientNet-B3 advantage over ViT-B/16 (p = 0.010). ViT-B/16 thus remained a strong, stable performer that significantly outperformed the three remaining CNNs, while the cross-validated ranking placed the most compact model, EfficientNet-B3, first: a Vision Transformer was highly competitive with, but not superior to, the strongest CNN. A consistent, architecture-agnostic misclassification pattern was identified between dyed-lifted polyps and dyed-resection margins across all five models, consistent with a task-level visual ambiguity that may also reflect overlapping class definitions and annotation factors, with direct clinical implications. Grad-CAM analysis, quantified by attention-entropy and concentration metrics, showed that model attention remained focused on relevant stained tissue regardless of whether predictions were correct, indicating that the dyed-class confusions reflect genuine visual ambiguity rather than a localisation failure. Conclusions: Under cross-validation, EfficientNet-B3 achieved the highest accuracy on the Kvasir V2 five-class task, significantly outperforming all other architectures, with ViT-B/16 being a close and competitive second. The identified confusion between post-procedural chromoendoscopic classes is unlikely to be fully resolved by architectural changes alone and may require higher-resolution imaging or domain expert re-annotation. These findings contribute to the evidence base for explainable deep learning in gastrointestinal endoscopy; external, multi-centre validation remains necessary before clinical adoption. Full article

Background: HER2 status guides targeted therapy in breast cancer but is currently determined by invasive biopsy. Imaging-based HER2 prediction from dynamic contrast-enhanced MRI (DCE-MRI) could provide a non-invasive adjunct decision-support signal, but published models are typically single-center with heterogeneous preprocessing that limits reproducibility. Methods: We trained a Triple-Head Dual-Attention ResNet (THDA-ResNet) that processes three DCE phases (pre-contrast, early post-contrast, and late post-contrast) on the multicenter BreastDCEDL dataset (n = 1149, I-SPY trials), and we compared it with Vision Transformer (ViT) and Convolutional Vision Transformer (CvT) baselines, all ImageNet-pretrained. We benchmarked 14 preprocessing strategies, with and without N4 bias-field correction. External validation used the independent BreastDCEDL_AMBL cohort (43 lesions). AUC confidence intervals used stratified bootstrap; model comparisons used DeLong’s test. Results: THDA-ResNet achieved the highest AUC, 0.74 (95% CI 0.65–0.83), versus 0.66 for ViT and 0.63 for CvT, with the advantage reaching borderline significance over CvT ($p=0.054$) and not significant over ViT ($p=0.14$). At a threshold of 0.7, it retained discrimination (sensitivity 0.41, specificity 0.86), while transformers collapsed to near-trivial classifiers. External AUC was 0.66 (0.49–0.81). N4 correction did not improve performance. Conclusions: Attention-driven CNNs are a strong default for HER2 prediction from DCE-MRI on medium-sized cohorts, and N4 correction can be omitted, simplifying the pipeline. Full article