Lightweight Attention-Based Architecture for Accurate Melanoma Recognition

Abstract

Dermoscopy, a non-invasive imaging technique, has transformed dermatology by enabling early detection and differentiation of skin conditions. Integrating deep learning with dermoscopic images enhances diagnostic potential but raises computational challenges. This study introduces APNet, an attention-based architecture designed for melanoma detection, offering fewer parameters than conventional convolutional neural networks. Two baseline models are considered: HU-Net, a trimmed U-Net that uses only the encoding path for classification, and Pocket-Net, a lightweight U-Net variant that reduces parameters through fewer feature maps and efficient convolutions. While Pocket-Net is highly resource-efficient, its simplification can reduce performance. APNet extends Pocket-Net by incorporating squeeze-and-excitation (SE) attention blocks into the encoding path. These blocks adaptively highlight the most relevant dermoscopic features, such as subtle melanoma patterns, improving classification accuracy. The study evaluates APNet against Pocket-Net and HU-Net using four large, annotated dermoscopy datasets (ISIC 2017–2020), covering melanoma, benign nevi, and other lesions. Results show that APNet achieves faster processing than HU-Net while overcoming the performance loss observed in Pocket-Net. By reducing parameters without sacrificing accuracy, APNet provides a practical solution for computationally demanding dermoscopy, offering efficient and accurate melanoma detection where medical imaging resources are limited.

1. Introduction

Skin cancer, one of the most prevalent forms of cancer, has become a major concern in public health, involving over 5 million cases annually in the US [1,2]. Melanoma is the most dangerous type of skin cancer, according to the American Cancer Society: delayed diagnosis can dramatically increase the five-years chance of fatality to 73%; however, early detection can improve the survival chance to 99%. Many different non-invasive imaging modalities have been explored to help melanoma diagnosis [3,4,5,6,7,8,9,10,11].

From the lesion appearance, there are several diagnostic markers to identify melanoma, such as asymmetry, irregularity of borders, unevenness of distribution of color, and diameter [12]. However visual comparison of lesions on the same person and experience are the common tools that many clinicians use. Dermoscopy image classification using novel deep learning models can assist in the diagnosis of skin cancer. Advancements in deep learning architecture design has led to the development of highly sophisticated models that are capable of analyzing and interpreting medical images. However, nearly all deep learning models are computationally intensive, requiring significant processing power and memory resources. Training these complex models may demand specialized hardware, such as complex Graphics Processing Units (GPUs) and a very large memory. Another formidable challenge is the acquiring, labeling, and storing of extensive data, demanded by more complex and deeper models, which can be a formidable challenge, especially in the medical domain, where patient data are sensitive and privacy regulations are stringent. On top of all that, deeper models can also require lengthy training times: up to hours, days, or even weeks.

A potent model in the field of biomedical image processing is the U-Net architecture that was first applied to biomedical image segmentation tasks, specifically biomedical image analysis difficulties like the ISBI cell tracking challenge [13]. The U-Net architecture derives its name from its U-shaped structure, which consists of a contracting path and an expanding path. Because of its special design, it may generate very accurate segmentation results even with a small amount of training data, which makes it very useful for medical imaging tasks where annotated datasets are sometimes hard to come by and expensive to purchase. The U-Net architecture consists of two main components: the contracting path and the expansive path. The contracting path, an encoder in a typical convolutional neural network (CNN), includes multiple convolutional and pooling layers, which reduces the spatial dimensions of the input image while capturing its hierarchical features [13]. This process effectively extracts high-level representations of the input image. The expansive path mimics the decoder in a CNN, and comprises up-sampling and convolutional layers that gradually recover the spatial resolution of the feature maps produced by the contracting path. Although CNNs have demonstrated impressive performance in a variety of computer vision tasks, the growing size of models has made it more difficult to use them in embedded vision, mobile, and real-time applications. By concatenating feature maps from the contracting path at corresponding layers in the expansive path, the U-Net model incorporates both local and global contextual information, facilitating precise localization of objects of interest in the input image. Moreover, the skip connections between the layers enable the U-Net model to preserve fine-grained details during the up-sampling process, thereby mitigating issues such as information loss and gradient vanishing commonly encountered in deeper networks. Half U-Net (HU-Net) is a modified version of the U-Net architecture where only a portion of the original model is utilized for a specific task [13,14,15,16,17]. HU-Net consists solely of the encoding (contracting) path of the U-Net architecture for classification tasks, with the decoding (expanding) path removed. More specifically, the HU-Net architecture first encodes the input dermoscopy image into a latent representation, capturing essential hierarchical features relevant to classification; then a fully connected model can be utilized for discrimination of two or more classes.

Recently, researchers have investigated distributed and privacy-preserving paradigms to enable scalability of learning and data sharing in healthcare. For instance, Selvaraj et al. (2025) [18] proposed a federated learning and Digital Twin-enabled distributed intelligence framework for 6G autonomous transport systems, demonstrating how decentralized architectures could facilitate collaborative purposes without exchanging data. While this approach highlights the potential of distributed intelligence for privacy-sensitive domains like medical imaging, it also introduces challenges related to communication overhead, synchronization between clients, and the need for powerful edge hardware.

Recently, Song et al. (2024) [19] presented a framework that employs annotations created using AI to improve quality in medical image segmentation. The work addresses issues of limited and inconsistent manual labeling. Their results demonstrate that synthetic annotation can be effective in improving the quality of training, especially in the context of large and diverse datasets. However, depending on AI-generated labels can transfer bias/error from the generative model, and there must be proper validation and human oversight. Issues of label reliability in medical imaging tasks are also important to consider in our work, as mentioned in the data Section 3.2, where we describe the use of well-curated ISIC datasets, while ensuring the data distribution was balanced using oversampling methods where necessary.

Google created the MobileNet [20] family of lightweight deep neural networks for embedded and mobile vision applications with constrained processing power. Depth-wise separable convolutions, which divide the conventional convolution operation into depth-wise and point-wise convolutions, are its key contribution. They greatly lower computational costs and model sizes while preserving high accuracy. Through the use of width and resolution multipliers, MobileNet offers developers the freedom to scale the model to meet certain hardware limitations [21]. The reason behind developing these models is to have a variant of the existing light models which will be easily deployable in memory-constrained scenarios. Pocket-Net [22], a lightweight version of U-Net with fewer parameters, is designed to address computational and memory constraints while maintaining competitive performance in medical images segmentation and classification tasks. Pocket-Net is typically lighter than MobileNet. By using fewer filters or more efficient layers, Pocket-Net minimizes the model size and computational requirements. Light models (e.g., MobileNet and Pocket-Net) are also designed for embedded board implementation. The limited memory of micro-controller units is one of the limitations of these architectures for hardware deployment. Despite being lighter than other CNN models, MobileNet typically has inferior classification accuracy. Therefore, more hyperparameters for modifying the model’s size have lately been proposed in an effort to strike a compromise between accuracy and complexity [23]. However, models become thick and unsuitable for devices with limited processing resources when the number of parameters is greatly increased.

We aim to reduce computational cost and memory usage by utilizing the Pocket-Net approach and incorporating attention mechanisms, resulting in our proposed Attention Pocket-Net (i.e., APNet). APNet integrates squeeze-and-excitation (SE) blocks into Pocket-Net, enabling the model to focus adaptively on the most relevant features for melanoma detection, further enhancing classification accuracy. This development holds significant promise for researchers who are end-users of neural networks in the domain of dermatology image analysis. The streamlined workflow facilitated by APNet’s resource-efficient architecture not only enhances the model training speed but also enables researchers to handle large and complex datasets more easily.

2. Related Works

Modern deep CNNs are convoluted and require a lot of computational power, especially when the convolutional layers that dominate runtime during prediction are being evaluated. By reducing the size of these networks, specifically by means such as pruning off complete feature maps, inference times can be made much faster. This is essential for performing real-time or near-real-time applications [24]. Although large neural networks are complex, they require large amounts of storage, memory bandwidth, and computational resources. Reducing the size of data makes it possible to use these images on devices with limited resources, such as smartphones, IoT devices, and other edge computing platforms. This not only reduces demands on computer hardware but also extends battery life in mobile devices by reducing power requirements for sample calculations [25]. By interpolating greedy criteria-based pruning and fine-tuning via backpropagation we can achieve a balance where the pruning mesh retains its effort while being efficient. Additionally, reducing the mesh complexity can help reduce overfitting because the thumbnails may not be susceptible to noise capture and lack samples in training data. Deep networks with a large number of parameters have the capability to learn complex features and functions. However, more parameters need more data. On the other hand, shallow networks with few trainable parameters restrict network learning capabilities and will cause overfitting. Network pruning techniques eliminate less-important connections of the network [26], but numerous conditional operations and additional representations are required to indicate the position of zero or non-zero parameters.

Alongside attempts to design novel architectures of deep networks, the need for computational speed and memory requirements for large dermoscopy datasets encourages scholars to develop methods to reduce the number of parameters for these networks A summary is provided in

Table 1. Summary of popular neural network size reduction methods, highlighting their mechanisms, advantages, and limitations for achieving computational efficiency and maintaining performance.

Method	Description	Advantageous	Disadvantageous
Pruning [36,37]	Pruning aims to remove unnecessary components of a network, such as weights or neurons, without significantly affecting accuracy.	➢ Reduces memory and computational requirements. ➢ Retains much of the original model’s performance. ➢ Effective for sparse or over-parameterized networks.	➢ Time-intensive to tune the pruning process ➢ Often requires fine-tuning after pruning to recover performance.
Knowledge Distillation [38]	Trains a smaller model (student) to mimic the outputs of a larger, pre-trained model (teacher).	➢ Produces smaller models with high accuracy, often close to the teacher model. ➢ Can transfer robustness and generalization from teacher to student.	➢ Requires training an additional large teacher model. ➢ Performance depends on the similarity of tasks and quality of the teacher.
Lightweight Architecture Design [20,39,40]	Develop models with efficiency in mind by rethinking the architecture.	➢ Designed from scratch to be resource-efficient. ➢ Combines techniques like depth-wise separable convolutions and bottleneck blocks.	➢ May underperform compared to larger models on complex tasks. ➢ Requires careful design and optimization.
Decomposes weight matrices into lower-rank approximations, reducing the number of parameters [41,42]	Decomposes weight matrices into lower-rank approximations, reducing the number of parameters.	➢ Effective for reducing computational complexity in matrix operations. ➢ Maintains reasonable performance when applied correctly.	➢ Limited by the rank approximation; aggressive factorization can degrade accuracy. ➢ Suitable mainly for over-parameterized layers.
Neural Architecture Search (NAS) [43]	Uses automated algorithms to design optimal neural network architectures.	➢ Finds architecture with an optimal balance of accuracy and efficiency. ➢ Can be customized for specific hardware.	➢ Extremely computationally expensive during the search phase. ➢ Requires advanced expertise and resources.

. Briefly, these attempts include efficient networks, which are often designed to perform well while using fewer parameters and computations [27,28]; model compression, which aims to reduce the computational requirements of deep networks while maintaining performance [29,30,31,32]; transfer learning and pre-trained models that leverage the knowledge gained from training a neural network on one task and apply it to a different but related task [33,34,35]; and architectural innovation, which might include modifications to well-known architectures, such as skip connection variations, and pruning. Pruning refers to eliminating less important neurons or connections by means of reducing the model’s computational complexity and memory footprint [24,25]. Pruning is typically applied to a pre-trained model or after a model has been trained on a large dataset. Models are typically trained on large, diverse datasets for generic tasks such as image classification or language modeling. This means that the model might have learned features or patterns that are not essential for the specific problem at hand.

3. Materials and Methods

3.1. HU-Net to Pocket-Net to APNet

The conventional HU-Net architecture [13], while highly effective, includes a substantial number of parameters, leading to increased memory and GPU requirements [44]. The approach of holding the number of feature maps constant for each convolution layer (in other words the number of output channels is the same as the number of input channels for all convolutional layers, except the final output layer), named “Pocket-Net” to imply the network’s compactness (small enough to fit in one’s back pocket), employs a hierarchical, scale-dependent strategy to eliminate the need for doubling the number of features at each down-sampling step. By keeping the number of feature maps constant, the number of network parameters is severely reduced, resulting in a significant computational cost reduction. Figure 1 shows how Pocket-Net can be used to modify a typical HU-Net structure. Pocket-Net is a compelling alternative in this scenario as it offers the advantage of significantly reduced parameters, thereby optimizing computational resources [22].

For many convolutional neural network (CNN) architectures, including popular methods like DenseNet [45], ENet [46], and fully convolutional networks (FCNs) [47], the number of feature maps in each convolution operation is doubled when the image resolution decreases. This strategy aims to compensate for the information loss caused by down-sampling by increasing the number of feature maps. However, traditional methods, such as wavelets and multigrid techniques, have long employed a hierarchical framework for scale-dependent information decomposition. This approach, involving the construction of resolution grids, efficiently preserves information without requiring the exponential growth of channels during down-sampling. Yet U-Net contains this hierarchical scale-dependent approach, suggesting there is no need for doubling the number of feature maps in each convolution operation, and that there is already present the ability to maintain effective multi-scale information handling.

While Pocket-Net achieves resource efficiency, APNet takes this further by incorporation of squeeze-and-excitation (SE) blocks [48,49,50,51] to enhance performance accuracy. The attention mechanism can be considered as a bias that allocates processing resources towards the most informative components of the input image. These SE blocks allow APNet to focus on critical features within dermoscopy images, improving the model classification accuracy over the original Pocket-Net model. To emphasize important channels of the features, a channel-specific descriptor learned by global average pooling has been used to take account of the spatial dependency.

Let the input image be represented as $X\in {\mathbb{R}}^{H\times W\times C}$ where $H$ and $W$ are the height and width, and $C$ is the number of channels (e.g., $C =1$ for greyscale images). In APNet, we use depth-wise separable convolutions (Figure 2) to reduce the number of parameters and computational load in each convolutional layer. For a given layer with input feature maps $X$ and convolutional filter $F$, a depth-wise separable convolution operation can be represented as two consecutive operations:

Depth-wise Convolution: Each channel in $X$ is convolved separately with a kernel of size $k\times k$, producing intermediate feature maps $X_d\in {\mathbb{R}}^{H^{’}{\times W}^{’}\times C}$.

$$X_d=X\ast F_d$$

(1)

where $F_d\in {\mathbb{R}}^{k\times k\times C}$ is the depth-wise filter applied to each channel individually.

Point-wise Convolution: A $1\times 1$ convolution combines the depth-wise features across channels, producing the final output $X_d\in {\mathbb{R}}^{H^{’}{\times W}^{’}\times C}$, where $D$ is the number of output channels.

$$X_p=X_d\ast F_p$$

(2)

where $F_p\in {\mathbb{R}}^{1\times 1\times C\times D}$.

This depth-wise separable convolutional structure reduces the parameters and computation significantly compared to standard convolutions. To enhance feature selection, APNet integrates SE blocks in specific layers. The SE block first “squeezes” spatial information and then “excites” or re-weighs each channel. For an input feature map $X\in {\mathbb{R}}^{H\times W\times C}$:

Squeeze Operation: Global average pooling (GAP) is applied to each channel, reducing $X$ to a vector $z\in {\mathbb{R}}^C$, where each element $z_c$ represents the average activation of channel $c$:

$$z_c={\displaystyle \frac{1}{H\times W}}\sum _{i =1}^H\sum _{j =1}^W{X}_{i,j,c}$$

(3)

Excitation Operation: The channel descriptor $z$ is passed through two fully connected (dense) layers with weights $W_1\in {\mathbb{R}}^{{\displaystyle \frac{c}{r}}\times C}$ and $W_2\in {\mathbb{R}}^{C \times {\displaystyle \frac{c}{r}}}$, where $r$ is the reduction ratio. These layers learn the channel dependencies, producing a reweighting vector $s \in {\mathbb{R}}^C$:

$$s =\sigma \left(W_2\delta \left(W_{1}z\right)\right)$$

(4)

where $\delta$ is the ReLU activation function and $\sigma$ is the sigmoid function. The output vector $s$ contains weights for each channel that are then used to rescale $X$.

Scaling: Finally, each channel in $\mathrm{X}$ is reweighted by $\mathrm{s}$, yielding the output ${\mathrm{X}}_{\mathrm{S}\mathrm{E}}$:

$$X_{SE}=X.s$$

(5)

where the dot operator ($.$) denotes channel-wise multiplication.

In the above formulation, all variables are dimensionally consistent with the SE literature [48]. The feature maps $X\in {\mathbb{R}}^{H \times W \times C}$ are unitless activations derived from convolutional operations. The global average pooling (Equation (3)) produces a dimensionless channel descriptor $z\in {\mathbb{R}}^C$, where each element represents the normalized average response of one channel. The reduction ratio $r$ is a scalar hyperparameter that controls the bottleneck dimension in the excitation operation. Following the SE formalism, the mapping $s =\sigma \left(W_2\delta \left(W_{1}z\right)\right)$ (Equation (4)) yields a reweighting vector $s\in {\mathbb{R}}^C$, where each element $s_c\in \left[\mathrm{1,0}\right]$ serves as a multiplicative coefficient applied to the corresponding feature channel in Equation (5).

The reduction ratio r in the squeeze-and-excitation (SE) block controls the dimensionality reduction applied to the channel descriptor during the “squeeze” operation. Specifically, the number of neurons in the first fully connected (FC) layer is reduced from (the number of input channels) to $C/r$, allowing the network to learn inter-channel dependencies while significantly lowering computational cost and overfitting risk.

$$S=\sigma \left(W_2\delta \left(W_{1}z\right)\right)$$

(6)

where $W_1\in {\mathbb{R}}^{C/r\times C}$ and $W_2\in {\mathbb{R}}^{C/r\times C}$, $\delta$ is the ReLU activation function, and $\sigma$ is sigmoid function. The reduction ratio $r$ was kept fixed at $r = 16$, which is a commonly used value in the original SE-Net paper and subsequent lightweight CNN studies [52,53].

This choice was motivated by the need to balance two factors: (1) avoiding an excessive increase in parameters that would undermine the lightweight design philosophy of APNet, and (2) retaining enough capacity in the excitation step to model meaningful channel interdependencies.

3.2. Datasets

We have tested the model separately on several widely regarded dermoscopy image datasets provided by The International Skin Imaging Collaboration (ISIC). These datasets include various types of skin diseases. The decision to analyze images from each dataset separately stems from the fact that subsequent datasets include prior images and thus it is not possible to train on one dataset and utilize a different dataset for testing. In addition, there are unique characteristics and challenges posed by each dataset. ISIC datasets represent a spectrum of dermatological cases, showcasing variations in skin conditions, lighting conditions, and imaging technologies. By individually examining these datasets, we aim to capture the nuances intrinsic to each set, allowing for a more nuanced evaluation of HU-Net, Pocket-Net, and proposed model (i.e., APNet) performance. This approach not only provides a robust assessment of the model’s generalizability across different datasets but also enables insights into its adaptability to varying data distributions and characteristics commonly encountered in real-world dermatology practices.

Class imbalance, where one class significantly outnumbers the others, can lead to poor generalization and performance degradation, as the model may prioritize accuracy on the majority class at the expense of minority classes. Therefore, different data augmentation methods are proposed to resolve such an issue. To address this issue, an oversampling method is employed. Oversampling involves augmenting the minority class (melanoma) by generating synthetic samples or replicating existing ones until the class distribution is more balanced. This ensures that the model receives sufficient exposure to the minority class during training, preventing it from being overlooked or underrepresented. The skewed distribution of classes within a dataset leads to biased model training, poor generalization, and misleading evaluation metrics. To address this problem various techniques can be employed, such as oversampling the minority class or undersampling the majority class. Based on our experience, we employed oversampling of minority classes in the training stage. The detailed information of classification datasets after augmentation and resizing is given in

Table 2. Detailed information of classification dataset after augmentation. A balanced representation of dataset classes helps the model learn discriminative features from both classes equally, preventing poor generalization and reducing bias towards the majority class. After oversampling, the dataset now contains an equal number of samples for both the melanoma and benign classes, ensuring a balanced distribution during training. For consistency across datasets and models, all images were resized to 256 × 256 pixels before being fed into the network.

Dataset	Melanoma (Augmented)	Benign (Augmented)	Training Set	Validation Set	Test Set
ISIC 2017	347 (1627)	1627 (same)	2602	325	325
ISIC 2018	6705 (8902)	8902 (same)	14,244	1780	1780
ISIC 2019	4522 (20,808)	20,810 (same)	33,295	4161	4162
ISIC 2020	584 (32,542)	32,542 (same)	52,068	6508	6508

. Examples of dermoscopy image datasets provided by The International Skin Imaging Collaboration (ISIC) are shown in Figure 3. ISIC 2017 [54] consists of melanoma and benign nevi images made for three tasks: lesion segmentation, dermoscopic feature classification, and disease classification. The ISIC 2018 [55] images were collected over 20 years from the Department of Dermatology at the Medical University of Vienna, Austria, and the skin cancer practice of Cliff Rosendahl in Queensland, Australia. Images taken before introduction of digital cameras were digitized with a Nikon Coolscan 5000 ED scanner at a 300 DPI quality and 15 × 10 cm size. These images were then manually cropped to 800 × 600 pixels at 72 DPI with the lesion centered and histogram corrections were applied to enhance the quality of the images. The other part of the data was collected with a digital camera by the University of Vienna by the MoleMax HD dermoscopy system. The dataset includes skin lesions of melanocytic nevi, melanoma, benign keratosis-like lesions, basal cell carcinoma, actinic keratoses, vascular lesions, and dermatofibroma, providing a diverse set of conditions for training and evaluating skin lesion classification models. ISIC 2019 [56] has been collected over 16 years (2010–1016) by the Department of Dermatology at the “Hospital Clínic de Barcelona” on their patients. They captured images via their high-resolution cameras and filtered the images to have better visualization; finally they then manually revised the images to check corresponding diagnoses against a reference table. The data can be categorized into nevus, melanoma, basal cell carcinoma, seborrheic keratosis, actinic keratosis, squamous cell carcinoma, dermatofibroma, vascular lesion, and “other” (lesions not contained in any of the other categories). ISIC 2020 [57] images have been collected from six different centers over 22 years (1998–2020) with or without polarized light in contact or non-contact dermoscope conditions. Notably, polarized light enables a clearer visualization of deeper skin structures, even in cases where there is no direct skin contact with the imaging interface or the utilization of a liquid interface. For some of the images, diagnosis labeling and internal biopsies were reviewed, and for the others the lesions were monitored over six months and considered benign if they remained without further granularity [58]. These images were categorized into nevi, atypical melanocytic proliferation, café-au-lait macule, lentigo NOS, lentigo simplex, solar lentigo, lichenoid keratosis, and seborrheic keratosis.

All procedures are implemented in a single Nvidia Geforce RTX 4090 GPU with 64 GB RAM, using the Tensorflow 2.10 framework. We used an Adam optimizer with a learning rate of 0.001, categorical cross-entropy loss function, and a batch size of 32 and 1000 epochs for all datasets.

4. Results

Using the implementation details mentioned above, we report performance metrics of HU-Net, Pocket-Net and APNet architectures with ISIC challenge classification task datasets from years 2017 to 2020. Note that we performed all experiments with the same GPU and API. The metrics used for evaluation include accuracy, sensitivity, specificity, the area under the receiver operating characteristic curve (AUC), ROC curve, and confusion matrix. Overall, while all architectures demonstrated high levels of performance, HU-Net consistently achieved slightly higher accuracy scores on most datasets. All datasets had reasonably similar sensitivity, meaning they correctly predicted presence of melanoma 96% of the time or better (although HU-Net performance for 2019 was closer to 95% sensitivity). The difference was in specificity: Pocket-Net had about double the number of false positives in each dataset (somewhat more for 2020), meaning while HU-Net specificity ranged from 92.2 to 99.9%, Pocket-Net specificity ranged from 85.7 to 98.7% (both are far better than any individual dermatologist). APNet enhanced the performance of Pocket-Net in terms of specificity, achieving values ranging from approximately 88% to 99.2%. The performance of HU-Net, Pocket-Net, and APNet, as well as some other complexity reduction methods including MobileNet [20], 3D-DenseUNet-569 [37], and Alex-Net Weight pruning [59], in terms of number of parameters, accuracy, sensitivity, specificity, and AUC on the four datasets is reported in

Table 3. Performance comparison between HU-Net, Pocket-Net, APNet. Different network parameter reductions mentioned in Table 1.

Dataset	Architecture	Number of Parameters	Accuracy	Sensitivity	Specificity	AUC
ISIC 2017	Mobile net	1,882,498	95.36%	93.85%	96.90%	0.9819
	3D-DenseUNet-569	802,178	54.80%	69.34%	40.11%	0.5676
	Alex-Net Weight pruning	656,511	73.83%	96.34%	51.20%	0.8957
	HU-Net	1,304,866	96.30%	97.50%	95.07%	0.986
	Pocket-Net	70,114	93.55%	96.90%	90.20%	0.977
	APNet	70,762	94.5%	98.14%	90.80%	0.986
ISIC 2018	Mobile net	1,882,498	98.90%	98.20%	99.56%	0.9989
	3D-DenseUNet-569	802,178	95.20%	93.70%	96.73%	0.9648
	Alex-Net Weight pruning	656,511	78.12%	98%	58.30%	0.9457
	HU-Net	1,304,866	98.44%	99.66%	97.17%	0.996
	Pocket-Net	70,114	96.80%	99.20%	94.40%	0.988
	APNet	70,762	97.90%	99.76%	96.04%	0.9908
ISIC 2019	Mobile net	1,882,498	96.40%	94.78%	98.05%	0.9917
	3D-DenseUNet-569	802,178	92.21%	9.09%	93.26%	0.9584
	Alex-Net Weight pruning	656,511	82.75%	94.40%	71.14%	0.9389
	HU-Net	1,304,866	93.80%	95.36%	92.24%	0.976
	Pocket-Net	70,114	91.26%	96.88%	85.70%	0.96
	APNet	70,762	92.6%	97.27%	87.9%	0.965
ISIC 2020	Mobile net	1,882,498	99.56%	99.37%	99.70%	0.9999
	3D-DenseUNet-569	802,178	98.60%	99.50%	97.70%	0.9834
	Alex-Net Weight pruning	656,511	97.75%	100%	95.46%	0.9999
	HU-Net	1,304,866	99.95%	100.00%	99.90%	0.999
	Pocket-Net	70,114	99.20%	99.76%	98.73%	0.999
	APNet	70,762	99.60%	100%	99.20%	0.999

. To assess the impact of minority class oversampling, we conducted an additional experiment where APNet was trained without any oversampling during the training stage, while keeping all other parameters identical. The results, presented in Table S1 (Supplementary Section), demonstrate a noticeable degradation in performance, particularly in sensitivity and balanced accuracy when oversampling was omitted. This confirms that minority class oversampling plays a critical role in improving the model’s ability to correctly identify melanoma cases and achieve a more balanced performance across classes.

To provide a visual interpretation of the suggested network and comparison between former architectures (e.g., HU-Net and Pocket-Net), we applied Gradient-weighted Class Activation Mapping (Grad-CAM) to highlight the regions of the focus for the classification task that most strongly influence the model’s predictions, allowing for an intuitive comparison of feature attention between classes. As shown in Figure 4, the heatmaps reveal that APNet effectively compensates for weakness of Pocket-Net and focuses on critical lesion regions, capturing discriminative patterns of the images.

The network parameters are reduced from 1,304,866 (HU-Net) to 70,114 (Pocket-Net) and 70,762 (APNet), resulting in both Pocket-Net and APNet having fewer parameters and thereby requiring less computation during both training and inference. This can lead to faster training times and lower inference latency, which is beneficial for real-time applications and resource-constrained environments. As shown in Figure 5, these models also consume less memory, which is crucial for deployment on devices with limited memory resources, such as mobile phones and micro-controllers. However, reducing the number of parameters in these models led to a loss of performance, limiting the model’s ability to capture fine-grained features in the data. Figure 5a compares training speed and memory usage of the original HU-Net structure and Pocket-Net and APNet for different batch sizes. Pocket-Net sped up training time for ISIC 2017, 2018, 2019, and 2020 datasets by 40% on average. Increasing batch size is a straightforward way to boost GPU speed, but is not always successful. Also, bigger batch sizes computed in parallel on the GPU means the batch size can be large as long as there is enough GPU memory. Figure 5b demonstrates the effect of batch sizes and architecture on memory usage.

The deep learning frameworks and GPUs tend to allocate GPU memory up front, especially during model initialization and graph construction. This pre-allocation ensures efficient memory utilization and avoids potential memory fragmentation issues during training. However, this behavior can sometimes lead to seemingly excessive memory consumption, particularly when working with large datasets or models. The HU-Net model, with over a million parameters, involves more complex operations or layers that require larger memory buffers or additional memory overhead during computation. Therefore, the model with a larger number of parameters would tend to go out-of-memory.

To provide a hardware-independent measure of computational cost, we calculated the number of floating-point operations (FLOPs) for each model using TensorFlow’s built-in profiler. The FLOPs metric quantifies the total number of arithmetic operations required for a single forward pass and offers a more generalizable comparison than GPU memory usage, which can vary across hardware and framework configurations. Using an input size of 256 × 256 × 3, the total FLOPs were approximately 1.95 GFLOPs for HU-Net, 0.68 GFLOPs for Pocket-Net, and 0.42 GFLOPs for the proposed APNet. These results demonstrate that APNet achieves a substantial computational reduction of nearly 78% compared to HU-Net while maintaining competitive classification performance, confirming its suitability for deployment in resource-constrained or real-time clinical applications. The ROC curves shown in Figure 6 demonstrate that HU-Net significantly outperforms Pocket-Net, while APNet shows the capability to improve the performance of the light model (i.e., Pocket-Net), highlighting the importance of incorporating attention mechanisms for improved feature prioritization and class discrimination. For Pocket-Net, the smooth rise in the ROC curve at the start of the curve (near the origin) of ISIC 2017–2019 indicates less sensitivity at low false positive rates compared to HU-Net. APNet compensated for the degradation of Pocket-Net by sharpening the curves, showing a more pronounced bow compared to Pocket-Net, reflecting its higher AUC and improved capability for distinguishing between classes.

The confusion matrices in Figure 7 demonstrate that APNet effectively addressed misclassification issues observed in the benign class with Pocket-Net for ISIC 2017–2019, highlighting its ability to enhance feature representation and improve class differentiation.

5. Discussion

The proposed APNet architecture demonstrates a promising balance between efficiency and performance in dermoscopic image classification. By integrating squeeze-and-excitation (SE) blocks into a lightweight Pocket-Net backbone, APNet enhances channel-wise feature recalibration, allowing the model to focus on more discriminative regions while maintaining a drastically reduced number of trainable parameters. This design enables faster inference, lower memory usage, and reduced computational demand—making APNet particularly suitable for deployment in clinical environments with limited hardware resources or real-time diagnostic applications.

Compared to traditional U-Net-derived architectures, APNet achieves comparable or superior classification accuracy, with a 94.6% reduction in parameters and approximately 40% less computational time. These results underscore the potential of attention-integrated lightweight models to deliver high diagnostic reliability without the overhead of large-scale networks. Furthermore, APNet’s ability to generalize across multiple ISIC datasets indicates robustness against inter-dataset variability, a key challenge in dermatological image analysis. As our goal was to demonstrate the generalizability of a single lightweight configuration rather than tailoring hyperparameters per dataset, we did not perform specific tuning across ISIC 2017–2020 datasets,

However, some limitations should be acknowledged. First, despite its efficiency, APNet exhibited a slight drop in specificity compared to the full HU-Net, likely due to reduced feature representation capacity in deeper layers. Second, while the SE block improves feature weighting, it primarily captures channel dependencies and may not fully exploit spatial relationships—an aspect where hybrid attention mechanisms (e.g., CBAM or ECA) could offer further improvement. Additionally, the current evaluation focused on publicly available ISIC datasets; real-world deployment would benefit from testing on diverse, clinically acquired data to validate generalizability under different imaging conditions and device settings.

Overall, APNet provides an effective trade-off between model complexity and diagnostic accuracy. Its compact design offers a practical foundation for future extensions, such as integrating hybrid attention modules, self-supervised pre-training, or federated learning frameworks to further enhance reliability and scalability in real-world dermatology applications.

With the help of advanced 3D imaging modalities such as optical coherence tomography and image processing we hope to improve the performance of dermoscopy for melanoma detection [60,61,62,63,64].

6. Conclusions

While deep learning models more often offer impressive results against problems (e.g., classification and segmentation), they come with several drawbacks, including computational complexity and limited memory sizes for large datasets due to the massive number of network trainable parameters. Hence, parameter reduction could be a solution to mitigate these disadvantages and improve the efficiency, provide the possibility to work with big datasets, and enable embedding these complex models into portable devices. In this study we investigated the effects of network size reduction methods on trainable parameter and comparing computational costs of HU-Net, Pocket-Net, and our proposed model, APNet, for analyzing dermatology image data. The pursuit of practical and resource-efficient solutions for accurate diagnosis is of paramount importance, especially in clinical settings. This study addressed some of the computational challenges presented by the traditional HU-Net and other network size reduction methods for classification of dermatology images while maintaining their performance. APNet, with its integration of SE blocks, demonstrated enhanced feature focus and accuracy, with a 94.6% reduction in parameters and approximately 40% reduction in computational time compared to HU-Net. This result indicates that high numbers of parameters may not be necessary for dermatology image classification. APNet’s performance across multiple dermoscopic datasets further confirms that the reduced architecture can effectively capture critical features. While Pocket-Net demonstrated this to some extent, it exhibited limitations in performance that APNet successfully addressed, ensuring both efficiency and accuracy are maintained. Specifically, APNet achieved around 1% classification accuracy improvement for ISIC 2017–219 and 0.4% for ISIC 2020 compared to Pocket-Net. Meanwhile, Pocket-Net showed dramatic specificity degradation of around 6.5% for the ISIC 2019 dataset; APNet compensates for this issue with a 4.3% sensitivity loss. The same results can be seen for ISIC 2018, where APNet reduced the 2.77% specificity degradation of Pocket-Net to 1.13% and achieved a slight improvement (around 0.5%) for ISIC 2017 and ISIC 2020. The attention mechanism in APNet enhances feature selection, contributing to improved classification accuracy while preserving efficiency.

It is important to note that while APNet achieved substantial gains in efficiency and overall accuracy, HU-Net occasionally maintained higher specificity due to its larger representational capacity. This highlights the trade-off between model complexity and performance: APNet is best suited for resource-limited or real-time clinical settings where computational cost and memory usage are critical, whereas HU-Net may still be preferable when maximum specificity is required and computational resources are not a constraint.