A Hybrid Model for Fluorescein Funduscopy Image Classification by Fusing Multi-Scale Context-Aware Features

Abstract

With the growing use of deep learning in medical image analysis, automated classification of fundus images is crucial for the early detection of fundus diseases. However, the complexity of fluorescein fundus angiography (FFA) images poses challenges in the accurate identification of lesions. To address these issues, we propose the Enhanced Feature Fusion ConvNeXt (EFF-ConvNeXt) model, a novel architecture combining VGG16 and an enhanced ConvNeXt for FFA image classification. VGG16 is employed to extract edge features, while an improved ConvNeXt incorporates the Context-Aware Feature Fusion (CAFF) strategy to enhance global contextual understanding. CAFF integrates an Improved Global Context (IGC) module with multi-scale feature fusion to jointly capture local and global features. Furthermore, an SKNet module is used in the final stages to adaptively recalibrate channel-wise features. The model demonstrates improved classification accuracy and robustness, achieving 92.50% accuracy and 92.30% F1 score on the APTOS2023 dataset—surpassing the baseline ConvNeXt-T by 3.12% in accuracy and 4.01% in F1 score. These results highlight the model’s ability to better recognize complex disease features, providing significant support for more accurate diagnosis of fundus diseases.

1. Introduction

Chronic fundus diseases are common and serious diseases in ophthalmology, which in advanced stages can lead to severe loss of vision or even complete blindness. Studies report that approximately 196 million people worldwide are affected by age-related macular degeneration (AMD), with cases expected to rise to 288 million by 2040 [1]. Diabetic retinopathy (DR) is a major cause of blindness [2], highlighting the need for early detection and intervention. Traditional fundus disease diagnosis relies on expert experience, which is time-consuming and limited by expertise. Manual analysis requires extensive knowledge and is prone to observer variability, increasing the risk of misdiagnosis. Thus, the demand for efficient and accurate automated diagnostic tools is growing [3].

Artificial intelligence (AI), especially deep learning, is advancing rapidly and is increasingly used in ophthalmic disease screening and classification [4,5]. However, most studies focus on a single or a few diseases, such as DR [6], AMD [7], and glaucoma [8]. These approaches are difficult to cope with the diversity and complexity of retinal diseases, especially when multiple pathologies coexist. For example, uveitis, a condition prevalent among young adults, has diverse types and complex etiologies, but research on its automated classification remains limited [9,10].

Furthermore, research progress varies across imaging modalities. In fundus image classification, attention mechanisms and multi-scale feature fusion strategies are widely used and have been proven to improve classification performance effectively. Li et al. [11] proposed a neural network model based on attention mechanisms and feature fusion for multi-label fundus image classification. The model achieves outstanding performance on the ODIR dataset, with an accuracy of 94.23% and a recall of 99.23%. It offers a novel approach to multi-label fundus disease diagnosis and shows big improvements in attention mechanisms and multi-scale feature fusion. Yan et al. [12] proposed Fundus-DANet, a multi-disease recognition model based on dilated convolution and fused attention mechanisms. Experiments on the OIA-ODIR dataset achieved an accuracy of 93%. The results of their study show that fused attention mechanisms can help with problems like morphological feature extraction and lesion variability.

In retinal vascular imaging, FFA images are the gold standard for visualizing the retinal vascular system [13,14]. However, studies on multi-lesion categorization based on FFA images are still few, and several focus on DR. Pan et al. [15] used DenseNet for four types of DR classification through transfer learning. The proposed model achieved area under the curve (AUC) values ranging from 0.8703 to 0.9647 in experiments on a cohort of 4067 FFA images acquired from the Second Affiliated Hospital of Zhejiang University. Gao et al. [16,17] first proposed a VGG16-based framework for automatic detection of five types of DR lesions, and subsequently developed a novel DR grading system using VGG16 with a nine-grid input strategy. This limits its generalizability and clinical value. Although the FFA-Lens developed by Veena K.M et al. [18] based on YOLO achieved high precision and recall values in detecting 25 lesions, their model relied on a traditional object detection framework and still has room for improvement. Lesions in FFA images vary in size and shape. Traditional CNNs, limited by fixed receptive fields, struggle to capture both local details and global structural information simultaneously [19]. Multi-scale feature fusion strategies can alleviate this challenge. Meanwhile, FFA image lesions are closely associated with vascular structures, and disease progression often involves long-range interactions [20]. Accurate classification requires models to capture long-term dependencies and contextual relationships. The attention mechanism enhances classification performance by capturing vascular details in FFA images [21].

Therefore, it is very important in terms of FFA images to come up with a new deep learning model that can handle classifying multiple lesions, combining features from different scales, and gathering global contextual information. This study proposes the EFF-ConvNeXt network to address this gap. It does this by using better attention mechanisms and multi-scale feature fusion strategies to correctly classify 23 common fundus lesions. Experimental results demonstrate the model’s superior performance across multiple evaluation metrics, highlighting its potential for fundus disease classification. The innovations of this study are as follows:

• Our model can classify 23 common retinal lesions, unlike traditional methods that group retinal diseases into a limited number of categories. This type of classification better reflects the complexity and diversity of diseases in clinical practice.
• We proposed the CAFF module to enhance multi-scale feature fusion in FFA images. It fixes a major problem with current methods by effectively capturing both shallow and deep features. This makes it easier to identify lesions and fine details at all scales.
• We proposed the IGC module to enhance global context extraction. By adding IGC, the model can focus better on important areas, which is very important for finding small lesions in FFA images. This approach significantly improves classification accuracy and robustness.

2. Model Architecture Design

This study proposes a novel fundus lesion FFA image classification model, named EFF-ConvNeXt. Built upon the ConvNeXt model, it integrates the VGG16 network, attention mechanisms, and multi-scale feature fusion. The overall network architecture is shown in Figure 1. The model consists of two main modules: the VGG16 network, which extracts low-level features such as edges and textures, and the improved ConvNeXt (im-ConvNeXt) module, which captures high-level features, including semantic information and global context. In im-ConvNeXt, the IGC Block enhances the positional encoding of single-scale features, the CAFF module fuses shallow and deep features at each stage to establish cross-stage correlations, and SKNet is incorporated to further enhance the network’s ability to adapt to multi-scale feature fusion.

2.1. The VGG16 Network

The VGG16 [22] is a classical CNN architecture proposed by the Visual Geometry Group at the University of Oxford in 2014. It achieved remarkable results in the ImageNet challenge and has been widely used in image classification and feature extraction. As shown in Figure 1, VGG16 primarily consists of 3 × 3 convolutional filters. These small filters capture local image features while maintaining a large receptive field, avoiding excessive computational complexity. VGG16 has 13 convolutional layers and 3 fully connected layers, enabling the extraction of complex image features. It has become a cornerstone for various computer vision tasks and a powerful pre-trained model for high-quality feature extraction. Taking advantage of this advantage, we employed VGG16 for the shallow extraction of features from FFA images. When used with the im-ConvNeXt model for deep semantic feature extraction, this method made classification much better.

2.2. The Improved ConvNeXt Network

Fair et al. [23] introduced the ConvNeXt model in 2022, which integrates the strengths of ResNet and Vision Transformer (ViT) [24]. This model outperformed others in the ImageNet-1K benchmark. Its design was inspired by the success of the Swin Transformer (SwinT) [25] model. Researchers [23] explored the limitations of pure ConvNets and proposed several key components for constructing and improving the ConvNeXt network. Inspired by the block stacking ratio of 1:1:3:1 in SwinT-tiny, they adjusted the stacking ratio of the ConvNeXt model to 3:3:9:3. Traditional 3 × 3 convolutions are replaced by grouped convolutions in ConvNeXt. It also has an inverted bottleneck mechanism, like MobileNetV2 [26], to make feature representation better. Additionally, it substitutes $ReLU$ with $GELU$ and replaces batch normalization (BN) with layer normalization (LN). These modifications optimize both computational efficiency and accuracy, resulting in the final ConvNeXt architecture.

The ConvNeXt offers models of varying scales. In this study, we selected the smaller ConvNeXt-T model for more efficient deployment of FFA in clinical applications. As illustrated in Figure 1, the improved ConvNeXt module incorporates two key enhancements. First, a CAFF module is introduced between adjacent stages to fuse multi-scale contextual information from shallow and deep layers. Second, the model’s ability to adapt to multi-scale fused features is enhanced by the addition of an SKNet module prior to classification.

2.2.1. The Context-Aware Feature Fusion Module

In the ConvNeXt model, the output of each stage is directly downsampled and sent to the next stage. This makes feature interactions weak and the ability to classify lesions poor. To enhance feature utilization, we fuse the previous stage’s output with its downsampled features and feed them into the next stage to improve lesion representation. Figure 2 illustrates the structure of the CAFF module.

The specific process of the CAFF module is as follows: First, shallow features pass through the IGC Block to capture global context and fine-grained details, generating richer deep features. Then, average pooling is applied to halve the width and height of the feature map, followed by a $1\times 1$ convolution to double the number of channels. Finally, an add operation fuses the deep features with the processed shallow features. The specific calculation process is shown in Equation (1):

$$X=LN\left(Conv2D\left(avgpool\left(Y\right)\right)\right)+X_d$$

(1)

where X denotes the CAFF module output, Y represents the IGC Block output features, $avgpool$ is average pooling, $Conv2D$ is a $1\times 1$ convolution, $LN$ is Layer Normalization, and $X_d$ signifies the deep features.

2.2.2. The Improved Global Context Network Module

Medical images are known to be of high complexity. Global contextual feature modeling is widely recognized as crucial in medical image vision tasks. In 2019, Yue Cao et al. [27] proposed GCNet based on Non-Local Neural Networks (NLNet) [28] and Squeeze-and-Excitation Networks (SENet) [29]. GC block primarily comprises the Context Modeling and Transform modules, with its architecture illustrated in Figure 3a. The Context Modeling captures global context information through adaptive weight allocation, enabling noise suppression and selection of critical features. It addresses the limitations of traditional convolution, enhancing model performance while maintaining computational efficiency. The transform module significantly improves feature representation by enhancing convolutional features to balance local details with global context. This improvement makes it suitable for integration into existing convolutional neural networks to optimize performance. GC block is good at capturing global context, but it lacks the local detail ability to meet the needs of fine-grained classification tasks that need detailed features, such as FFA image classification.

The Mamba [30] model does a good job of handling long-range and multi-modal data and figuring out the subtle dependencies and relationships between pieces of information. Inspired by the Mamba model, we propose IGC module in this study. The architecture of the IGC block is shown in Figure 3b. We integrate the context modeling and transformation components of the GC block. This procedure reduces the feature processing redundancy of GC block and helps speed up model convergence and optimization efficiency. Subsequently, we replace the LN with a BN that performs better in convolutional neural networks. After attention pooling modulation, the features are weighted or re-emphasized. Finally, these weighted features are spatially reorganized through an additional Conv2D layer. With these improvements, IGC block more effectively captures the relationships and dependencies between FFA image details, providing finer and more accurate information. The IGC block is implemented as follows:

The input feature is $X_j\in {\mathbb{R}}^{C\times H\times W}$, where C represents the number of channels, and H and W are the height and width.

The feature extraction path applies multiple convolutions to the input X, starting with a $1\times 1$ convolution, followed by $BN$ and $ReLU$, then another $1\times 1$ convolution. The equation is as follows:

$$\begin{array}{c}{X}_1=W_{v_1}X\end{array}$$

(2)

$$\begin{array}{c}{X}_2=ReLU\left(BN\left(X_1\right)\right)\end{array}$$

(3)

$$\begin{array}{c}{X}_3=W_{v_2}{X}_2\end{array}$$

(4)

where $W_{v1}$ is the weight of the first $1\times 1$ convolution, and $X_1$ is the intermediate result. $BN$ refers to batch normalization, and $ReLU$ is the activation function. $W_{v2}$ is the weight of the second $1\times 1$ convolution, and $X_3$ is the final result.

The global context path uses a $1\times 1$ convolution on the input X to get global features. It then uses $softmax$ to make global context weights. The equation is given below:

$$\begin{array}{c}G=W_{k}X\end{array}$$

(5)

$$\begin{array}{c}{G}_w=\mathrm{s}oftmax\left(G\right)=\sum _{j=1}^{N_p}{\displaystyle \frac{e^{W_k{X}_j}}{{\sum }_{m=1}^{N_p}{e}^{W_k{X}_m}}}\end{array}$$

(6)

where G is the result of applying the weight matrix $W_k$ to the input features X. $X_j$ represents the feature vector at position j. The term ${\displaystyle \frac{e^{W_k{X}_j}}{{\sum }_{m=1}^{N_p}{e}^{W_k{X}_m}}}$ weights the features at each position using $softmax$ normalization. ${\sum }_{j=1}^{N_p}$ represents the global range over which features are weighted and summed to produce a global feature representation for context modeling.

The features obtained from the feature extraction path are element-wise multiplied by the global context weights. This operation yields a weighted context feature representation. The formula is shown in Equation (7):

$$F_g=X_3\otimes G_w=\sum _{j=1}^{N_p}{\displaystyle \frac{e^{W_k{X}_j}}{{\sum }_{m=1}^{N_p}{e}^{W_k{X}_m}}}·W_{v_2}\left(\mathrm{R}eLU\left(BN\left(W_{v_1}{X}_j\right)\right)\right)$$

(7)

the symbol ⊗ represents element-wise multiplication, or the Hadamard product, indicating the multiplication of two matrices or vectors at corresponding positions.

Finally, a $1\times 1$ convolution integrates the weighted features, and the original input X is added to the global context features $F_c$ to obtain the final output. The process equation is as follows:

$$\begin{array}{c}{F}_c=W_{v_3}{F}_g\end{array}$$

(8)

$$\begin{array}{c}Y=X+F_c\end{array}$$

(9)

where, in Equation (8), $W_{v3}$ is the third weight matrix, and in Equation (9), X is the initial input. The steps are integrated into a single equation:

$$Y_i=W_{v_3}\left(\sum _{j=1}^{N_p}{\displaystyle \frac{e^{W_k{X}_j}}{{\sum }_{m=1}^{N_p}{e}^{W_k{X}_m}}}·W_{v_2}\left(\mathrm{R}eLU\left(BN\left(W_{v_1}{X}_j\right)\right)\right)\right)+X_i$$

(10)

in Equation (10), $X_i$ represents the pixel at the current position on the feature map, with j and m enumerating all possible positions. $X_j$ and $X_m$ represent the pixels at corresponding positions. $W_k$, $W_{v1}$, $W_{v2}$, and $W_{v3}$ are the linear transformation matrices, and $N_p=H\times W$ is the number of positions. $ReLU$ is the activation function, and $BN$ refers to batch normalization.

2.2.3. The SKNet Attention Module

Fundus diseases exhibit complex heterogeneity, with significant variations in lesion location, number, size, and depth. These complexities challenge CNNs in retaining critical local information, leading to performance degradation. To mitigate this issue, we introduced the SKNet attention mechanism [31]. The SKNet attention mechanism adaptively selects the convolutional kernel size and dynamically adjusts the receptive field so that it can effectively pick up features at different scales. The SKNet architecture comprises three key operations: Split, Fuse, and Select. These operations are responsible for splitting, fusing, and selecting convolution kernels at different scales. The structure of SKNet is shown in Figure 4. This adaptive mechanism enhances the network’s ability to capture and integrate multi-resolution features. Because of this, adding it to the EFF-ConvNeXt model can make finding and classifying lesions more accurate.

2.3. Loss Function Optimization

The cross-entropy loss function [32] is commonly used in image classification, and its formula is given in Equation (11):

$$L_{CE}=-\sum _{i=1}^{m}p\left(x\right)logq\left(x\right)$$

(11)

where m is the batch size, i denotes the i-th FFA image, $p\left(x\right)$ is the true probability distribution of the 23 fundus disease types, and $q\left(x\right)$ is the predicted probability distribution.

There is a big class imbalance in the APTOS2023 dataset. This could make the model favor the majority class and ignore the minority class, which would limit its practical performance. The traditional cross-entropy loss function struggles to optimize the model effectively. To improve model performance, we introduced Focal Loss (FL) [33] to address the class imbalance issue. Its mathematical expression is given in Equation (12). FL increases the weight of difficult-to-classify samples so that the model pays more attention to challenging samples and improves the accuracy and generalization ability. This strategy optimizes the intra-class distance, which can further improve the FFA image classification performance.

$$FL\left(p_t\right)=-{\alpha }_t{\left(1-p_t\right)}^{\gamma }log\left(p_t\right)$$

(12)

where, in Equation (12), $p_t$ is the predicted probability for class t, while ${\alpha }_t$ and $\gamma$ are hyperparameters. ${\alpha }_t$ balances class ratios, and $\gamma$ reduces the weight of easy samples. We set $\alpha =0.25$ and $\gamma =1.0$. To focus on both easy and hard examples, we combine cross-entropy loss with Focal Loss, as shown in Equation (13):

$$L_{CE}=\lambda L_{CE}+(1-\lambda )FL$$

(13)

where $L_{CE}$ denotes cross-entropy loss, $FL$ represents Focal Loss, and $\lambda$ is a weight coefficient ranging from 0 to 1. The model achieves optimal performance at $\lambda =0.5$.

3. Experimental Results and Analysis

3.1. Dataset and Data Preprocessing

The dataset was sourced from the 2023 Asia Pacific Tele-Ophthalmology Society (APTOS) Big Data Competition. The dataset utilized in this study was retrospectively collected from real-world clinical practice, comprising 55,361 multi-modal fundus angiography images from both eyes of 3179 distinct patients. In order to minimize the memory footprint of the model, we conducted experiments using only FFA images. The contest only provided annotated data for the training set, so we used 33,559 images from 1921 patients in the training set in this study. The APTOS dataset (APTOS2023) includes 23 types of lesion classifications. Figure 5 illustrates the data distribution used in our study. The number of images per class ranges from 55 to 15,231, indicating a significant class imbalance.

According to Figure 5, the number of images in the “macular neovascularization” category is significantly higher than in other categories. The “central retinal vein occlusion” and “central retinal artery occlusion” categories have relatively fewer images. This class imbalance may lead to substantial discrepancies in classification accuracy across categories. We implemented data balancing strategies and preprocessing methods to cope with this phenomenon. The data balancing strategy involves adjusting the sampling frequency for each category, with higher sampling probabilities to underrepresented classes and lower probabilities to those with excessive samples. We determine the sampling rate according to Equation (14) as shown in

Table 1. Sampling rates for each category in the APTOS2023 Dataset.

Category	Rate	Category	Rate	Category	Rate
BRVO	10	DR	8	PCV	6
CRAO	10	DAMD	7	PDR	10
CRVO	10	EM	10	RD	10
CSC	3	EM	10	RPED	8
CA	10	MN	1	ROV	10
CS	8	myopia	10	UC	5
CM	10	other	10	uveitis	6
CME	10	PPE	10

Note: Due to the length of the full names, some lesions are abbreviated in the table. The full names of these lesions can be found in Abbreviations.

.

$$p_k={\displaystyle \frac{1/n_k}{{\sum }_{j=1}^K(1/n_j)}}$$

(14)

where $p_k$ is normalized sampling probability, $n_k$ is the number of samples in class k, and K is total number of classes.

For image preprocessing, we used five data enhancement techniques: random cropping, random flipping, Gaussian blur, brightness and contrast adjustment, and sharpness adjustment. Each of these techniques was applied to the original images with a random probability. The effect of data enhancement is shown in Figure 6. The augmented images effectively retain the detailed features of the original images. Additionally, we divided the dataset into training, validation, and test sets in a ratio of 6:2:2. These subsets were randomly partitioned on a patient-wise basis and are mutually exclusive to ensure their independence and prevent any overlap of images from the same patient across different subsets.

3.2. Implementation Details and Performance Metrics

All experiments were conducted on a workstation equipped with an NVIDIA Quadro RTX 5000 GPU with 15GB VRAM and an Intel(R) Xeon(R) Gold 521 processor. The software environment includes the Windows 10 operating system and Python 3.9.0 as the programming language. We employed transfer learning by initializing the main model with pretrained weights from the ImageNet dataset. We resized the FFA images to a resolution of $224\times 224$. Each model was trained using the AdamW optimizer and the cross-entropy-focal loss function. The initial learning rate was set to $5\times {10}^{-4}$, with a final learning rate of $1\times {10}^{-8}$. The weight decay parameter (L2 regularization) was set to $1\times {10}^{-4}$. To ensure stable evaluation results, we performed five-fold cross-validation exclusively on the training set, while keeping the validation and test sets completely independent throughout the process. After several rounds of hyperparameter tuning, the batch size was set to 64, and the training calendar elements to 200.

Several performance metrics were considered for model evaluation, including accuracy ($Acc$), precision ($P_r$), recall ($R_e$), and F1 score ($F1$). The formulas are provided in Equations (15)–(18). Macro-averaging was used for model evaluation. Specifically, after calculating the metrics for each class, the arithmetic mean of these values was computed as the model performance indicator.

$$\begin{array}{c}Acc={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}\end{array}$$

(15)

$$\begin{array}{c}{P}_r={\displaystyle \frac{TP}{TP+FP}}\end{array}$$

(16)

$$\begin{array}{c}{R}_e={\displaystyle \frac{TP}{TP+FN}}\end{array}$$

(17)

$$\begin{array}{c}F1={\displaystyle \frac{2\times P_r\times R_e}{P_r+R_e}}\end{array}$$

(18)

where $TP$ is the count of true positives (both label and prediction are positive), $FN$ is false negatives (positive label, negative prediction), $FP$ is false positives (negative label, positive prediction), and $TN$ is true negatives (both label and prediction are negative).

3.3. Experimental Results

All experiments were conducted on the APTOS2023 dataset under identical conditions. Each model was evaluated through three independent trials, with the average of the results taken as the final outcome to ensure stability.

3.3.1. Comparison Between SOTA Baseline Model

The EFF-ConvNeXt model achieves 92.5% accuracy, 92.46% precision, 92.56% recall, and 92.30% F1 score on the APTOS2023 dataset. It also demonstrated an inference time of 0.007 s per image, balancing high classification performance with computational efficiency.

Table 2. Classification performance of different models on APTOS2023.

Model	Average $\mathit{Acc}$ (%)	Average ${\mathit{P}}_{\mathit{r}}$ (%)	Average ${\mathit{R}}_{\mathit{e}}$ (%)	Average $\mathit{F}1$ (%)	Inference Time (t/s)
ViT	70.15	70.06	70.37	70.18	0.004
SwinT	89.36	89.43	89.36	88.29	0.004
VGG19	66.86	64.52	65.33	65.98	0.003
ResNet50	78.64	78.65	78.52	78.39	0.003
DenseNet121	75.41	74.81	76.24	74.12	0.004
EffcientNet-B3	68.40	68.43	68.29	68.26	0.005
ConvNeXt	89.38	89.36	89.03	89.30	0.005
EFF-ConvNeXt	92.50	92.46	92.56	92.30	0.007

Note: The bold formatting here is used to highlight the best performance results.

compares EFF-ConvNeXt with SOTA baseline methods for FFA image classification. Among all models, VGG16 shows relatively lower performance, while ConvNeXt and Swin Transformer perform better, achieving accuracies of 89.38% and 89.36%, respectively. Our EFF-ConvNeXt model outperforms ConvNeXt by 3.12% in accuracy, with a slight increase in inference time of 0.002 s.

Table 3. The performance of EFF-ConvNeXt on the classification task across different lesion subsets of the APTOS2023 dataset.

Lesion Category	$\mathit{Acc}$ (%)	${\mathit{P}}_{\mathit{r}}$ (%)	${\mathit{R}}_{\mathit{e}}$ (%)	$\mathit{F}1$ (%)
BRVO	98.02	97.79	98.67	98.23
CRAO	96.98	95.66	96.33	95.99
CRVO	97.33	97.58	97.78	97.68
CSC	75.86	78.19	75.86	77.01
CA	94.44	93.67	91.67	92.66
CS	90.91	93.10	93.18	93.14
CM	86.56	86.15	87.10	86.62
CME	96.72	97.86	95.07	96.45
DR	93.54	94.91	93.75	94.32
DAMD	86.72	92.16	85.45	88.68
EM	98.79	89.79	96.70	93.11
MN	95.33	97.98	96.64	97.30
myopia	95.82	94.66	96.63	95.63
other	98.33	97.98	98.55	98.26
PPE	94.67	96.43	93.10	94.74
PCV	92.33	91.04	91.04	91.04
PDR	99.59	99.24	99.01	99.12
RAM	91.83	91.45	91.30	91.37
RD	97.90	97.67	97.67	97.67
RPED	48.72	47.67	47.37	47.52
RVO	81.82	82.08	81.82	81.95
UC	95.67	97.23	96.25	96.74
uveitis	73.08	72.57	71.28	71.92

shows the classification performance of EFF-ConvNeXt across different lesion categories on the APTOS2023 dataset. The model achieves high precision and recall in most classes. Notably, for common and well-represented categories such as BRVO, CRVO, CME, and PDR, the model achieves high precision, recall, and F1 scores, all exceeding 95%. However, relatively lower performance is observed in challenging categories such as RPED and uveitis, where the F1 scores drop to 47.52% and 71.92%, respectively. This may be attributed to the limited sample size, ambiguous visual features, or inter-class similarities. These findings highlight the model’s overall effectiveness while also suggesting that future improvements could focus on enhancing performance for underrepresented or complex lesion types.

To intuitively compare model performance, we plotted the accuracy and loss curves of EFF-ConvNeXt, ConvNeXt, Swin-T, and ResNet50 (as shown in Figure 7). All models achieved over 95% accuracy on the training set, with most converging after 100 epochs. ResNet50 exhibited significant fluctuations in accuracy and eventually stabilized at a relatively low value of 78.64%. In contrast, EFF-ConvNeXt achieved a stable accuracy of 92.5%, outperforming ConvNeXt by 3.12% with reduced variance. Regarding loss, ResNet50 performed noticeably worse on both the training and validation sets. EFF-ConvNeXt demonstrated superior stability and reached the lowest loss value of 0.384, highlighting its overall performance advantage.

To evaluate performance improvements, we conducted ten independent runs per model and plotted box plots for key metrics. As shown in Figure 8, the proposed EFF-ConvNeXt achieved the highest median values and lowest variance across all metrics, showing superior performance and robustness. Compared to ConvNeXt, it consistently improved by over 3%, with statistically significant gains (p < 0.01). In contrast, conventional models such as VGG19 and ViT exhibited lower performance and higher variability, demonstrating the effectiveness of our enhanced multi-scale and global context fusion design.

3.3.2. Visualization Analysis

We applied Grad-CAM++ [34] to visualize model decisions on the APTOS dataset. The activation maps highlight key regions by capturing response intensity in the last convolutional layer. Figure 9 compares activation maps of ResNet, Swin Transformer, ConvNeXt, and EFF-ConvNeXt on the same image. The ResNet50 captures local features but struggles with complex lesions. SwinT and ConvNeXt extract global features more effectively. In contrast, EFF-ConvNeXt achieves precise lesion localization with stronger responses, demonstrating the effectiveness of CAFF in enhancing feature extraction.

We plotted the macro-averaged ROC curve (Figure 10) and confusion matrix (Figure 11) for EFF-ConvNeXt. The warm diagonal distribution in Figure 11 shows that it is very good at classifying most types of fundus diseases, showing that it works well for complex FFA image classification. Figure 10 shows that the macro AUC reaches 0.9859, approaching 1, indicating excellent discriminative ability and stability. The model also keeps a high true positive rate (TPR) and a low false positive rate (FPR) across thresholds, showing that it can be used to classify fundus diseases and is reliable.

3.3.3. Comparison with Different Pretrained Models

Table 4. Comparison of the Performance of Various Pretrained Models Using im-ConvNeXt as the Baseline.

Model	Average $\mathit{Acc}$ (%)	Average ${\mathit{P}}_{\mathit{r}}$ (%)	Average ${\mathit{R}}_{\mathit{e}}$ (%)	Average $\mathit{F}1$ (%)	Inference Time (t/s)
im-ConvNeXt	92.03	92.05	92.13	92.37	0.005
SwinT	89.36	89.43	89.36	88.29	0.004
swinT+im-ConvNeXt	91.78	91.86	91.77	92.02	0.008
ResNet50	78.64	78.65	78.52	78.39	0.003
ResNet50+im-ConvNeXt	92.12	91.08	91.68	92.18	0.007
VGG16	63.91	64.12	63.93	63.98	0.003
VGG16+im-ConvNeXt (ours)	92.50	92.46	92.56	92.30	0.007

Note: The bold formatting here is used to highlight the best performance results.

further reveals that SwinT+im-ConvNeXt leads to a performance drop. VGG16 performs poorly on the APTOS2023 dataset. However, as a pre-trained model, it outperforms ResNet and SwinT. Moreover, SwinT+im-ConvNeXt exhibits a decline in performance. We attribute the limited improvement in ResNet+im-ConvNeXt to functional redundancy, while the lack of significant gain in SwinT+im-ConvNeXt may result from overlapping functionalities. In contrast, VGG16’s simple convolutional stacking effectively captures edges and textures, complementing ConvNeXt’s semantic and global context modelling, leading to better overall performance.

3.3.4. Comparison with Previous Attention Mechanisms

In addition, we conducted comparative experiments by replacing the IGC module in the CAFF module with other attention mechanisms to validate the superiority of the IGC block. Each model was evaluated through three independent trials, with the average results reported. The best results are highlighted in bold in

Table 5. Performance comparison of proposed attention module adopting im-ConvNeXt as backbone with different cutting-edge attention modules.

Model	Average $\mathit{Acc}$ (%)	Average ${\mathit{P}}_{\mathit{r}}$ (%)	Average ${\mathit{R}}_{\mathit{e}}$ (%)	Average $\mathit{F}1$ (%)	Inference Time (t/s)
baseline	90.96	90.95	91.10	90.93	0.005
+CBAM [35]	87.83	87.86	87.65	87.12	0.005
+ECA [36]	91.84	91.98	91.80	91.91	0.005
+SimAM [37]	91.06	91.05	91.06	91.18	0.005
+SE [29]	91.56	91.24	91.43	91.23	0.005
+GC [27]	89.43	89.36	89.48	89.46	0.005
+IGC	92.50	92.46	92.56	92.30	0.005

Note: The baseline indicates that no attention mechanism was added to CAFF, while other models replaced different attention mechanisms in CAFF. The bold formatting here is used to highlight the best performance results.

. As shown in Table 5, all models exhibit comparable inference times, while ConvNeXt+IGC outperforms all other models across the remaining evaluation metrics. Compared to the baseline model, the average Acc improves by 1.54%, and the average F1 score increases by 1.37%. ConvNeXt+SE and ConvNeXt+ECA achieve strong performances, with average accuracies of 91.56% and 91.84%, respectively, but still lag behind ConvNeXt+IGC. In contrast, ConvNeXt+CBAM and ConvNeXt+GC show weaker results, with accuracies of 87.83% and 89.43%. The IGC block improves accuracy by 3.07% and the F1 score by 2.84% over the GC block. These results demonstrate that IGCNet more effectively focuses on key areas of FFA images, accurately capturing lesion features.

3.3.5. Ablation Studies

A series of ablation experiments were conducted to analyse the impact of gradually adding or removing each module on the performance of EFF-ConvNeXt.

Table 6. Performance comparison of ablation experiments.

Type	Configuration	VGG16	CAFF	SKNet	Loss Opt.	Average $\mathbf{Acc}$ (%)	Average ${\mathit{P}}_{\mathit{r}}$ (%)	Average ${\mathit{R}}_{\mathit{e}}$ (%)	Average $\mathit{F}1$ (%)	Inference Time (t/s)
Additive	Baseline					89.38	89.36	89.03	89.30	0.005
	+ VGG16	✓				90.10	90.36	89.68	90.16	0.006
	+ VGG16 + CAFF	✓	✓			91.81	91.80	91.26	91.75	0.006
	+ VGG16 + SKNet	✓		✓		91.05	91.03	91.17	91.02	0.006
	+ VGG16 + Loss Opt.	✓			✓	90.96	90.86	91.10	91.98	0.006
Ablative	− Loss Opt.	✓	✓	✓		92.05	92.19	91.97	92.03	0.007
	− SKNet	✓	✓		✓	91.81	91.82	91.68	91.78	0.006
	− CAFF	✓		✓	✓	90.96	90.95	91.10	90.93	0.006
	− VGG16		✓	✓	✓	92.03	92.05	92.13	92.37	0.006
	Completed model	✓	✓	✓	✓	92.50	92.46	92.56	92.30	0.007

Note: ✓ indicates that this module is included in the baseline model. “+” indicates modules included in the baseline model. ”−” indicates reduction of modules in the completed model. “Opt.” is short for optimization. The bold formatting here is used to highlight the best performance results.

reports the results of the APTOS2023 dataset. The baseline model achieved an average accuracy of only 89.38%. After incorporating VGG16 into the backbone network, the average accuracy improved to 90.10%, indicating that deep feature extraction offers significant advantages in fundus image classification. When the CAFF module was further integrated, the average accuracy reached 91.81%, with a recall of 91.26%, further highlighting the benefits of enhanced feature extraction. In contrast, introducing SKNet or optimizing the loss function individually led to relatively modest improvements, with accuracies of 91.05% and 90.96%, respectively.

The synergy between modules plays a critical role in system performance. When CAFF is combined with SKNet, the inference time increases by only 0.001 s, while the accuracy reaches 92.05%, which was better than just adding their individual scores together. The complete model, incorporating all components, achieved the best overall performance, with peak values in accuracy (92.50%), precision (92.46%), and recall (92.56%). Interestingly, when VGG16 was removed while retaining the other modules, the inference time decreased by 0.001 s, and the F1 score improved to 92.37%, indicating that the CAFF and SKNet combination is more efficient for lightweight applications.

These findings confirm that the CAFF module provides the most significant performance gains, while SKNet and the optimized loss function rely on the backbone network to be effective. Overall, collaborative optimization among modules is key to achieving optimal classification accuracy.

3.3.6. Comparison with Previous FFA Image Classification Methods

We evaluated the proposed EFF-ConvNeXt on the APTOS2023 dataset and compared its performance with SOTA methods for fundus disease classification, as shown in

Table 7. Comparison with previous automated FFA image classification methods on APTOS2023.

Model	$\mathit{Acc}$ (%)	${\mathit{P}}_{\mathit{r}}$ (%)	${\mathit{R}}_{\mathit{e}}$ (%)	$\mathit{F}1$ (%)	Recognition Time t/s
Pan et al. [15]	76.14	76.18	76.38	76.21	0.004
Gao et al. [17]	63.91	64.12	63.93	63.98	0.004
Gao et al. [16]	74.57	74.75	75.25	73.98	0.003
FFA-lens [18]	88.53	88.38	87.63	87.88	0.008
EFF-ConvNeXt	92.50	92.46	92.56	92.30	0.007

Note: The bold formatting here is used to highlight the best performance results.

. For a fair comparison, we reimplemented existing YOLO-based approaches [18] and CNN models [15,16,17] under similar experimental settings. As observed, EFF-ConvNeXt consistently outperforms previous YOLO-based and CNN-based methods on the ATOPS2023 dataset. Compared to traditional CNN models [15,16,17], EFF-ConvNeXt demonstrates superior capability in extracting discriminative features from FFA images, leading to significantly improved performance. This improvement is attributed to the integration of global feature extraction, multi-scale feature fusion, and pretrained CNN backbones, which enhance the representation of disease-specific and stage-relevant features while suppressing redundant information. Notably, the performance gain is achieved without a substantial trade-off in computational speed. Therefore, the proposed framework holds great promise as an efficient system for multi-lesion fundus disease classification and can serve as a valuable tool to assist ophthalmologists in verifying screening outcomes.

4. Discussion

4.1. Experimental Results and Clinical Relevance

Deep learning is widely applied in automated disease diagnosis across ophthalmology, orthopedics, cardiology, dermatology, and pathology [38,39]. In ophthalmology, AI plays a crucial role in aiding the diagnosis and treatment of diseases such as DR, AMD, and glaucoma. Its applications hold significant potential to transform the diagnosis and treatment of various ocular diseases in the future [40]. Although much progress has been made on these tasks, to the best of our knowledge, no previous work has targeted FFA image classification for more than 20 diseases. In this work, we investigated the importance of image differentiation of multiple lesions for automatic classification. Furthermore, we propose a multi-scale feature fusion network to capture lesion locations and enable multi-disease classification. It integrates VGG16, CAFF, and SKNet into the ConvNeXt framework to enhance lesion feature representation. VGG16 effectively captures edge and texture information, while ConvNeXt improves semantic and global context modeling, boosting overall performance. Additionally, CAFF and SKNet facilitate multi-scale feature fusion and selection, enhancing system stability and classification accuracy, particularly in complex FFA images.

To comprehensively validate the feasibility and effectiveness of the EFF-ConvNeXt model, extensive experiments were conducted and comparative analyses were performed against multiple benchmark models. As shown in Table 2 and Table 7 and Figure 7, Figure 8 and Figure 9, the proposed model outperforms the latest FFA classification methods in both classification accuracy and lesion recognition, with statistically significant improvements. In particular, it surpasses ConvNeXt and FFA-Lens by 3.12% and 3.97% in accuracy, respectively, delivering more focused and accurate lesion identification while maintaining high computational efficiency. According to the results in Table 4, Table 5 and Table 6, the VGG16 and IGC modules contribute most significantly to performance enhancement. Although VGG16 alone performs poorly in FFA classification, its combination with ConvNeXt and IGC outperforms other configurations. Meanwhile, the results demonstrate that while our method introduces a slight increase in inference time due to the added modules, it achieves a significantly better classification performance, thus providing a favorable trade-off between accuracy and efficiency.

Although the present experiments were conducted solely on the APTOS2023 dataset—an inherent limitation for assessing cross-domain generalisation—we employed extensive data augmentation, preprocessing, transfer learning and regularisation strategies to maximise the model’s capacity to learn domain-invariant features. The strong internal-validation results already obtained provide initial evidence of its generalisability. Furthermore, Figure 9 indicates that the model concentrates on anatomical regions relevant to the target pathologies rather than device-specific artefacts or background patterns, implying that it has captured biologically meaningful features. Taken together, these observations strongly suggest that EFF-ConvNeXt can be transferred to data acquired from other hospitals or imaging devices. We are actively seeking collaborations to obtain multi-centre datasets and plan to incorporate advanced domain-generalisation techniques to comprehensively evaluate and further enhance the robustness and real-world utility of our model in diverse clinical environments.

Based on Table 3 and Figure 10 and Figure 11, the proposed model achieved promising results on the APTOS2023 dataset. In the confusion matrix, it is noteworthy that the classification accuracy for RPED was only 47.37%, whereas PRD reached 99.01%. As shown in Figure 5, this significant disparity is primarily attributed to class imbalance. RPED and MN are often different stages or co-manifestations of the same pathological process in clinical settings. Since MN accounts for a disproportionately large number of samples in the dataset, the model tends to predict ambiguous or borderline cases as the more frequently encountered class—namely, MN. To mitigate this misclassification issue, we introduced fine-grained feature enhancement, which helped alleviate some errors. However, inter-class imbalance remains a challenge. Such imbalance and misclassification issues can lead to delayed or missed diagnoses in clinical deployment, posing potential risks to patient outcomes. Future research will focus on expanding the dataset and reducing inter-class performance disparities to enhance model robustness and clinical reliability.

4.2. AI Ethics in Ophthalmology

The application of AI models in ophthalmic diagnostics has led to significant performance improvements. However, their integration into clinical practice necessitates rigorous consideration of ethical frameworks and regulatory compliance, which are critical to ensuring patient safety, fairness, and real-world reliability [41]. The U.S. Food and Drug Administration (FDA), together with the UK’s Medicines and Healthcare products Regulatory Agency (MHRA) and Health Canada, has outlined ten Good Machine Learning Practice (GMLP) guiding principles [42]. These emphasize core elements such as data governance, transparency, independent evaluation, and human-centered interface design to ensure the quality and safety of AI-enabled medical devices. In this study, we adhered to a structured development process that included traceable data sourcing, hierarchical validation, and continuous performance monitoring. For future clinical translation, we plan to adopt a Predetermined Change Control Plan (PCCP) [43] to manage model updates and mitigate performance drift in accordance with the latest FDA guidelines.

Another key aspect is the standardization of study reporting and model validation. Internationally recognized guidelines such as CONSORT-AI [44] offer detailed recommendations for trial design, results disclosure, and model reproducibility, thereby enhancing the transparency and comparability of AI-related clinical studies [45]. In the present work, we have followed CONSORT-AI recommendations to report model inputs and outputs, conduct error analysis, and clarify limitations of use. In future stages, we intend to perform prospective validation using multicenter external datasets.

4.3. Limitations and Future Clinical Applications

Despite significant performance improvements on the experimental dataset, the model has limitations. First, current data augmentation techniques cannot fully simulate the complex pathological conditions of real-world fundus images, and the APTOS2023 dataset alone cannot fully demonstrate the generalization of EFF-ConvNeXt. As such, the reported results may be overly optimistic. Future studies will focus on collecting additional external datasets to enhance sample diversity and improve both robustness and generalizability. Second, despite high accuracy, computational speed and resource consumption may limit practical applications. Future research will design a more lightweight model, optimizing the architecture to reduce computational load and improve processing speed. Ultimately, integrating longitudinal or progression data offers irreplaceable advantages in predicting disease evolution. However, this study is limited to single-time-point FFA images, which fail to capture the dynamic changes of lesions over time. Due to the scarcity of longitudinal FFA data, we did not incorporate follow-up sequences for modeling disease progression. As part of our future work, we plan to collect multi-timepoint data and explore temporal modeling approaches to enhance the prediction of disease trajectories.

5. Conclusions

This study proposes an improved ConvNeXt model for FFA image classification, retaining the core structure of the original ConvNeXt while incorporating VGG16, CAFF, and SKNet modules, along with an optimized loss function. These enhancements significantly improve the model’s accuracy and robustness in medical image classification tasks. Experimental results show that EFF-ConvNeXt achieves an accuracy of 92.50% and an F1 score of 92.30% on the APTOS2023 dataset, which are 3.12 and 4.01 percentage points higher than the original ConvNeXt-T model, respectively. The model accurately distinguishes 23 common retinal diseases while maintaining high efficiency, providing clinicians with a reliable diagnostic aid. However, current data augmentation techniques cannot fully capture the diversity of real-world lesions. Future work will focus on expanding FFA image datasets to enhance model stability and generalization. Additionally, lightweight model designs will be explored to facilitate integration into FFA diagnostic systems, offering more efficient support for retinal disease screening and diagnosis.