Image Stitching of Low-Resolution Retinography Using Fundus Blur Filter and Homography Convolutional Neural Network

Abstract

Great advances in stitching high-quality retinal images have been made in recent years. On the other hand, very few studies have been carried out on low-resolution retinal imaging. This work investigates the challenges of low-resolution retinal images obtained by the D-EYE smartphone-based fundus camera. The proposed method uses homography estimation to register and stitch low-quality retinal images into a cohesive mosaic. First, a Siamese neural network extracts features from a pair of images, after which the correlation of their feature maps is computed. This correlation map is fed through four independent CNNs to estimate the homography parameters, each specializing in different corner coordinates. Our model was trained on a synthetic dataset generated from the Microsoft Common Objects in Context (MSCOCO) dataset; this work added an important data augmentation phase to improve the quality of the model. Then, the same is evaluated on the FIRE retina and D-EYE datasets for performance measurement using the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). The obtained results are promising: the average PSNR was 26.14 dB, with an SSIM of 0.96 on the D-EYE dataset. Compared to the method that uses a single neural network for homography calculations, our approach improves the PSNR by 7.96 dB and achieves a 7.86% higher SSIM score.

1. Introduction

While the World Health Organization, as estimated by the 2019 World Sight Report [1], says that 2.2 billion people worldwide suffer from some domain of visual impairment, nearly 1 billion of the cases are avoidable or treatable. In this scenario, early detection and accurate diagnosis of eye pathologies become critical for the prevention of loss of vision and the maintenance of patients’ quality of life. One of the biggest challenges to eye health is the lack of ophthalmologists, especially in remote distanced places and developing countries. This is causing a deficiency in access to timely diagnosis and high-quality management of eye disorders. On the other hand, advances in technology—like the one represented by the empowerment of hand-held mobile phones equipped with the D-EYE device for digital imaging of the retina [2]—open up new opportunities for early diagnosis.

Eye screening by visualizing the retina through fundus imaging offers an opportunity to non-invasively examine the systemic microcirculation in the human retina. Detailed clinical observations of the characteristics of the retinal fundus contribute not only to detecting eye diseases but also to identifying early indicators of a wide range of pathologies, such as diabetes, stroke, hypertension, arteriosclerosis, cardiovascular, neurodegenerative, renal and fatty liver diseases [3].

One challenge that limits the early detection of vision pathologies is the need for more professionals in regions far from large urban centers, especially in developing countries. A limited amount of the literature exists pertaining to the number of optometrists and related ophthalmic personnel. According to a recent study conducted in 2019, which examined the ophthalmology workforce in 198 countries, representing 94% of the global population, it was found that despite the increasing number of practicing ophthalmologists in most countries, there exists an uneven distribution and a significant shortfall in the current and projected number of ophthalmologists [4]. According to the International Agency for the Prevention of Blindness (IAPB), there are critical human resource shortages for allied ophthalmic personnel, especially in sub-Saharan Africa [5].

Considering this gap, methods that can help in the screening process for ocular pathologies can be great allies for ophthalmologists and patients. One viable option for retinal screening is using mobile devices, such as the D-EYE device [6].

Although devices like the D-EYE have been developed as a low-cost tool to capture images of the fundus with just a simple attachment to a smartphone, these still have limitations in image quality. Figure 1 shows some frames of a video obtained with the D-EYE device, and as can be observed, the images contain a perceptible amount of noise and cannot always allow a complete visualization of the optic disk, which is the region of interest for a possible diagnosis, and it is already just a tiny part of the image.

Among the few studies in the literature related to the problem of low-resolution retinographies, image stitching is the one developed by [7], presenting a comprehensive method that includes the steps of retinal segmentation, identification and matching of reference points, image alignment, and fusion. Variation in the video capture conditions, for example, due to changes in the retinal illumination, impacts the consistency of the results realized with this method. Furthermore, investigating the detection and montage of low-quality retinal images [8], brought out the approach of Super Retina by [9]. This method used semi-supervised deep learning model training techniques for keypoint detectors and descriptors involving the proposed Keypoint Progressive Expansion (KPE) technique, which addresses incompleteness issues arising from the manual labeling of data used in training. Although Super Retina was mostly tested on high-resolution images, it achieved remarkable feature matching and produced well-aligned results. Another technique reviewed by [8] was the stitching of several images suggested by [10], expected to give a complete image with captures obtained from an attached smartphone. This may be a very promising approach for retrieving a complete exposure of the retina, using Uniform Robust Scale-Invariant Feature Transform (UR-SIFT) [11] for feature detection and combining techniques for feature matching and refinement. However, the effectiveness of low-quality images, such as those obtained with D-EYE, may require the application of image pre-processing and image stitching techniques to improve visualization and analysis, both by experts and by automatic methods.

Therefore, this paper aims to address the problem of low-resolution retinal images by employing the image stitching technique to improve the visualization of images captured by the D-EYE. Thus, a method for image stitching between low-resolution retinographies via homography estimation is proposed based on the homography estimation model proposed by [12] with relevant modifications. Thus, we can highlight the following contributions: (1) we proposed a Fundus Blur Filter (FBF) originally designed in this study to mitigate the variations arising from the image capture conditions, which is fundamental to the success of the method; (2) we proposed a modification to the homography estimation network architecture in [12], utilizing four independent semi-siamese convolutional neural networks for more precise coordinate regression, instead of the original single regression.

The rest of this paper is organized as follows. Section 2 presents the proposed method, including the method for generating the synthetic dataset used to train the deep learning model used in homography estimation and the FBF applied to the input images during the model training, originally created for and used in this work, which is essential for the model to work. Section 3 presents the results of the proposed method and discusses the results. Section 4 presents the conclusions and future work.

2. Materials and Method

This section describes the proposed method for image stitching on low-resolution retinal images. First, we describe the dataset used to train the deep learning model. Next, we discuss the different stages of the proposed method, including data augmentation, feature extraction, and homography estimation, as well as the evaluation metrics and experimental setup used to validate the method.

2.1. Synthetic Dataset

Training a neural network for image stitching typically necessitates a large dataset [13,14]. Due to the scarcity of datasets specifically related to stitching low-resolution retinal images, the approach presented by [13] was used as a base for the generation of the synthetic dataset used in this work, modified to suit the needs of this application. For this, 80,000 images from the MS COCO dataset [15] were used.

Each image in the MS COCO dataset is cropped, adjusted to a square format, and resized to 274 × 274 pixels. Subsequently, a new crop is taken from the center of each image, represented in Figure 2A by the white square, which becomes the first input to the neural network. A position—blue square in Figure 2A—for the second crop is marked to ensure overlap with the first. The corners of this position are adjusted to simulate slight variations in the viewing angle, causing deformation relative to the initial marking; the deformed blue square is represented by the orange square in Figure 2B. The pixel distances between the corners of the original second crop position and the deformed marking are recorded as the label for the deep learning model, used in calculating the displacement needed in each corner of the second image to align with the first. The image is then warped so that the corners of the second crop form a square (orange square in Figure 2C), which serves as the second input to the network. The label for the stitching is the union of white and blue square contents in Figure 2B, as can be seen in Figure 2D. Experiments were conducted by adjusting the overlap between images within 50% to 100%. Such adjustments consider the characteristics of images obtained through D-EYE, which are captured from videos, allowing for increased overlap when selecting adjacent frames.

2.2. Proposed Method

The proposed method for estimating homography in low-resolution retinal images was inspired by the work of [12], with some adaptations.

A Fundus Blur Filter is applied exclusively during the training phase to deal with variations in the image capture condition, a Siamese convolutional neural network [16,17] for feature extraction, a feature correlation layer, and a regression network to predict the points that will be used to calculate homography using the Direct Linear Transformation (DLT) algorithm [18]. Figure 3 shows the proposed method pipeline.

2.2.1. Data Augmentation

Similar approaches for synthetic dataset generation are common in training homography estimation models [12,19]. However, it has disadvantages for predicting homography between image crops obtained under different lighting conditions, as a deep learning model usually performs optimally when it is fed with data closely similar to those samples seen during training [20]. In order to avoid this limitation and achieve generalization, especially with low-resolution images coming from the same retina captured at different moments, a significant innovation was introduced. Thus, we proposed a filter, denominated Fundus Blur Filter (FBF) (Figure 4), consisting of a black image with a white ellipse and a circle surrounding the ellipse, which was applied randomly in terms of position and size around the center of the image. This filter varies the input images’ intensity toward the ellipse, making the model more robust to variations that D-EYE images may contain. Such modification generalizes the model between crops of the same image and between images of the same retina captured at different moments, making the approach adaptable to varying image acquisition conditions.

An improvement was introduced to overcome the limitations in predicting homography between images obtained under different lighting conditions in low-resolution retinal images, such as those faced by [7]. This improvement provides an effective strategy for ensuring the robustness and generalizability of the homography estimation model.

The proposed solution involves applying a blur filter consisting of a black mask containing a white ellipse and a surrounding circle, with random positions and sizes around the center of the image. This filter is then applied to the input images during training, gradually adjusting the intensity towards the ellipse, as shown in Figure 4. To do this, Equation (1) is used, where $f_0$ is the input image at position x, $f_1\left(x\right)$ is the filter at position x, $g\left(x\right)$ is the resulting image at position x, and $\alpha$ controls the blending ratio [21].

$$g\left(x\right)=(1-\alpha )f_0\left(x\right)+\alpha f_1\left(x\right)$$

(1)

By gradually adjusting the intensity of the images towards the ellipse, we improve the model’s ability to deal with the nuances in the images captured by the D-EYE device. The FBF allows the model to generalize between clippings of the same image and between images captured at different times with variations in retinal illumination, thus improving the quality and consistency of the stitching process. The implementation details of the FBF are outlined in Algorithm 1, while the parameters used in the experiments conducted in this work are presented in

Table 1. Parameters of the Fundus Blur Filter.

Parameter	Description	Variation
image_size	Size of the image	128
image_center	Center of the image	(64,64)
ellipse_offset_x	Random integer offset in the X-axis	−25 to 25
ellipse_offset_y	Random integer offset in the Y-axis	−25 to 25
circle_offset_x	Random integer offset in the X-axis	−40 to 40
circle_offset_y	Random integer offset in the Y-axis	−40 to 40
ellipse_axis_range_x	Random integer X-axis of the ellipse	38 to 42
ellipse_axis_range_y	Random integer Y-axis of the ellipse	35 to 42
ellipse_angle_range	Random integer angle of the ellipse	0° to 360°
blur_kernel	Size of the blur kernel	(51, 51)
blur_sigma_range	Range of sigma for Gaussian blur	20 to 30
center_threshold_range	Range of center offset	−5 to 5

.

Algorithm 1. Fundus_Blur_Filter

1: Initialize the filter:   2: filter ← matrix(image_size, image_size) filled with 0   3:   4: Define the center for the ellipse with some offset:   5: $ellipse\_center\_x\leftarrow \left(image\_center\_x+\mathrm{ellipse}\_\mathrm{offset}\_\mathrm{x}\right)$   6: $ellipse\_center\_y\leftarrow \left(image\_center\_y+\mathrm{ellipse}\_\mathrm{offset}\_\mathrm{y}\right)$   7: $ellipse\_center\leftarrow \left(ellipse\_center\_x, ellipse\_center\_y\right)$   8:   9: Defines the circle center around the ellipse: 10: circle_center_x ← ellipse_center_x + circle_offset_range_x 11: circle_center_y ← ellipse_center_y + circle_offset_range_y 12: 13: Define the axes of the ellipse with random values within the axis ranges: 14: ellipse_axes $\leftarrow (\mathrm{ellipse}\_\mathrm{axis}\_\mathrm{range}\_\mathrm{x}, \mathrm{ellipse}\_\mathrm{axis}\_\mathrm{range}\_\mathrm{y})$ 15: 16: Randomly select the angle for the ellipse: 17: ellipse_angle $\leftarrow \mathrm{ellipse}\_\mathrm{angle}\_\mathrm{range}$ 18: 19: Draw the ellipse on the filter at the defined center, axes, and angle: 20: draw_ellipse(filter, ellipse_center, ellipse_axes, ellipse_angle, color = 255) 21: 22: Draw the circle on the filter at the defined center: 23: draw_circle(filter, circle_center, radius = image_size/2, color = 255, thickness = 3) 24: 25: Apply Gaussian blur to the filter: 26: filter $\leftarrow \mathrm{GaussianBlur}(filter, \mathrm{blur}\_\mathrm{kernel}, \mathrm{blur}\_\mathrm{sigma}\_\mathrm{range})$

2.2.2. Feature Extraction

Feature extraction is one of the significant steps in image stitching, which helps match the features between images to estimate the geometric relationship. Therefore, an approach motivated by the work in [12] is used, in which a Siamese convolutional neural network was used for feature extraction along with a feature correlation layer.

In this work, the input images are transformed to grayscale, following most of the practices on training homography estimation models [19,22,23]. The architecture of the network consists of two identical sub-networks with the same weights, each one being composed of four blocks, where each block has a convolutional layer with kernel size $3\times 3$ and a stride of 1. The number of output filters in the first two layers is 64, and 128 in the last two layers. Batch normalization and max-pooling 2D with a pool size of $2\times 2$ are applied after each convolutional layer to reduce the data dimension, followed by a ReLU activation function. The sub-networks process the input images in parallel and extract their feature maps, which are then passed to the subsequent Feature Correlation phase. Figure 5 depicts this architecture, where Feature Map $F_A$ contains the features extracted from Input Image 1, and Feature Map $F_B$ contains the features extracted from Input Image 2.

2.2.3. Feature Correlation

The next step in the network is the feature correlation layer, following the feature extraction phase, where the correspondences between feature maps $F_A$ and $F_B$ extracted from Input Images 1 and 2, are calculated using the proposed method in [12].

Here, L2 normalization is applied to the feature maps $F_A$ and $F_B$, $\in W\times H\times C$; then, correspondences $D\in W\times H\times (W\times H)$ between these two maps are obtained. W, H, and C represent the dimensions of the feature maps, respectively, as well as the width, height, and number of filters (channels). The correspondence at every position in $F_A$ with regard to all positions in $F_B$ is calculated as follows:

$$D(x_1,x_2)={\displaystyle \frac{〈F_A\left(x_1\right),F_B\left(x_2\right)〉}{\left|F_A\left(x_1\right)\right|\left|F_B\left(x_2\right)\right|}},(x_1,x_2)\in {\mathbb{Z}}^2$$

(2)

where $(x_1,x_2)$ are positions in feature maps, while $F_A\left(x_1\right)$ is a one-dimensional feature vector at position $x_1$ in the feature map $F_A$. $D(x_1,x_2)\in [0,1]$, where a value closer to 1 it means that the features in the corresponding positions of the $F_A$ and $F_B$ maps correspond better, thus highlighting the regions where features are more alike.

2.2.4. Regression

To calculate the homography that maps points from one image to another, it is necessary to identify corresponding points between the two images. In this case, we perform the regression of the distances between the corners of an image and themselves in a position where the image aligns with the other image.

For the regression, four semi-Siamese convolutional neural sub-networks were employed. Unlike traditional Siamese networks, where branches share identical weights, semi-Siamese networks have the same structure but allow for different weights in each branch. Each sub-network calculates an ordered pair (x, y), which represents the distances between corners of the second input of the model regarding the deformed marking described in Section 2.1. These distances are the displacement required for each corner of the first input image to align with the second one. Figure 6 shows the Regression Network Architecture proposed. Unlike the [12] method that uses a single neural network for this task, this would allow the homography between low-resolution retinal images to be estimated with more precision since each coordinate is estimated individually.

These sub-networks take the tensor containing the correspondences of features as the input and are structured with four convolutional layers with 512 output filters each. Batch normalization and the ReLU activation function were applied after each convolutional layer, followed by two fully connected layers with a 50% dropout between them. The four networks will have their outputs concatenated to obtain a vector of size 8, which will be the distance between the four corners of the markings for the second input to the network. These are added to the initial coordinates of the corners and provide the final coordinates. The initial and final coordinates are used in computing the homography via the DLT algorithm, reported in the literature by [18].

2.2.5. Image Stitching

Image stitching, also known as image mosaicking, is a process that combines images with overlapping areas to form a wide-view, high-resolution image [24], often referred to as a panorama. The image stitching process involves five main steps: acquiring the images with overlapping regions, detecting distinctive features such as corners or edges in each image, matching corresponding features between the overlapping images, and aligning them based on those matches. After alignment, the images are blended to minimize visible seams and differences in exposure, ending in the generation of the final stitched image [21]. The resulting images must have overlapping areas and represent the same scene.

To perform the image stitching process, we put each 128 × 128 input image in the center of a black frame of size $274\times 274$ pixels. Then, we use the inverse of the homography matrix $\mathbf{H}$ to project each pixel $(u,v)$ of the second image to its new position $(u^{\prime },v^{\prime })$ in the resulting image. In Equation (3), w is the normalization or scaling factor that appears when we represent coordinates in a projective space. Just like w, w’ is used to transform the homogeneous coordinates back to Cartesian coordinates by dividing $x^{\prime }$ and $y^{\prime }$ by $w^{\prime }$. This ensures that the scale is removed and the points are represented correctly in the 2D plane [21]. The transformation is provided by Equation (3) in homogeneous coordinates.

$$\left(\begin{array}{c}{u}^{\prime }\\ v^{\prime }\\ w^{\prime }\end{array}\right)={\mathbf{H}}^{-\mathbf{1}}\left(\begin{array}{c}u\\ v\\ w\end{array}\right)$$

(3)

Subsequently, after applying the homography matrix, image fusion is carried out to blend the overlapping images seamlessly so that minimal differences show up at the places where they meet. Fusion is dealt with, pixel by pixel, in an elementary way just by picking out each color channel from both images for the maximum value. The implementation details of our proposed method are available at https://gitlab.com/jdallyson/fbf_homography_cnn.

2.3. Evaluation Metrics

To compare the quality between the result of stitching and the expected image, metrics that are widely used in the literature [25] were adopted: Structural Similarity Index, Peak Signal-to-Noise Ratio, and Root Mean Squared Error.

Natural image signals have strong spatial dependencies that convey essential structural information, which traditional metrics like RMSE and PSNR fail to capture due to their reliance on pointwise differences [26]. These metrics can yield the same error value for different types of distortions, making them less effective in assessing structural integrity [27]. In contrast, the Structural Similarity Index (SSIM) directly compares image structures by evaluating luminance, contrast, and correlation, aligning more closely with human perception [28]. While SSIM is better suited for capturing the perceptual quality critical to image stitching, RMSE and PSNR still provide valuable insight into pixel-level errors.

The Structural Similarity Index (SSIM) between two images $\alpha$ and $\beta$ is defined as

$$SSIM(\alpha ,\beta )={\displaystyle \frac{(2{\mu }_{\alpha }{\mu }_{\beta }+C_1)(2{\sigma }_{\alpha \beta }+C_2)}{({\mu }_{\alpha }^2+{\mu }_{\beta }^2+C_1)({\sigma }_{\alpha }^2+{\sigma }_{\beta }^2+C_2)}}$$

(4)

where ${\mu }_{\alpha }$ and ${\mu }_{\beta }$ represent the mean pixel values of images $\alpha$ and $\beta$, respectively, providing an average intensity for each image. The terms ${\sigma }_{\alpha }^2$ and ${\sigma }_{\beta }^2$ denote the variances of the pixel values in images $\alpha$ and $\beta$. The covariance between the images is represented by ${\sigma }_{\alpha \beta }$. Constants $C_1$ and $C_2$ are included to stabilize the division when the denominators are close to zero, typically defined as $C_1={\left(K_{1}L\right)}^2$ and $C_2={\left(K_{2}L\right)}^2$, where $K_1$ and $K_2$ are small constants, and L is the dynamic range of pixel values (e.g., 255). The SSIM index ranges from −1 to 1, with 1 indicating perfect similarity between the images.

The Peak Signal-to-Noise Ratio (PSNR) measures the quality between the label and the image resulting from the proposed method; the higher the PSNR, the better the resulting quality, while lower PSNR values mean more significant differences between the images. It is defined as

$$\mathrm{PSNR}=20lo{g}_{10}\left({\displaystyle \frac{MA{X}_f}{\sqrt{MSE}}}\right)$$

(5)

where $MA{X}_f$ is the maximum pixel value in the image, and $MSE$ is the Mean Squared Error, which is defined as

$$\mathrm{MSE}={\displaystyle \frac{1}{n}}\sum {(y-\widehat{y})}^2$$

(6)

where n is the number of pixels in each image, y is a pixel from the label image, and $\widehat{y}$ is the corresponding pixel in the resulting image.

The RMSE measures the difference between the result and the label and is defined as follows:

$$\mathrm{RMSE}=\sqrt{MSE}$$

(7)

2.4. Experiments

To estimate homography, we used the model proposed in this work, trained with 120,000 pairs of 128 × 128 pixel images, the same number of 274 × 274 pixel label images and distance vectors described in Section 2.1, for validation. Additionally, 12,000 pairs of images, labels and distance vectors were used for validation during training. These images were generated from a set of 80,000 images from the MSCOCO dataset [15], following the method described in Section 2.1. During the training, the model reached its best weights after 9 epochs with a learning rate of 0.0001 [29,30,31], a common value used to train deep learning image processing models, and a batch size of 7 as it was the maximum batch value that the machine used could support. The machine configuration used included an Intel i5-11400 processor, 16 GB of RAM and an NVIDIA GeForce RTX 3060 GPU with 12 GB of VRAM.

The tests were carried out on 40 pairs of images extracted manually from the D-EYE video dataset [2]. During extraction, care was taken to avoid frames with little or no relevant information for diagnosis, similar to those illustrated in Figure 7. The green channel of the retinal images contained more relevant information and was therefore used to estimate homography. The videos were taken without dilating the pupil, giving the patient greater comfort, but sacrificing image quality due to eye movements, pupil size and media opacity. The selected images only show the limits of the optic disc, which hinders the demonstration of the method in other structures of the posterior pole.

The label for each image was defined in GIMP [32] by manually aligning the image pairs. Afterward, the images were aligned again, swapping their positions. The resulting composite images were merged pixel by pixel, selecting the highest value for each channel.

Also, 49 pairs of high-resolution images from the FIRE dataset [33] were added to the study, specifically from subset ‘P’, which contains image pairs that could be useful for mosaicking applications. As far as these images are concerned, two kinds of tests were conducted: on the whole, FIRE ‘P’ images to test the alignment technique and, in a second phase, Regions Of Interest corresponding to the optical disk in each image were cut, producing a set of pairs of cut-outs that were then joined together. This was carried out just like with the D-EYE images in GIMP. The choice to include both the complete images and the optical disk cut-outs was aimed at verifying whether the proposed model is effective at different scales and levels of detail. Figure 8 shows one example of each dataset used in the tests.

3. Results and Discussion

This section describes and discusses the results obtained by the proposed method. To this end, the SSIM, PSN, MSE and RMSE metrics are presented in Section 2.3.

Table 2. Results using the homography estimation method proposed by [12].

Dataset	Metric	Average	Standard Deviation	Minimum	Median	Maximum
D-EYE	PSNR↑	18.18	2.56	13.00	18.67	23.09
	SSIM↑	0.89	0.03	0.77	0.83	0.89
	RMSE↓	4.89	0.33	4.09	4.91	5.81
Fire P	PSNR↑	19.01	2.60	15.26	18.33	25.85
	SSIM↑	0.80	0.04	0.71	0.81	0.90
	RMSE↓	51.52	13.94	22.52	53.52	76.22
Fire P crop	PSNR↑	17.84	2.36	14.11	17.15	22.71
	SSIM↑	0.85	0.03	0.75	0.85	0.92
	RMSE↓	58.53	14.64	32.29	61.27	86.95

shows the result with the homography estimation method of [12]. While it may seem that the values of the metrics are close to those in

Table 3. Results using the homography estimation method proposed by [12] with the addition of the Fundus Blur Filter during training.

Dataset	Metric	Average	Standard Deviation	Minimum	Median	Maximum
D-EYE	PSNR↑	22.9	3.29	18.04	22.69	32.4
	SSIM↑	0.89	0.04	0.77	0.90	0.97
	RMSE↓	3.92	0.69	2.45	3.90	5.43
Fire P	PSNR↑	23.62	5.02	14.63	23.64	36.03
	SSIM↑	0.82	0.09	0.57	0.85	0.96
	RMSE↓	33.99	19.11	6.98	29.06	81.94
Clipping Fire P	PSNR↑	20.29	3.10	14.11	20.23	27.65
	SSIM↑	0.89	0.04	0.77	0.89	0.97
	RMSE↓	45.35	15.93	18.29	42.99	86.98

, the original approach was unhelpful in predicting the homography in the test cases coarsely. Mostly, the approach just centers the images and overlays them over each other without accurate alignment. It was not until the FBF proposed in this work that the network aligned the images correctly. The introduction of the FBF was a breakthrough for the homography estimation model, representing an improvement over previous results.

Table 2 reflects the limits of the former method, and Table 3 and

Table 4. Results obtained from our method.

Dataset	Metric	Average	Standard Deviation	Minimum	Median	Maximum
D-EYE	PSNR↑	26.14	3.78	16.93	26.17	33.15
	SSIM↑	0.96	0.04	0.80	0.93	0.97
	RMSE↓	3.21	0.85	1.74	3.09	5.19
Fire P	PSNR↑	24.08	6.27	14.86	23.49	36.99
	SSIM↑	0.80	0.10	0.56	0.82	0.95
	RMSE↓	34.84	22.25	6.25	29.57	79.81
Clippling Fire P	PSNR↑	25.46	3.80	17.82	25.19	36.89
	SSIM↑	0.94	0.03	0.83	0.94	0.98
	RMSE↓	25.66	10.45	6.31	24.29	56.76

represent a qualitative leap in the precision of the estimates. The introduction of the blur mask increased PSNR significantly from 18.18 dB to 22.90 dB, indicating a big drop in the image reconstruction error. The SSIM improved from 0.82 to 0.89, indicating enhanced structural fidelity after alignment. Moreover, the reduction in the RMSE from 4.89 to 3.92 shows improved accuracy of estimated distances between images.

The proposed method goes further ahead in the improvements made. The average PSNR increased to 26.14 dB, the SSIM to 0.926, and the RMSE reduced to 3.21 for the D-EYE dataset. As observed in Table 4, it is standing apart and additionally proving that the blur mask is not some incremental improvement; rather, it is designed for playing a very critical role in the accurate and effective aligning of retinal images.

Figure 9 shows four examples of results where the input images have substantial illumination differences between them. Although the results are generally more accurate, minor displacements between the label and the result image are still present, as reflected in the metric values shown in Table 4.

Table 5. Result by average metric for each method.

	Proposed by [12]			Proposed by [12] + FBF			Our Method
	D-EYE	FIRE P	C FIRE P	D-EYE	FIRE P	C FIRE P	D-EYE	FIRE P	C FIRE P
PSNR↑	18.68	19.01	17.84	22.90	23.62	20.29	26.14	24.08	25.46
SSIM↑	0.89	0.80	0.85	0.89	0.82	0.89	0.96	0.80	0.94
RMSE↓	4.89	51.52	58.53	3.92	33.99	45.35	3.21	34.84	25.66

Highest values are indicated in bold.

provides a summary of the average values of each metric across the datasets, facilitating a clearer comparison between the methods.

Still, it was found that the individualized regression strategy for each point was really useful for correcting inaccuracies and improving alignment, as exemplified in Figure 10, where it is easy to see that the multiple regression method can provide results closer to the label than single regression.

Figure 11 presents comparative examples of the 3 methods; it is evident in the first row, where FBF is not used during model training, that the model is unable to estimate a homography matrix that can be used to align the images. In the second line, with the single regression that used FBF during the training in some images, such as (A), (D), (E) and (F), it is possible to observe inferior alignment quality compared to the proposed method.

Evaluating the proposed method on high-resolution images from the P subset of the FIRE dataset showed that the model could return a sufficiently accurate homography in 21 cases out of 49 image pairs. Most of the failures occurred with images with little overlap between them, an essential factor for accurate homography estimation. However, this is not a significant obstacle to the problem presented in this work since it is possible to use neighboring frames of the videos for stitching, ensuring sufficient overlap. Figure 12A shows a pair of images from subsection “P” of the FIRE dataset, where, with sufficient overlap between the pictures, the proposed method was able to estimate an approximate homography matrix. Figure 12B shows a pair of images with little overlap, in which the model could not estimate a homography matrix that aligned with the figures.

4. Conclusions

This research proposed a homography estimation method for stitching low-resolution retinal images. The method could extract relevant features from the input images and calculate a correlation feature, enabling the alignment of retinal images.

From the results discussed, we can see that the introduction of the Fundus Blur Filter (FBF) into the training process of the homography estimation model brought important improvements with regard to image alignment accuracy. The increase in PSNR and SSIM, as well as the reduction in RMSE, indicate that the proposed model was able to consistently improve the quality of the reconstruction, outperforming previously tested methods. Although the method proved effective, especially in the D-EYE dataset, when compared to the method proposed by [12], improving the PSNR by 7.96 dB and achieves a 7.86% higher SSIM score, there are still challenges in situations where there is little overlap between the images, as observed in the FIRE dataset. Despite these limitations, we believe that the use of neighboring frames, as discussed, can mitigate some of these difficulties, ensuring sufficient overlap for proper alignment.

As this current approach requires manual intervention in selecting frames from the videos obtained through D-EYE, acting as a drawback for use in a clinical setting, future work is expected to overcome the identified limitations, aiming at automating the selection of sections of the area of interest in the video frames captured using D-EYE to develop automatic algorithms for ordering frames to enable alignment and stitching of multiple images. This will enable better visualization of the retina by reconstructing low-resolution retinographies in a panorama, helping ophthalmologists identify eye diseases early enough to treat them.