Nested U-Net-Based GAN Model for Super-Resolution of Stained Light Microscopy Images

Abstract

The purpose of this study was to propose a deep learning-based model for the super-resolution reconstruction of stained light microscopy images. To achieve this, perceptual loss was applied to the generator to reflect multichannel signal intensity, distribution, and structural similarity. A nested U-Net architecture was employed to address the representational limitations of the conventional U-Net. For quantitative evaluation, the peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and correlation coefficient (CC) were calculated. In addition, intensity profile analysis was performed to assess the model’s ability to restore the boundary signals more precisely. The experimental results demonstrated that the proposed model outperformed both the signal and structural restoration compared to single U-Net and U-Net-based generative adversarial network (GAN) models. Consequently, the PSNR, SSIM, and CC values demonstrated relative improvements of approximately 1.017, 1.023, and 1.010 times, respectively, compared to the input images. In particular, the intensity profile analysis confirmed the effectiveness of the nested U-Net-based generator in restoring cellular boundaries and structures in the stained microscopy images. In conclusion, the proposed model effectively enhanced the resolution of stained light microscopy images acquired in a multichannel format.

1. Introduction

Light microscopy, which can be used to observe high-resolution microstructures, is an essential tool for tissue analysis in disease diagnosis and biological research [1,2]. However, light microscopes have inherent limitations in resolution owing to physical limitations that include diffraction. Performing a detailed analysis at high magnification after obtaining an overview of the entire structure at low magnification can result in sample damage and increased image acquisition time [3,4,5].

Staining techniques have been introduced to compensate for these limitations indirectly. Staining increases the contrast between the nucleus, cells, and the tissue matrix, clearly revealing the structure of the subject and enabling accurate pathological analysis [6,7]. In addition, quantitative evaluation using stain intensity and distribution is possible, and biological information such as protein expression locations can be visualized using specific markers [8,9,10]. Inappropriate staining can cause noise in images, blur cell boundaries, and overly intensify or weaken the signals from specific structures, which can reduce the accuracy of quantitative analysis [11,12].

Thus, various algorithms have been proposed to directly improve resolution. In particular, super-resolution algorithms demonstrate high image restoration performance, such as preserving boundary signals with improved blurring and noise in grayscale light-microscopy images.

Among conventional super-resolution algorithms, a variety of techniques have been developed to surpass the diffraction limit through fluorescence imaging grounded in physical principles [13,14,15,16]. These approaches enhance spatial resolution by combining precise illumination control or position-dependent data acquisition with advanced post-processing and image reconstruction procedures. Fluorescence fluctuation-based super-resolution methods further improve resolution by exploiting the temporal variability of fluorescence emission [17,18]. By statistically analyzing the intrinsic flickering and intensity fluctuations of fluorescent molecules, these techniques enable detailed structural reconstruction without the need for specialized optical configurations or learning-based models. Interference-based super-resolution techniques address diffraction limitations by leveraging the interference properties of light, allowing for the visualization of cellular microstructures that remain inaccessible under conventional optical microscopy [19,20,21]. Through the integration of multiple optical paths, phase modulation, and precise interference pattern analysis, these methods facilitate high-resolution three-dimensional imaging. Although they often require intricate optical alignment, rapid data acquisition, and complex reconstruction algorithms, they can achieve spatial resolutions on the scale of several tens of nanometers.

However, stained light microscopy images are typically composed of three channels (RGB) [22,23]. Compared to single channels, multiple channels generated by staining cause distortion signals, such as the point spread function (PSF), noise, and blurring, which are difficult to characterize, leading to difficulties in accurate correction [24,25]. In addition, staining can introduce nonlinear color distributions and boundary intensities across different structural regions. Consequently, applying inappropriate algorithms may exaggerate artificial boundaries driven by color contrast rather than true structural features [26,27]. In addition, optimization of the conventional super-resolution algorithm should be tailored to specific image characteristics, as the staining intensity can vary significantly across different samples.

Deep learning-based models have been proposed to overcome the limitations of conventional algorithms for improving the resolution of images. Deep learning models, which are trained on real image data, show superior performance in analyzing nonlinear and complex patterns and demonstrate high potential for noise and artifact reduction. These results indicate a strong suitability for processing stained light microscopy images. In particular, deep learning models have been actively applied to various analytical tasks in light microscopy imaging, such as segmentation, image restoration, and super-resolution, owing to their notable advantages in image processing [28,29,30,31].

Initial U-Net-based models have been actively used in light microscope images owing to their advantages, such as the superior restoration of high-frequency information and efficient computation owing to their simple structure [32]. However, the use of single-skip connections to transfer the encoded feature information often results in blurred boundary representations. In contrast, the nested U-Net model addresses the structural limitations of the U-Net by introducing nested skip pathways with intermediate convolution layers between the encoder and decoder, along with deep supervision [33]. This architecture enables multiscale learning and improves the accuracy of boundary and fine structure restoration. In particular, compared to the standard U-Net, the nested U-Net architecture further improves feature propagation and gradient flow through dense nested paths. These nested paths enable features extracted at various depths to be reused across scales, which contribute to minimizing the semantic gap between the encoder and decoder. The structure of nested U-Net is particularly suitable for restoring fine structures in images that contain complex and multichannel information, such as stained optical microscope images. Furthermore, applying deep supervision not only improves convergence speed during training, but also reduces the risk of overfitting by supervising intermediate outputs at multiple stages of the decoder. Thus, nested U-Net demonstrates improved generalization performance and robustness in high-resolution image restoration.

Generative adversarial network (GAN)-based models were trained to generate visually realistic high-resolution outputs by simultaneously leveraging adversarial and perceptual losses [34,35]. Because the discriminator evaluates visual plausibility based on features such as boundaries, high-frequency textures, and fine intracellular structures, the generator is encouraged to emphasize these components, leading to an enhanced restoration of fine details and edges. However, in the pursuit of visual sharpness, GANs may also produce artificial high-frequency information, potentially resulting in boundary distortions or the generation of false edges [36,37].

Therefore, in this study, we propose a GAN model that incorporates a nested U-Net architecture to improve the super-resolution of stained light microscopy images, effectively addressing the structural limitations of U-Net and the perceptual artifacts commonly observed in conventional GANs.

2. Materials and Methods

2.1. Dataset Construction

The light microscopy images of mouse embryo head sections at embryonic day 16.5 were used as the dataset. All tissue sections were prepared and stained with hematoxylin and eosin (H&E) using standard histological procedures. Light microscopy images were captured at 12.5× magnification using a digital microscope (DM500; Leica Microsystems, Heerbrugg, Switzerland) and saved in RGB format at a resolution of 1040 × 1392 pixels. All imaging parameters were kept consistent across samples.

From the acquired light microscopy images, the noise was reduced while preserving the edge signals by applying the non-local means (NLM) algorithm as follows:

$$NLM\left[f\right]\left(m\right)={{\displaystyle \sum }}_{n=1}\omega \left(m,n\right)f\left(n\right),$$

(1)

where $m$ and $n$ are the pixel indices in the stained light microscopy image, and $\omega \left(m,n\right)$ is a weight representing the similarity between pixels $m$ and $n$ constrained by $0 \le \omega \left(m,n\right)\le 1$. The weights were computed based on the distance between pixel neighborhoods as follows:

$$\omega \left(m,n\right)={\displaystyle \frac{1}{Z\left(m\right)}}{e}^{-{\displaystyle \frac{\parallel v\left(k_m\right)-v{\left(k_n\right)\parallel }_{2,a}^2}{d^2}}},$$

(2)

$$Z\left(m\right)={{\displaystyle \sum }}_n{e}^{-{\displaystyle \frac{\parallel v\left(k_m\right)-v{\left(k_n\right)\parallel }_{2,a}^2}{d^2}}},$$

(3)

Here, for a square kernel of size $k$ centered at pixel $i$, $v\left(k_i\right)$ represents the vectorized intensity values within the patch. The term $\parallel v\left(k_m\right)-v{\left(k_n\right)\parallel }_{2,a}^2$ is the squared Euclidean distance between two patches, computed with a Gaussian kernel of standard deviation $a$. $Z\left(m\right)$ is a normalization constant that ensures that the weights sum to one, and $d$ is a smoothing parameter that controls the degree of filtering. In this study, $d$ was set to 0.05, and the patch size and search window were set to 7 and 15, respectively.

Subsequently, two sets of images were prepared: one processed with NLM, and the other with a resolution degraded by a factor of 16. Both were converted into patches from 512 × 512 matrix-sized images using a stride of 256, forming the label and input datasets, respectively. We obtained 26,996 paired data points; 19,748, 2416, and 4832 paired data points were used for training, validation, and testing, respectively.

2.2. Nested U-Net-Based GAN Model

2.2.1. Comparison with Previous Super-Resolution Models in Microscopy

A variety of deep learning-based models have been proposed to achieve the super-resolution reconstruction of light microscopy images.

Table 1. Prior studies on deep learning-based super-resolution model development for optical microscopy images.

Author	Year	Input Type	Architecture	Generator Loss	Discriminator Loss	Notable Features
Rivvenson et al. [38]	2017	Multichannel	DCNN	L2		Registration-based dataset, direct image-to-image mapping, self-feeding
Nehme et al. [39]	2018	Single channel	FCN	L1 + L2		Localization-free reconstruction, sparse regression optimization
Wang et al. [40]	2019	Multichannel	U-Net + patchGAN	MSE, SSIM	BCE	Hybrid loss design, platform-adaptive, patch-based discriminator
Qiao et al. [41]	2021	Multichannel	cGAN + Fourier Channel Attention	MSE, SSIM, BCE	BCE	Spatial-frequency domain integration
Sun et al. [42]	2022	Multichannel	DCGAN	MSE, VGG19, Gram, TV	BCE	Multi-component loss for texture restoration
Chen et al. [43]	2023	Multichannel	U-Net + Residual-dense based patchGAN	L1, SSIM, VGG19	BCE	Dual-stage (signal enhancement + SR), U-Net discriminator, frequency domain L1
Qiao et al. [44]	2024	Multichannel	U-Net + 3D RCAN	MSE, Hessian Reg., Gap Amend. Reg.		Self-supervised with image re-corruption, dual-stage denoise + deconvolution
Guo et al. [45]	2025	Multichannel	3D RCAN	MSE		Multi-stage synthetic degradation for self-supervision, scalable multi-step restoration

summarizes prior representative studies, organized by input channel types, network architectures, loss functions, and key technical features.

Early approaches, such as those based on deep convolutional neural networks (DCNNs) and fully convolutional networks (FCNs), predominantly relied on pixel-wise losses and were often constrained to single channel data. As a result, their applicability to multichannel stained images was limited, particularly due to the lack of perceptual and adversarial components in their design. To overcome these shortcomings, U-Net and GAN-based architectures were introduced. These models adopted adversarial learning strategies, with later studies incorporating attention mechanisms and frequency-domain modules. Such developments enhanced the realism of reconstructed structures and improved edge sharpness. Nonetheless, most models were still optimized for grayscale images, and only a few directly addressed the nonlinear distortions introduced by staining in multichannel microscopy. More recent efforts have applied composite loss functions to better balance global image quality with local textural detail. In parallel, advances in architectural design, such as the use of attention modules, residual dense blocks, multi-scale feature fusion, and frequency-domain processing, have contributed to the improved restoration of fine structures and greater robustness to noise. However, explicitly modeling the semantic variability and nonlinear structural characteristics of complex stained microscopy data remains a significant challenge. Furthermore, most existing models have been evaluated on images with relatively mild degradation, where much of the original spatial and color information is still retained. As a result, their effectiveness is limited when applied to severely degraded inputs, which often occur in practical microscopy settings involving poor-quality or highly compromised data. To address these issues, the present study introduces a GAN-based model employing a nested U-Net generator, specifically optimized for stained light microscopy images. The generator utilizes densely connected skip pathways and deep supervision to enhance the recovery of structural details, while a patch-based discriminator enforces local realism in fine cellular features. The model was trained using a composite loss function allowing it to effectively handle both nonlinear chromatic distortions and semantic inconsistencies introduced by staining. Furthermore, the proposed model was designed to effectively restore both visual clarity and structural fidelity in severely degraded multichannel images, aiming to address challenging conditions where conventional models tend to underperform.

2.2.2. Generation Architecture

Figure 1 is an illustration of a super-resolution nested U-Net-based GAN model for stained light microscope images. The generator in the proposed GAN model was based on a nested U-Net architecture, which addresses the semantic gap limitations of the original U-Net by introducing densely connected skip pathways between the encoder and decoder at multiple depths. These nested skip connections enable the progressive refinement of features and facilitate the more effective fusion of multi-scale contextual information. Each convolutional unit comprises two 3 × 3 convolutional layers, followed by Batch Normalization and ReLU activation. Feature maps from deeper layers were up-sampled using bilinear interpolation and concatenated with shallower features to enhance the structural reconstruction. Furthermore, deep supervision was applied by generating auxiliary outputs at multiple decoding stages, which improved the gradient flow and accelerated convergence during training.

To achieve the high-quality super-resolution of stained light microscopy images, we employed a composite generator for perceptual loss that jointly accounted for pixel-level accuracy, perceptual similarity, structural consistency, and artifact suppression. The overall objective function is defined as follows:

$$L_G={\mathcal{L}}_{GAN}+{\lambda }_{L1}{\mathcal{L}}_{L1}+{\lambda }_{VGG}{\mathcal{L}}_{VGG}+{\lambda }_{SSIM}{\mathcal{L}}_{SSIM}+{\lambda }_{TV}{\mathcal{L}}_{TV},$$

(4)

where ${\mathcal{L}}_{GAN}$ denotes the adversarial loss, which encourages the generator to produce visually realistic outputs that are indistinguishable from real high-resolution images, and are particularly important for recovering fine-grained textures and staining-induced high-frequency variations. ${\mathcal{L}}_{L1}$ is the pixel-wise L1 loss between the generated image and ground truth, contributing to the overall structural alignment and reducing the global intensity deviations. ${\mathcal{L}}_{VGG}$ represents the perceptual loss, calculated as the L1 distance between the feature maps extracted from a pretrained VGG-19 network, enabling the preservation of semantic details and morphological patterns. ${\mathcal{L}}_{SSIM}$ measures structural similarity by comparing luminance, contrast, and texture between images, which is critical for maintaining biologically meaningful features such as cell boundaries and internal structures. Finally, ${\mathcal{L}}_{TV}$ is a total variation regularization term that reduces noise and suppresses checkerboard artifacts by enforcing local smoothness in the generated output.

In models for the super-resolution of stained microscope images, L1 loss is most important for accurately estimating the signal intensity of the restored image. However, applying L1 loss independently can cause excessive blurring. To resolve blurring and to clearly depict the boundaries of fine structures, SSIM loss was applied. Additionally, TV loss contributes to maintaining the local smoothness of the image by mitigating the artificially enhanced boundary signals caused by SSIM loss. VGG loss calculates the loss based on feature maps, thereby preserving the shapes of detailed structures and complementing L1 and SSIM loss. The weighting factors ${\lambda }_{L1}$, ${\lambda }_{VGG}$, ${\lambda }_{SSIM}$, and ${\lambda }_{TV}$ control the relative importance of each loss component and were empirically set to 50, 1, 5, and 0.01, respectively. This multi-objective formulation is particularly well-suited for stained light microscopy data, where the color distribution, structural integrity, and perceptual clarity must be simultaneously optimized to ensure both visual plausibility and quantitative reliability.

2.2.3. Discriminator Architecture

In this study, the discriminator adopted a patchGAN-based architecture that evaluates the realism of image patches rather than the entire image. Specifically, the discriminator receives a concatenation of the input image and either the ground truth or the generated image as input, and produces a two-dimensional map of real/fake predictions, where each element corresponds to a local receptive field. Structurally, the discriminator consists of a series of convolutional layers that progressively down-sample the input while preserving spatial information at the patch level. This enabled the model to focus on fine-grained, localized inconsistencies between real and generated image pairs. For optimization, a binary cross-entropy loss with logits (BCEWithLogitsLoss) was used to distinguish between the real and fake patches. The adversarial loss from this discriminator is back-propagated to the generator to encourage the production of a locally realistic output. In the proposed model, the discriminator structure was applied to generate images that were both structurally accurate and visually convincing. The discriminator structure penalizes elements such as unrealistic textures and color transitions. High-resolution label images were used as ground truth in the adversarial learning process, and the generator’s output was directly compared with ground truth to enable the more effective learning of high-frequency information and detailed structures in boundary areas.

For these reasons, patchGAN is particularly well suited for stained light microscopy images, where critical information is often concentrated in small, high-frequency structures, such as cell boundaries, nuclei, and stained subcellular regions. Unlike global discriminators that may overlook localized differences, patchGAN enforces local realism, which is essential for faithfully reconstructing biologically relevant textures and structures that are altered by staining. Localized adversarial supervision is crucial to improve the perceptual quality and interpretability of super-resolved microscopy images.

2.3. Quantitative Evaluation

To quantitatively evaluate the performance of each super-resolution model on the stained light microscopy images, the peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and correlation coefficient (CC) were measured as follows:

$$PSNR=10 · lo{g}_{10}{\displaystyle \frac{S_{peak}^2}{MSE}},$$

(5)

$$MSE=\sqrt{{\displaystyle \frac{{{\displaystyle \sum }}_{i=1}^N{\left(f_i-g_i\right)}^2}{N}}},$$

(6)

where $f_i$ and $g_i$ represent the reference and comparison images, respectively, $N$ is the number of pixels in the image, and $S_{peak}^2$ is the maximum signal intensity in the region of interest (ROI).

$$SSIM={\displaystyle \frac{\left(2{\mu }_f{\mu }_g+C_1\right)\left(2{\mu }_{fg}+C_2\right)}{\left({\mu }_f^2+{\mu }_{g }^2+C_1\right)\left({\sigma }_{f }^2+{\sigma }_g^2+C_2\right)}},$$

(7)

$$CC={\displaystyle \frac{{{\displaystyle \sum }}_{i=1}^N\left(f_i-\widehat{f}\right)\left(g_{i }-\widehat{ g}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^N{\left(f_i-\widehat{f}\right)}^2}\sqrt{{{\displaystyle \sum }}_{i=1}^N{\left(g_i-\widehat{g}\right)}^2}}},$$

(8)

where ${\mu }_f$ and ${\mu }_g$ represent the local mean intensities of the reference image $f$ and the comparison image $g$, respectively, and ${\mu }_{fg}$ represents the local covariance between the two images. ${\sigma }_f^2$ and ${\sigma }_g^2$ are the local variances, and $C_1$ and $C_2$ are small constants introduced to avoid instability when the denominators are close to zero. In addition, $\widehat{f}$ and $\widehat{g}$ are their global mean values, and $N$ is the total number of pixels.

SSIM comprehensively considers brightness, contrast, and structural elements, while CC measures the linear relationship between two images, enabling structural and statistical quality assessment of the entire image. In particular, in color images, structural information dispersion and color distortion occur in each channel. SSIM and CC are useful factors that can comprehensively evaluate consistency in structure and color between channels, including simple intensity differences. However, blurring effects can occur in detailed areas during the application of super-resolution models. Thus, the tooth region of the embryo was designated as the region of interest (ROI) to analyze variations in boundary sharpness and the signal intensity profiles were measured. The mean squared error (MSE), mean absolute error (MAE), and Pearson correlation coefficient (PCC) were calculated based on the obtained intensity profiles. In addition, to reduce the influence of noise on the label data during intensity profile extraction, a median filter was applied as a preprocessing step. The measured MSE, MAE, and PCC can analyze the restoration rate of high-frequency signals in local areas that cannot be measured by PSNR and SSIM. In particular, MSE and MAE can intuitively confirm the difference in signal intensity between the two images. In addition, PCC was measured to evaluate the consistency of detailed structural differences and intensity distribution patterns.

3. Results

Figure 2 presents the results of applying various super-resolution models to the stained light microscopy images. Box A indicates the enlarged tooth region shown in Figure 3.

Visually, the output images generated by all super-resolution models exhibited improved image quality compared to the input images. However, in some models, although noise reduction had been achieved, the blurring of fine structures remained unresolved. In particular, Figure 3 demonstrates that the U-Net model failed to preserve fine details in Circle A, indicating a degraded resolution performance. By contrast, both the nested U-Net and GAN-based models showed enhanced contrast and boundary sharpness in the embryonic tooth region.

Figure 4 illustrates another example of super-resolution applied to stained microscopy images, with Box B corresponding to the magnified salivary gland area shown in Figure 5.

In this case, the U-Net model exhibited a globally distorted signal intensity. Meanwhile, the GAN model based on nested U-Net clearly enhanced the contrast and boundary definition in subtle tissue structures. Figure 6 presents enlarged images of the three regions defined in Figure 5 to provide a visually intuitive comparison of the performance of each model.

For quantitative evaluation, the PSNR, SSIM, and Pearson correlation coefficient (PCC) were measured, and the results are summarized in Figure 7.

Among all the models, the GAN model based on the nested U-Net yielded the highest performance across all metrics. Notably, when applying the U-Net model for the super-resolution of stained light microscopy images, the PSNR, PCC, and SSIM were approximately 86.4%, 97.7%, and 99.6%, respectively, compared with the input image, indicating a degradation in reconstruction quality. In addition, to evaluate the degree of restoration of actual tissue boundaries, as shown in Figure 8, line A in Figure 3 was selected as the region of interest (ROI), and the intensity profiles were measured and analyzed (

Table 2. Quantitative evaluation results of super-resolution models based on intensity profiles.

Model	MSE	MAE	PCC
Input	$1.03\times {10}^{-3}$	$2.59\times {10}^{-2}$	0.949
U-Net	$4.12\times {10}^{-3}$	$5.81\times {10}^{-2}$	0.963
Nested U-Net	$1.32\times {10}^{-3}$	$3.14\times {10}^{-2}$	0.979
U-Net-based GAN	$6.91\times {10}^{-4}$	$2.12\times {10}^{-2}$	0.966
Nested U-Net-based GAN	$5.97\times {10}^{-4}$	$1.97\times {10}^{-2}$	0.972

).

The GAN model based on nested U-Net exhibited the best performance in terms of MSE and MAE. However, in terms of the PCC, although the model achieved high accuracy, it showed an approximately 0.7% lower correlation than the model using nested U-Net without GAN.

4. Discussion

Light microscopy is widely used in the biological and diagnostic fields to analyze cellular and tissue structures. However, structural differentiation is often challenging because most biological samples are colorless or translucent. To address this issue, staining is commonly employed to enhance the contrast and facilitate morphological identification. This process enables the more accurate detection of spatial features, such as location, size, and shape, and supports quantitative analysis. Furthermore, because staining agents selectively bind to specific biochemical components, they allow the investigation of various biological characteristics and molecular information within the sample.

In particular, stained light microscopy images demonstrate that each RGB channel corresponds to distinct biological structures (e.g., nuclei, cytoplasm), and each channel exhibits unique point spread functions (PSFs), noise characteristics, and intensity distributions. This causes nonlinear and heterogeneous signal distortion between channels. However, most traditional super-resolution algorithms are based on grayscale images and assume that the PSFs, noise distribution, and blur of the entire image are the same regardless of the channel. Thus, conventional methods fail to capture structural interactions across channels, and simple intensity-based restoration often leads to issues such as color boundary distortion, color loss, and the generation of artificial edges. Furthermore, the omission of structural information may lead to distortions in biologically important features such as cell boundaries. In addition, conventional U-Net models trained with a single loss function focus solely on minimizing pixel-wise differences, which makes it difficult to account for the complex structural and chromatic interactions inherent in color images. The use of a single skip connection also limits the model’s ability to comprehensively estimate inter-channel structural signals and morphological variations in stained microscopy data. To address this issue, the present study proposes a GAN model based on a nested U-Net architecture to effectively achieve super-resolution for stained light microscopy images. Furthermore, perceptual loss was employed to reflect diverse image characteristics, thereby enhancing the reconstruction performance of the model.

The quantitative evaluation results demonstrated a progressive improvement in performance across all factors in the order of U-Net, nested U-Net, U-Net-based GAN, and nested U-Net-based GAN (Figure 7). Specifically, the PSNR is a metric that quantifies the reconstruction error based on pixel-wise differences in signal intensity. The U-Net model recorded the lowest PSNR, and the nested U-Net model exhibited relatively poor performance compared to the GAN-based models. These results suggested that models using only the U-Net architecture were less effective in restoring the signal intensity and distribution in multichannel-stained light microscopy images than those incorporating GANs. SSIM, a metric that considers image brightness, contrast, and structural consistency to evaluate structural similarity, showed a similar trend, with the GAN-based models outperforming the others. In contrast, CC, which focuses on the linear relationship between pixel intensities rather than the absolute signal magnitude, showed relatively small performance differences between the nested U-Net and GAN-based models.

This suggests that the integration of GAN architectures is more effective than conventional U-Net structures in enhancing the resolution of stained-light microscopy images [46,47]. The basic U-Net architecture employs a skip connection scheme that transfers features directly from each encoder block to its corresponding decoder block, without adequately addressing the semantic gap between encoder and decoder features. This makes it insufficient for restoring fine details such as boundary information and internal cellular structures [48]. In particular, since the input data in this study were down-sampled by a factor of 16, the U-Net lacked sufficient structural means to effectively recover the severe loss of high-frequency information. Additionally, the model processed all input channels through a unified feature flow after merging them in the initial convolution layer, without considering the distinct biological meanings or signal distribution characteristics of each channel. This could have resulted in the distortion of boundary signals and tissue structures due to an imbalance in the ratio of signal intensities between channels. To address these limitations, the nested U-Net model improved structural restoration by employing nested skip connections and deep supervision. However, it still relied on pixel-wise-based static feature flows and lacked a feedback structure to guide the recovery of boundary sharpness or visual clarity. These limitations became more pronounced in stained light microscopy images, where each channel carries unique biological information. Without explicitly modeling the nonlinear color distributions and boundary intensities across channels, the nested U-Net may have failed to maintain the structural consistency and relative intensity ratios, leading to blurred or distorted reconstructions [49,50,51]. On the other hand, GAN-based models leverage the adversarial interplay between the generator and discriminator to more accurately reconstruct not only visual similarity but also sharp boundaries and intricate intracellular structures [52,53,54]. This was achieved by training the generator to produce stained light microscopy images that were evaluated by the discriminator across all color channels, thereby promoting the restoration of channel-specific features that resembled actual biological structures. In this process, the discriminator provided implicit feedback regarding the visual plausibility of the generated outputs, which guided the generator toward reducing artifacts such as color imbalance and channel-dependent edge degradation. In particular, the patchGAN discriminator, which operates on local image patches, contributes to the enhancement of fine structures such as cell boundaries and subcellular regions, enabling perceptually coherent and structurally faithful reconstructions.

The reliability of this analysis was further supported by the intensity profile results presented in Table 2. According to the intensity profile evaluation, the GAN models based on the U-Net architecture exhibited superior performances in terms of the MSE and MAE. This indicated that the GAN-based models achieve higher quantitative accuracy in estimating the signal intensity and distribution in multichannel images. In particular, the accurate restoration of signal distribution across multiple channels can significantly contribute to enhancing both the contrast and resolution between structures in stained light microscopy images. On the other hand, with respect to structural similarity evaluated by the Pearson correlation coefficient (PCC), the nested U-Net model demonstrated the highest correlation with the label data compared to conventional U-Net and GAN-based models. This suggests that the nested U-Net architecture partially overcame the limitations of the basic U-Net in representing structural patterns.

On the other hand, the GAN model incorporating a nested U-Net architecture yielded slightly lower PCC values than the standalone nested U-Net. This result indicated that, in the process of enhancing visual sharpness, the GAN may have overemphasized high-frequency details, leading to the appearance of artificial edges or distortions that deviated from the actual structural features. These findings imply that although the nested U-Net performs well in restoring boundary information, its integration within a GAN framework requires more careful tuning of the perceptual loss. Moreover, introducing a global discriminator loss in parallel may contribute to suppressing excessive high-frequency enhancement and supporting the preservation of overall structural continuity. With further refinement, the model could be developed into a more advanced framework that more reliably restores both the signal intensity and spatial distribution in multichannel stained light microscopy images, while also preserving fine structural detail.

In addition, the proposed super-resolution model demonstrated overall strong performance in high-frequency restoration; however, several limitations were also clearly observed. As illustrated in Figure 5 and Figure 6, when magnifying the salivary gland region, the model successfully reconstructed prominent boundary signals; however, small and fine edge structures were either blurred or partially omitted. This limitation was presumed to result from the extreme downscaling of input data during training, which likely caused the loss of detailed boundary information. Additionally, certain fine boundary signals exhibited statistical properties similar to background noise, leading the model to mistakenly suppress them as noise [55,56,57]. To address these shortcomings, it is anticipated that incorporating pre- or post-processing techniques such as deblurring algorithms could enhance the preservation of fine structures and enable more precise edge restoration.

In addition, although multiple-slice images were used for model training, the dataset was constructed based on a single embryo specimen. Therefore, further validation using a more diverse set of embryo samples is required to ensure the generalizability of the results. In addition, while perceptual loss was employed to capture structural similarities along with signal intensity and distribution across channels, the super-resolution model followed the conventional U-Net-based architecture [58,59,60]. To further enhance the super-resolution performance proposed in this study, future work should consider structural modifications or extensions of the model, such as optimized layer configurations, the use of multi-resolution inputs, and the integration of color correction subnetworks [61,62,63].

5. Conclusions

In this study, we proposed a nested U-Net-based GAN model for the super-resolution reconstruction of stained light microscopy images. We first identified the structural representation limitations of conventional U-Net-based models and addressed them by employing a nested U-Net architecture as the generator, which was advantageous for restoring cellular boundaries and fine structures. The perceptual loss was incorporated to enhance the visual quality of the reconstructed images. The experimental results demonstrated that the proposed model outperformed the conventional U-Net and GAN-based models in terms of quantitative metrics such as PSNR, SSIM, and PCC. Furthermore, the model showed improved performance in restoring the signal intensity, distribution, and structural information in the stained microscopy images.