An Adaptive Semantic Segmentation Network for Adversarial Learning Domain Based on Low-Light Enhancement and Decoupled Generation

Abstract

Nighttime semantic segmentation due to issues such as low contrast, fuzzy imaging, and low-quality annotation results in significant degradation of masks. In this paper, we introduce a domain adaptive approach for nighttime semantic segmentation that overcomes the reliance on low-light image annotations to transfer the source domain model to the target domain. On the front end, a low-light image enhancement sub-network combining lightweight deep learning with mapping curve iteration is adopted to enhance nighttime foreground contrast. In the segmentation network, the body generation and edge preservation branches are implemented to generate consistent representations within the same semantic region. Additionally, a pixel weighting strategy is embedded to increase the prediction accuracy for small targets. During the training, a discriminator is implemented to distinguish features between the source and target domains, thereby guiding the segmentation network for adversarial transfer learning. The proposed approach’s effectiveness is verified through testing on Dark Zurich, Nighttime Driving, and CityScapes, including evaluations of mIoU, PSNR, and SSIM. They confirm that our approach surpasses existing baselines in segmentation scenarios.

1. Introduction

Semantic segmentation is a fundamental task in computer vision where each pixel of a given image is labeled with an object category. It is widely used in various applications such as autonomous driving [1], medical imaging [2], and human parsing [3]. In recent years, the performance of semantic segmentation of daytime scene images has substantially improved due to the rapid progress in deep learning and computing power. As researchers have tackled more challenging image segmentation scenarios under various limited, adverse, and degraded conditions, semantic segmentation of nighttime images [4] has emerged as a prominent research focus. However, nighttime semantic segmentation poses unique challenges; for example, low contrast of the input images makes it difficult to obtain clear and complete segmentation boundaries, and the variation in lighting conditions might lead to changes in the brightness and color of the objects within the same scene. Additionally, the manual labeling of a high-quality training set of nighttime images is also a formidable task, contributing to the degradation of segmentation model performance. The present study seeks to address the above-mentioned bottlenecks via proposing a nighttime semantic segmentation network that is suitable for real-scenario applications such as autonomous driving and security monitoring.

Nighttime images, as a class of low-light images, have many regions of foreground pixels that are not obvious or recognizable to the human eye, and it is difficult to perform high-quality pixel-level annotation on this part of the image. Consequently, a sufficient quantity of accurate segmentation instances is the basis for realizing efficient learning of segmentation models. To tackle this, current schemes such as domain adaptation [5,6], synthetic datasets [7], and style transfer [8] are commonly employed. Due to the low brightness and contrast of nighttime images, this paper transfers daytime images to nighttime images by domain adaptation. However, large differences in scene feature distributions and foreground types between the source and target domains, primarily concerning light intensity, can often lead to the distortion of crucial spatial semantic details during domain transfer. In view of this, some studies have proposed to establish domain transfer from the daytime domain to the nighttime domain using an intermediate domain as a smooth transition, such as the twilight domain. In [4,9,10], the twilight domain serves as a bridge, allowing the model trained in the daytime domain to adapt progressively to the nighttime domain by extracting features from twilight images and performing transfer alignment learning. In [11], a model adaptation method based on course learning is proposed, which adapts the model to light changes and noise in nighttime scenes by gradually increasing the complexity of the nighttime images. Building [11], research conducted in [12] utilizes feature maps to provide prior knowledge about nighttime scenes, aiding the model in understanding objects and structures and guiding adaptive training. In [13], an encoder–decoder structure for semantic segmentation of nighttime images is introduced which uses a domain map approach for mapping synthetic to real data. In [14,15,16], a generator network is trained using adversarial learning to translate daytime images to nighttime images. Subsequently, the feature extractor of the generator network is adopted for the semantic segmentation network to extract transform-based regularized features from nighttime images. Furthermore, to enhance the generalization performance of the model, research in [17] employs Adversarial Generative Networks (GANs) to translate daytime images to nighttime images, and random transformations are then applied to those images, followed by joint training using the adversarial and semantic segmentation loss functions.

Among the above-mentioned methods, refs. [4,9,10,11,12] leverage intermediate domains to create a smooth transition, thereby improving model generalization and potentially reducing dataset labeling costs. Nonetheless, introducing intermediate domains may entail additional preprocessing and model training and fail to fully cover all variations from daytime to nighttime. On the other hand, refs. [13,14,15,17] employ techniques such as style transfer and build synthetic datasets to address the difficulty of labeling nighttime (low-light) samples. While building synthetic datasets offers advantages, it also carries the risk of introducing bias and noise. Furthermore, these schemes focus only on the statistical representation of the overall image style in style transfer and thus are prone to the loss of spatial details. In addition, generating a transferred image with the same semantics as the original image, especially when dealing with a relatively large domain gap, remains a challenging aspect of image translation.

To address these problems, this paper uses pairs of day and night images in similar scenes as target domains and tries to transfer the source domain generalized model to the scene-specific multi-target domain without introducing an intermediate domain, synthetic datasets, or style migrations so as to improve the segmentation quality by joint adversarial learning and multi-domain co-training. Based on this, this paper proposes an adaptive semantic segmentation network for the adversarial learning domain based on low-light enhancement and decoupled generation (DLA-Net). At the front end of the model, a lightweight low-light image enhancement network (LIE-SubNet) is embedded to elevate foreground contrast in nighttime images and accomplish spatial feature alignment across different illuminance datasets. Existing segmentation models typically treat the foreground target as a unified entity; however, foreground boundary regions usually contain richer spatial details with higher-frequency feature information, whereas non-boundary regions exhibit fewer spatial details featured by low-frequency distributions. Inspired by [5], this paper leverages a generative network capable of decoupling the foreground body and edge to predict segmentation masks and uses two discriminators for adversarial training between the source and target domains. Additionally, a small pixel reweighting strategy [18] is implemented to process the input images and reduce prediction uncertainty, thus improving the segmentation accuracy for small targets. In our experiments, the Dark Zurich dataset [4] is employed, which contains pairs of daytime and nighttime images based on rough GPS positional alignment. Through extensive testing on the Dark Zurich, CityScapes, and Nighttime Driving [9] datasets, the proposed method is verified to demonstrate improved performance in low-light nighttime semantic segmentation. The primary contributions of our work are summarized as follows:

•

In this paper, a multi-domain model joint training network for semantic segmentation, DLA-Net, is introduced which transfers the source domain to the multi-target domain of a specific scene without requiring an intermediate domain. It accomplishes joint adversarial training of the multi-domain model, supported by the low-light image enhancement sub-network, on the multi-target domain;

•

The low-light image enhancement sub-network, LIE-SubNet, which combines deep learning and mapping curve iteration, is proposed to enhance pixel contrast and spatial feature alignment of nighttime images. In the segmentation network, a generative network capable of decoupling subjects and edges is utilized to guide segmentation prediction via exploiting the adversarial loss in the daytime and nighttime domains;

•

To effectively utilize both low-frequency and high-frequency information of foreground targets, the segmentation mask is decoupled into the body generation branch and the edge preservation branch. These branches can focus on different attributes of the regional features during training. The resulting masks are then composited and reconstructed to achieve a complete semantic segmentation mask capable of retaining the details while removing the void noise.

2. Related Work

Domain adaptation for semantic segmentation: Domain adaptation seeks to transfer knowledge learned in the source domain to the target domain, where the object classes are similar but the distribution of data statistics differs. Currently, a portion of domain adaptive schemes adopt adversarial learning frameworks, introducing an adversarial loss function between the source and target domains to guide the model in aligning feature representations across different data domains. For example, in [19], Hoffman et al. proposed a new approach to semantic segmentation using category-constrained [20] full convolutional domain adversarial learning. AdaptSegNet [5] utilizes adversarial training to achieve feature alignment in the source and target domains. Additionally, several approaches employ joint training and multiple task learning strategies to improve model generalization by sharing parameters among source and target domains. In BDL [21], images from both source and target domains are input into a shared convolutional neural network, with the last layer divided into two branches for semantic segmentation tasks in the respective source and target domains. Through sharing the feature extraction layer of the network, the source and target domains can leverage the underlying image feature representation, thereby improving adaptation to redundant representations of the target domain.

Unlike adversarial learning, style transfer [17] and image translation from source image to target image are also widely used for domain adaptation. They typically incorporate a domain invariance loss function into the generator network to enforce domain-invariant image generation in the target domain [22,23]. This type of loss function commonly comprises both an adversarial loss and a domain invariance loss. Specifically, the generator network is trained using the adversarial loss, while the domain invariance loss is used to teach the generator network to learn the shared features between source and target domains, resulting in domain-invariant representations. Some other studies have explored the combination of self-training strategy and fine-tuning strategy through multiple rounds of network training. However, the self-training strategy may introduce noise when using pseudo-labeling, thus impacting model performance [24]. To mitigate the influence of noise, researchers have proposed several improved self-training techniques [25,26], such as using model ensembles to reduce noise or refining pseudo-labeling to reduce mislabeling. Alternatively, some studies have employed course-based learning [27,28] to acquire simple attributes in the target domain before using them to normalize semantic segmentation models. However, the significant visual disparities between daytime and nighttime images pose a formidable challenge for these methods, making them ill suited to effectively handling domain adaptation in scenarios with markedly different illumination intensities. Consequently, they often fall short of delivering satisfactory performance in nighttime semantic segmentation. This paper explores more efficient techniques to minimize the domain gap so that transfer models can achieve accurate segmentation predictions.

Nighttime (low-light) semantic segmentation: Some studies have demonstrated the effectiveness of employing intermediate domains for the progressive adaptation of semantic models trained on daytime scenes to nighttime scenes. For example, Dai et al. [9] proposed a step-by-step adaptive approach based on intermediate domains. This approach leverages an intermediate twilight domain as a bridge between daytime and nighttime scenes and trains an intermediate model on the twilight domain, which is then applied to the semantic segmentation of nighttime scenes. Later, Sakaridis et al. [4,5] extended the approach to a class of guided curriculum adaptation frameworks, incorporating synthetic and unlabeled real images to establish correspondences in scene images at various times. However, it is worth noting that this progressive adaptation approach often necessitates training several semantic segmentation models. For instance, in [4], three models were trained separately for three different domains, potentially making the training process less efficient. Building upon this methodology of using intermediate domains for progressive domain adaptation, some studies have trained some additional image translation models. CycleGAN [17] is a good case in point and enables the inter-transfer of daytime and nighttime images before training semantic segmentation models, thus introducing different visual features through diverse augmented data and aiding in the adaptation of the transferred models to various scenes and environments. Furthermore, ref. [29] introduced a nighttime semantic segmentation method based on image translation by translating nighttime images into daytime ones and utilizing semantics with the model trained on the daytime domain.

More recently, to improve the semantic segmentation of night scenes, researchers have explored the use of different sensors to capture the same image as an auxiliary input. Vertens et al. [30] proposed to utilize the insensitivity of thermal infrared to changes in illumination as a supplemental input to segmented images to provide additional information for nighttime semantic segmentation. Additionally, other studies have devised specialized scene semantic segmentation methods. For example, Ref. [31] proposed a two-stage adversarial training approach that employs domain adaptation techniques to transform between pairs of daytime and nighttime scenes, particularly for rainy and nighttime scenarios. Likewise, Ref. [32] introduced an adaptive network capable of automatically adapting its internal architecture based on the attributes of input images to different environmental conditions, including nighttime and rainy ones. Differing from the above methods, this paper introduces a network structure designed to train semantic segmentation for low-light images via end-to-end adversarial learning without resorting to intermediate domains or auxiliary images.

3. Method

3.1. Framework Overview

The domain adaptive method proposed in this paper involves two key domains: a source domain S for pre-training, which can be any normal lighting scene, and a target domain $T=\{T_d,T_n\}$ containing two roughly aligned subdomains, $T_d$ and $T_n$, representing daytime and nighttime scenes, respectively. In the pre-training phase, a labeled image set $S\triangleq \{X_S,Y_S\}$ from the source domain was used to optimize the semantic segmentation network parameters. Subsequently, two discriminators, $D_{S\to T_d}$ and $D_{S\to T_n}$, were employed to bootstrap the domain adaptive model transfer from S to $T_d$ and from S to $T_n$ to efficiently model semantic segmentation of the nighttime scene $T_n$ in the target domain. The domain adaptive semantic segmentation network in this paper comprises three modules: (1) a low-light image enhancement network $N_{en}$, (2) a pre-trained semantic segmentation network $G_{pre}$ and a transfered semantic segmentation network $G_{seg}$, which decouples the body and edge during segmentation and provides predicted image dimensions of ${\mathbb{R}}^{H\times W\times C}$, with C denoting the total number of image categories, and (3) a segmented mask activation network $N_Y$, which consists of a convolutional layer and a sigmoid normalization function, as shown in Figure 1. The network input contains the source domain image $X_S$ and the target domain images $X_{T_d}$ and $X_{T_n}$, consisting of three types of domain samples. Among them, $X_{T_n}$ was additionally passed through a nighttime (low-light) enhancement network $N_{en}$, which generated an enhancement loss $L_{en}$ to optimize the enhancement result and brought the output closer to the daytime domain. The network uses image annotations $X_S,Y_S$ in the source domain S dataset to compute the segmentation loss $L_{seg}$ and then obtains the segmentation prediction masks $\tilde{F}=\{{\tilde{F}}_S,{\tilde{F}}_{T_d},{\tilde{F}}_{T_n}\}$ and segmentation loss $L_{de}$ by $G_{seg}$. After that, two discriminators, $D_{S\to T_d}$ and $D_{S\to T_n}$, perform adversarial transfer learning, and the final segmentation masks $\widehat{Y}=\{{\widehat{Y}}_S,{\widehat{Y}}_{T_d},{\widehat{Y}}_{T_n}\}$ are obtained via activating the network $N_Y$, i.e., $\widehat{Y}=\{N_Y\left({\tilde{F}}_S\right),N_Y\left({\tilde{F}}_{T_d}),N_Y\left({\tilde{F}}_{T_n}\right)\right\}$. The whole network guides the domain adaptive alignment of the model based on the composite total loss $L_{total}$.

3.2. Low-Light Image Enhancement Sub-Network

In the realm of image illumination enhancement, the majority of research commonly employs methods like mapping curves or neural networks. However, this paper has the initiative to fit mapping curves with neural networks to design a low-light image enhancement sub-network. The objective was to homogenize the intensity distribution of the input image $X_{T_n}$ from the nighttime target domain $T_n$ and generate the enhanced image ${\widehat{X}}_{T_n}$, ensuring that the predictions of different domain samples align after passing through the segmentation network. Inspired by [13], we utilized an iterative pixel enhancement mapping curve to adjust the brightness and contrast of the image through the pixel grayscale mapping relationship, as shown in Equation (1).

$$Ie(X_{T_n}\left(x\right);\alpha )=log(1+X_{T_n}\left(x\right)+\alpha X_{T_n}\left(x\right)(1-X_{T_n}\left(x\right))),$$

(1)

where x is the pixel coordinate, and the $\alpha$ parameter ensures that each pixel value in the enhanced image falls within the normalized range of $[0,1]$, preventing any loss of information due to overflow. By setting $\alpha$ to a value between −1 and 1, the $Ie$ curve can be controlled within the range of $[0,1]$. For example, when $\alpha =-1$, $Ie(X_{T_n}\left(x\right);-1)=log(1+{X_{T_n}\left(x\right)}^2)$, i.e., each value is within $[0,1]$.

To adapt to more challenging low-light conditions, iterating the quadratic curve $Ie$ could result in a higher-order curve. Although the higher-order curve is able to adjust the image over a wider area, it still applies a global adjustment as the $\alpha$ value is applied to all pixels, resulting in over-enhancement or diminution of localized regions. To solve this problem, we used a separate curve for each RGB channel of the input image to perform an iterative transformation so that each channel has a corresponding optimal $\alpha$ value for image enhancement, as shown in Equation (2).

$$I{e}_m(X_{T_n}\left(x\right);\mathcal{A})=log(1+I{e}_{m-1}\left(X_{T_n}\left(x\right)\right)+{\mathcal{A}}_{m}I{e}_{m-1}\left(X_{T_n}\left(x\right)\right)(1-I{e}_{m-1}\left(X_{T_n}\left(x\right)\right))),$$

(2)

where m, set to 8 in this paper, signifies the number of iterations and controls the curvature, and $\mathcal{A}$ is a parametric mapping with the same size as the given image used to represent the optimal $\alpha$ value for each channel. To obtain the mapping relationship among the input image and its optimal curve parameter mapping, this paper proposes a depth curve fitting network, as illustrated in Figure 2.

To evaluate the quality of the enhancement image, we used the following three losses to train the image enhancement network.

To suppress overexposure or underexposure of certain areas, we designed an exposure control loss $L_{ec}$ to regulate the level of exposure. $L_{ec}$ quantifies the disparity between the mean luminance value of a specific area and the intended exposure level e. e was set to a grayscale value in the RGB color space following existing methods [33,34] in this paper. This loss brought the enhancement closer to the desired exposure level, mitigated overexposure or underexposure, and hence obtained a more visualized and higher-quality image, as shown in Equation (3).

$$L_{ec}=\frac{1}{V}\sum _{i=1}^V\left|{\widehat{I}}_i-e\right|,$$

(3)

where V denotes the number of non-overlapping regions with a size of $16\times 16$, and $\widehat{I}$ represents the average luminance value of localized region V in the augmented image ${\widehat{X}}_{T_n}$. e was set to 0.5 in the experiment.

The color constancy loss employed in this paper was based on the Gray-World [35] color constancy assumption, which posits that each color channel is averaged as gray over the whole image. This loss rectifies potential color deviations in the enhanced image, recovers color information affected by changes in illumination, improves the quality and visual perception of the image, and determines the relationship among the three color channels, as shown in Equation (4).

$$L_{cc}=\sum _{\forall \left(a,b\right)\in \tau }{({\overline{E}}^{\left(a\right)}-{\overline{E}}^{\left(b\right)})}^2,$$

(4)

where ${\overline{E}}^{\left(a\right)}$ and ${\overline{E}}^{\left(b\right)}$ denote the average intensity values of the a-channel and the b-channel, respectively, in the enhanced image ${\widehat{X}}_{T_n}$, and $(a,b)$ denotes a pair of channels, $\tau =\left\{\right(R,G),(R,B),(G,B\left)\right\}$. The smaller value of $L_{cc}$ indicates that the color of the brightened image is more balanced, and the larger $L_{cc}$ indicates that the brightened image may have the problem of color bias.

In this paper, an illumination smoothness loss [36] was built into each curve parameter mapping $\mathcal{A}$ to maintain a monotonic relationship between adjacent pixels. The loss assists the model in learning that the illumination changes in the neighboring regions exhibit both consistency and smooth transition and improving image processing performance and image quality. It is shown in Equation (5).

$$L_{is}=\frac{1}{M}\sum _{m=1}^M\sum _{s=1}^{H\times W\times C}{\left(\left|{\nabla }_x{\mathcal{A}}_m^{\left(s\right)}\right|+\left|{\nabla }_y{\mathcal{A}}_m^{\left(s\right)}\right|\right)}^2,$$

(5)

where M stands for the number of iterations. Specifically, C denotes the RGB color channels, and $C=3$. ${\nabla }_x$ and ${\nabla }_y$ denote the horizontal and vertical gradient operations, respectively. The smaller the value of $L_{is}$, the smoother the light of the brightened image, and vice versa, which indicates that there are mutations or artifacts in the light of the brightened image.

The total enhancement loss is shown in Equation (6).

$$L_{en}=L_{ec}+{\lambda }_1{L}_{cc}+{\lambda }_2{L}_{is},$$

(6)

where ${\lambda }_1$ and ${\lambda }_2$ are hyperparameters used to balance the size of the loss and were set to 0.5 and 20 in the experiments, respectively.

Contributing to the realm of LIE-SubNet, this paper explores the combination of a set of higher-order curves that can be iterated with a deep learning network for different numbers of iterations to verify the optimal performance and enhance nighttime pixel contrast. The method reduces the domain gap among the daytime and nighttime domains without resorting to an intermediate domain or the training of multiple distinct models and feeds the segmentation network with smaller differences in illumination images.

3.3. Semantic Segmentation Network for Decoupling Body and Edge

Currently, mainstream semantic segmentation methods primarily focus on enhancing the internal consistency of the object through global modeling or refining the object details along the boundaries through multi-scale feature fusion. However, it is worth noting that foreground boundary regions typically harbor more spatial detail and higher-frequency feature information. In view of this, we introduced the semantic segmentation network $G_{seg}$ for decoupling body and edge, which contains a body generation branch $\rho$ and an edge preservation branch $\delta$. Unlike previous studies, we do not require the input image’s ground truth map and trained two branches with distinct losses to predict the body feature map and edge feature map, respectively. The implementation details are described below.

Decoupling segmentation framework: In this paper, we assume that the spatial features of the image conform to the addition rule, i.e., $\tilde{F}=F_{\mathrm{body}}+F_{\mathrm{edge}}$. Accordingly, the body feature $F_{body}$ can be generated first, and the edge feature $F_{edge}$ can be obtained by a specific subtraction operation. If we make $F_{body}=\rho \left(F\right)$, then $F_{edge}=F-F_{body}$, as shown in Equation (7).

$$\begin{array}{cc}\hfill \tilde{F}& =\rho \left(F\right)+\delta \left(F_{edge}\right)\hfill \\ \hfill & =F_{body}+\delta (F-F_{body}),\hfill \end{array}$$

(7)

where $\rho$ represents the body generation branch mapping which is used to aggregate contextual information within objects to form a distinct body for each object. On the other hand, $\delta$ denotes the edge preservation branch mapping, which is designed to extract spatially detailed features from the boundary region.

Body generation branch: This branch is responsible for the generation of more consistent feature representations for pixels that are part of the same object in an image. Low-resolution feature maps typically contain low-frequency terms, with the low-spatial-frequency portion representing the image as a whole. Therefore, the low-resolution feature maps represent the most salient parts. In order to achieve this goal, as illustrated in Figure 3, X is the input image, and we utilized an encoder–decoder architecture after the backbone to extract F. Specifically, the encoder downsamples F using dilated convolution, which downsamples F into a low-resolution representation of the low-spatial-frequency portion, denoted as $F_{low}$. In some cases, low-resolution features might still contain high-frequency information. We assumed that this compressed representation encapsulates the most obvious object portions and leads to rough representation which ignores details or high-frequency portions. Therefore, we used bilinear interpolation to upsample $F_{low}$ to the same size as F to obtain $F_{up}$. Then, we cascaded F and $F_{up}$ and used a $1\times 1Conv$ to adjust the channel dimensions to ${\mathbb{R}}^{H\times W\times C}$ to obtain $F_{conv}$, i.e., $F_{conv}=h_{conv}\left(F\right||F_{up})$, where $h_{conv}$ denotes the $1\times 1$ convolutional layer, and $\left|\right|$ denotes the channel dimensionality join operation. This branch also contains an average pooling layer by average pooling $F_{conv}$ to generate a feature map $F_{ap}$ with a more distinct body, i.e., $F_{ap}=h_{ap}\left(F_{conv}\right)$ and $F_{ap}\in {\mathbb{R}}^{H\times W\times C}$, where $h_{ap}$ denotes the average pooling operation.

To increase the spatial accuracy of body features in segmentation results, we first mapped each pixel p in the default spatial grid ${\Omega }_l$ on $F_{ap}$ to a new pixel point p via feature relocation. Then, we used a variable bilinear sampling mechanism [37,38] to approximate the value of each pixel point p in $F_{body}$, i.e., $F_{body}\left(\widehat{p}\right)={\sum }_{p\in l\left(o\right)}{F}_{ap}\left(p\right)$, where l denotes the pixels in the four fields around p, o is the center point, and $F_{body}\in {\mathbb{R}}^{H\times W\times C}$. In addition, to ensure smoother performance of the body feature and reduce noise and discontinuities in the prediction results, we applied the $L_2$ loss [39] to bootstrapping the body generation branch learning, as shown in Equation (8).

$$L_{body}=\sqrt{{\displaystyle \sum _{s=1}^{H\times W\times C}}{\left(F_{body}^{\left(s\right)}\right)}^2},$$

(8)

where s denotes the positional index of the element in $\forall F\in {\mathbb{R}}^{H\times W\times C}$, and $s=1,2,..., H\times W\times C$.

Edge preservation branch: This branch is dedicated to handling high-frequency terms $F_{high}$ in the image, where high-frequency features usually encompass more detailed edge information. To obtain the high-frequency edge feature, we subtracted the body feature $F_{body}$ from the original feature F, i.e., $(F-F_{body})$. Drawing inspiration from recent work on decoder design [40], we outputted a low-level feature $F_{detail}$ through the backbone’s low layer, which served as a complement to the missing fine-detail information and augmented the high-frequency terms in $F_{edge}$. Finally, $(F-F_{body})$ and $F_{detail}$ were cascaded, and then a $1\times 1Conv$ was used for channel adjustment to obtain $F_{edge}$. The implementation is expressed in Equation (9).

$$F_{edge}=h_{\mathrm{conv}}((F-F_{body})\left|\right|F_{detail}),$$

(9)

where $F_{edge}\in {\mathbb{R}}^{H\times W\times C}$.

The edge preservation branch focuses more on edge detail features and does not require body features. Unlike the $L_2$ loss, the $L_1$ loss can obtain a sparse solution so that certain features have zero weight. This makes the boundary sparser and reduces unnecessary body features, contributing to an accurate boundary prediction feature map. Therefore, the $L_1$ loss [39] was utilized to guide the learning of the edge preservation branch, as shown in Equation (10).

$$L_{edge}=\sum _{s=1}^{H\times W\times C}\left|F_{edge}^{\left(s\right)}\right|.$$

(10)

The final decoupling loss is:

$$L_{de}=L_{body}+{\lambda }_3{L}_{edge},$$

(11)

Both the $L_{body}$ and $L_{edge}$ losses complement each other by sampling pixels from different regions of the image, which was beneficial for showing the performance of the experimental results. Since the edge portion is not a large part of the overall image, ${\lambda }_3$ is used to balance the weight of $L_{edge}$ in $L_{de}$, which was set to 0.4 in the experiments.

In this paper, $G_{seg}$ acquired the body feature map $F_{body}$ and edge feature map $F_{edge}$ of the input image X through the body generation branch $\rho$ and the edge preservation branch $\delta$, respectively. Moreover, the edge features were supplemented by the high-frequency detailed features $F_{detail}$ output from the lower layer of the backbone network. By employing distinct body and edge losses, the segmentation performance was enhanced, and the final segmentation map F was obtained through $F_{body}+F_{edge}$.

3.4. Multi-Target Domain Adversarial Learning Strategy

During the multi-target domain adversarial learning strategy, in order to ensure relatively close feature distributions after spanning different domains and to better achieve transfer alignment between source and target domains, this paper added the adversarial loss terms $L_{S\to T_d}$ and $L_{S\to T_d}$ to the outputs of the daytime domain $T_d$ and the nighttime domain $T_n$, respectively. Both discriminators had identical structures, weights, and training protocols, where the identification source domain image was 1, and the target domain image was 0. The binary cross-loss function [41] was utilized to make both ${\tilde{F}}_{T_d}$ and ${\tilde{F}}_{T_n}$ close to ${\tilde{F}}_S$. The antagonistic loss is defined as:

$$L_{adv}=L_{S\to T_d}(X_S,X_{T_d})+L_{S\to T_n}(X_S,X_{T_n}),$$

(12)

In the experiments, we trained the generator and the discriminators alternately. The generator used in the source domain $G_{pre}$ was pre-trained, and the target domain $G_{seg}$ was transfered. The objective functions of $D_{S\to T_d}$ and $D_{S\to T_n}$ are defined as:

$$\begin{array}{cc}\hfill L_{S\to T_d}(X_S,X_{T_d})=\underset{G_{seg}}{min}\underset{D_{S\to T_d}}{max}(& E_{X_S\sim p_{data}(X_S)}(log{D}_{S\to T_d}(G_{pre}(X_S)))+\hfill \\ & E_{X_{T_d}\sim p_{data}(X_{T_d})}(1-log{D}_{S\to T_d}(G_{seg}(X_{T_d})))),\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill L_{S\to T_n}(X_S,X_{T_n})=\underset{G_{seg}}{min}\underset{D_{S\to T_n}}{max}(& E_{X_S\sim p_{data}(X_S)}(log{D}_{S\to T_n}(G_{pre}(X_S)))+\hfill \\ & E_{X_{T_n}\sim p_{data}(X_{T_n})}(1-log{D}_{S\to T_n}(G_{seg}(X_{T_n})))),\hfill \end{array}$$

(14)

We used cross-entropy loss to train the semantic segmentation loss of the source domain. Moreover, we introduced the small pixel reweighting $w_k$ to address the small target category imbalance, as shown in Equation (15).

$$L_{seg}=-\frac{1}{NC}\sum _{t=1}^{H\times W}\sum _{k=1}^C\left|\right|w_k{GT}^{(t,k)}·log({\widehat{Y}}_S^{(t,k)}){\left|\right|}_1,$$

(15)

where N is the total number of image pixels, k denotes category, $\left|\right|·{\left|\right|}_1$ is the $L_1$ norm that sums up all the pixels, $w_k$ is the pixel weight, ${\widehat{Y}}_S^{\left(k\right)}$ is the prediction map ${\widehat{Y}}_S$ from the kth channel of the source domain image obtained from the activation network $N_Y$, i.e., $\widehat{Y}=N_Y\left(\tilde{F}\right)$, and $G{T}^{\left(k\right)}$ is the ground truth of the kth category of the one-hot encoding. Specifically, for each category k, we first defined a weight $w_k^{\prime }=-log\left(p_k\right)$, where $p_k$ denotes the percentage of all valid pixels that are labeled as category k in the source domain. Then, $w_k$ was further normalized by $w_k=(\left(w_k^{\prime }-\overline{w}\right)/{\theta }_k)·std+avg$, where $\overline{w}$ and ${\theta }_k$ are the mean and standard deviation of $w_k^{\prime }$, respectively, and $std$ and $avg$ are preset constants to limit the value of $w_k$ to positive. Finally, $w_k$ was multiplied by the corresponding category channel in $\tilde{F}$ to generate the weighted probability map, and then the segmentation result was yielded via $N_Y$, as shown in Equation (16).

$${\widehat{Y}}^{\left(k\right)}=N_Y(w_k·{\tilde{F}}^{\left(k\right)}),$$

(16)

where ${\widehat{Y}}^{\left(k\right)}\in {\mathbb{R}}^{H\times W\times C}$.

Therefore, the total loss of the whole network is:

$$L_{total}=L_{en}+L_{seg}+L_{de}+L_{adv}.$$

(17)

In summary, we designed a segmentation network for decoupling body and edge. It predicted the body and edge features of the input image, applied $L_1$ and $L_2$ losses to constrain them, respectively, and was then synthesized into a segmentation feature map. After that, two discriminators were used to distinguish different domain outputs between source and daytime image and source and nighttime image in a multi-objective domain adversarial learning strategy. Additionally, probabilistic reweighting was used to optimize the segmentation prediction for small targets.

4. Experiments

4.1. Experimental Settings

To assess the performance of the proposed DLA-Net and its components, the Mean Intersection-to-Noise Ratio (mIoU), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index (SSIM) were employed as the evaluation metrics in the experiments and compared with the advanced methods. The mIoU is a widely used metric for evaluating the accuracy of pixel-level semantic segmentation models which calculates the ratio of intersection and union between predicted segmentation results and true labels. PSNR and the SSIM are commonly used metrics for evaluating image enhancement work. Both metrics compare the differences between the original and compressed/distorted images. PSNR measures image quality by comparing the Peak Signal-to-Noise Ratio, while the SSIM evaluates it in terms of structure, brightness, and contrast similarity. In addition, the following datasets were used for the training of all segmentation models and performance evaluation during the daytime–nighttime domain adaptive transfer process.

• CityScapes [24]: The CityScapes dataset comprises 5000 street view images with a resolution of $2048\times 1024$ divided into 2975 training images, 500 validation images, and 1525 testing images. Each image is annotated at the pixel level with 19 categories. We used the CityScapes training images as the training data in the training phase and the comparative experimental dataset for the decoupled body and edge segmentation modules;
• Dark Zurich [4]: The Dark Zurich dataset comprises 2416 nighttime images, 2920 twilight images, and 3041 daytime images for training, all with a resolution of $1920\times 1080$. The images in these three domains are roughly aligned using GPS localization of neighboring locations and panning/zooming operations in all directions. In this paper, 2416 nighttime images were utilized to train the network model (without utilizing twilight images). In addition to the above images used for training, Dark Zurich contains 201 annotated nighttime images, of which 50 were used for validation (Dark Zurich-val) and 151 for testing (Dark Zurich-test) and evaluation;
• Nighttime Driving [9]: In the experiments, we exclusively utilized the Nighttime Driving test set, which comprises 50 nighttime images with a resolution of $1920\times 1080$. All images are pixel-level annotated using 19 cityscape categories;
• SICE dataset [42]: The Part 2 subset of the SICE dataset was utilized in this paper, which comprises of 229 multi-exposure sequences and the reference image corresponding to each sequence. In the experiments, only low-light images from the Part 2 subset were used.

In this study, we implemented the proposed adversarial learning domain adaptive semantic segmentation network using PyTorch on a single NVIDIA 3060 GPU, and all networks were trained using the same settings. Following [43], we trained the networks using an SGD optimizer and set the SGD optimizer momentum to 0.9 and a decay of $5\times {10}^{-4}$. The base learning rate of the network was $2.5\times {10}^{-4}$, and then the learning rate was reduced using a polynomial learning rate strategy with a decay power of 0.9. The batch size was 2. We used an Adam optimizer [44] to train the discriminators with the $\beta$ set to (0.9, 0.99). The learning rate of discriminators followed the same decay strategy as the generator. The total enhancement loss $L_{en}$ incorporates weights ${\lambda }_1$ and ${\lambda }_2$, which are selected from the intervals $[0.1-1.0]$ and $[20-25]$, respectively. These values were chosen based on previous similar work and different loss characteristics. After experimentation on the validation set, ${\lambda }_1$ and ${\lambda }_2$ were set to 0.4 and 20, respectively. In $L_{de}$, the weight of $L_{edge}$ is determined by ${\lambda }_3$. This hyperparameter is set because $L_{body}$ and $L_{edge}$ are complementary, with fewer pixels in the edge part relative to the body part. The default value of ${\lambda }_3$ is 1, but, in our experiments, we found that a value of 0.4 resulted in the best segmentation. To ensure the positivity of the values of $w_k$, we set $std=0.05,avg=1.0$ in the experiments. ResNet-101 [45] was used as the backbone. To facilitate smoother convergence during training, we used a total of 180,000 pre-training epochs on the CityScapes dataset with three different semantic segmentation models.

Table 1. The performance of three distinct semantic segmentation models on the validation sets of CityScapes and Dark Zurich.

Method	Dark Zurich-Val	CityScapes-Val
RefineNet [46]	14.46	64.50
PSPNet [47]	11.44	64.97
DeepLab-v3+ [48]	11.58	63.77

presents the performance of the three distinct semantic segmentation models on the validation sets of CityScapes and Dark Zurich.

4.2. Comparison with Other Methods

Comparison on Dark Zurich-test: In this paper, we first compare the proposed DLA-Net with several state-of-the-art methods on Dark Zurich-test [4], including CPSL [49], ProCA [50], and DiGA [51], as well as some other domain adaptation methods [5,21,52]. The performance results are summarized in

Table 2. Results of the current state-of-the-art method and the DLA-Net proposed in this paper for each category in the Dark Zurich test set. CityScapes→ DZ-night denotes the adaptation from CityScapes to Dark Zurich-night. Bold font indicates the best, and underlining indicates the second-best.

Method	Road	Sidewalk	Building	Wall	Fence	Traffic Light	Traffic Sign	Vegetation	Terrain	Sky	Person	Rider	Car	Truck	Bus	Train	Motocycle	Bicycle	Pole	mIou
RefineNet-CityScapes [46]	68.8	23.2	46.8	20.8	12.6	30.4	26.9	43.1	14.3	0.3	36.9	49.7	63.6	6.8	0.2	24.0	33.6	9.3	29.8	28.5
PSPNet-CityScapes [47]	79.0	21.8	53.0	13.8	11.2	20.2	21.9	43.5	10.4	20.2	37.4	33.8	64.1	6.4	0.0	52.3	30.4	7.4	22.5	28.8
DeepLab-v3+-CityScapes [48]	78.2	19.0	51.2	15.5	10.6	28.9	22.0	56.7	13.3	20.8	38.2	21.8	52.1	1.6	0.0	53.2	23.2	10.7	30.3	28.8
AdaptSegNet-CityScapes→ DZ-night [5]	86.1	44.2	55.1	22.2	4.8	5.6	16.7	37.2	8.4	1.2	35.9	26.7	68.2	45.1	0.0	50.1	33.9	15.6	22.1	30.4
ADVENT-CityScapes→DZ-night [52]	85.8	37.9	55.5	27.7	14.5	14.0	21.1	32.1	8.7	2.0	39.9	16.6	64.0	13.8	0.0	58.8	28.5	20.7	23.1	29.7
BDL-CityScapes→ DZ-night [21]	85.3	41.1	61.9	32.7	17.4	11.4	21.3	29.4	8.9	1.1	37.4	22.1	63.2	28.2	0.0	47.7	39.4	15.7	20.6	30.8
CPSL [49]	75.0	28.6	48.1	20.8	13.8	36.3	29.4	48.9	13.3	0.4	42.8	49.7	68.9	17.9	0.0	27.1	34.4	11.4	33.8	31.6
ProCA [50]	81.2	46.4	58.3	21.5	19.5	40.0	41.1	64.3	30.5	31.6	53.0	47.0	75.0	38.7	0.0	49.1	30.2	20.5	40.7	41.5
DiGA [51]	79.8	48.8	65.7	7.3	10.5	38.4	38.4	63.6	17.5	55.3	51.6	53.0	74.2	62.0	0.0	37.0	28.6	22.0	40.6	42.1
DLA-Net (RefineNet)	88.5	53.3	69.7	33.9	19.9	31.4	35.8	69.4	32.1	82.2	44.1	43.6	54.0	21.9	0.0	40.8	35.9	24.0	24.9	42.4
DLA-Net (PSPNet)	89.2	53.0	74.0	40.2	20.3	26.0	29.4	71.2	25.4	83.2	46.2	33.1	67.4	18.2	0.3	65.6	37.5	22.8	24.2	43.5
DLA-Net (DeepLab-v3+)	89.5	59.2	70.1	32.7	22.0	33.4	32.8	69.6	30.9	79.3	44.8	40.7	66.5	15.9	0.1	72.1	30.7	22.0	29.7	44.3

.

In Table 2, CPSL, ProCA, and DiGA are shown to have utilized the same baseline RefineNet, while the other methods employed DeepLab v3+. Additionally, all methods utilized ResNet-101 as the backbone [45], and the experimental dataset was Dark Zurich-test. DLA-Net with DeepLab-v3+, RefineNet, or PSPNet achieved superior or equivalent performance compared to existing methods on this dataset. It attained an overall improvement in mIoU of 2.2% compared to the highest score obtained by existing methods (DiGA). Furthermore, the DLA-Net proposed in this paper excelled in various categories, such as roads, sidewalks, and sky. For example, in the sky category, DLA-Net outperformed ProCA and DiGA by 51.6 mIoU and 27.9 mIoU, respectively, demonstrating its ability to accurately segment these categories despite a large daytime–nighttime domain gap. Figure 4 provides the visualization results of the comparison experiments with ProCA [50] and DiGA [51], highlighting the superior performance of DLA-Net in the categories of sky, road, and sidewalk.

Comparison on Nighttime Driving: We compared the proposed DLA-Net with some other baseline methods on Nighttime Driving test [9], and results are reported in

Table 3. Comparison results of the proposed DLA-Net and some baseline methods on Nighttime Driving test. Bold font indicates the best, and underlining indicates the second-best.

Method	mIoU
RefineNet-CityScapes [46]	32.75
PSPNet-CityScapes [47]	25.44
DeepLab-v3+-CityScapes [48]	27.65
AdaptSegNet-CityScapes→ DZ-night [5]	34.5
ADVENT-CityScapes→DZ-night [52]	34.7
BDL-CityScapes→DZ-night [21]	34.7
CPSL [49]	38.2
ProCA [50]	46.7
DiGA [51]	49.9
DLA-Net (RefineNet)	43.82
DLA-Net (PSPNet)	44.59
DLA-Net (DeepLab-v3+)	47.08

.

It is important to note that the Nighttime Driving test dataset is not as finely labeled as the Dark Zurich test dataset as some elements like buildings and vegetation are not labeled. Of the 50 images in the Nighttime Driving test dataset, only two are labeled with the sky category. Despite the limited number of labeled categories and the small dataset size, the DLA-Net with DeepLab-v3+ still achieved the second-best performance (DiGA was the top performer) on this dataset. Figure 5 reports the visualization results of the ProCA [50] and DiGA [51] comparison experiments. This underscores that the DLA-Net proposed in this paper can produce superior results even when working with a small number of samples and labeled categories for segmentation tasks.

Comparison of decoupling body and edge segmentation module with other advanced segmentation methods: This paper uses ResNet-101 as a backbone [45] on the CityScapes dataset [24] to compare the decoupling body and edge segmentation module with some state-of-the-art techniques. The experimental results are listed in

Table 4. Comparison of decoupled body and edge segmentation module with other advanced segmentation methods. Bold font indicates the best, and underlining indicates the second-best.

Method	Backbone	mIoU
DFN [53]	ResNet-101	79.3
PSANet [54]	ResNet-101	80.1
DenseASPP [55]	DenseNet-161	80.6
DANet [56]	ResNet-101	81.5
CCNet [57]	ResNet-101	81.4
BAFNet [58]	ResNet-101	81.4
ACFNet [59]	ResNet-101	81.9
GFFnet  [60]	ResNet-101	82.3
X. Li et al. [61]	ResNet-101	82.8
Ours	ResNet-101	83.1

.

As shown in Table 4, the decoupling body and edge segmentation method proposed in this paper achieved the highest mIoU among all methods, reaching 83.1 mIoU. This demonstrates the effectiveness of the body generation branch and the edge preservation branch in segmentation. The body branch with $L_2$ loss constraints obtained prominent body features, while the edge branch with $L_1$ loss constraints captured clear edge features. The combination of these features resulted in an overall segmentation map that has been experimentally proven to yield better segmentation performance. Ablation experiments between different components are discussed in the next section.

Comparison of low-light image enhancement sub-network with other methods: In this paper, reference image quality assessment metrics PSNR and SSIM were used to quantitatively compare the performance of different methods on the SICE Part 2 test set [42]. Higher values of SSIM and PSNR indicate that the enhanced image is closer to the ground truth in terms of structural properties and pixel-level image content, respectively. The experimental results are presented in

Table 5. Comparison of low-light image enhancement sub-networks with other methods. Bold font indicates the best, and underlining indicates the second-best.

Method	PSNR↑	SSIM↑
MBLLEN [33]	14.78	0.534
RetiexNet [62]	15.56	0.525
RUAS [63]	16.40	0.500
ZeroDCE [36]	14.86	0.559
SCI [64]	14.78	0.522
EnlightenGAN [41]	17.48	0.651
Ours	18.10	0.638

.

Table 5 reveals that, despite not using any paired or unpaired training data, the LIE-SubNet proposed in this paper still achieved the best PSNR and second-best SSIM results (EnlightenGAN was the top performer). Combining the mapping curve and the depth network resulted in a 1.1% improvement compared to the second-best performance, and the mapping curve with multiple iterations made the overall pixels of the low-light images more uniform (see Figure 5).

In this subsection, we present comparative experiments of the overall method using the Dark Zurich dataset and the Nighttime Driving dataset. The experimental results validate the excellent performance of the proposed method. However, it should be noted that the Dark Zurich dataset mostly contains unobstructed image foregrounds, and there exists a one-to-one correspondence between the daytime and nighttime images. As a result, DLA-Net is able to perform optimally and obtain excellent results. When there is occlusion in the foreground of an image, LIE-SubNet and decoupled subject and segmentation networks may not be effective in enhancing the image and performing decoupled segmentation.

4.3. Ablation Study

To demonstrate the effectiveness of the different components of the proposed DLA-Net in this paper, several ablation experiments were conducted on several model variants. The results of the ablation experiments on different components are detailed below.

Ablation study on decoupling body and edge modules: The effectiveness of the two branches in the decoupling body and edge segmentation network is illustrated in

Table 6. Comparison of decoupled body and edge segmentation module with other advanced segmentation methods. ✓ indicates that $L_{body}$ or $L_{edge}$ was used.

Method	${\mathit{L}}_{\mathbf{body}}$	${\mathit{L}}_{\mathbf{edge}}$	mIoU	$\mathsf{\Delta }\mathsf{(}\mathsf{\%}\mathsf{)}$
DeepLab-v3+ [48]			74.6	-
+$\rho$ & $\delta$	-	-	75.9	1.7↑
	✓	-	76.3	0.5↑
	-	✓	76.6	0.4↑
	✓	✓	77.6	1.7↑
w/o $F_{detail}$	✓	✓	77.1	0.7↓

, where $\rho$ and $\delta$ denote the body generation branch and the edge preservation branch, respectively. The direct addition of the body generation and edge preservation branches in DeepLab-V3+ [48] improved the segmentation effect by 1.7%, implying that both branches are effective. After adding $L_{body}$ and $L_{edge}$, respectively, there were further improvements of 0.5% and 0.4% in performance, demonstrating that using $L_2$ and $L_1$ loss constraints can facilitate the model’s learning of different features. Finally, when all losses were combined, the performance was further improved by 1.7%. This paper also investigated the necessity of the $F_{detail}$ module, and its removal resulted in a decrease in segmentation performance of about 0.7%.

The results of the ablation studies for each component of the decoupling body and edge segmentation module are shown in

Table 7. Ablation study on effect of each component.

Method	mIoU	$\mathbf{\Delta }\mathsf{(}\mathsf{\%}\mathsf{)}$
DeepLab-v3+ [48] +$\rho$ & $\delta$	77.6	-
w/o $\rho$ average pooling	76.4	1.5 ↓
w/o $\rho$ encoder–decoder	76.8	1.0 ↓
w/o $\delta$	77.2	0.4 ↓

. After removing the average pooling layer and the encoder–decoder in the body generation branch, the model performance decreases by 1.5% and 1.0% accordingly. After removing the edge preservation branch, the model performance decreased by 0.4%. Therefore, removing the three modules individually leads to varying magnitudes of degradation in segmentation network performance. This indicates that average pooling and codecs help predict the body feature in the body generation branch and that average pooling improves the performance to a greater extent, while the entire edge preservation branch can also elevate the performance of the segmentation network.

Ablation study for each loss in the low-light image enhancement sub-network: The visualization results of the ablation experiments with different loss functions in the LIE-SubNet are presented in Figure 6. After removing the exposure control loss $L_{ec}$, the image exhibited overexposure in areas with strong lighting, underscoring the effectiveness of exposure constraints in the network. The removal of the color consistency loss $L_{cc}$ resulted in the overall image’s severe color deviation. With the removal of the light smoothing loss $L_{is}$, artifacts appeared between adjacent regions in the image. These experiments highlight the critical contributions of each loss function used in this paper in the LIE-SubNet.

Ablation study on different components of DLA-Net: As shown in

Table 8. Ablation study of several DLA-Net (DeepLab-v3+) modules proposed in this paper on Dark Zurich-val.

Method	mIoU
ProCA [50]	25.47
DiGA [51]	25.21
AdaptSegNet-CityScapes→ DZ-night [5]	19.13
w/o $X_{T_d}$	22.58
w/o LIE-SubNet & $L_{en}$	32.45
w/o $L_{en}$	33.86
w/o $L_{de}$	20.19
w cross-entropy loss in $L_{de}$	32.96
w/o probability reweighting	31.68
w/o pre-trained segmentation model	29.78
DLA-Net	35.74

, AdaptSegNet [5] was used as the baseline and DLA-Net as the full model. It was observed that, although $X_{T_d}$ was unlabeled, using roughly aligned $X_{T_d}$ to predict $X_{T_n}$ was quite important and also played a key role in DLA-Net. It reduced the segmentation results by 36.8% without using $X_{T_d}$, indicating that the training in the daytime domain is quite critical in the network. The LIE-SubNet and the corresponding loss function $L_{en}$ also contribute to the whole network. Meanwhile, the utilization of the decoupling body and edge loss $L_{de}$ in this paper yielded superior results when compared to applying the cross-entropy loss directly to computing the segmentation loss. The performance disparity between the two methods is notable, and not using the $L_{de}$ loss outperformed using the cross-entropy loss by 38%. In addition, the adoption of probabilistic reweighting in experiments enhanced the segmentation performance, affirming its effectiveness as an auxiliary tool.

This chapter presents comparison and ablation experiments on DLA-Net and its components across multiple datasets. The results demonstrate that DLA-Net efficiently segments images in nighttime scenes without using labeled images or synthetic datasets. LIE-SubNet effectively brightens low-light images, and the decoupling body and edge segmentation effectively predict feature maps with a uniform body and clear edge. However, DLA-Net struggles with domain adaptation when faced with large domain gaps caused by differences in styles and inherent variations between datasets, such as in urban street scenes. Future work will focus on conducting in-depth research to better understand these differences and adapt to a wider range of scenes and datasets.

5. Conclusions

In this paper, we proposed the DLA-Net, an adversarial learning domain adaptive semantic segmentation network capable of performing semantic segmentation of nighttime low-light images. DLA-Net leverages a combination of mapping curve iteration and a deep network to enhance low-light images, ensuring their distributions align with those from different domains. The segmentation network employs the decoupling body and edge modules that can efficiently obtain body and edge features, respectively. After the segmentation network, two discriminators are used to differentiate outputs from different domains. Therefore, a multi-target domain adversarial learning strategy is constituted between the generator and the discriminators to realize the adversarial learning domain adaption for multi-target domains. The experimental results underscore the efficacy of each designed component, showcasing outstanding performance on datasets such as Dark Zurich and Nighttime Driving. State-of-the-art performance is also obtained on unlabeled or thin labeled datasets, and segmentation performance is better on recognizable classes with large domain gaps. However, DLA-Net does not perform well in adapting to the different styles and inherent differences between datasets. Future work will investigate how to adapt to more scenarios and datasets.