Road Agglomerate Fog Detection Method Based on the Fusion of SURF and Optical Flow Characteristics from UAV Perspective

Abstract

Road agglomerate fog seriously threatens driving safety, making real-time fog state detection crucial for implementing reliable traffic control measures. With advantages in aerial perspective and a broad field of view, UAVs have emerged as a novel solution for road agglomerate fog monitoring. This paper proposes an agglomerate fog detection method based on the fusion of SURF and optical flow characteristics. To synthesize an adequate agglomerate fog sample set, a novel network named FogGAN is presented by injecting physical cues into the generator using a limited number of field-collected fog images. Taking the region of interest (ROI) for agglomerate fog detection in the UAV image as the basic unit, SURF is employed to describe static texture features, while optical flow is employed to capture frame-to-frame motion characteristics, and a multi-feature fusion approach based on Bayesian theory is subsequently introduced. Experimental results demonstrate the effectiveness of FogGAN for its capability to generate a more realistic dataset of agglomerate fog sample images. Furthermore, the proposed SURF and optical flow fusion method performs higher precision, recall, and F1-score for UAV perspective images compared with XGBoost-based and survey-informed fusion methods.

1. Introduction

Agglomerate fog is a special fog that is much denser and has lower visibility, appearing in the range of tens to hundreds of meters [1]. Since agglomerate fog can severely impair a driver’s vision, it is difficult to promptly discern the road conditions ahead, and it is highly prone to triggering chain-reaction traffic accidents, inducing a significant threat to traffic safety. Traditional agglomerate fog monitoring methods can be classified into two types: (1) active remote sensing technology, including meteorological sensors, LiDAR, infrared imaging devices [2,3], and meteorological satellites [4]; and (2) vision-based detection technology [5,6,7,8,9,10,11,12,13,14]. However, active remote sensing technology faces challenges in signal attenuation, low spatial resolution, coverage constraints, and high implementation costs, which prevent it from being widely adopted in the field of traffic management. Vision-based detection technology can obtain richer feature information for more accurate environment monitoring using existing road surveillance equipment without additional cost investment. Hence, the vision-based methods hold broader application prospects in real-time monitoring of agglomerate fog.

Currently, vision-based research on agglomerate fog mainly focuses on (1) the fog generation and removal methods [5,6], (2) recognition and tracking methods of traffic objects under agglomerate foggy conditions [7,8,9], and (3) the detection of agglomerate fog in the road environment. For the detection of agglomerate fog, the approaches can be classified into traditional image processing methods and AI (artificial intelligence)-based methods. Traditional image processing methods primarily employ image feature extraction and classification approaches to accomplish fog state recognition, such as utilizing gray-level co-occurrence matrix [10,11], edge detection [12], or texture analysis [13,14]. Meanwhile, AI-based methods leverage large-scale patchy fog datasets for model training and perform state classification guided by gradient convergence principles, with predominant models including support vector machines (SVM), clustering analysis, K-nearest neighbor (KNN), and convolutional neural networks (CNN), and other models.

In [15], the authors propose a fog-density-level distinction method using an SVM classification framework that incorporates both global and local fog-related features, such as entropy, contrast, and dark channel information. The method achieves a classification accuracy of 85.8%, with greater robustness compared to traditional image processing methods, such as edge detection or texture analysis. To comparatively analyze the performance of SVM, K-NN, and CNN, the authors in [16] examine the three methods for identifying foggy and clear weather images based on the gray-level co-occurrence matrix extracted from ground-moving onboard scenario images. In [17], the authors propose an Attention-based BiLSTM-CNN (ABCNet) model, which integrates attention mechanisms with BiLSTM and CNN architectures to predict atmospheric visibility caused by heavy fog. Current research shows that artificial intelligence methods, which extract effective image features and convolutional characteristics and construct gradient descent convergence classifiers, can achieve efficient fog classification and atmospheric visibility prediction. This approach has now been a mainstream research direction.

In recent years, unmanned aerial vehicles (UAVs) have been widely applied in traffic detection, including traffic flow status, traffic events, and road environmental conditions, using technologies such as visual analysis, multi-sensor fusion, and edge computing [18,19,20]. In the field of fog research, the authors in [21] employed a classification module and a style migration module to address the challenge of non-uniform dense fog removal in UAV imagery without relying on paired foggy and corresponding fog-free datasets. The proposed method significantly enhanced the visual clarity and interpretability of complex foggy scenes using the classification-guided thick-fog-removal network and style migration mechanism. In [22], a multi-task framework named ODFC-YOLO was proposed, which seamlessly integrated image dehazing into a YOLO detection subnetwork via end-to-end joint training. This combination significantly improved detection accuracy and maintained high real-time performance for UAV-based object recognition in dense fog environments. However, existing methods primarily analyze static scenario images without consideration of the challenges caused by moving UAVs in dynamic flight conditions, such as motion-induced blurring, viewpoint variation, and fog density gradation. Furthermore, in real-world scenarios, the scarcity of agglomerate fog datasets limits model training performance and generalization capability.

To address these challenges, this paper develops a UAV-based framework for agglomerate fog detection, including a FogGAN model for generating agglomerate fog images to alleviate data scarcity, and an agglomerate fog detection model based on Bayesian fusion of SURF and optical flow characteristics for the fog state judgement. In the new UAV aerial perspective, the proposed fusion method integrates both the static and dynamic features of UAV images and performs higher precision, recall, and F1-score compared with traditional approaches.

2. Agglomerate Fog Sample Dataset Generation Method Based on GAN

Since agglomerate fog has characteristics of rapid formation and dissipation, limited spatial coverage, and shifting locations with air currents dynamically, it is difficult to acquire substantial valid samples in actual road environments. This paper adopts the generative adversarial network (GAN) [23] to synthesize an agglomerate fog sample dataset based on a limited number of field-collected images.

To obtain a high-quality and realistic agglomerate fog dataset, this paper proposes an improved generative adversarial network named FogGAN, in which the generator is enhanced by injecting physical image cues into its feature modulation layers for producing agglomerate fog with more authentic tonal and structural characteristics. The architecture of the FogGAN is shown in Figure 1.

Figure 1. The generation and working principle of GAN model.

2.1. Generator

The agglomerate fog image samples acquired from actual road scenarios are linearly normalized to the unit interval [0,1], as expressed by Equation (1).

$${\tilde{I}}_c(x,y)={\displaystyle \frac{I_c(x,y)-I_{\min }}{I_{\max }-I_{\min }}},\hspace{1em}c\in \{R,G,B\}$$

(1)

In Equation (1), $I_{\min }$ and $I_{\max }$ are denoted as the minimum and maximum light intensity values of the agglomerate fog image samples, respectively.

To simulate variations in viewpoint and sensor noise characteristics in actual road scenarios, random rotation and horizontal/vertical flipping, color jittering, and Gaussian noise injection are carried out for the augmentation of agglomerate fog images.

The color jittering is expressed by Equation (2).

$$I_c^{\mathrm{jitter}}(x,y)={\tilde{I}}_c^{\mathrm{jitter}}(x,y)\left(\begin{array}{c}1+{\delta }_c\end{array}\right) {\delta }_c~U(-0.1,0.1)$$

(2)

In Equation (2), $I_c^{\mathrm{jitter}}(x,y)$ is the color jittering image. ${\delta }_{\mathrm{c}}$ represents independent perturbations. Color jittering modifies image properties, including brightness, contrast, and saturation.

The Gaussian noise is expressed by Equation (3):

$$I_c^{\mathrm{aug}}(x,y)=I_c^{\mathrm{jitter}}(x,y)+{\eta }_c(x,y) {\eta }_c~N(0,{\sigma }^2)$$

(3)

In Equation (3), $I_c^{\mathrm{aug}}(x,y)$ is the image after adding Gaussian noise. ${\eta }_c(x,y)$ is the random noise following the Gaussian distribution of standard deviation $\sigma$, which is generally valued in the range $[0.01,0.05]$.

These augmentations enrich the training set with diverse lighting, color balance, and noise conditions. Based on the augmented image, the depth map describes the distance from each pixel to the camera and is produced by a CNN-based monocular depth estimation model that is pretrained on UAV perspective images.

The global photometric statistics from $\tilde{I}$ are extracted and further converted into a grayscale image following the ITU-R BT.601 [24] standard by Equation (4). In this study, the photometric statistics are computed from the real images to maintain the photometric consistency of the generated images. The grayscale conversion is performed as follows:

$$I_{\mathrm{gray}}(x,y)=0.299{\tilde{I}}_R(x,y)+0.587{\tilde{I}}_G(x,y)+0.114{\tilde{I}}_B(x,y)$$

(4)

Referring to Equation (4), the mean gray level is calculated by Equation (5).

$$I_{\mathrm{gray}}^{ave}={\displaystyle \frac{1}{HW}}{\displaystyle \sum _{x=1}^H{\displaystyle \sum _{y=1}^W{I}_{\mathrm{gray}}}}(x,y)$$

(5)

where H is the height of the image and W is the width. Further, the gray-level contrast, i.e., the standard deviation, is calculated by Equation (6).

$$K=\sqrt{{\displaystyle \frac{1}{HW}}{\displaystyle \sum _{x,y}{\left(I_{\mathrm{gray}}(x,y)-I_{\mathrm{gray}}^{ave}\right)}^2}}$$

(6)

For each color channel in $c\in \{R,G,B\}$, the average and the standard deviation of light intensity are calculated by Equations (7) and (8), respectively.

$${\mu }_c={\displaystyle \frac{1}{HW}}{\displaystyle \sum _{x,y}{\tilde{I}}_c}(x,y)$$

(7)

$${\sigma }_c=\sqrt{{\displaystyle \frac{1}{HW}}{\displaystyle \sum _{x,y}{\left({\tilde{I}}_c(x,y)-{\mu }_c\right)}^2}}$$

(8)

After normalization, the physical parameter vector of the agglomerate fog image sample (8) can be expressed by Equation (9).

$$P={\left[\begin{array}{c}\widehat{g},\widehat{K},{\widehat{\mu }}_R,{\widehat{\mu }}_G,{\widehat{\mu }}_B,{\widehat{\sigma }}_R,{\widehat{\sigma }}_G,{\widehat{\sigma }}_B\end{array}\right]}^T$$

(9)

The depth map M, the physical parameter vector P, and the random noise vector Z collectively form the input of the Generator. Since the proposed method incorporates the physical features of the image samples, FogGAN can ensure that the generated agglomerate fog images exhibit enhanced realism.

In practice, $g(Z;M;P)$ is instantiated as a 1-D latent vector concatenating the embeddings of the depth map M, the photometric vector P and the noise Z. This vectorized input g is fed to the fully connected layer in Equation (10).

$$f^{(1)}=W^{(1)}g+b^{(1)}$$

(10)

where $W^{\left(1\right)}$ and $b^{(1)}$ are the weight matrix and bias vector, respectively.

To achieve physics-driven fog attenuation, these features are further modulated using a depth-dependent transmission coefficient $\alpha (M;{\theta }_M)$ and a learnable mapping $\beta (P;{\theta }_P)$ of the photometric parameters using Equation (11).

$$f^{(2)}=\alpha \left(\begin{array}{c}M;{\theta }_M\end{array}\right)\odot f^{(1)}+\beta \left(\begin{array}{c}P;{\theta }_P\end{array}\right)$$

(11)

In Equation (11), $\alpha (M;{\theta }_M)$ is a per-pixel mask of size 1 × H × W that multiplies all channels at location $(x,y)$ (element-wise multiplication). $\alpha (M;{\theta }_M)$ is calculated using Equation (12).

$$\alpha (M;{\theta }_M)=\exp \left(-\kappa \widehat{M}\right),\hspace{1em}\widehat{M}=\mathrm{normalize}(M)$$

(12)

where ${\theta }_M=\kappa$ and $\kappa$ > 0 control the attenuation strength.

$\beta (P;{\theta }_P)$ is a per-channel bias of size C × 1 × 1 that is broadcast over H × W. $\beta (P;{\theta }_P)$ is calculated using Equation (13).

$$\beta (P;{\theta }_P)=\mathrm{MLP}(P)\in R^{C\times 1\times 1}$$

(13)

where ${\theta }_P$ denotes the learnable parameters of the MLP (weights and biases).

As mentioned in the aforementioned analysis, the physical cues injection process is shown in Figure 2.

Figure 2. Physical cue injection process.

The modulated feature $f^{(2)}$ is passed through L successive up-sampling blocks, which consist of convolution, BatchNorm, and ReLU activation, as shown in Equation (14).

$$f^{(l+1)}=\mathrm{ReLU}\left(\mathrm{BatchNorm}(\mathrm{Conv}(f^{(l)}))\right),\hspace{1em}l=2,\dots ,L-1$$

(14)

The output fake images are expressed by Equation (15).

$$\widehat{x}=\mathrm{TConv}(f^{(L)})$$

(15)

The entire generator is trained to minimize the combined loss as given by Equation (16).

$$\left\{\begin{array}{l}{L}_G=L_{\mathrm{adv}}+{\lambda }_{\mathrm{phy}}{L}_{\mathrm{phy}}+{\lambda }_{\sup }{L}_{\sup }\\ L_{\mathrm{adv}}=E_{Z,M,P}[-\log D(\widehat{x})]\\ L_{\mathrm{phy}}={‖B(\widehat{x})-P‖}_2^2\\ L_{\sup }={‖\widehat{x}-x‖}_1\end{array}\right.$$

(16)

where $D(\cdot )$ is the discriminator, and $B(\cdot )$ extracts the same global photometric statistics defined in Equations (4)–(9). $E_{Z,M,P}$ denotes expectation over $Z~p(z)$, and $(M,P)~p_{\mathrm{data}}$, ${\lambda }_{\mathrm{phy}}$, and ${\lambda }_{\sup }$ are the weights.

2.2. Discriminator

Taking either the real foggy image $x$ or the fake foggy image $\widehat{x}=G(g(Z;M;P))$ created by the Generator as input, a scalar $D(\cdot )\in [0,1]$ is produced to describe the reality probability. The discriminator architecture consists of Q down-sampling blocks and a fully connected layer, as expressed by Equations (17)–(19).

$$h^{(0)}=x\hspace{1em}\mathrm{or}\hspace{1em}\widehat{x}$$

(17)

$$h^{(q+1)}=\mathrm{LeakyReLU}\left(\mathrm{BatchNorm}({\mathrm{Conv}}^{(q)}(h^{(q)}))\right)\hspace{1em}q=0,\dots ,Q-1$$

(18)

$$D(x)=S\left(W^{(e)}{h}^{(Q)}+b^{(e)}\right)$$

(19)

where each ${\mathrm{Conv}}^{(q)}$ is a convolution with stride 2. $W^{(e)}$ and $b^{(e)}$ are the weights and bias of the final linear layer. $S$ is the sigmoid activation.

The discriminator is trained to minimize the standard binary-cross-entropy loss as given by Equation (20).

$$L_D=-E_{x~p_{\mathrm{data}}}\left[\log D(x)\right]-E_{g~p_g}\left[\log \left(1-D(G(g(Z;M;P)))\right)\right]$$

(20)

By alternately minimizing $L_D$ with respect to $D$ and minimizing the generator loss $L_G$ with respect to $G$, the two networks play the minimax game that drives $G$ to produce increasingly realistic foggy images.

3. Dense Fog Detection Method Based on SURF Feature Extraction and Optical Flow Analysis

3.1. ROI Division for UAV Perspective

The agglomerate fog has characteristics of small spatial scale and localized distribution, whereas the forward-facing perspective of drones during UAV cruise has a broad distant view. It is imperative to define the region of interest (ROI) for fog detection, which is beneficial for accurately locating where the fog occurs and for reducing computational overhead. In this paper, trapezoidal ROI along the road within different image depth ranges are preset and applied for further research.

For the image resolution of $W_{IMG}\times H_{IMG}$, n trapezoidal ROIs are divided according to Equation (21).

$$ROI(i)=\left\{\begin{array}{l}(x_{i,1},y_{i,1})=({\displaystyle \frac{\left(n-i\right)x_{\mathrm{TL}}+\left(i-1\right)x_{\mathrm{BL}}}{n-1}},{\displaystyle \frac{i-1}{n}}{H}_{IMG})\\ (x_{i,2},y_{i,2})=({\displaystyle \frac{\left(n-i\right)x_{\mathrm{TR}}+\left(i-1\right)x_{\mathrm{BR}}}{n-1}},{\displaystyle \frac{i-1}{n}}{H}_{IMG})\\ (x_{i,3},y_{i,3})=({\displaystyle \frac{\left(n-i\right)x_{\mathrm{TR}}+\left(i-1\right)x_{\mathrm{BR}}}{n-1}},{\displaystyle \frac{i}{n}}{H}_{IMG})\\ (x_{i,4},y_{i,4})=({\displaystyle \frac{\left(n-i\right)x_{TL}+\left(i-1\right)x_{BL}}{n-1}},{\displaystyle \frac{i}{n}}{H}_{IMG}) i,j=1,2,\dots ,n\end{array}\right.$$

(21)

where $(x_{i,j},y_{i,j})$ are the pixel coordinates of the j-th vertex of the i-th ROI (j = 1, 2, …, n; i = 1, 2, …, n); $x_{\mathrm{TL}}$, $x_{\mathrm{BL}}$, $x_{\mathrm{TR}}$, and $x_{\mathrm{BR}}$ denote the horizontal pixel positions of the trapezoid’s top-left, top-right, bottom-left, and bottom-right base points, respectively; and $W_{IMG}$ and $H_{IMG}$ are the image width and height as shown in Figure 3.

Figure 3. Distribution of regions of interest.

3.2. SURF Extraction of Road Agglomerate Fog

For each ROI, the SURF (speeded-up robust features) [25] algorithm is employed for robust detection and description of interest points. The integral image at any location (x,y) is defined as the cumulative sum of pixel intensities above and to the left of (x,y), denoted by Equation (22).

$$I_{\Sigma }(x,y)={\displaystyle \sum _{i=0}^x{\displaystyle \sum _{j=0}^{y}I}}(i,j)$$

(22)

Based on the integral image acquired by Equation (22), box filters are applied to approximate the second-order Gaussian derivatives $L_{xx}(x,y,s)$, $L_{xy}(x,y,s)$, and $L_{yy}(x,y,s)$ at scale s. The resulting responses are combined into the Hessian matrix, as expressed by Equation (23).

$$\mathrm{Hessian}(x,y,s)=\left[\begin{array}{cc}{L}_{xx}(x,y,s)& L_{xy}(x,y,s)\\ L_{xy}(x,y,s)& L_{yy}(x,y,s)\end{array}\right]$$

(23)

As shown in Equation (23), the determinant of this $\mathrm{Hessian}(x,y,s)$ effectively approximates the response of the Laplacian-of-Gaussian operator. In this paper, the points with $\mathrm{Hessian}(x,y,s)$ values above the threshold in both spatial and scale dimensions are defined as key points.

The orientation of each feature point is assigned using the Haar wavelet transform on the circular neighborhood around the key point. Figure 4 shows the differentiated response of the Haar wavelet in the x and y directions in the same road scenario.

Figure 4. Haar responses of the road scenario: (a) Horizontal Haar response H_x; (b) Vertical Haar response H_y.

The wavelet responses are aggregated in both horizontal and vertical directions, and the orientation $\theta$ is acquired using Equation (24).

$$\theta =\mathrm{atan}2\left({\displaystyle \sum H_y},{\displaystyle \sum H_x}\right)$$

(24)

where $w_x$ and $w_y$ denote the Haar wavelet transform in the horizontal and vertical directions, respectively, and $\mathrm{atan}2$ is the four-quadrant inverse tangent function.

Finally, to construct the SURF descriptor, a square region centered at each key point is divided into a 4 × 4 grid of subregions, as shown in Figure 5a. Within each of the $4 \times 4$ subregions, four statistical values of the Haar responses are accumulated. Concatenating these four values from all 16 subregions yields a 64-D descriptor, as shown in Figure 5b.

Figure 5. Construction of SURF descriptors: (a) 4 × 4 subregion response matrix; (b) Mean horizontal response per subregion.

After detection, the count of SURF key points within the region of interest is denoted by $N_f$ for the i-th ROI and is used as one evidential variable in the subsequent Bayesian fusion.

3.3. Optical Flow Analysis of Road Agglomerate Fog

The SURF algorithm is primarily applied for static feature extraction of individual images. To improve the accuracy of agglomerate fog detection during UAV adaptive cruise, this paper further integrates optical flow features for dynamic feature extraction. The Lucas–Kanade [26] algorithm is used to capture dynamic features of agglomerate fog by pixel motion across consecutive image frames for each ROI.

Referring to the brightness constancy assumption, pixel values exhibit minimal variation over short time intervals between consecutive frames, as expressed by Equation (25).

$$I(x,y,t)=I(x+\mathrm{\Delta }x,y+\mathrm{\Delta }y,t+\mathrm{\Delta }t)$$

(25)

Using the first-order Taylor series, the optical flow constraint equation is obtained and expressed by Equation (26).

$$I_{x}u+I_{y}v+I_t=0$$

(26)

In Equation (26), $I_x$, $I_y$, and $I_t$ are partial derivatives of image intensity in the spatial (x,y) and temporal t domains. u and v represent horizontal and vertical components of the optical flow vector, respectively.

By establishing an overdetermined system of equations using multiple neighboring pixels, Equation (26) can be further transformed into Equation (27), and the solution of the optical flow constraint equation can be treated as a least squares problem.

$$\left[\begin{array}{cc}{\displaystyle \sum I_x^2}& {\displaystyle \sum I_x}{I}_y\\ {\displaystyle \sum I_x}{I}_y& {\displaystyle \sum I_y^2}\end{array}\right]\left[\begin{array}{c}u\\ v\end{array}\right]=-\left[\begin{array}{c}{\displaystyle \sum I_x}{I}_t\\ {\displaystyle \sum I_y}{I}_t\end{array}\right]$$

(27)

Referring to the previously identified SURF key points of consecutive frames, optical flow vectors are computed for capturing displacement and directionality of feature movements. In agglomerate fog areas, atmospheric particles typically exhibit subtle and coherent motion, resulting in smaller magnitudes and directionally consistent optical flow vectors. In contrast, non-fog areas tend to display irregular and diverse motion features.

Let $w(x,y,t)={[u(x,y,t),v(x,y,t)]}^T$ denote the Lucas–Kanade flow vector at pixel (x,y) between two consecutive frames t and t + 1. Its magnitude is expressed by Equation (28).

$$M_v=\Vert w(x,y,t){\Vert }_2=\sqrt{u{(x,y,t)}^2+v{(x,y,t)}^2}$$

(28)

The mean magnitude of optical flow vectors is denoted by $M_v$ for the i-th ROI and is used as the other evidential variable in the subsequent Bayesian fusion.

3.4. Bayesian-Based Data Fusion for Agglomerate Fog Detection

For each ROI in the frame of the road image detected by UAV, calculate the number of SURF $N_f$ and the mean optical flow magnitude $M_v$. The likelihoods under the “there is agglomerate fog in the ROI” hypothesis are expressed by Equations (29) and (30), respectively.

$$p_f\left(N_f\mid \mathrm{ROI}\right)={\displaystyle \frac{1}{\sqrt{2\pi }{\sigma }_f}}\exp \left(-{\displaystyle \frac{{(N_f-{\mu }_f)}^2}{2{\sigma }_f^2}}\right)$$

(29)

$$p_m(M_v\mid \mathrm{ROI})={\displaystyle \frac{1}{\sqrt{2\pi }{\sigma }_m}}\exp \left(-{\displaystyle \frac{{\left(M_v-{\mu }_m\right)}^2}{2{\sigma }_m^2}}\right)$$

(30)

where ${\mu }_f$ and ${\sigma }_f$ denote the mean and standard deviation of SURF counts, respectively. ${\mu }_m$ and ${\sigma }_m$ are the mean and standard deviation of motion magnitudes, respectively. These parameters are derived from statistical analysis of the samples by FogGAN.

Assuming the prior agglomerate fog probability $P_0$ is conditionally independent of $N_f$ and $M_v$, the posterior probability of agglomerate fog can be expressed by Equation (31).

$$P(\mathrm{fog}\mid N_f,M_v,\mathrm{ROI})={\displaystyle \frac{P_0{p}_{f(N_f\mid \mathrm{ROI})}{p}_m(M_v\mid \mathrm{ROI})}{P_0{p}_{f(N_f\mid \mathrm{ROI})}{p}_m(M_v\mid \mathrm{ROI})+(1-P_0)\left[\begin{array}{c}1-p_{f(N_f\mid \mathrm{ROI})}\end{array}\right]\left[1-p_m(M_v\mid \mathrm{ROI}\right]}}$$

(31)

When the posterior probability is larger than the preset threshold $\tau$, it can be considered that agglomerate fog has occurred, as expressed by Equation (32).

$$P\left(\mathrm{fog}\mid N_f,M_v,\mathrm{ROI}\right)\ge \tau$$

(32)

4. Experimental Results and Analysis

4.1. Experimental Conditions

In this paper, the DJI Mavic 3T UAV is used for original agglomerate fog image collection, and the configurations presented in Table 1 are collected from mileage post number K470 to K480 of the Jingtai Highway. The road scenario is shown in Figure 6.

Table 1. Parameters of the experiment.

Category	Parameter	Value
Environmental conditions	wind scale	1–4 F
	temperature	10–25 °C
	visibility	4–10 km
UAV configurations	pitching angle	60°
	resolution of the camera	3840 × 2160 Px
	frame rate	30 FPS
	locate mode	RTK ± 0.1 m
Flight configurations	height	100–150 m
	speed	15 m/s

Figure 6. UAV and experimental scenario.

4.2. FogGAN-Based Dataset Generation Results and Analysis

To evaluate the performance of the proposed FogGAN, comparative experiments were conducted against three benchmark methods: DCGAN [27], CycleGAN [28], and Pix2Pix [29]. The visual effects of agglomerate fog images generated by different methods are presented in Figure 7.

Figure 7. Agglomerate fog visual effects using different methods.

To carry out a quantitative evaluation of the results generated by different methods, this paper applies three indexes for further comparative analysis, including peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Frechet inception distance (FID). The indexes are calculated as follows:

① PSNR: Generated images with a greater PSNR have higher fidelity to the reference. The PSNR is calculated by Equation (33).

$$PSNR=10{\log }_{10}\left({\displaystyle \frac{{({\mathrm{MAX}}_I)}^2}{MSE}}\right)$$

(33)

where MAX_I is the maximum possible pixel value. MSE is the mean squared of the referenced image X and the generated image Y, as expressed by Equation (34).

$$MSE={\displaystyle \frac{1}{3H_{IMG}{W}_{IMG}}}{\displaystyle \sum _{i=1}^{H_{IMG}}{\displaystyle \sum _{j=1}^{W_{MG}}{\displaystyle \sum _{k=1}^3{\left(X(i,j,k)-Y(i,j,k)\right)}^2}}}$$

(34)

② SSIM: Generated images with a greater SSIM mean they are closer to the reference in luminance, contrast, and structure. The SSIM is calculated using Equation (35).

$$SSIM(X,Y)={\displaystyle \frac{\left(2{\mu }_X{\mu }_Y+V_1\right)\left(2{\sigma }_{XY}+V_2\right)}{\left({\mu }_X^2+{\mu }_Y^2+V_1\right)\left({\sigma }_X^2+{\sigma }_Y^2+V_2\right)}}$$

(35)

where ${\mu }_X$ and ${\mu }_Y$ are local means; ${\sigma }_X^2$ and ${\sigma }_Y^2$ are local variances; ${\sigma }_{XY}$ is the covariance; and $V_1={(l_1{\mathrm{MAX}}_I)}^2$, $V_2={(l_2{\mathrm{MAX}}_I)}^2$ with l₁ = 0.01, l₂ = 0.03.

③ FID: Generated images with a lower FID indicate the generated distribution is closer to the reference. The FID is calculated using Equation (36).

$$FID=\Vert {\eta }_X-{\eta }_Y{\Vert }^2+Tr({\sum }_X+{\sum }_Y-2{\left({\sum }_X{\sum }_Y\right)}^{1/2})$$

(36)

where $({\eta }_X,{\Sigma }_X)$ and $({\eta }_Y,{\Sigma }_Y)$ are the mean and covariance of deep features for the referenced and generated images, respectively.

The PSNR, SSIM, and FID of different methods are presented in Table 2.

Table 2. PSNR, SSIM, and FID of different methods.

Method	PSNR (dB)	SSIM	FID
DCGAN	20.5	0.65	35.2
CycleGAN Pix2Pix FogGAN	22.3 23.1 24.1	0.70 0.72 0.77	30.1 28.4 25.3

It is evidence that the proposed FogGAN achieves higher PSNR and SSIM values while exhibiting lower FID compared with DCGAN, CycleGAN, and Pix2Pix. Since the generator in FogGAN is enhanced by injecting physical image cues, the generated images exhibit greater similarity to the original images in terms of structural features, with high values in PSNR and SSIM. Meanwhile, FogGAN improves the pixel-level fidelity and creates a more realistic road background with lower FID.

4.3. Agglomerate Fog Detection Results Based on the Fusion of SURF and Optical Flow

4.3.1. SURF Extraction Results and Analysis

In this paper, four ROIs are configured for agglomerate fog detection. The SURF extraction results for partial samples are shown in Figure 8.

Figure 8. SURF extraction results: (a) Reference (clear) image. (b–h) Detection results under agglomerate fog, with yellow trapezoids marking road ROIs, and R1–R4 (near → far) and red dots indicating SURF key points. All panels use identical detector parameters and are aligned to the reference in (a).

The number of SURF $N_f$ and density $\rho$ for different images and ROIs are presented in Table 3 and Figure 9.

Table 3. SURF distributions for different images and ROIs.

Image	ROI(1)		ROI(2)		ROI(3)		ROI(4)
	${\mathit{N}}_{\mathit{f}}$	$\mathit{\rho }$	${\mathit{N}}_{\mathit{f}}$	$\mathit{\rho }$	${\mathit{N}}_{\mathit{f}}$	$\mathit{\rho }$	${\mathit{N}}_{\mathit{f}}$	$\mathit{\rho }$
a (no fog)	2711	7.68 × 10−4	6012	1.06 × 10−3	8910	1.14 × 10−3	9598	9.62 × 10−4
b	117	3.31 × 10−5	452	8.56 × 10−5	1411	2.01 × 10−4	3996	4.56 × 10−4
c	72	2.04 × 10−5	239	4.53 × 10−5	1024	1.46 × 10−4	1585	1.81 × 10−4
d	135	3.82 × 10−5	467	8.84 × 10−5	577	8.21 × 10−5	1552	1.77 × 10−4
e	93	2.63 × 10−5	670	1.27 × 10−4	957	1.36 × 10−4	1847	2.11 × 10−4
f	94	2.66 × 10−5	403	7.63 × 10−5	1884	2.68 × 10−4	2218	2.53 × 10−4
g	138	3.90 × 10−5	452	8.56 × 10−5	2000	2.84 × 10−4	5406	6.16 × 10−4
h	149	4.21 × 10−5	756	1.43 × 10−4	1622	2.31 × 10−4	3881	4.43 × 10−4

Figure 9. SURF density variation of different samples and ROIs.

Referring to Figure 8, Figure 9, and Table 3, the distribution of SURF points follows a pattern where the density is lower in long-distance ROIs and higher in short-distance ROIs. The reason is that agglomerate fog concentration is higher in long-distance regions, allowing the image to capture limited features that mainly originate from the agglomerate fog. In contrast, visibility is relatively better in short-distance regions, enabling ground targets to generate more feature points. Therefore, SURF features can effectively describe the state of agglomerate fog in an actual scenario.

4.3.2. Optical Flow Extraction Results and Analysis

Taking any two consecutive frames as examples, the optical flow is detected and is shown in Figure 10. In the figure, the optical flow magnitude is represented by the brightness.

Figure 10. Optical flow extraction results: (a) Reference (clear) image. (b–h) Detection results under agglomerate fog. The visualization encodes magnitude only: darker = larger motion, brighter = smaller motion. All panels share the same contrast range for comparability and are aligned to the reference in (a).

The optical flow magnitude $M_v$ for different images and ROIs is presented in Table 4.

Table 4. Optical flow magnitude distribution for different images and ROIs.

Image	ROI(1)	ROI(2)	ROI(3)	ROI(4)
a (no fog)	2.52	2.25	2.42	2.46
b	0.21	0.24	0.61	1.31
c	0.33	0.48	0.49	1.59
d	0.29	1.20	0.71	1.53
e	0.26	0.47	0.80	1.68
f	0.30	0.57	0.99	2.03
g	0.36	0.77	0.95	2.23
h	0.30	0.53	0.75	2.48

Referring to Figure 10, Figure 11, and Table 4, the optical flow magnitude exhibits a distribution pattern where values are smaller in long-distance ROIs and larger in short-distance ROIs. The reason is that higher agglomerate fog concentration in long-distance ROIs results in minimal pixel variation between consecutive frames captured by the moving UAV. In contrast, better visibility at close range leads to significant pixel changes due to dynamic UAV perspective shifts.

Figure 11. Optical flow magnitude variation of different samples and ROIs.

4.3.3. Fusion Method Experimental Results and Analysis

Based on the aforementioned analysis, the SURF and optical flow features can be independently applied to the judgement of the agglomerate fog state by setting certain thresholds. Referring to Equations (28)–(30), the agglomerate fog state is detected for different images, including 300 training samples and 100 test samples. The precision, recall, and F1-score of the SURF-based method, optical flow-based method, and the fusion method are presented in Table 5.

Table 5. Agglomerate fog detection performance of the proposed method.

Method	Precision	Recall	F1-Score
SURF-based method	0.82	0.75	0.78
Optical flow-based method Fusion of SURF and optical flow method	0.79 0.88	0.78 0.85	0.76 0.82

It is evidence that both the SURF-based method and the optical flow-based method can achieve high precision for the agglomerate fog state judgement. However, the fusion of the two features presents better performance, where the precision increases by 7.3% compared to the SURF-based method and by 11.4% compared to the optical flow-based method; the recall improved by 13.3% and 9.0%, respectively; and the F1-score improved by 5.1% and 7.9%, respectively.

To further evaluate the performance of the proposed method, the XGBoost-based method [13] and the survey-informed fusion method [12] are selected for the comparison of agglomerate fog detection. All methods are tested under identical experimental settings using the same mixed dataset generated by FogGAN. The precision, recall, and F1-score are presented in Table 6.

Table 6. Agglomerate fog detection results of different methods.

Method	Precision	Recall	F1-Score	Processing Time (ms/Image)
XGBoost-based method	0.78	0.74	0.76	140
Survey informed fusion method Fusion of SURF and optical flow method	0.80 0.88	0.76 0.85	0.78 0.82	135 180

The experimental results show that the proposed SURF and optical flow fusion method presents higher precision, recall, and F1-score compared with the XGBoost method and the survey-informed fusion method. Different from the XGBoost-based method, which relies on a dynamic light source, and the survey-informed fusion method, which requires high texture agglomerate fog images, the implement of the fusion of SURF and optical flow is able to detect both the static and dynamic physical characteristics of agglomerate fog in localized ROIs, and thus it yields better performance under rapidly changing viewpoints of UAVs. Since optical flow detection requires the comparison of two consecutive images, the CPU time is longer than that of the other two methods. Nevertheless, the millisecond-level additional CPU time will not substantially impact the practical engineering application of agglomerate fog detection; for example, the dissemination of fog status.

5. Conclusions and Future Work

This paper proposes an improved generative adversarial network (FogGAN) to obtain a high-quality and realistic agglomerate fog dataset, in which the generator is enhanced by injecting physical image cues into its feature modulation layers. Using the data sample generated by FogGAN, a fusion method of SURF and optical flow for detecting agglomerate fog in UAV-captured road scenarios is further presented. The method describes static features using SURF and dynamic features using optical flow magnitude, achieving better performance for agglomerate fog detection under the flight viewpoint of UAVs.

Future work includes the following: (1) Experiments will be further carried out under different UAV flight scenarios, such as different flight altitudes and camera resolutions, to evaluate the performance of the proposed method. (2) The integration of deep convolutional features of images with SURF and optical flow should be further explored to improve the accuracy of agglomerate fog detection.