Multi-Scale Dual Discriminator Generative Adversarial Network for Gas Leakage Detection

Abstract

Gas leakages pose significant safety risks in urban environments and industrial sectors like the Oil and Gas Industry (OGI), leading to accidents, fatalities, and economic losses. This paper introduces a novel generative AI framework, the Multi-Scale Dual Discriminator Generative Adversarial Network (MSDD-GAN), designed to detect and localize gas leaks by generating thermal images from RGB input images. The proposed method integrates three key innovations: (1) Attention-Guided Masking (AttMask) for precise gas leakage localization using saliency maps and a circular Region of Interest (ROI), enabling pixel-level validation; (2) Multi-scale input processing to enhance feature learning with limited data; and (3) Dual Discriminator to validate the thermal image realism and leakage localization accuracy. A comprehensive dataset from laboratory and industrial environment has been collected using a FLIR thermal camera. The MSDD-GAN demonstrated robust performance by generating thermal images with the gas leakage indications at a mean accuracy of $81.6\%$, outperforming baseline cGANs by leveraging a multi-scale generator and dual adversarial losses. By correlating ice formation in RGB images with the leakage indications in thermal images, the model addresses critical challenges of OGI applications, including data scarcity and validation reliability, offering a robust solution for continuous gas leak monitoring in pipeline.

1. Introduction

Gas risks from leaks, emissions, and fatalities due to exposure to gas in confined spaces are among the most common causes of accidents in urban areas with complex gas distribution networks, as well as in the Oil and Gas Industry (OGI). Additionally, leaks and losses during gas transportation contribute to significant costs and environmental issues for the industry. Detecting gas leaks is especially challenging because gases are invisible, unlike other fluids, making traditional detection methods less effective-particularly given the vast distances pipelines cover and the often harsh environments in which they operate. To address these challenges, our approach leverages widely available RGB cameras and advanced Machine Learning (ML), offering a cost-effective solution that does not require specialized thermal imaging equipment. This method can improve detection accuracy and accessibility compared to conventional techniques, which often rely on expensive sensors or fiber optic systems.

This article is proposed within the digitalization framework led by Universidad Carlos III de Madrid, Spain (UC3M), through the AMPL laboratory. It focuses on urban and industrial environments, where gas leak detection has been identified as a critical enhancement to the creation of digital twins. By integrating gas leakage detection into digital twin process, the proposed work aims to improve monitoring and safety in these complex settings. In these environments, the detection and localization of these events are of vital importance, which is due to the personal and material risks involved. The proposal seeks to provide a tool that leverages digitalization capabilities onboard a ground vehicle equipped with systems such as cameras and LiDAR, in order to identify events as dangerous as those mentioned here. Furthermore, this research contributes to the growing field of AI-driven environmental monitoring and industrial safety, opening new possibilities by stating the challenges in various industrial applications beyond gas leakage detection.

The generative artificial intelligence (GenAI), particularity through thermographic imaging techniques, has shown potential in creating enhanced visual representations of otherwise imperceptible phenomena. This approach draws motivation from related applications, such as using GenAI to colorize old movies [1] or generate human facial images through future lifespan projections [2]. In our case, we extend these concepts to the critical domain of gas leakage detection. By developing a system that can generate thermal images from RGB input, we aim to make gas leakage detection more accessible, efficient, and widespread. This could significantly improve safety measures, reduce economic losses associated with undetected gas leakage, and contribute to environmental protection efforts.

The solution proposed is inspired by recent advances in GenAI to develop an innovative model that visualizes and detects gas leaks, making these invisible hazards perceptible to the human eye. Conditional Generative Adversarial Networks (cGANs) are a prominent example of GenAI models, capable of learning complex data distributions and generating realistic synthetic data across various domains. In this work, we leverage the generative capability of cGANs to bridge the gap between RGB imaging and thermal imaging, enabling accurate detection of gas leakage points in pipeline networks. While cGANs have been widely employed in the literature to generate synthetic images, and addressing the issue of data scarcity [3,4]. The application of computer-vision based methods in OGI has been constrained by limited availability of relevant high-quality data. To address this, our paper introduces a novel GenAI-based approach that utilizes RGB images to generate corresponding thermal images, thereby enabling the detection and localization of gas leaks. Our model includes attention-guided masking (AttMask) strategy, which directs the model’s focus to regions with a high probability of leakage. Moreover, multi-scale input processing method is proposed to learn better features with limited data. Acting as a digital twin, the proposed model serves as a digital gas detector and has demonstrated its effectiveness in OGI field applications.

Detailed contribution: The four-fold contribution of this work consists of the following:

• Comprehensive comparative analysis of existing state-of-the-art (SOTA) methods: We provide an in-depth comparison of the traditional, machine learning (ML) and deep learning (DL)-based techniques employed for monitoring and detection of gas leakage, including digital gas detection platforms.
• Novel dataset collection: A unique dataset of paired RGB and thermal imaging is collected in both laboratory and industrial environment for gas leakage detection, addressing data scarcity and digital gas detector capabilities in this field.
• MSDD-GAN model: We propose a novel GenAI model named Multi-Scale Dual Discriminator Generator Adversarial Network (MSDD-GAN) for gas leakage detection, functioning as a digital platform for thermal imaging.
• Leveraging cGAN’s model: We leverage the SOTA model of cGAN to generate thermal images from RGB images, serving as a digital gas detector platform. We also provide its comparison with the proposed MSDD-GAN.

Paper organization: This paper is organized as follows: Section 2 provides a comprehensive review of existing SOTA methods and challenges in gas leakage detection. Section 3 formulates the problem statement, defining the research objectives and scope. Section 4 discusses the dataset collection process, detailing the acquisition of thermal and RGB image pairs in both laboratory and industrial environments. Section 5 presents the proposed methodology, elaborating on the architecture and implementation of cGANs and MSDD-GAN. Section 6 highlights the implementation parameters of the proposed model as well as discuss the results and findings of our work, offering a detailed analysis of the model’s performance employing different performance matrices. Finally, Section 7 concludes this work by highlighting the future directions.

2. Literature Review

Effective gas leak detection and pipeline monitoring are important challenges in the OGI, where traditional techniques often require high maintenance costs and struggle with timely identification of leaks and corrosion [5]. As pipelines age or sustain external damage, the likelihood of leaks increases, underscoring the need for regular and reliable inspection methods to ensure safety and efficiency.

Table 1. Theoretical comparison of the objectives, methods, and limitations of SOTA methods used for gas leakage detection in OGI sector.

Ref.	Objective	Method/Tech	Dataset	Limitations
[6]	Address cross-domain issues in leakage detection	Semi-supervised domain adaptation	Cross-domain pipeline data	Lack visual localization of leakage, not robust with limited data
[7]	Semi-supervised anomaly detection in gas pipeline	LSTM-AE, OCSVM	Real gas pipeline sensor data	Focused only on numerical signals, does not provide visual localization of leaks
[8]	Percussion-based, leak detection	MobileNetV2 (DL on audio)	Pipe hammering audio sound	Limited to accessible sites, not suitable for image-based detection or remote monitoring
[9]	CH4 leakage from marine decommissioned wells	Seismic and hydro- acoustic sensing	Data from 43 wells in the North Sea	Requires specialized equipment, does not support real-time or visual detection
[10]	Manifold learning for leakage diagnosis	VMD, SPD matrix, LLE, SVM	Pipeline sensor condition signal	Relies on numerical data, limiting practical deployment in complex environments
[11]	Enhance small leakage detection using ML	CNN, BiLSTM, attention mechanism	SCADA batch data from pipelines	Requires extensive labeled data, less effective for visually ambiguous leaks
[12]	Develop low cost WSN for real-time leak detection	2D-CNN, LSTM- AE, field tests	Experimental pipeline network	Sensor deployment and maintenance is challenging, not suitable for non-invasive detection
[13]	Investigate MFL for detecting welding defects in pipelines	Electromagnetic simulation	406 pipelines samples	Limited to certain defect types, restricting broader applicability
[14]	Predict pipeline failures using ML and hybrid models	ANNs, SVM, biometric analysis	Pipeline sensor signal data	Not design for real-time leak detection, limiting its use for monitoring
[15]	Predict CH4 emission trends in China’s OGI sector	Emission estimation, inventories analysis	Chinese OGI operational data	Provide macroscopic estimations; not suitable for local leak detection

provides a comparison of several relevant studies that address these challenges. This section is structured into three subsections which discuss traditional, machine learning (ML), and deep learning (DL)-based approaches, as outlined below.

2.1. Traditional Methods

A variety of traditional and sensor-based methods have been explored for gas leak detection and pipeline monitoring in OGI. Early approaches relied on numerical simulations and electromagnetic analysis. For example, Maxwell electromagnetic simulation software has been effectively used for the numerical analysis of welding defects, allowing for the validation of the reliability of magnetic flux leakage (MFL) techniques for diagnosing pipeline welding defects using experimental data [11,16]. These techniques involve significant maintenance costs and have a very limited scope.

Remote sensing technologies, such as satellite-based monitoring, have also gained attention and have been employed to analyze the atmospheric methane (CH4) and nitrogen dioxide (NO2) concentrations in the industrial area of Zhoushan, China [17]. These studies have revealed variations in CH4 due to petroleum activities and identified NO2 as a tracer for CH4 emissions. This technology allows for the analysis of surface temperature and provides information on the urban heat island effect but unable to identify the specific location of gas leaks.

Gas leakage, particularly methane, poses significant environmental risks in marine environments. Recent studies using seismic and hydro-acoustic data reveal that shallow gas accumulations near wells drive methane releases in regions like the North Sea, though these remain locally significant rather than globally impactful [18,19].

Nanotechnology-based sensors are an emerging area, with nanoparticles (NPs) being incorporated into sensors for downhole reservoir studies [20]. However, NP-based methods still face limitations in achieving real-time, accurate leakage detection in natural gas pipelines. To address this, three-dimensional capacitance array sensors have been proposed for early detection [21], but their efficiency degrades due to sediment build-up in pipelines.

Optical and infrared-based detection methods have also seen significant development. Multispectrum infrared (MSIR) flame detectors, such as FL4000H and its variants, are increasingly used for heat hotspot detection in OGI [18]. The transition from traditional UV/IR to triple infrared (IR3) detection has notably improved detection performance, offering superior immunity to false alarms and a wider field of view. IR3 detectors can identify heat hotspots at long distances and maintain reliability even in harsh environments [22]. Full-scale experiments have shown that infrared energy intensity at $4.504$ $\mathsf{\mu }$m correlates with the area of thermal hotspots, with detector performance depending more on this intensity than on temporal variations, thereby improving thermal mapping and design evaluations. Optical gas imaging (OGI) is another promising method, utilizing spectrally filtered thermal imaging to visualize and quantify gas leaks, making otherwise invisible emissions detectable from a safe distance [23,24]. These methods rely on complex and costly hardware, demand precise calibration, and often exhibit limited generalization and adaptability, resulting in high maintenance requirements-particularly in complex environments.

Other specialized detectors, such as those using Cobalt-60 and Sodium Iodide with Thallium, have been proposed to enhance data collection for gas leakage detection [7]. Wireless sensor networks (WSNs) have also emerged as a promising solution for real-time pipeline condition monitoring, corrosion detection, and gas leakage detection. Recent studies have focused on WSN-based systems for monitoring anomalous events across upstream, midstream, and downstream sectors [25,26], outlining the deployment requirements and industrial challenges associated with these systems.

The adoption of digital twining, cloud and edge computing in OGI has been explored to further enhance monitoring capabilities. However, implementation has been slow due to high costs and persistent security concerns [15,27]. From a manufacturing perspective, solutions such as composite flexible pipelines have been introduced [8] to prevent leaks, though these approaches are often cost-prohibitive due to the use of expensive materials. Ultimately, a fundamental challenge remains: many gases are invisible to the naked eye, complicating detection and necessitating the development of more advanced, reliable, and non-invasive monitoring technologies.

The aforementioned discussion highlights the limitations of traditional methods, particularly their high-maintenance costs, reliance on specialized equipment and materials, and sensitivity to environmental factors. These challenges indicate the need for more adaptive and intelligent solutions. ML and DL algorithms offer a transformative approach by enabling automated, accurate, and real-time detection of gas leaks using standard RGB data. Such models are capable of learning complex correlations between visible cues and thermal signatures, providing a scalable and cost-effective alternative to conventional techniques. Subsequent subsections discuss the SOTA ML and DL methods used for gas leak detection.

2.2. ML-Based Methods

The integration of multiple inspection techniques with machine learning (ML) algorithms has demonstrated improved performance for defect detection in OGI pipelines which are a primary cause of gas leakage [8,28]. A wide range of ML algorithms, including support vector machines (SVMs), random forests, and hybrid-ML algorithms, have been successfully applied to pipeline leakage detection and corrosion prediction [10]. Recent studies highlight the potential of combining different ML algorithms to improve accuracy and robustness [29]. In [30,31], authors reviews the application of unsupervised machine learning techniques in active infrared thermography (AIRT) for defect detection in composite materials, focusing on image denoising, segmentation, and other post-processing tasks. It highlights recent advances in models for thermogram analysis and provides guidance for researchers entering the field. Advanced feature extraction techniques are also evolving, with manifold learning-based approaches proposed for capturing complex, non-linear relationships in pipeline data. These methods have shown promise in improving the accuracy and reliability of leak detection models [6]. For instance, CH4 leakages from oil wells are now classified into categories such as active, plugged and abandoned, and inactive wells [32]. However, current supervised learning methods often struggle with cross-domain generalization and limited availability of labeled fault samples. To address this, metric learning with pseudo-label strategies has demonstrated improved data utilization and adaptability in challenging scenarios [33].

The aforementioned ML-based approaches have been extended to mobile applications. For example, an improved MobileNetV2 model has been developed to detect pipeline leaks by analyzing hammering sounds recorded via smartphones [34]. This approach leverages the Mel-spectrogram features for early detection of small leakages from underground storage tanks (USTs). One notable approach is digital twinning [15], which integrates ground system data with cloud-based processing for real-time gas leak monitoring. However, cloud-related data processing delays can hinder the real-time effectiveness of such systems.

Despite the progress achieved with ML-based algorithms, these methods depend heavily on well-annotated data, which is often scarce or impractical to obtain in real-world OGI fields. Additionally, traditional ML approaches typically rely on hand-crafted features or shallow models, which struggle to capture the complex, non-linear relationships inherent in pipeline sensor and image data. Most existing ML techniques are primarily applied to numerical or acoustic signals, with limited capability for advanced visualization or direct image-based leak localization. As a result, these methods fail to detect or accurately localize gas leaks that do not produce strong sensor signals or occur in visually complex, noisy, or data-scarce environments [10,35].

2.3. DL-Based Methods

The aforementioned discussion highlights the limitations of ML-based techniques, DL methods have emerged as efficient and adaptive alternatives for gas leakage detection in OGI. Unlike traditional ML approaches, which often depend on manual feature engineering and struggle to capture complex, non-linear relationships in pipeline data, DL models—particularly those leveraging advanced neural network architectures—can automatically learn hierarchical representations directly from raw data, including images and time-series data. This capability enables more robust, scalable, and automated leak detection and localization, even in challenging scenarios where ML techniques fall short [36,37].

Recent years have seen a surge in the adoption of data-driven, AI-based techniques for pipeline monitoring. Deep neural networks such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), including Long Short-Term Memory (LSTM) models [38], have demonstrated strong performance in analyzing both numerical and virtual-sensor data. For example, hybrid frameworks combining 1D-CNNs with LSTMs have been proposed to exploit both spatial and temporal features in pipeline monitoring, enabling automatic extraction of relevant features from complex, multi-dimensional data streams [39,40]. Semi-supervised approaches, such as improved LSTM-autoencoders (LSTM-AE) coupled with one-class Support Vector Machines (OCSVM), have addressed the challenge of limited labeled data by learning normal operational patterns and flagging deviations as potential leaks [29]. Field validations have shown the effectiveness of these models for real-time detection and classification of pipeline conditions [41]. Furthermore, advanced architectures that integrate CNNs, bi-directional LSTMs (BiLSTM), and multi-head attention mechanisms have been developed to capture both local and long-term dependencies in time-series data, further improving anomaly detection accuracy [14].

Deep learning models typically require large volumes of high-quality, labeled data for effective training, which is often scarce in industrial OGI applications. Model validation and generalization remain difficult, especially when data is limited or highly variable due to changing pipeline conditions or sensor noise. Moreover, many existing DL approaches are primarily focused on numerical or acoustic data, with relatively limited research dedicated to leveraging image-based or visualization techniques for tracking invisible gas leaks. This restricts the ability to detect and localize leaks that do not produce strong sensor signals, particularly in visually complex or data-constrained environments [8,29].

These limitations motivate the need for innovative, data-efficient, and non-invasive solutions that can leverage standard RGB images for robust gas leakage detection. Generative deep learning models, such as Generative Adversarial Networks (GANs) [42], offer a promising direction by learning complex correlations between visible features around leakage points and their corresponding thermal signatures. By generating synthetic thermal images from RGB inputs, these models can make invisible gas leaks perceptible and actionable, enabling continuous, low-cost, and real-time monitoring of pipeline networks. In this context, we propose an efficient alternative for gas leak monitoring using a digital twin-based generative AI model that synthesizes thermal images to detect leakage spots in pipelines. The Multi-Scale Dual Discriminator Generative Adversarial Network (MSDD-GAN) is designed to map ice by-products accumulated at leakage regions to their respective gas volumes, effectively serving as a digital gas detector. By incorporating multi-scale inputs and dual discriminators, the MSDD-GAN achieves high accuracy in both thermal image generation and leak localization, even with limited training data. This approach combines the strengths of advanced AI techniques with thermal imaging, offering a more efficient and accurate method for detecting and visualizing gas leaks in pipelines. Leveraging AI-generated thermal images as a digital twin, our method addresses the shortcomings of traditional and existing DL-based detection techniques, providing a comprehensive and scalable solution for pipeline integrity management in the OGI field.

3. Problem Statement

In OGI, the use of image-based models for gas leakage detection are still evolving. As we know, thermal cameras that can detect gases, are expensive. Therefore, this paper focuses on detecting gas leakage spots in pipelines by generating thermal images from RGB images. For this purpose, a GenAI model is proposed, which could transform regular CCTV camera images into thermal images, and allow localization of gas leakage spots. This method expands coverage beyond traditional handheld thermal cameras, leveraging existing CCTV networks to detect leaks in areas where conventional methods are ineffective.

To optimize computational resources, we incorporate turbo processing, as shown in Figure 1. Since gas leakage is not a continuous event, the incorporation of turbo processing reduces computational complexity by processing the CCTV RGB-images only when triggered by the SCADA system. The SCADA system will trigger our proposed GenAI model upon receiving an indication of potential leaks based on real-time data of critical parameters such as pressure and temperature. This comprehensive system offers an efficient solution for detecting gas leaks in extensive pipeline networks, addressing scenarios where traditional methods may prove inadequate.

4. Experimental Dataset

Gas pipeline leaks are a significant concern, but they can only be detected by OGI professionals who specialize in identifying potential issues in pipeline infrastructure. Several signs commonly indicate the presence of a gas leak. One of the most noticeable signs is the accumulation of ice around the leaking area, which forms due to the rapid expansion of gas and its interaction with surrounding moisture in cold temperatures. Another indication is the appearance of water droplets around the joints of the pipeline, which occur as a result of condensation from gas leakage. In some cases, oil spots may also emerge around the pipeline joints, indicating leaks at these specific locations.

Despite the importance of gas leakage detection in OGI, there is a notable scarcity of publicly available datasets in this field. This poses significant challenges for researchers working on AI-based leakage detection models. The sensitive nature of industrial operations and potential safety concerns often restrict the sharing of data. To address this limitation, we have collected the datasets in two different environments, D1: laboratory and D2: industrial. The following subsections provide descriptions of these datasets.

4.1. Laboratory Data (D1)

This data is collected at the Lab of Universidad Carlos III de Madrid (UC3M) in Spain, which specializes in advanced sensor technology. Thermal images were captured using FLIR E 96 thermal camera while CO2 gas was released through a hose mounted on a stand in an outdoor area, as shown in Figure 2. The objective was to capture leakage that may occur at various time intervals. As the gas flows from the cylinder through the hose, the experiment aims to monitor both the gas propagation within the hose and the area around the leakage. Given that gas leakage is often associated with ice formation as a by-product, the experiment also seeks to explore the correlation between the quantity of ice formed and the volume of gas leaked.

During the experiment, RGB images along with thermal images were captured to observe the gas flow inside the hose at different time slots, synchronously. Figure 3 provides some of the examples of RGB-thermal image pairs. These images illustrate the relationship between ice formation and gas propagation, as time passes. The initial dataset comprises 238 images, which were extracted at a sampling rate of 15 frames/s.

The dataset consists of two distinct phases of the leakage; (i) before the leakage occurs (when no gas is present) and (ii) during gas propagation within the hose. The dataset is divided into three subsets; training, validation, and testing set with a split-ratio of 60:20:20. This split-ratio ensures that the model is trained on pre-leak and early-leak conditions, while being tested on more advanced stages of gas leakage, thus evaluating its ability to generalize to different leak scenarios. The collected dataset can be assessed via the following link: https://github.com/saifkhazraji/thermal_images_gas_detection.git (accessed on 2 September 2025).

4.2. Industrial Data (D2)

Another set of data is collected in the industrial environment of the Halfaya Oil and Gas Field in Iraq; some examples of the data are shown in Figure 4. This dataset provides clear examples of gas leakage indicators that have been reported in the field. Figure 4 depicts a leakage scenario, highlighting the accumulation of ice and visible water droplets around the affected pipeline joints. These images were captured from the site, recorded during one of their routine pipeline investigation using FLIR thermal cameras. The total number of collected images is 167 RGB-thermal pairs which were extracted from the video at a sampling rate of 15 frames/s.

This figure highlights the importance of regular monitoring and maintenance of gas pipelines, especially in areas prone to environmental challenges such as temperature fluctuations and moisture accumulation. Timely identification and remediation of these signs can significantly mitigate the risk of more severe pipeline damage or hazardous leaks, which ultimately enhancing safety and operational integrity.

5. Proposed Methodology

The proposed methodology is organized into the following subsections: pre-processing steps, including data augmentation and attention-guided masking; a modified SOTA model of cGANs to generate thermal images, the proposed MSDD-GAN model and its performance comparison with cGANs; and finally, discussion of the employed performance metrics for this work.

5.1. Data Augmentation

In this study, data augmentation was applied to both laboratory and industrial datasets to maximize their utility.

• Laboratory data (D1): The data includes 238 RGB-thermal image pairs that were augmented to create 560 pairs by rotating, resizing, and adding varying levels of noise. This data augmentation allows the model to generalize better and perform effectively even with noisy data.
• Industrial data (D2): Similarly, the images collected from the Halfaya Oil and Gas Field were augmented by rotating, resizing, and adding varying level of noise, resulting in a total of 560 RGB-thermal image pairs. This process not only increases the volume of data but also improves the model’s robustness by exposing it to a wider variety of scenarios, thereby enhancing its ability to detect gas leakage under diverse conditions.

5.2. Attention-Guided Masking (AttMask)

In this work, we introduce a novel binary segmentation masking process within our model’s framework to generate thermal images for gas leakage detection. The AttMask strategy provides a saliency map to guide our model to focus on specific informative areas of the image. The focus of this method is on accurately identifying and verifying the location of gas leaks by introducing a secondary discriminator responsible for detecting the leakage spots and ensuring their precise positioning in the generated thermal images.

To achieve this, we implement an attention-guided masking process wherein only predefined positions of potential gas leaks are filtered from both target and generated thermal images. This filtering is implemented using a circular Region of Interest (ROI) mask applied to both images. The ROI ensures that the evaluation process focuses solely on critical regions where gas leakage occurs, as shown in Figure 5. The mask is created by determining the centroid coordinates of the gas leakage location in the target thermal image using the impixelinfo tool of MATLAB which provides precise information about the pixel values and coordinates in the target thermal image. This precise localization enables pixel-by-pixel comparison between the corresponding regions of the generated and target thermal images, which allowing both accurate gas leakage detection and localization.

5.3. Conditional GANs (cGANs)

The first GAN model for this kind of application was introduced in [43], which provides the structure and implementation details. GANs have since been applied to various tasks, such as image-to-image translation [44], where RGB images are transformed into other types of images. For our work, this transformation is crucial to identify potential areas of gas leakage. This subsection provide a brief background of cGANs by highlighting the modifications performed to generate thermal images for gas leakage detection before explaining the MSDD-GAN in subsequent subsection.

cGANs plays a central role in the proposed model and underlying the relationship between the thermal images, indicating gas leakage and the corresponding RGB images showing ice formations around the leakage points. This correlation arises because leaking gas lowers the surrounding temperature, which causes the moisture to freeze and accumulate as ice at the leakage point. An example RGB images with ice are shown in Figure 6. Ice formation around a leak serves as a visual indicator of leakage intensity: the more ice observed around a spot, the greater the likelihood of a gas leakage. Our model learns this correlation and is capable of generating thermal images from RGB images.

Inspired by GANs, cGANs architecture comprises two core components: a generator $\left(G\right)$ and a discriminator $\left(\mathcal{D}\right)$. The generator G synthesizes thermal images from input RGB images $\left(B\right)$, while the discriminator evaluates whether a thermal image is real $\left(C\right)$ or generated (C′). During training, G learns to generate realistic thermal images that deceive $\mathcal{D}$, while $\mathcal{D}$ improves the ability to distinguish between real and synthetic images. If $\mathcal{D}$ consistently identifies the generated image as fake $\mathcal{D}(C\prime |B)\approx 0$, the generator update its parameters to improve realism until equilibrium is achieved.

When an indication signal from SCADA is received, an RGB image B, which depicts the visible ice accumulation around pipeline leaks is fed as an input to G, it generates a thermal image, s.t., $C\prime =G\left(B\right)$, highlighting the areas of potential gas leakage. The $\mathcal{D}$ receives a pair of images as input: a real thermal image $\left(C\right)$ and the generated thermal image $(C\prime )$, where $C\prime$ is generated by G. The $\mathcal{D}$ then computes the probability indicating whether the thermal image in the pair is real or fake, given the input RGB image, i.e., $\mathcal{D}\left(C\right|B)$ and $\mathcal{D}(C\prime |B)$, respectively. In the processing block (from Figure 7), the discriminator then evaluates the accuracy of $C\prime$ generated by G. Figure 7 illustrates the training process of cGANs to learn the optimal thermal image generation from the RGB image. During training, G and $\mathcal{D}$ engage in a minimax game: the generator seeks to produce thermal images that are indistinguishable from real ones, while the discriminator strives to correctly classify each pair as real or fake. The subsequent blocks in Figure 7 calculate the adversarial loss for the cGAN passes through two loss functions. The discriminator loss $\left({\mathcal{L}}_{\mathcal{D}}\right)$ combines binary cross-entropy terms for real and fake classifications as follows:

$${\mathcal{L}}_{\mathcal{D}}=-{\mathbb{E}}_{B,C}\left[log\mathcal{D}\left(C\right|B)\right]-{\mathbb{E}}_B[log(1-\mathcal{D}(C\prime |B)],$$

(1)

where $\mathbb{E}$ denotes the expected value. The generator loss $\left({\mathcal{L}}_G\right)$ focuses on making generated images appear realistic to $\mathcal{D}$:

$${\mathcal{L}}_G=-{\mathbb{E}}_B\left[log\mathcal{D}(C\prime |B)\right].$$

(2)

Once the loss computed, the validation block compares the generated thermal image with the target thermal images by using the L1 reconstruction loss as follows:

$${\mathcal{L}}_{recon}={\mathbb{E}}_{B,C}\left[\right||C-{G\left(B\right)\left|\right|}_1].$$

(3)

The proposed model employs L1 reconstruction loss because the objective is to generate thermal images that preserve edge details and produce more realistic results. In contrast, L2 loss tends to overly smooth or blur the outputs due to its averaging effects. The total generator loss then becomes

$${\mathcal{L}}_G^{total}={\mathcal{L}}_G+\lambda {\mathcal{L}}_{recon},$$

(4)

where $\lambda$ controls the reconstruction weight. Total loss in cGANs ${\mathcal{L}}_{\mathrm{total}}$ is the summation of generator and discriminator losses, with the objective of minimizing the generator loss ${\mathcal{L}}_G$ and maximizing the discriminator loss ${\mathcal{L}}_{\mathcal{D}}$. Mathematically, it can be written as follows:

$${\mathcal{L}}_{\mathrm{total}}=\arg \underset{G}{min}\underset{D}{max}\left[{\mathcal{L}}_D+{\mathcal{L}}_G\right].$$

(5)

Minimizing generator loss enables the model to generate more realistic thermal images that can fool the discriminator and can accurately indicate the gas leakage at ice-accumulating spots, whereas maximizing the discriminator loss improves the ability to better distinguish the real and generated thermal images.

Training proceeds until equilibrium is reached, where the discriminator cannot reliably distinguish real from generated images. Upon convergence, the process terminates successfully, providing a validated thermal image $(C^{\prime }=G\left(B\right))$ that highlights probable gas leakage locations correlated with the RGB input’s ice accumulation patterns.

5.4. Multi-Scale Dual Discriminator Generative Adversarial Network (MSDD-GAN)

The proposed MSDD-GAN model is designed to generate thermal images from RGB inputs for gas leakage detection, by taking into account the foundational principles of cGAN. While traditional cGANs require intensive training with large dataset; on the contrary MSDD-GAN is capable of providing reliable results even when only small datasets are available. The proposed architecture of MSDD-GAN is illustrated in Figure 8.

Both the generator and discriminator components of the proposed model have been modified. On the generator side, the noise image A and multi-scale resolution RGB images B are provided as inputs. Specifically, image pairs at three different resolutions $256\times 256$, $192\times 192$, and $64\times 64$ pixels-are used, representing $100\%$, $75\%$, and $25\%$ of the original image resolution, respectively. This multi-scale approach allows the generator to capture features across different levels of features, which improve its ability to generate realistic thermal images even with the small dataset. This multi-scale input strategy can be mathematically formalized as

$$C\prime =G_{\mathrm{multi-scale}}(B,A),$$

(6)

where $G_{\mathrm{multi-scale}}$ comprises of the network layers ${\mathcal{F}}_G$, detailed in

Table 2. Network topography of proposed MSDD-GAN model.

Models	Layers	Activation	Parameters
Generator	1. Input layer	A and C	Image sizes: 256 × 256, 192 × 192, 64 × 64
	2. Encoder with 4 Conv2D	LeakyRelu: $\alpha$ = 0.2	Kernel: $4\times 4$, stride: 2, padding: same Filter sizes for 4 layers: 64, 128, 256, 512
	3. Decoder with 4 Conv2DTranspose	Relu	Kernel: $4\times 4$, stride: 2, padding: same Filter sizes for 4 layers: 512, 256, 128, 64
Discriminator	1. Input layer	C and C’	Image sizes: 256 × 256
	2. Conv2D	leakyRelu: $\alpha$ = 0.2	Filter: 64, kernel: $4\times 4$, stride: 2
	3. Conv2D	LeakyRelu: $\alpha$ = 0.2	Filter: 128, kernel: $4\times 4$, stride: 2
	4. BatchNorm	—	[0, 1]
	5. Conv2D	LeakyRelu: $\alpha$ = 0.2	Filter: 256, kernel: $4\times 4$, stride: 2
	6. BatchNorm	—	[0, 1]
	7. Conv2D	LeakyRelu: $\alpha$ = 0.2	Filter size: 512, kernel: $4\times 4$, padding: same
	8. BatchNorm	—	[0, 1]
	9. Output layer	sigmoid	Single output per batch

, to generate the thermal image $C\prime$.

The proposed MSDD-GAN architecture incorporates a dual discriminator approach to improve the accuracy of thermal image generation and gas leakage localization. The first discriminator $\left({\mathcal{D}}_1\right)$, is responsible for validating the correlation between the target thermal image $\left(C\right)$ and the generated thermal image $\left(C^{\prime }\right)$. This process can be expressed as

$${\mathcal{D}}_1={\mathcal{F}}_{\mathcal{D}}(C,C^{\prime }),$$

(7)

where ${\mathcal{F}}_{\mathcal{D}}$ represents the network layers of the discriminator, as described in Table 2. In contrast, the second discriminator ${\mathcal{D}}_2$, focuses on verifying the generator’s output with a binary mask $\left(M\right)$, ensuring that the generator accurately localizes the gas leakage. The binary mask, obtained from AttMask (described in Section 5.2) allows ${\mathcal{D}}_2$ to focus attention on gas leakage spots and isolate them as ROI. The formulation for ${\mathcal{D}}_2$ is given by:

$${\mathcal{D}}_2={\mathcal{F}}_{\mathcal{D}}\left(M\left(C\right),M\left(C^{\prime }\right)\right).$$

(8)

Both discriminators share the same architecture; however, they process different inputs. The detailed network topology of MSDD-GAN, along with the encoder-decoder of G and $\mathcal{D}$ is presented in Table 2. The ${\mathcal{D}}_2$ plays a pivotal role in accurately validating the location of gas leakage by comparing the predefined leakage locations in the masked target images with their corresponding locations in the generated images and computing the loss (${\mathcal{L}}_{\mathcal{D}2}$). This comparison ultimately improves the accuracy for gas leakage detection and localization. The total discriminator loss $\left({\mathcal{L}}_{\mathcal{D}}\right)$ is computed as follows:

$${\mathcal{L}}_{\mathcal{D}}={\mathcal{L}}_{\mathcal{D}1}+{\mathcal{L}}_{\mathcal{D}2},$$

(9)

where ${\mathcal{L}}_{\mathcal{D}1}$ and ${\mathcal{L}}_{\mathcal{D}2}$ calculated as

$${\mathcal{L}}_{\mathcal{D}1}={\mathbb{E}}_{B,C}\left[log\left({\mathcal{D}}_1\left(C\right|B)\right)\right]+{\mathbb{E}}_B\left[log(1-{\mathcal{D}}_1\left(C^{\prime }\right|B))\right],$$

$${\mathcal{L}}_{\mathcal{D}2}={\mathbb{E}}_{B,C}\left[log\left({\mathcal{D}}_2\left(C\right|B)\right)\right]+{\mathbb{E}}_B\left[log(1-{\mathcal{D}}_2\left(C^{\prime }\right|B))\right].$$

The generator loss ${\mathcal{L}}_G$ is computed similarly to the cGAN, as described in Equations (2)–(4). The overall loss ${\mathcal{L}}_{total}$ is computed as described in Equation (5). The dual-discriminator structure of the proposed MSDD-GAN model significantly improves performance in thermal image generation, which allows to detect and localize the gas leakage from the generated thermal images $C^{\prime }$. Moreover, it effectively addresses the challenges posed by limited training data in this field.

5.5. Performance Metrics

Evaluating image-to-image translation algorithms involves several key metrics, including Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Mean Absolute Error (MAE). These metrics provide a comprehensive assessment of the quality and accuracy of the generated images compared to the ground-truth images.

SSIM evaluates the similarity between two images by considering luminance, contrast, and structure changes. Values range from −1 to 1, with high values indicating greater structural similarity between target C and generated thermal image $C^{\prime }$. Mathematically, SSIM can be defined as follows:

$$\mathrm{SSIM}(C,C^{\prime })={\displaystyle \frac{(2{\mu }_C{\mu }_{C^{\prime }}+{\gamma }_1)(2{\sigma }_{C{C}^{\prime }}+{\gamma }_2)}{({\mu }_C^2+{\mu }_{C^{\prime }}^2+{\gamma }_1)({\sigma }_C^2+{\sigma }_{C^{\prime }}^2+{\gamma }_2)}},$$

(10)

where $\{{\mu }_C$, ${\sigma }_C^2\}$ and $\{{\mu }_{C^{\prime }}$, ${\sigma }_{C^{\prime }}^2\}$ are the mean and variance of the target C and generated thermal images $C^{\prime }$, respectively. ${\sigma }_{C{C}^{\prime }}$ defines the covariance. ${\gamma }_1$ and ${\gamma }_2$ are small constants that are included to avoid division by zero.

PSNR measures the quality of generated thermal images wrt target thermal images, higher PSNR value indicates $C^{\prime }$ is similar to C, suggesting better performance of our model. PSNR can be defined as follows:

$$\mathrm{PSNR}=20·{log}_{10}\left({\displaystyle \frac{{\mathrm{MAX}}_I}{\sqrt{\mathrm{MSE}}}}\right),$$

(11)

where $MA{X}_I$ is the maximum possible pixel value in thermal images (e.g., 255 for 8-bit images) and Mean Squared Error (MSE) is calculated as follows:

$$MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{(C_i-C_i^{\prime })}^2,$$

(12)

where n is the dimension of the thermal images. Mean Absolute Error (MAE) directly measures the average error magnitude between C and $C^{\prime }$ as follows:

$$\mathrm{MAE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|C_i-C_i^{\prime }\right|.$$

(13)

6. Results and Discussions

The results section presents the implementation details of the proposed model, along with both qualitative and quantitative evaluations for a comprehensive analysis. In addition, a comparison with state-of-the-art GAN models is provided.

6.1. Implementation Details

During the initial training stages ($t\to 0$), the generator loss ${\mathcal{L}}_G$ is notably high, around 50. This high loss is characteristic of GANs models, as the generator starts learning with random initialized weights, producing low-quality generated thermal images that the discriminator can easily distinguish from real samples. As training advances, ${\mathcal{L}}_G$ decreases significantly as shown in Figure 9 and stabilizes between approximately 5 dB and 10 dB. This reduction indicates that the generator is improving its ability to generate more realistic thermal images that challenge the discriminator’s ability to distinguish from real images. The fluctuations in loss reflect the dynamic nature of MSDD-GAN training, where both generator and discriminator continuously adjust weights to improve output.

Table 3. Summary of MSDD-GAN training parameters.

Parameter	Configurations
Epochs	100
# (training samples)	560
Batch size	1
Loss target	0.01 dB
# (generators)	1
# (discriminators)	2
Total training time	37 h
Total training steps	56,000
Hardware description	GPU: AMD Ryzen 7 5800H with Radeon Graphics (3.20 GHz) RAM: 32 GB DDR5

presents a summary of the model’s training parameters. Figure 10 shows the generated thermal images obtained at different stages of the training process.

MATLAB was used during the training phase to generate AttMask images leveraging the impixelinfo (see Section 5.2), which were then provided as input to the GAN models (e.g., MSDD-GAN and other GAN variants) implemented in Python. This required manual coordination between the two environments, as the training process relied on external model architectures outside MATLAB. In contrast, the testing phase was fully automated: the pre-trained weights were loaded, and inference was performed using an end-to-end pipeline without the need for manual intervention. Thus, while training involved interaction between MATLAB and Python environments, the testing was entirely automatic.

6.2. Qualitative Results

The test set consists of a video sequence provided as input to the model. Frames are extracted at a fixed frame rate and processed through the attention-guided masking module, after which the trained weights of the GAN models are employed to make inference. Evaluation of the model’s performance relies on three key metrics, i.e., SSIM, PSNR, and Accuracy. The efficacy of the proposed MSDD-GAN model in successfully generating thermal images from RGB images is visually demonstrated in Figure 11 and Figure 12. These figures showcase the model’s ability to generate thermal images that closely match the target images in terms of gas leakage representation.

6.3. Quantitative Results

Quantitatively, the MSDD-GAN achieved an average accuracy that is 3.78% higher than the baseline cGAN (see,

Table 4. Quantitative comparison of the proposed MSDD-GAN and baseline cGAN.

Dataset	Model	ACC (%)	PSNR (dB)	SSIM	MAE
D1	cGAN	78.64	53.35	0.688	0.049
	MSDD-GAN	81.60	59.98	0.800	0.035
	Pix2Pix GAN	48.31	21.09	0.687	0.145
	CycleGAN	49.42	12.41	0.308	0.391
D2	cGAN	72.96	48.04	0.640	0.057
	MSDD-GAN	75.20	54.11	0.753	0.043

and Figure 13). This performance improvement is particularly notable given the model’s improved resilience to noisy images. These results collectively highlight the MSDD-GAN’s robustness and superior performance in thermal image generation for gas leakage detection, even under challenging input conditions.

The MSDD-GAN demonstrates superior performance with a median SSIM (Equation (10)) of 0.801, compared to cGAN’s median SSIM of 0.72 for dataset collected in laboratory environment D1 (Figure 14). This higher median and broader range of SSIM values for the MSDD-GAN indicate that the generated thermal images have higher structural similarity with target thermal images The variability in SSIM scores suggests that while MSDD-GAN can produce images of better quality, it also encounters variability in performance across diverse samples. In contrast, cGAN consistently generates low-quality images but with less variability.

PSNR (from Equation (11)) is a measure often used to quantify image quality, where higher PSNR values indicate better accuracy and less distortion. MSDD-GAN consistently outperforms cGAN with higher PSNR values (see Figure 15). Notably, MSDD-GAN attains a median PSNR of 60.5dB, significantly higher than the cGAN value of 53.3dB. This indicates that MSDD-GAN generates thermal images with superior quality, lower reconstruction error and improve the preservation of thermal image characteristics.

Accuracy of generated images is calculated by comparing them to target images. Pixel errors are evaluated by identifying differences between generated and target images. Pixels in a generated image that differ from the corresponding pixels in the target image are marked as errors. Accuracy is then determined by the proportion of zero values in the error matrix, considering the total number of pixels in the image. Figure 16 illustrates the accuracy scores with MSDD-GAN outperforming cGAN, achieving an accuracy of 84.6%.

It is observed that the model’s ability to generate thermal images using D2, i.e., industrial dataset, is slightly less than with D1, i.e., the laboratory dataset. This is evident from the performance metrics demonstrated in above figures. The discrepancy is attributed to the nature of D2, which contains a greater variety of pipelines than D1, Additionally, D2 has fewer training samples, which may impact the model’s performance.

6.4. Additional Comparison

Further comparative results with Pix2Pix GAN [45] and CycleGAN [46] models show that baseline Pix2Pix GAN yielded lower SSIM (0.6876) and PSNR (21.09 dB) demonstrating inferior performance, especially in gas leak region generation. CycleGAN attained even lower SSIM (0.3084) and PSNR (12.41 dB), generating thermal images with reasonable background quality but poor localization in gad leakage areas. This highlights the necessity for a masked loss function to direct the model’s focus and penalize error more strongly in relevant regions. Figure 17 and Figure 18 present the qualitative results corresponding to the Pix2Pix GAN [45] and CycleGAN, respectively. These images show that the state-of-the-art models fail to accurately generate thermal images, particularly in the gas leakage areas.

6.5. Robustness Test

A comprehensive robustness analysis is performed to evaluate the performance of the model with varying noise levels, assessing its ability to generate high-quality outputs despite input degradation. The test protocol involves distorting input images with two distinct noise levels prior to model processing, i.e., 3 dB (low noise) and 7 dB (moderate noise). This setup simulates real-world scenarios where thermal images may be subject to various forms of noise or distortion. The model’s performance was quantitatively monitored using SSIM. SSIM values were computed for each noise level, and scatter plots were generated to visualize the distribution of these values which provides insights into the consistency and stability of the model’s performance.

The scatter plots in Figure 19a and Figure 19b display the SSIM behavior of test images with added noise of 3 dB and 7 dB, respectively. The results demonstrate that the cGAN performance (see Figure 19) degrades as the noise level increases. In contrast, Figure 20 shows that the MSDD-GAN exhibits greater noise immunity, maintaining approximately constant SSIM values even with increased noise levels. For this analysis, 3 dB and 7 dB noise was added in the images, with the corresponding SSIM values depicted in Figure 20a and Figure 20b respectively. At 3 dB and 7dB noise levels, the SSIM values are 0.795 and 0.790, respectively, compared to an SSIM of 0.800 for MSDD-GAN without noise. This minimal decline in SSIM as noise increases reflects the robustness of the proposed model to noise, which is why Figure 20a,b display very similar plots across various noise levels.

7. Conclusions

This work contributes significantly to the field of OGI by integrating a GenAI concept for thermal images generation using RGB images. The proposed MSDD-GAN model, utilizing a dual discriminators architecture, demonstrates significant improvements over baseline method of cGAN in producing high-quality thermal images. Moreover, SOTA cGAN model is modified to generate the thermal images and provide its comparison with proposed MSDD-GAN model. Our model trained with target thermal and RGB image pairs, successfully understands the correlation between indications of gas leakage from RGB images such as ice formation. Performance metrics for the MSDD-GAN are promising, with mean values of 81.6% accuracy, 59.97 dB PSNR, and 0.799 SSIM across test dataset collected in laboratory environment. Similarly, it shows superior performance on the industrial dataset. Moreover, robustness test was conducted to analyze the model’s performance with different noise levels.

The proposed model acts as a digital platform and offers a cost-effective, continuous monitoring solution for gas leaks, enhancing both safety measures and economic efficiency in industrial settings. In conclusion, MSDD-GAN represents a substantial advancement in thermal imaging technology for gas leakage detection. Its ability to generate thermal images from RGB images opens new avenues for widespread industrial implementation. The proposed method is currently limited to detecting and localizing gas leaks only after leakage indications become visible in RGB images. In the future, however, this work could be extended to predict leaks before any visual indicators appear by integrating GenAI models with the traditional sensors already embedded in the OGI system. Moreover, future work could focus on expanding the model’s applicability to diverse environmental conditions and exploring its potential in other industrial safety applications.