A Novel Oil Spill Dataset Augmentation Framework Using Object Extraction and Image Blending Techniques

Abstract

Oil spills pose significant threats to marine and coastal ecosystems, biodiversity and local economies, necessitating efficient and accurate detection systems. Traditional detection methods, such as manual inspection and satellite imaging, are often resource-intensive and time consuming. This study addresses these challenges by developing a novel approach to enhance the quality and diversity of oil spill datasets. Several studies have mentioned that the quality and size of a dataset is crucial for training robust vision-based deep learning models. The proposed methodology combines advanced object extraction techniques with traditional data augmentation strategies to generate high quality and realistic oil spill images under various oceanic conditions. A key innovation in this work is the application of image blending techniques, which ensure seamless integration of target oil spill features into diverse environmental ocean contexts. To facilitate accessibility and usability, a Gradio-based web application was developed, featuring a user-friendly interface that allows users to input target and source images, customize augmentation parameters, and execute the augmentation process effectively. By enriching oil spill datasets with realistic and varied scenarios, this research aimed to improve the generalizability and accuracy of deep learning models for oil spill detection. For this, we proposed three key approaches, including oil spill dataset creation from an internet source, labeled oil spill regions extracted for blending with a background image, and the creation of a Gradio web application for simplifying the oil spill dataset generation process.

1. Introduction

Detecting oil spills is critically important due to their devastating environmental, economic, and social consequences. Oil spills can cause severe damage to the marine and coastal ecosystems, threatening biodiversity, aquatic life, and human livelihoods. When oil spills occur, they release toxic substances that can suffocate marine species, contaminate food chains, and cause long-lasting distributions to habitats. Additionally, oil spills impact local economies, particularly those dependent on fishing and coastal resources. Cleanup and recovery efforts are often costly and time consuming, and in many cases, the full ecological restoration may take a long time. In this context, the ability to detect oil spills swiftly and accurately using a vision-based deep learning (DL) model has become increasingly significant. Traditional methods, such as manual inspection and satellite imaging, can be resource intensive and take time. Automated detection systems based on computer vision have the potential to significantly reduce the response time by providing real-time monitoring and analysis. Vision-based deep learning models can analyze vast amounts of visual data from aerial imagery, satellites, and other sensors to identify oil spill patterns, estimate their size, and track their spreads. This capability not only enhances the effectiveness of early intervention but also helps to minimize the impact by enabling quicker and more targeted cleanup efforts. However, the success of artificial intelligence (AI)-based models hinges on the availability of large, high-quality datasets. For any deep learning model to achieve high accuracy, it needs to be trained on diverse and comprehensive datasets that represent various oil spill scenarios under different environmental conditions. Unfortunately, existing oil spill datasets are often limited in size and variety, which can result in models that fail to generalize well in real-world conditions. For example, oil spills occurring in calm waters may differ significantly in appearance from ones happening during rough seas or in polar regions. Without enough data covering these scenarios, models may misidentify oil spills or miss them entirely. A well-structured and expansive dataset is critical to overcoming these challenges. Larger datasets enable models to learn the nuanced differences between oil spills and similar phenomena, such as algae blooms or shadowed areas in water bodies. In particular, larger datasets that include variations in oil spill size, shape, color, and environmental conditions such as lighting or wave patterns, which provide the necessary diversity for robust model training.

Moreover, augmenting existing data with techniques like synthetic data generation, or blending real-world data with simulated spill environments can further enhance the dataset’s richness, enabling the development of more accurate and reliable AI-based detection systems. However, the total volume of oil spilled into the environment in 2023, which was estimated to be around 2000 tons, demonstrates the urgency of enhancing these systems. Looking back to history, there were several severe oil spill related disasters. For example, on 10 February 1983, a tanker collided with an Iranian oil platform situated above the Nowruz oil field in the northern Persian Gulf, resulting in significant structural damage [1]. The impact caused the platform to tilt at a 45 degree angle and over time corrosion and wave action led to the collapse of the platform, rupturing the wellhead. As a result, approximately 1500 barrels of oil (63,000 gallons) were spilled into the Persian Gulf each day until the leak was finally capped in September 1983 [1]. Similarly, in terms of land-based spills, the largest in history took place in Uzbekistan on 2 March 1992. A blowout at a well in the Fergana Valley released a massive quantity of oil. As estimated 88 million gallons of oil gushed out into the surrounding area, causing an uncontrollable fire that burned for two months until the pressure subsided. With regards to marine oil spill incidents, the Deepwater Horizon incident, which occurred in 2010 in the Gulf of Mexico, stands as the most notorious marine oil spill in history. In this incident, an explosion led to the release of an estimated 4.9 million barrels of oil (about 700,000 tons) into the ocean, causing unprecedented environmental damage. These incidents highlight the global risk posed by oil spills, both on land and sea. From the Persian Gulf, to Uzbekistan and the Gulf of Mexico, oil spills have caused widespread ecological damage and economic loss [1]. Regarding ocean oil spill related disasters, there have been several big accidents. In the case of American history, some major oil spills stand out for their impact on the environment and their scale. In 1969, an offshore platform blowout near Santa Barbara, California, released more than four million gallons of oil into the ocean. Then, in 1989, the Exxon Valdez oil tanker ran aground in Alaska’s Prince William Sound, causing an even larger spill of over 11 million gallons of oil. Each of these disasters marked a significant moment in the history of oil spills in U.S. waters [2].

Currently, two primary approaches are utilized for oil spill detection [3]. The first involves direct surveillance using maritime vehicles [4]. In this method, marine environmental management agencies and coast guard units deploy coastal patrol vessels or strategically place oil sensor buoys to routinely monitor sea conditions [5,6,7,8]. In the event of an oil spill, these patrol vessels provide real-time monitoring and relay crucial information to central command, while sensor buoys transmit data to onshore monitoring stations. But this method faces significant limitations in terms of mobility and the accuracy of detecting and quantifying oil spill incidents. Navigating between oil spill areas, or from the mainland to the spill site, can be time consuming and logistically challenging, particularly for spills located in remote or offshore regions.

Furthermore, onboard visibility is often constrained, making it difficult to effectively observe and cover the entire affected area. The equipment used on ships for measuring the extent of oil spills is often rudimentary, limiting the ability to accurately assess spill size, shape, direction, and composition. Consequently, this approach tends to focus on providing basic information, such as the spill’s location, type of oil involved, and cause of the accident. Nevertheless, this approach falls short in offering detailed insights into the spill area, the specific types of oil present, the distribution and size of oil patches, and the trajectory of the spill. Another drawback is the intermittent nature of patrol operations, which can lead to delays in detecting oil spills, especially those occurring in distant or less monitored areas. These limitations underscore the need for more efficient, accurate, and continuous monitoring systems. The second approach to oil spill detection utilizes remote sensing technologies [3]. This method primarily leverages satellite imagery, radar data, or aerial photographs to monitor and identify oil spills in marine environments [9]. Remote sensing offers the advantage of covering vast oceanic areas with high-frequency, real-time monitoring, making it highly effective for the early detection and continuous surveillance of oil spills. Among these, Synthetic Aperture Radar (SAR) has emerged as one of the most widely used tools for oil spill detection due to its ability to capture high-resolution images even under adverse weather conditions and at night. SAR’s capacity to distinguish oil slicks from surrounding waters has been extensively documented in the literature, further underscoring the pivotal role of remote sensing in modern spill detection. According to the analysis [10], there are several limitation regarding the use of the SAR image application for further processing, which will be described in the next section in detail.

This paper proposed oil spill dataset creation by covering traditional and advanced augmentation methods. Combining various methods significantly increased the oil spill dataset. The main contributions of our study are as follows:

• Collecting oil spill represented images from internet sources because currently there are no open-source, colorful oil spill datasets.
• Labeling and extracting oil spill regions from annotated images and blending them into ocean images.
• Developing a user-friendly Gradio web application for blending extracted oil spill images (target image) with various ocean background images (source image).

In Section 2, we include a comprehensive review of oil spill related research works, datasets, and algorithms, and a comparative analysis of various techniques. Section 3 describes our proposed augmentation techniques, providing equation explanations along with graphical representations of oil spill dataset generation. In Section 4, we discuss our research work and make final conclusions on the oil spill dataset generation process with plans for future directions.

2. Related Works

2.1. SAR-Imaging-Based Research and Its Limitations

The reduction in speckle noise without losing the details of SAR images is mentioned as a difficult task. This is because concrete details about radar speckles the noise effects in SAR imaging, which in turn results in the effect of a granular pattern distribution. Also, more disadvantages of SAR imaging could be mentioned, such as the rough surface scattering or that specular reflection causes no radar to return. SAR images are highly sensitive to surface roughness and background variations, which makes it challenging for vision-based model applications to differentiate oil spills from other marine phenomena, such as algae blooms, low-wind areas (wind slicks), biogenic films, and reflections from vessels. Debris can also resemble oil spills in SAR imagery, which can also lead to a high false-positive rate. Additionally, SAR images do not provide spectral information, as they rely on radar backscatter rather than optical properties. Due to the impact of short-wavelength gravity waves which are generated by local winds and contribute to the dispersion of sea spectrum energy, along with capillary waves which arise from friction and are influenced by wind speed and sea surface conditions [11], the backscattering from the sea surface diminishes. This reduction in backscatter causes oil slicks to manifest as dark spots with intricate patterns in SAR imagery. Consequently, distinguishing between different oil types and lookalikes is difficult.

However, since there are limitations related to SAR imaging plenty of research work has been conducted with SAR imaging, as shown in

Table 1. SAR imaging-based research works.

Satellite Name	Operator	Polarization	References
RADARSAT-1	Canadian Space Agency (CSA)	Single-HH	[12,13,14,15]
RADARSAT-2	Canadian Space Agency (CSA)	Quad	[16,17,18,19,20,21,22]
RISAT-1	India	Quad	[23,24,25]
Kompsat-5	Korea	Dual	[26,27,28]
Sentinel-1	European Space Agency (ESA)	Dual	[28,29,30,31,32]

, and parameters analysis-based research works on SAR imaging, as represented in

Table 2. SAR imaging feature category-based research works.

Feature Category	Feature	References
Geometric(Shape)	Area Perimeter Spreading Shape factor	[33,34,35,36,37]
Statistical	Object standard deviation Object mean value Max contrast Mean border gradient Max gradient	[38,39,40,41,42]
Texture	Contrast Homogeneity Entropy Correlation Dissimilarity	[13,15,39,43]

.

Table 2 below includes key features used in SAR imaging-based research, separated into three main groups of geometric shape, statistical features, and texture features.

Overall, SAR-based research offers robust performance in adverse conditions. However, it has certain limitations, including speckle noise, lower spatial resolution compared to optical imagery, and challenges in interpreting backscatter intensities for complex surfaces. Moreover, colorful imaging provides higher spectral resolutions, enabling more detailed visual analysis and improve feature extraction. The synergy of optical imaging’s high resolution and natural visualization makes it more suitable for applications like fine-grained classification and detailed object detection, but segmentation through its reliance on favorable weather and lighting conditions is a notable limitation. One possible solution here is the generation of oil spill dataset in various ocean weather conditions.

2.2. Commonly Used Machine Learning Methods for Oil Spill Detection

Traditional methods for detecting and monitoring marine oil spills, such as satellite imagery, aerial surveillance, and manual reporting continue to play an essential role [44]. In the past, traditional on-site monitoring methods were commonly employed to detect oil spills. Various machine learning (ML) models have been developed to solve complex classification related issues through recursive training samples [45], such as the Support Vector Machine (SVM) [46], Decision Tree (DT) [47], K-nearest neighbor (KNN) [48], Random Forest (RF) [49], and other ML algorithms [50].

However, this approach presented several challenges, including the risk of direct exposure to hazardous oil substances and other on-site dangers. For example, Zhang et al. [51] conducted a comparison analysis of the three most widely used supervised classifiers, such as Artificial Neural Network (ANN) and SVM, for oil spill classification in the case of SAR images. Since the main application SVMs are designed for are binary classification tasks where they identify the optimal hyperplane, this approach is limited due to its handling of complex nonlinear data. To address this issue, the concept of kernel tricks was introduced which allows SVMs to map input data into a higher-dimensional feature space. This transformation enables the construction of an optimal separating hyperplane that can effectively handle non-linear decision boundaries. Therefore, various kernel functions have been developed to facilitate this, including linear, polynomial, sigmoid, and radial basis function (RBF) kernels. In the context of oil spill classification, RBF and polynomial kernels are widely employed due to their ability to handle the nonlinear patterns inherent in remote sensing and marine environments [20,52].

Careful consideration of these elements is essential when applying SVMs to oil spill detection tasks. Reported accuracies of SVM models in oil spill studies vary, ranging from 71% to 97%, highlighting the method’s potential while also emphasizing the importance of proper model tuning [53].

2.3. State-of-the-Art Methods in Oil Spill Detection

DL models have demonstrated exceptional performance in detecting oil spills from SAR and optimal imagery by automatically extracting discriminative features that can effectively differentiate between oil spills and lookalikes. Unlike traditional methods that often rely on hand-engineered features and case-specific adjustments, DL models offer superior generalization capabilities, making them more adaptable to a wide range of scenarios and conditions. This flexibility addresses the inherent limitations of conventional techniques, which tend to be limited to specific cases or environments. Since 2017, there has been a noticeable surge in the adoption of DL models in oil spill related research.

For instance, Jiaoa et al. [54] employed a deep CNN to improve the accuracy of oil spill detection. Their approach enhanced the standard CNN architecture by introducing an additional convolutional layer, along with a fully connected layer, to improve feature extraction and classification performance. The input data comprised images collected via Unnamed Aerial Vehicles (UAVs), supplemented with metadata such as GPS coordinates, pitch angle, and other relevant parameters, which provided contextual information for better detection accuracy. The study results demonstrated the robustness of the model, achieving an impressive 99.33% accuracy in mAP, highlighting its potential applicability for real-world oil spill detection tasks. Moreover, the researchers addressed the issue of overfitting, a common challenge in DL models, by incorporating techniques such as optimized dropout rates and L2 norm regularization. Similarly, Kerf et al. [55] not only succeeded in identifying oil spills during daylight but also introduced a novel methodology that leveraged both visible and infrared imagery captured by UAVs to facilitate oil spill detection at night. Their experimental setup involved simulating a controlled oil spill in a small but realistic seaport environment, where UAVs equipped with cameras were used to capture both visible and infrared image data. The research focused on training a model by combining various backbone architectures with segmentation networks. After extensive experimentation, the team found that the most effective oil detection model for night-time conditions utilized the MobileNet backbone integrated with the fully convolutional network with an 8 pixel (FCN8) segmentor. This model achieved an impressive 89% mean Intersection over Union (mIoU) in their experimental context. Overall, the study highlighted the significant potential of UAV-based oil spill detection systems, demonstrating that these systems can operate effectively regardless of the time of day. By incorporating both visible and infrared imagery, the methodology holds promise for improving oil spill monitoring in low-light or night-time conditions, expanding the versatility and applicability of UAV technology in environmental disaster management.

The application of remote sensing technology for detecting and monitoring marine oil spills has been extensively documented by several researchers [7] as can be seen in

Table 3. Deep learning model applications in oil spill detection description.

Application	DL Model	Data	Architecture	Data Size	Reference
Segmentation	AEs	SLAR	LSTM/Selectional AE	256 × 256	[24,56]
	GANs	ERS-1/2, ENVISAR ASAR	Adversarial f-divergence	256 × 256	[18]
	CNN	Radarsat-1/2	DeepLabv3+/SegNet	321 × 321/256 × 256	[8,20,57]
Classification	AEs	AVIRIS/RADARSAT-2	Stacked AE/DBN	20 × 20	[51,58]
	DBN	Radarsat-2	DBN with RBM	32 × 32	[59]
	RNN	SLAR	MLPs, LSTM, Vanilla-RNN, B-LSTM	256 × 256	[18]
Detection	CNN	SLAR	Two-stage CNN	28 or 50 per side	[60]
		Unmanned aerial vehicle	Faster R-CNN	-	[61]
		VV and VH	Res-Net 101	1024 × 1024	[62]

. Among these technologies, SAR remains the most prevalent due to its ability to operate under all weather conditions and its insensitivity to cloud cover, unlike optical sensors, such as LandSat and Sentinel-2 [20]. Despite these advantages, SAR-based oil spill detection faces challenges, particularly in minimizing false positives caused by similar visual attributes shared by non-oil slick elements, like low-wind areas, natural films, rain cells, current shear zones, and biological algae blooms and weed beds. In addressing these limitations, advancements in DL have demonstrated their versatility in oil spill detection, with applications spanning image-patch-based classification, semantic segmentation, and object detection. For instance, semantic-segmentation models, such as AEs applied to SLAR data using LSTM architectures and GANs with ERS-1/2 and ENVISAR ASAR datasets, effectively classify oil spills at the pixel level. Additionally, classification models like stacked AEs and DBNs are employed for patch-level analysis, while detection models have been used to enhance oil spill detection from both satellite and Unmanned Aerial Vehicle data sources. However, the performance of traditional ocean surveillance systems using aircraft and coastguards is often limited by adverse weather, logistical challenges, and delayed data transmission. These challenges underscore the importance of not only curating a high-quality dataset but also employing advanced methodologies to maximize the utility of available data, while achieving larger datasets can mitigate issues related to oil spill detection.

3. Advances in Research, Materials and Proposed Methodology

3.1. Challenges in Oil Spill Detection

Numerous studies have demonstrated the benefits of data augmentation in ML for training DL models. The performance of ML models, particularly in computer vision (CV) tasks, is closely tied to the quality and quantity of the training data. It is well established that larger datasets generally lead to better performing DL models. Research has shown that models trained on extensive datasets often achieve superior accuracy and robustness compared to those trained on smaller datasets [7,63,64,65,66,67,68].

This is because larger datasets offer a more comprehensive representation of various scenarios, which reduces the risk of overfitting and enhances the model’s ability to generalize to new unseen data. One common challenge with smaller datasets, particularly in CV tasks, is the difficulty of achieving good generalization on validation and test sets. Models trained on limited data frequently over fit, performing well on the training data but struggling to adapt to new data in real-world settings. This problem is especially prevalent in fields where data collection is challenging, expensive, or time consuming such as medical imaging or remote sensing, leading to reliance on small datasets. To address the limitations of small datasets in DL model development, several advanced techniques have been proposed. These include methods such as dropout regularization, batch normalization, and transfer learning. Dropout, for instance, involves randomly deactivating certain units during training, which prevents the model from becoming overly dependent on specific neurons and thus reduces overfitting. Batch normalization helps to stabilize the training process by normalizing the inputs to each layer, ensuring a mean of zero and a standard deviation of one, which also aids in speeding up convergence.

Most CV-based models heavily rely on high-quality images too. It is well established that larger datasets often contribute to better-performing DL models, as observed by various researchers [69,70,71,72,73,74,75,76,77]. Models trained on substantial datasets tend to exhibit higher accuracy and robustness, primarily because they are exposed to a wider variety of examples. This broader dataset representation enables the model to better capture the diversity of real-world scenarios, which in turn reduces the chance of overfitting and enhances the model’s capacity to generalize effectively to new and unseen data. In contrast, one of the most persistent challenges in CV is the difficulty models face when training on small datasets, such as when they struggle to generalize well to validation and test sets [78]. Overfitting is a common issue in such cases, where the model becomes highly attuned to the nuances of the training data but lacks the flexibility to adapt to new unseen inputs. This problem is especially pronounced in domains where data collection is difficult, costly or time consuming, leading to reliance on smaller datasets. For example, in specialized tasks like detecting rare events (e.g., oil spills in various ocean scenarios), obtaining sufficient labeled data can be a significant barrier. Therefore, in this research we are proposing a user-friendly oil spill dataset generating web application. By using this technique, we can save time by avoiding oil spill labeling. In other words, we augment the oil spill dataset by extracting our manually labeled 2417 oil spill images and blending them with various ocean environments, such as hazy, sunny, low light, etc. This approach will help to create or apply oil spills to any kind of ocean image and create a big amount of oil spill images. The dataset generation process is shown in Figure 1 below.

In

Table 4. Challenges for the identification and detection of oil spills.

Challenges	Overcome
The process of preparing considerable amount of labeled data	yes
Limitation or absence of accessible open-source annotated oil spill datasets	yes
Hyper-parameter tuning		no
Generalization across diverse datasets	yes

we included several challenges related to oil spill datasets, which was described by Rami et al. [7]. In our proposed method, we can overcome three main issues, such as oil spill dataset preparation, creation of an open-source annotation dataset, and the ability to generalize across various datasets.

3.2. Proposed Methodology

To solve the limitations posed by small datasets, various advanced techniques have been developed to improve the robustness and generalization of DL models [79,80,81,82,83]. For example, in the use case of a transfer learning application, this technique offers an effective solution by leveraging pre-trained models on large-scale datasets and fine-tuning them on task-specific, smaller datasets. This kind of approach allows models to utilize pre-trained features from a rich dataset, which can be invaluable when the target dataset lacks a substantial amount of labeled data. In the realm of data augmentation, several studies have underscored its importance in boosting model performance. Wang et al. [58] compared various augmentation techniques and demonstrated their effectiveness in improving accuracy, particularly in image classification tasks. Data augmentation can be especially useful when generating synthetic examples of rare or difficult to capture scenarios, which would otherwise require significant manual effort for data collection and labeling. In the context of oil spill detection, the creation of high-quality, diverse datasets is crucial for developing robust CV-based DL models. Oil spills manifest in a variety of forms, depending on factors such as the type of oil, the volume spilled, and the environmental conditions like sea state, weather, and lighting. For instance, oil spills can look significantly different in calm versus rough seas, or in bright sunlight compared to overcast or night-time conditions as can be seen in Figure 2 and Figure 3, which represent our first contribution to collecting oil spill images with various types. Such variability necessitates a comprehensive dataset that captures these diverse scenarios to ensure that DL models can generalize well across different conditions.

Furthermore, the representation of oil spills under various environmental conditions, such as different water turbidity, reflections from the water surface, cloud coverage, and oil water mixing states—can greatly influence model accuracy.

A well-constructed oil spill dataset should include not only high-resolution images from different environmental contexts but also varying oil types and dispersion patterns.

Table 5. Oil spill and ocean image dataset description.

Oil Spill (Source) Types	Tanker Spills	Pipeline Spills	Offshore Spills	Natural Spills
Oil spill types	Crude oils	Refined oils	Medium oils	Heavy oils
Environmental context	Marine	Fresh water
Spill characteristics	Surface oil
Total source images	2417
Different ocean scenes	Open sky	Cloudy	Rainy	Foggy (Hazy)
Total target images	500

above describes our dataset details about oil spill types in the case of source image and target ocean images. Combinational augmentation technique application will significantly increase the dataset size because 2417 oil spill images are extracted from background in data augmentation process and blended to 500 target ocean images.

This variability helps models learn more intricate features, thus making them more robust in detecting oil spills in real-world situations. In addition, datasets should account for edge cases, such as partial or obscured spills, to train the model to recognize a broader range of oil spill characteristics. High-quality representation across a wide spectrum of oil instances is paramount to ensuring that the DL models are not only accurate but also resilient to the diverse and often unpredictable nature of real-world scenarios.

3.3. Applied Data Augmentation Techniques

In oil spill dataset creation, background removal serves as a critical technique for isolating the oil spill region from the surrounding oceanic environment. This method allows us to focus on the specific area of interest—namely, the oil spill—while excluding non-essential parts of the image, such as the background water, clouds, or other unrelated elements. The image background removal library we utilize, developed by Daniel Gatis, leverages advanced algorithms including thresholding, edge detection, segmentation, and a DL-based model to effectively distinguish the oil spill from the ocean background.

The process begins by targeting the primary region of interest, the oil spill as shown in Figure 4. For a given image I, represented as a matrix of pixel values:

$$I=\left\{p\right(i,j){\}}_{i,j=1}^{M,N}$$

(1)

where M and N are dimensions of the image, such as spatial resolution in M(columns), N(rows), and $p(i,j)$ denotes pixel values at position $(i,j)$, or equal to $i\in \left[1,M\right] \mathrm{a}\mathrm{n}\mathrm{d} j\in [1,N]$.

The segmentation algorithm is then applied to divide the image into regions, resulting in a segmentation mask S (Figure 5):

$$S=\left\{s\right(i,j){\}}_{i=1,j=1}^{M,N}$$

(2)

where $s(i,j)$ is a binary values indicating whether the pixel at $(i,j)$ belongs to the foreground (oil spill) or background (ocean). This segmentation mask allows us to precisely isolate the oil spill region from the rest of the image.

After segmentation, the processed image I′, which contains the oil spill without the background, is obtained though element-wise multiplication of the original image I with the segmentation mask S:

$${\mathrm{I}}^{\prime }=\mathrm{I} \odot \mathrm{S}$$

(3)

where ⊙ denotes the Hadamard product, which is an element wise multiplication equation. This ensures that only the oil spill region remains while the background is effectively removed.

To enhance the dataset’s diversity and improve robustness, we apply various data augmentation techniques, blending the extracted oil spill regions with different oceanic scenarios such as hazy, rainy cloudy, sunny, etc., ocean environment conditions. This augmentation helps simulate real-world conditions, which are crucial in training models to detect oil spills under varying environmental circumstances.

The blending function integrates the oil spill image onto different ocean backgrounds while preserving transparency and visual consistency. It converts the image to the RGBA format to handle transparency effectively. The alpha channel in the oil spill image acts as a mask, ensuring that only the non-transparent parts of the oil spill are blended into the ocean background. This allows for seamless augmentation of the oil spill dataset, maintaining the visual integrity of both the oil spill and the underlying ocean scenarios. These augmentations not only increase the size of the dataset but also enable models to generalize better to differentiate in real-world conditions, thus improving the accuracy and reliability of oil spill detection systems. By focusing on these augmentations, we ensure that the dataset is robust enough to account for varying weather and oceanic conditions, which is crucial for effective oil spill detections.

3.4. Target Image Resizing and Positioning in Blending to Source Image

In the process of oil spill dataset augmentation, image resizing and positioning, in the case of an extracted oil spill, play a critical role in adapting the extracted oil spill regions to various oceanic backgrounds. This approach not only enhances the dataset’s diversity but also ensures that the model can recognize oil spills across different scales and perceptiveness, which is crucial for real-world applications. To preserve the transparency of the oil spill region during compositing, the input image is converted to the RGBA mode. This conversion ensures that the image contains four channels—red, green, blue, and alpha—where the alpha channel is specifically responsible for maintaining transparency. The inclusion of the alpha channel is particularly important when overlaying the resized oil spill on different ocean backgrounds, as it ensures that the spill blends seamlessly with the background without artifacts from the edges of the original image. The dimensions of the resized oil spill image are computed by multiplying the original width and height of the image by a scaling factor. The scaling factor is randomly selected from a predefined range to simulate different scales and perspectives, providing variability in the dataset. For instance, smaller spills can be simulated by reducing the size, while larger spills can be created by increasing the scale, making the model more resilient to detecting spills of varying sizes in ocean environments, as can be seen in Figure 6 below.

The resizing of the oil spill image is performed using the LANCZOS [84] resampling algorithm, which is known for its high-quality image scaling. The LANCZOS algorithm ensures that the resized image maintains sharpness and minimizes the introduction of blurring or artifacts during the scaling process. Mathematically, the resizing operation can be expressed as scaling by a factor s, where the coordinates (x, y) in the original image map to new coordinates (${\mathrm{x}}^{\prime },{\mathrm{y}}^{\prime }$) in the resized image:

$${\mathrm{x}}^{\prime }=s·x, {\mathrm{y}}^{\prime }=s·y$$

(4)

The LANCZOS resampling algorithm is defined as:

$${\mathrm{p}}^{\prime }\left({\mathrm{x}}^{\prime },{\mathrm{y}}^{\prime }\right)=\sum _i\sum _{j}p\left(x_i,y_j\right)·L\left(x^{\prime }-x_i\right)·L\left(y^{\prime }-y_i\right)$$

(5)

Equation (5) ensures that each resampled pixel values $p^{\prime }\left({\mathrm{x}}^{\prime },{\mathrm{y}}^{\prime }\right)$ is computed based on the weighted contributions of nearby original pixel values $p\left(x_i,y_j\right)$, with L representing the LANCZOS kernel. This kernel is designed to minimize distortion and ensure a smooth transition when rescaling the image. Once the oil spill image is resized, it is positioned onto the ocean background at randomly selected coordinates. The random positioning of the spill within the image simulates natural variations in where oil spills might appear in different ocean contexts. This variability is key for creating a more robust dataset, as it trains the model to detect oil spills not only in the center of the image but across various locations, angles, and perspectives.

This resizing and positioning strategy, when applied systematically across multiple backgrounds such as hazy, rainy, cloudy, and sunny ocean conditions, contributes significantly to the creation of a diverse oil spill dataset. By varying the scale and placement of the oil spills, we ensure that oil spills may occur at different sizes and positions relative to the camera’s perspective or environmental context.

3.5. Random Flip Application

The next technique we applied for data augmentation is the random rotation of oil spills when blending extracted oil spills with various background ocean images. This technique enhances the dataset for creating variability in the appearance of the oil spills, allowing the model to detect spills from different angles. While rotated, oil spill images are combined with the target ocean backgrounds using advanced blending techniques that ensure a smooth integration between the spill and its environments. These techniques preserve the natural appearance of the scene, maintaining visual consistency, and preventing any harsh boundaries or unnatural transitions between the spill and the ocean.

After applying the rotation, the final augmented image is saved for further use in training and validation. This augmented image serves as an important part of the dataset, helping the model become more robust and capable of recognizing oil spills in diverse contexts. In addition to rotation, other transformations such as horizontal and vertical flips are also applied to create further variability. A horizontal flip is performed by mirroring the imaging along the vertical axis:

$$I^{\prime }\left(x,y\right)=I\left(W-x,y\right)$$

(6)

from the equation I(x, y) represent the pixel at coordinates (x, y) in the original image, and W is the width of the image. Similarly, a vertical flip is applied by mirroring the image along the horizontal axis:

$$I^{\prime }\left(x,y\right)=I\left(x, H-y\right)$$

(7)

where H is the height of the image. Both these operations simulate different orientations of the oil spill, ensuring that the model is trained to detect spills in various configurations. Rotation of the oil spill image is conducted by rotating the image by a random angle $\theta$:

$$x^{\prime }=x·\cos \left(\theta \right)-y·\sin \left(\theta \right)$$

(8)

$$y^{\prime }=x·\sin \left(\theta \right)+y·cos\left(\theta \right)$$

(9)

where (x, y) are the original coordinates and ($x^{\prime },y^{\prime }$) are the new coordinates in the rotated image.

The random flip application of oil spill images into ocean image is graphically represented in Figure 7. The oil spill is rotated in 360 degrees with scaling in its size. Moreover, oil spill location is also changeable from place to another place in the background ocean image.

3.6. Pyramid Blending of Oil Spill

As can be seen in Figure 6, oil spill images are not well blended to the background image. Therefore, to make augmentation more natural we applied an alpha blending technique. Alpha blending is a weighted average of the pixel values in the foreground and background images. Alpha values are normalized to fall within [0, 1] range, converting from 0 to 255 to a fraction by dividing by 255. The core blending equation is implemented in a loop over each of the RGB channels. For each channel c, the pixel value from the overlay image is multiplied by a_(i,j) to retain a proportion of the overlay color based on its opacity. The background pixel value at the same location is scaled by (1 − a_(i,j)), meaning if the overlay pixel is partially transparent, the background color will show as less accurate. The result is added together for each color channel, creating the final color value for that pixel in the blended image.

These augmentation techniques, such as random rotation of the oil spill while blending it to background ocean image is also crucial for the effective simulation of real-world conditions. When the augmented oil spill images are integrated into various ocean backgrounds (e.g., hazy, rainy, sunny, fog, cloudy, or open air), they help the dataset represent a wide range of environmental scenarios, which assist models to generalize across different types of images.

3.7. Blur Effects Application in Augmentation

Additionally to the above applied data augmentation techniques, we also applied a Gaussian blur effect [85] for blended images to simulate various levels of atmospheric and depth-of-field effects. This technique is essential for creating more realistic scenarios where spills may be viewed in different environmental settings, like fog, haze, or camera focus variations. By introducing the Gaussian blur, we aim to mimic the atmospheric distortion that can occur in real-world imaging, ensuring that the model is capable of detecting oil spills even in challenging visual conditions. Moreover, we also applied noise injection to apply random noise to the image, typically drawn from a Gaussian distribution. Noise injection plays a crucial role in making the model more robust by preventing it from learning spurious noise patterns that might be present in the training data. Without this, models may lead to overfitting, learning to recognize noise rather than the actual features of interest. Which means, by injecting noise, the model learns to ignore irrelevant variations and instead focus on the essential patterns, improving its generalization ability on unseen data. By this technique, we aim to reach stability for the model training process by enhancing the model’s resilience to noisy inputs.

Mathematically, the noise injection can be represented as:

$$I^{\prime }=I+n$$

(10)

from Equation (10) $I^{\prime }$ is the noise image, I is the original image, and n $~$ N(m, ${\sigma }^2$) is Gaussian noise with m and variance ${\sigma }^2$. The noise is added at random intensities to different images, ensuring that the model becomes accustomed to varying noise levels.

Following noise injection, we applied the Gaussian blur filter to further augment the image. Gaussian blur is a smoothing technique that reduces high-frequency components, which can simulate atmospheric effects such as fog or camera defocus. The blur radius was set randomly between 1 and 20 pixels, and it was applied to both the oil spill and various oceanic backgrounds, ensuring that each image has a unique degree of atmospheric distortion. This randomization helps to create a wide range of conditions under which the model must detect oil spills, including scenarios where the oil spill and background may be partially or completely blurred due to environmental factors. The Gaussian blur function operates by convolving the image with a Gaussian kernel, which smooths the pixel values and reduces sharp edges, mimicking real-world effects like haze or distance. The equation for the Gaussian blur applied to an image is as follows:

$$G(x, y)={\displaystyle \frac{1}{\underset{.}{2\pi {\sigma }^2}}}{e}^{-{\displaystyle \frac{x^2+y^2}{{2\sigma }^2 }}}$$

(11)

where G(x, y) is the value of the Gaussian function at point (x, y) and $\sigma$ is the standard deviation, controlling the amount of blur application. A larger $\sigma$ results in a more pronounced blur effect, simulating thicker atmospheric interference.

3.8. Changing Color Channels and White Balance Adjustment

Lighting biases pose a significant challenge in image recognition tasks, particularly in datasets involving complex natural environments such as oceans. Digital image data is typically represented as a tensor with dimensions corresponding to the height, width and color channels of the image. These channels carry critical information that affects how objects within the image are perceived. In the context of oil spill detection, lighting inconsistencies across ocean backgrounds can lead to diminished model performance, as variations in brightness, shadows, and color temperature can distort the appearance of oil spills. Chatfield et al. [86] observed a 3% accuracy drop between grayscale and RGB image classification, emphasizing the importance of using RGB color channels for image-based tasks. This drop highlights how color information contributes significantly to object recognition and for applications like oil spill detection, maintaining accurate color representation across different environments becomes critical. Therefore, implementing augmentations within the color channel space becomes an effective and straightforward approach to enhance the dataset’s robustness. To mitigate lighting biases and ensure consistency between oil regions and diverse oceanic background, we apply white balance adjustments. White balance aims to correct the color imbalance caused by lighting conditions, ensuring that the colors in the image appear natural and consistent. This is especially important when blending extracted oil spill regions onto different ocean scenes where lighting can vary significantly. Proper white balance adjustment ensures that the appearance of oil spills remains realistic, regardless of the background scenario. To achieve this, we calculate the average intensity for each color channel. For a given image with three color channels, the pixel intensities for each channel are represented as:

$${\mathrm{I}}_{\mathrm{c}\mathrm{o}\mathrm{l}\mathrm{o}\mathrm{r}\_\mathrm{c}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\mathrm{d}}\left(\mathrm{x},\mathrm{y}\right)=(\mathrm{R}\left(\mathrm{x},\mathrm{y}\right),\mathrm{G}\left(\mathrm{x},\mathrm{y}\right),\mathrm{B}(\mathrm{x},\mathrm{y}\left)\right)$$

(12)

where R, G, and B represent the red, green, and blue channels, respectively. The aim of white balance adjustment is to normalize these channels to ensure color accuracy. The average intensity for each color channel is computed as:

$${Avg}_c={\displaystyle \frac{1}{N}}\sum _{i=1}^N{I}_{c,i}$$

(13)

where ${Avg}_c$ represents the average intensity of channel c, $I_{c,i}$ represents the intensity of the i-th pixel in the channel, and N is the total number of pixels in that channel. Next, a scaling factor for each channel c is computed based on the average intensity across all channels. The scaling factor is designed to equalize the color balance across the channels, thus compensating for any lighting-induced biases:

$$S_{c }={\displaystyle \frac{{Avg}_{all}}{{Avg}_c}}$$

(14)

where ${Avg}_{all}$ is the intensity computed across all channels. This scaling $S_{c }$ adjusts each channel’s intensity to ensure a balanced and neutral color distribution in the image, thus mitigating lighting discrepancies. Finally, the pixel intensities in each channel c are adjusted using the computed scaling factor:

$${I^{\prime }}_{c,i}=I_{c,i}·S_{c }$$

(15)

From Equation (15) we get adjusted intensity for c channel of each pixel as ${I^{\prime }}_{c,i}$. $S_{c }$ scales the intensity values to align with overall balance. This process ensures that the overall color distortion of the image becomes more uniform, effectively reducing cast and lighting disparities. It is important to mention that this process is useful while we are working with colorful oil spill images, where consistent illumination and color neutrality are critical.

3.9. Gradio Web Interface for Oil Spill Data Augmentation

Table 6. Gradio web application description for oil spill image augmentation.

Description	Values (Range)
Number of images	1–1000
Target image x-axis	0–1500
Target image y-axis	0–1500
Output folder	Specify
Contrast factor	0.5–3
Sharpness factor	0.5–3
Blur radius	0–20
Rotation angle	0–360
Equalize histogram	yes/no
Apply white balance	yes/no
Random crop width	10–500
Random crop height	10–500
Apply random crop	yes/no

below describes a set of parameters for a Gradio web interface specifically designed for augmenting oil spill images by blending them with ocean background images. The detailed configuration options enable us to manipulate several aspects of the image, allowing for the generation of a diverse, high-quality dataset suitable for developing robust DL models for oil spill detection and analysis.

By leveraging various augmentation techniques outlined in the table, the interface facilitates the creation of a comprehensive oil spill dataset, capturing a wide array of environmental conditions. For example, to simulate realistic environmental scenarios, we can use images of rainy or hazy ocean conditions as background layers. These backgrounds are combined with existing oil spill images to create augmented versions that depict oil spills under challenging weather conditions. The augmentation process involves overlaying the oil spill images onto these background images, with random adjustments applied to the scale and positioning of the oil spill regions. This ensures that the augmented dataset includes diverse variations, replicating the random nature of oil spill appearances in real-world scenarios. This is a crucial technique in data augmentation, because in such kinds of ocean scenarios it is cumbersome to capture actual oil spills. Therefore, we set the number of images for blending in the range of 1 to 1000. A very high number of data augmentations can also lead to an overfitting issue, therefore we set maximum range to 1000. Target image x and y axis parameter defines the horizontal (x-axis) and vertical (y-axis) coordinate in pixels for positioning the oil spill target image on the ocean background. Positioning ranges in from 0 to 1500 along with the background image. Combining variations in x and y coordinates allows for a realistic simulation of oil spills in diverse locations on the water, aiding in creating a spatially varied dataset. We also included an output folder section to save specific oil spills with their names, such as thin or thick oils, or rainbow and black oil spills. This can help to manage images systematically, facilitating easy access for model training. Contrast adjustment is crucial in simulating different conditions, which may impact the visibility of an oil spill in an ocean setting. The contrast factor controls the level of contrast enhancement applied to the blended image, with values between 0.5 (low contrast) and 3 (high contrast). Contrast application allows for the creation of images that represent oil spills in various lighting and visibility conditions, such as cloudy, foggy, or high-glare situations. Moreover, we applied a sharpness factor with the same range as contrast adjustment. This parameter is valuable for simulating scenarios where the camera focus varies, or environmental factors affect the sharpness of the image. Enhanced sharpness may highlight oil edges, while reduced sharpness can represent distant or low visibility spills, adding a layer of diversity to the dataset. For more augmentation techniques, blur radius is one of the most commonly used method. This method introduces Gaussian blur to the image, simulating conditions such as motion blur or low-focus scenarios. A blur of radius 0 applies no blur, while a higher radius can mimic adverse conditions, like wind or camera shape. In our case, the range is set to 0–20. The next applied technique is rotation angle of oil spill image in the range of 0–360. This parameter rotates the oil spill target image before blending it with the ocean background. Various orientations make the dataset more comprehensive, capturing irregular shapes and orientations of typical oil spills.

Figure 8 is the Gradio [87] library-based web application for oil spill augmentation. Figure 8 is the illustrative look of Table 6. The figure covers five different square colors, including red, green, black, blue, and violet.

The figure consists of two main parts, which is that the green color area covers information about input image processing and the blue line covers area that represents the processed input oil spill image and background ocean image. In red square, we have oil spill image (a), which is background removed from an actual ocean image, and (b) is the ocean image. Violet and black line squares give the same information about data augmentation techniques. The random crop width specifies the width of a cropped section taken from the augmented image. This is useful for generating partial views of oil spills, simulating situations where only fragment of the spill is visible. By setting a range from 10 to 500 pixels, this parameter helps in training the model on different scales and perspectives of oil spills, enhancing detection accuracy. Similar to the width, this parameter defines the height of the randomly cropped section. The included yes and no application buttons enable or disable the random cropping function. In the black square, it is available to customize image augmentation, include or remove specific augmentation techniques, locate and scale up and down the oil spill image. In the violet section, we input sample images for augmentation. In the blue color section, we can get sample augmentation images with applied augmentation techniques. The most importance step in using the Gradio oil spill web application is to input proper oil spill images with extraction from the background and to input proper background ocean image. Specifying saving folder name and creating customized oil spill images are adjustable.

In data augmentation, we also included a histogram equalization technique. Histogram equalization is a preprocessing technique used to enhance contrast by distributing the intensity values of an image more evenly. Application of this technique enables or disables histogram equalization, which can improve visibility in images taken under low-contrast conditions. When applied, this setting ensures that the model is exposed to a range of contrast levels, thereby enhancing its ability to detect oil spills in diverse lighting environments. There are a lot of research works related to histogram equalization. For example, Keun et al. [88] performed a comprehensive survey about contrast enhancement techniques based on histogram equalization, such as dualistic sub-image and recursive mean-separate methods.

4. Quantitative Evaluation

Table 7. Comparative analysis of oil spill detection accuracy by data size.

	Precision		Recall		F-1		Precision-Recall mAP@0.5
2419/14507	Before	After	Before	After	Before	After	Before	After
Yolo-v8m-seg	0.939	0.923	0.94	0.989	0.95	0.987	0.955	0.995
Yolo-v11m-seg	0.924	0.871	0.95	0.99	0.95	0.99	0.954	0.995

below includes baseline and augmented datasets-based model training results with bold letters representing outperforming results. We trained two SOTA models, such as Yolov8m-seg [89] and Yolov11m-seg [89]. For comparison, we analyze precision, recall, F-1, and mean Average Precision (mAP) at Intersection over Union (IoU) threshold value 0.5, which is a key evaluation metric used in object detection tasks. In the table, we highlight outperforming results in a bold style.

Figure 9 below illustrates the comparative performance of the Yolov8m-seg model training achievement results for oil spill detection by using two datasets, the baseline and the augmented oil spill dataset. The performance is evaluated across four metrics: precision, recall, F-1 score, and mAP@0.5. The augmented dataset demonstrates a significant improvement in almost all metrics compared to the baseline dataset. The only advantage of baseline-based model training is that the model’s performance was slightly better in the metric of precision, reaching to 93.9% before augmentation and 92.3% after augmentation, respectively. In recall, F-1 scores, and mAP@0.5 metrics, the augmentation results were showed to be outperforming baseline results, from 94% to 98.9% in recall, 95% to 98.7% in F-1, and 95.5% to 99.5% in mAP scores.

Similarly, in the Yolov11m-seg model training case, oil spill detection measurement in the same metrics showed similar advantages. The only advantage of the not-augmented model training was in precision score, which was at 92.4% in the base dataset and 87.1 in the augmented dataset. Other metrics achieved higher scores in the augmented dataset model training (Figure 10).

In Figure 11, training and validation performance metrics for the Yolov11-seg model fine-tuning results performed better with the augmented dataset applied to oil spill detection. The training and validation loss plots show consistent declines, indicating effective learning and no overfitting. Precision and recall metrics for both bounding boxes and segmentation masks steadily increase, demonstrating improved accuracy and coverage in identifying oil spills. Additionally, the mAP metrics (mAP@50 and mAP@50-95) for both detection and segmentation rise continuously, reflecting robust performance across varying IoU thresholds. Overall, the trends confirm that the model is successfully learning to localize, classify, and segment oil spills with strong generalization to unseen data.

Figure 12 shows the same metrics are presented for the baseline dataset related oil spill model training. However, less data shows lower accuracy. In comparison, in Figure 11, the training and validation losses decrease more rapidly and stabilize at lower values across all categories, such as in B for bounding box and in M for segmentation tasks. Notably, the mAP@50 and mAP@50-95 metrics in the augmentation dataset consistently achieve higher values after augmentation model training, which reflects improvement in detecting and segmenting under varying IoU thresholds.

In contrast, Figure 12 shows relatively slower convergence and lower metric scores, suggesting limited variability in the baseline dataset.

5. Discussion

In this study, we addressed the critical need for high-quality and diverse datasets to improve the accuracy [77,90,91,92] and robustness of deep learning models for oil spill detection in various ocean environments. By integrating advanced object extraction methods with traditional data augmentation techniques, we successfully created a comprehensive dataset of realistic oil spill images across ocean environments. The application of image blending techniques ensured the generation of visually seamless and authentic images, which are vital for training models capable of generalizing to real-world scenarios. Additionally, the development of a Gradio-based web application provides a user-friendly platform for practitioners to generate customized augmented datasets by selecting target and source images and adjusting augmentation parameters. By leveraging a systematic augmentation pipeline that includes blending oil spill images with varied ocean backgrounds and applying transformations such as contrast adjustment, sharpness enhancement, rotation, and Gaussian blur, the augmented dataset successfully simulates diverse and realistic environmental scenarios. This increased variability allows the models to generalize better to unseen data, as evidenced by significant improvements in recall, F1 scores, and mAP metrics across the Yolov8m-seg and Yolov11m-seg models. Notably, while the baseline dataset yielded slightly higher precision scores, the augmented dataset achieved superior performance in all other metrics, highlighting the augmented dataset’s ability to capture the random and irregular nature of real-world oil spills. Furthermore, training and validation loss trends confirm effective model learning with reduced overfitting and higher mAP scores across IoU thresholds reflect robust detection and segmentation capabilities. This work demonstrates the critical role of augmentation in addressing the inherent limitations of small or homogenous datasets.

6. Conclusions

Overall, the contributions of this research assist the development of more effective vision-based oil spill detection systems. By enhancing the availability of diverse and realistic training data, our approach has the potential to early detection of oil spill spread in ocean environments while scenes might be rainy, snowy, foggy, or any other kind of weather condition that impacts and hides oil spill visualization. Integration of the target image with the source image enables it to overcome data scarcity of such ocean environment related oil spills.

In our future work, we will focus on further optimizing the augmentation techniques by incorporating additional environmental variables, including GAN-based augmentation approaches, and check the validation of the dataset’s impact on deep learning performance in operational settings. To improve the functionality of the Gradio web application interface, we aim to enhance the organization of the generated augmented images by introducing the splitting into a training and testing set and specifying oil spill types. When augmented images are generated, the application will automatically divide them into two separate folders, such as a folder for training data and another for testing data. This feature ensures that the dataset is pre-structured for machine learning workflows, simplifying the process of preparing the data for model training and evaluation. Users will have the option to specify the ratio of training to testing images, such as 80% for training and 20% for testing. With the oil spill type saving option, the application will allow categorizing the augmented images based on oil spill types and create subfolders within the training and testing directories for each spill type.