2.2. FLUX+LoRA-Based Data Augmentation Method
2.2.1. Overall Framework of the Proposed Method
To address the problems of high acquisition cost, annotation difficulty, and imbalanced sample distribution across target categories in side-scan sonar imagery, this paper proposes a side-scan sonar image generation method based on the FLUX.1 diffusion model and LoRA fine-tuning. The method is designed to construct new samples with realistic sonar imaging characteristics, thereby supporting subsequent data augmentation. The overall workflow of the proposed method is shown in
Figure 1 and can be divided into two parts: the training stage and the inference stage.
During the training stage, an image–text paired dataset is first constructed using side-scan sonar images and their corresponding textual descriptions. The input images are mapped into the latent space through the Flux VAE encoder to obtain the corresponding latent representations. Noise is then sampled in the latent space to construct intermediate noisy states. Meanwhile, the text descriptions corresponding to the images are fed into a text encoder to obtain conditional vectors that characterize the image semantics and imaging features. Subsequently, the noisy latent variables and the text conditions are jointly input into the FLUX.1 base model. With the parameters of the base model frozen, only the LoRA modules inserted into key layers are trained, enabling the model to gradually adapt to the highlight regions, shadow regions, background textures, and overall grayscale distribution characteristics of side-scan sonar images. After training, a FLUX.1-LoRA image generation model tailored to the side-scan sonar scenario is obtained.
During the inference stage, new text prompts and randomly sampled initial noisy latent variables are provided as inputs. Under the joint constraints of the LoRA weights and the text conditions, the model progressively evolves and denoises the noisy latent variables, ultimately producing latent target image representations that are consistent with the target semantics and exhibit side-scan sonar style characteristics. These latent representations are then decoded by the Flux VAE decoder to recover the final output images. Through this process, the model can generate new side-scan sonar image samples, thereby supplementing the original dataset with a wider variety of target structural patterns and background combinations.
Unlike conventional data augmentation methods, the proposed approach does not simply perform rotations, scaling, or grayscale perturbations on existing images. Instead, it generates new target samples by learning the latent distribution of side-scan sonar images while preserving the realistic imaging style. Therefore, this method can increase the structural diversity of the training data to a certain extent and provide a richer sample basis for subsequent target detection tasks.
2.2.2. Diffusion Model Based on FLUX
Diffusion models are a class of generative models that have been widely used in image generation tasks in recent years. Their core idea is to gradually add noise to the original samples and learn the reverse denoising process, so as to recover new samples that conform to the real data distribution from random noise [
17,
20]. Compared with traditional generative models such as generative adversarial networks, diffusion models offer advantages including more stable training, higher generation quality, and better sample diversity, and therefore exhibit strong performance in image synthesis tasks [
21].
For an input image xt0, the diffusion model first progressively adds noise to it through the forward diffusion process, yielding a noisy sample xt at the t time step. This process can generally be expressed as follows:
where
denotes the noise schedule parameter at step
, and
is the identity matrix. As the number of diffusion steps increases, the noise in the sample gradually intensifies, and the sample eventually approaches a standard Gaussian distribution.
For convenience in training, the above process is often rewritten as follows:
where
is the cumulative noise coefficient, and
denotes Gaussian noise. The objective of model training is to learn a parameterized function
, which predicts the noise term at a given time step
under the condition information
, thereby approximating the reverse denoising process. The corresponding loss function can typically be written as follows:
where
denotes the conditional information, which in this paper mainly corresponds to the conditional vector obtained by encoding the text descriptions.
In the reverse diffusion process, the model gradually reconstructs the original image from the noisy image by learning the noise prediction function. This process can be expressed as
where
is the noise predicted by the neural network.
In this study, FLUX.1 is adopted as the base generative model. FLUX.1 is a conditional image generation framework based on latent-space modeling. Its overall generation process does not perform diffusion directly in the pixel space; instead, the image is first mapped into a low-dimensional latent space through a variational autoencoder (VAE), and noise modeling as well as progressive generation are then carried out in the latent space [
18,
22]. Compared with performing operations directly in the original image space, latent-space generation can significantly reduce computational complexity, improve training and inference efficiency, and at the same time preserve the main structural information of the image [
23]. For side-scan sonar images, target recognition depends not only on local intensity variations but also heavily on the spatial relationships among the target highlight regions, shadow regions, and background textures. For example, ships, aircraft wreckage, and human targets in sonar images are all typically characterized by strong highlight scattering regions followed by shadows, yet different target categories exhibit clear differences in length, shape, shadow extent, and coupling patterns with the background. Therefore, a generative model must learn not only the grayscale distribution but also the structural relationship among the target, shadow, and background. FLUX.1 possesses strong conditional modeling capability and latent-space structural representation ability, making it well suited to such image generation tasks with pronounced structural characteristics.
During the training stage, side-scan sonar images are first mapped into the latent space through the Flux VAE encoder to obtain latent image representations. These latent representations compress the information in the original images while preserving, as much as possible, the main target contours, light–dark relationships, and background texture features. Noise is then added to the latent representations, which are subsequently combined with the conditional vectors output by the text encoder and fed into the FLUX.1 model for training. By learning noise prediction, the model realizes the mapping from “noisy latent representation + text condition” to “denoised latent representation”, thereby gradually capturing the distribution characteristics of side-scan sonar images.
During the inference stage, the model starts from randomly sampled initial noisy latent variables and progressively performs the reverse denoising process under the guidance of newly provided text prompts, ultimately obtaining target latent representations, which are then restored into images by the Flux VAE decoder. The generated images are semantically controlled by the prompts, while their style is constrained by the sonar-domain characteristics introduced through LoRA fine-tuning. As a result, the model can produce new samples that simultaneously contain target semantic information and side-scan sonar style characteristics.
2.2.3. LoRA Fine-Tuning Strategy
Although FLUX.1 has strong image generation capability, its parameter scale is relatively large. If full-parameter fine-tuning is performed directly on a small-scale domain dataset such as side-scan sonar imagery, it not only requires substantial computational and storage resources, but also easily leads to overfitting due to the limited number of training samples, thereby reducing the stability of the generated results. To address these issues, this paper adopts the LoRA (Low-Rank Adaptation) method to perform parameter-efficient fine-tuning of FLUX.1.
The basic idea of LoRA is to introduce a low-rank incremental term on top of the original weight matrix, rather than directly updating all the parameters of the original model [
12,
24]. Let the original weight matrix of a linear mapping layer be W ∈ Rm × n. Its fine-tuned weight can then be expressed as
where
,
, and
is a low-rank dimension much smaller than
and
. During training, the original weight matrix
is kept fixed, and only the low-rank matrices
and
are updated, so that model adaptation can be achieved with a relatively small number of trainable parameters.
Compared with full-parameter fine-tuning, LoRA offers several advantages. First, it requires training only a small number of additional parameters, which significantly reduces the GPU memory usage and computational resources needed for training, making it suitable for experiments under limited hardware conditions. Second, LoRA preserves the general knowledge already contained in the large-scale pretrained model and avoids excessively disrupting the original model distribution under small-sample training conditions, thereby improving training stability. Third, LoRA modules are highly pluggable, and the resulting weight files are relatively compact, which facilitates rapid switching and comparison across different tasks or data conditions.
In this study, the LoRA modules are mainly inserted into the key linear transformation layers of the FLUX.1 model, enabling it to gradually learn the highlight structures, shadow distributions, and background texture patterns specific to side-scan sonar images while retaining the original image generation capability of the base model [
12]. It should be noted that the purpose of employing LoRA fine-tuning in this study is not to enable the model to learn image generation from scratch, but rather to achieve lightweight adaptation to the sonar image domain style on the basis of the pretrained model’s original generation capability [
25]. Therefore, LoRA is particularly well suited to the present side-scan sonar image generation task, where the sample size is limited but the requirement for domain-specific style adaptation is clear.
From the perspective of the training pipeline, the image latent variables, noisy states, and text conditions are jointly fed into the FLUX.1 base model. During noise prediction, the model mainly relies on the LoRA modules to capture the domain differences between side-scan sonar images and natural images. After training, the LoRA weights encode the model’s adaptation to the sonar image style. During inference, it is only necessary to load the trained LoRA weights into the base FLUX.1 model, after which images with sonar imaging characteristics can be generated under new text prompts.
2.2.4. Construction of Image–Text Paired Data and Prompt Design
Diffusion models are conditional generative models, and their generation performance depends not only on the domain adaptation capability of the model itself but also closely on the quality of the image–text paired data used during training. To enable the model to learn both the target semantics and the imaging characteristics of side-scan sonar images, this study constructs an image–text paired dataset composed of side-scan sonar images and their corresponding textual descriptions, on the basis of which LoRA fine-tuning is performed.
During dataset construction, real side-scan sonar image samples are first collected, followed by sample screening and preprocessing. Samples with excessively low quality, blurred targets, or severely missing structural information are removed. The retained images mainly cover typical target categories such as ships, aircraft wreckage, and human bodies, while also attempting to account for variations in target scale, orientation, and background conditions. Unlike ordinary natural images, side-scan sonar images are typically grayscale, and target structures often depend on the combined relationship between strong highlight echo regions and the shadow regions behind them. Therefore, during data selection, particular attention is paid to whether the target and shadow structures are clear and whether the background textures are representative.
With regard to textual description construction, this study does not use only simple category labels. Instead, by considering the imaging characteristics of side-scan sonar images, a text description containing both category semantics and imaging features is designed for each image. Specifically, the text descriptions usually include the target category, the sonar image type, and necessary structural cues, such as “side-scan sonar image of a shipwreck with bright highlight and dark shadow” or “side-scan sonar image of a human-shaped target on seafloor background”. In this way, the text condition conveys not only what the target is, but also, to some extent, guides the model to learn the expected light-dark relationships and spatial structures of the target in sonar imagery.
To further strengthen the model’s ability to recognize the domain style, a unified trigger word is introduced into the text descriptions to explicitly indicate that the image belongs to the side-scan sonar style domain. For example, a style cue such as “sonar image” can be included in the description, allowing the model to establish a mapping between this phrase and the side-scan sonar imaging style during fine-tuning. After training, this trigger word can also be used during inference to improve the consistency of the generated results in terms of sonar style.
In addition to the text descriptions used during training, prompt design in the inference stage is equally important. Inference prompts determine not only the target category of the generated image, but also influence the target size, pose, and background representation in the generated image. For the task considered in this study, a suitable prompt should simultaneously contain two types of information: one is target semantic information, such as ship, plane, and human; the other is sonar imaging information, such as side-scan sonar, highlight, shadow, and seafloor texture. If only the target name is provided, the model is more likely to rely on its original pretrained semantic knowledge while neglecting the sonar style constraints. By contrast, when sonar imaging descriptions are included, the model is more likely to generate images consistent with the visual characteristics of side-scan sonar imagery.
Therefore, this study adopts a prompt design strategy based on category semantics + imaging structure + style trigger words to enhance both the controllability of target generation and the consistency of sonar style in the generated samples. This design is beneficial not only for establishing a more stable image–text alignment during training, but also for providing a foundation for generating richer samples under different prompting conditions in subsequent stages.
2.2.5. Sonar Image Generation and Sample Screening
After training the FLUX.1-LoRA model, the trained generative model is further used to generate new side-scan sonar image samples in order to expand the original dataset. The generation process starts from randomly initialized noisy latent variables. Under the joint action of the input prompts and LoRA weights, the model progressively removes noise and evolves the latent structure in the latent space, ultimately obtaining the latent representation of the target image, which is then restored into a side-scan sonar image through the Flux VAE decoder.
From the perspective of the generation mechanism, the inference stage can be divided into four steps. First, the corresponding text conditional vector is obtained according to the predefined prompt. Second, an initial noisy latent variable is randomly sampled from a Gaussian distribution as the starting point for generation. Third, under the joint constraints of the text condition and the LoRA weights, the model iteratively denoises the noisy latent variable over multiple steps, so that the latent variable gradually evolves from a random noise distribution into a latent representation with target structure and sonar style. Finally, the resulting target latent representation is fed into the Flux VAE decoder to recover the final image.
To improve the diversity of the generated samples, this study appropriately varies the random seeds and prompt formulations during generation, enabling the model to produce differences in target scale, orientation, shadow length, and background texture. Since targets in side-scan sonar images are typically represented by the combination of highlight regions and shadow regions, special attention must be paid during generation to the correspondence between the target body and its shadow region. If the target body is relatively clear but the shadow structure is missing, or if the shadow position is obviously unreasonable, such samples are usually of limited value for subsequent tasks.
Considering the stochastic nature of diffusion-based generation, not all generated samples are directly suitable for downstream detector training. Therefore, after image generation, a two-stage filtering procedure consisting of label auditing and quality screening is adopted. First, each generated image is checked for the existence and validity of its corresponding YOLO-format label file. Samples with missing label files, empty labels, invalid class indices, or invalid bounding-box coordinates are removed. Second, the remaining candidates are screened according to their visual usability for side-scan sonar target detection. The screening mainly considers whether the generated sample contains a clear target structure, whether the target-highlight and acoustic-shadow patterns are visually plausible, whether the seabed background is consistent with the sonar imaging style, and whether the target can be reliably annotated with a bounding box.
In the final generation process, 2317 raw images were produced by the FLUX+LoRA model. Among them, 2302 images had corresponding label files, 15 images had missing labels, and 223 images contained empty labels. After label auditing, 2079 valid candidates remained. The final screened subset contained 1780 generated images, corresponding to an acceptance rate of 76.82% relative to all raw generated images and 85.62% relative to the valid candidates. The selected generated samples contained 635 body objects, 309 plane objects, and 836 ship objects. These samples were added only to the training set in the downstream detection experiments, while the validation and test sets were kept unchanged and contained only real side-scan sonar images.
This screening strategy helps reduce the influence of low-quality generated samples and ensures that the augmented training set contains sonar-style samples with usable target structures. The real-image split and augmentation composition used for evaluation are described in
Section 2.3, and the effect of the screened generated samples on downstream detection performance is evaluated in
Section 3. The screening statistics of the FLUX+LoRA-generated samples are summarized in
Table 1.
2.3. Dataset
The side-scan sonar image dataset used in this study contains three target categories: body, plane, and ship. All images were annotated in YOLO format using bounding boxes and category labels. To evaluate the effect of different augmentation strategies in a fair and leakage-free manner, the dataset was reorganized into a canonical real-image split. Specifically, 1340 real images were used for training, 383 real images were used for validation, and 192 real images were used for testing. The validation and test sets were kept unchanged for all experiments and contained only real side-scan sonar images.
Different augmentation strategies were applied only to the training set. For the original setting, only the 1340 real training images were used. For conventional augmentation, two training sets were constructed: a target-region augmentation setting and a full-image augmentation setting. The target-region augmentation setting added 1832 augmented samples to the training set, while the full-image augmentation setting added 2680 augmented samples. For the proposed diffusion-based augmentation, FLUX+LoRA-generated samples were filtered through label auditing and quality screening, and 1780 screened generated samples were added to the training set. In all settings, no augmented or generated samples were added to the validation or test sets.
This protocol differs from a random split after augmentation. By adding augmented or generated samples only to the training set and keeping the validation/test sets real-only and unchanged, the evaluation more directly reflects whether the generated samples improve detector generalization to real sonar images. The detailed dataset composition is shown in
Table 2.
2.4. Experimental Settings
To evaluate the effect of different augmentation strategies on side-scan sonar target detection, all detection experiments were conducted using a fixed YOLOv8n detector [
26]. The purpose of this design is to isolate the influence of the training data source and avoid confounding the evaluation with differences among detector architectures. The compared training settings include the original real-image training set, two conventional augmentation settings, and the proposed FLUX+LoRA-generated augmentation setting.
For all experiments, the validation and test sets were kept unchanged and contained only real side-scan sonar images. Augmented or generated samples were added only to the training set. This setting ensures that the reported performance reflects the generalization ability of the detector on real sonar images rather than on augmented or synthetic validation/test samples.
The input image size was set to 640 × 640, the number of training epochs was set to 100, and the batch size was set to 24. The SGD optimizer was used with an initial learning rate of 0.01. To reduce the influence of random initialization and training stochasticity, each experiment was repeated with three random seeds, namely 42, 43, and 44, and the results are reported as mean ± standard deviation. Built-in random augmentation operations in the detector training pipeline were disabled so that the comparison mainly reflected the effect of the explicitly constructed augmentation datasets. The experiments were implemented using Python 3.10, PyTorch 2.5.1 with CUDA 12.4, and Ultralytics YOLOv8.
In addition to the main augmentation comparison, generated-sample amount ablation, screening-strategy ablation, LoRA-rank sensitivity analysis, and a controlled diffusion-backbone comparison with FID/KID-based image quality evaluation were also conducted under the same fixed-detector evaluation protocol.
2.5. Evaluation Metrics
To objectively evaluate the influence of different data augmentation methods on side-scan sonar target detection performance, commonly used evaluation metrics for object detection tasks were adopted in this study, including Precision, Recall, mAP@0.5, and mAP@0.5:0.95.
Among them, Precision is used to measure the proportion of predicted positive bounding boxes that are truly positive, while Recall indicates the proportion of real targets that are successfully detected by the model. mAP@0.5 represents the mean average precision calculated under an intersection over union (IoU) threshold of 0.5, whereas mAP@0.5:0.95 denotes the mean average precision calculated over IoU thresholds ranging from 0.5 to 0.95 with a step size of 0.05, which more comprehensively reflects the overall performance of the model under different localization accuracy requirements.
Compared with any single metric, the joint use of Precision, Recall, and mAP under different thresholds can provide a more comprehensive evaluation of the detection model in terms of classification accuracy, recall capability, and bounding box localization accuracy. Therefore, the comparative analysis of different augmentation strategies in this study is mainly based on the above metrics.