Generative AI and Simulation-Based Data Augmentation for Enhanced Object Detection in Low-Data Forestry Environments

Abstract

Detecting rare ground-level obstacles (e.g., large boulders) in dense boreal forests from low-altitude UAV RGB imagery is challenging due to limited annotated data, strong background clutter, and expensive field labeling. This paper evaluates two complementary synthetic-data augmentation pipelines for low-data forestry object detection: segmentation-guided diffusion inpainting, where SegFormer-derived forest-floor masks constrain Stable Diffusion inpainting to plausible insertion regions, and simulator-based generation in Unreal Engine 5 with controlled domain randomization and automatic annotations. We conduct a ten-fold cross-validation study on a real UAV dataset of 64 images and report both accuracy and stability across folds. Compared to real-only training (mean mAP50 ≈ 0.579; mAP50-95 ≈ 0.350), inpainting improves mean performance (mAP50 ≈ 0.647; mAP50-95 ≈ 0.435) while substantially reducing cross-fold variance and lifting the worst-case fold from 0.301 to 0.619 in mAP50. Simulator augmentation yields slightly lower mean accuracy (mAP50 ≈ 0.546; mAP50-95 ≈ 0.344) but markedly improves robustness by mitigating collapse on difficult splits (minimum mAP50 0.496 vs. 0.301). These results indicate that carefully curated generative augmentation can reduce failure risk and improve generalization in extremely data-limited forestry detection settings without additional field data collection.

1. Introduction

Forests are incredibly valuable natural resources and play a vital role as carbon sinks, making it essential to monitor and manage them effectively. Thanks to recent advancements in Unmanned Aerial Vehicle (UAV) imaging, forestry research has been revolutionized, offering high-resolution spatial and temporal data. Commercial UAVs fitted with RGB cameras can capture images with resolutions ranging from 1 to 6 cm per pixel over extensive areas. This level of detail allows for the analysis of individual trees and ground features that was once impossible to achieve with satellite or ground surveys. Modern AI techniques can take advantage of these large, high-resolution datasets; for instance, deep learning has been utilized for tree detection, species classification, and monitoring disturbances in forests.

In practical applications, object detection using UAV imagery can significantly enhance safety and efficiency. A key example is the identification of obstacles on the forest floor, such as rocks, stumps, and debris, before harvesting takes place. Spotting large boulders or rock formations ahead of time helps planners design skidding trails and prevents damage to machinery or accidents. Recent studies have even employed UAV-based surveys to map outcroppings and boulder fields in forested areas. Additionally, low-altitude UAV flights have proven effective in identifying rocks and boulders that could obstruct large equipment, allowing crews to clear hazards before heavy machinery arrives. These automated detections contribute to Geographic Information System (GIS) planning; for example, high-resolution terrain models derived from UAV data are used to optimize logging routes and improve overall extraction efficiency. Beyond operational planning, UAV-RGB sensing is also highly beneficial for environmental monitoring, including assessing forest health, mapping biomass, and detecting disturbances, which further underscores the need for robust object detection tools tailored specifically for forestry.

Motivated by the emerging ability of UAV platforms to operate at low altitudes underneath the canopy, this study focuses on low-altitude, ground-level imaging that contrasts with conventional aerial or top-down remote sensing. The close-up views enable capture of forest-floor features and rare objects that are typically hidden in overhead imagery. In addition to contextual vegetation, the UAV records high-quality RGB imagery (and logs complementary onboard sensors such as LiDAR/GNSS during flight) of functionally important obstacles, including large boulders and stumps [1]. By concentrating on near-ground image collection instead of wide-area mapping, this work addresses rare-object visual detection and 2D instance localization in dense vegetation environments, where small and non-diverse training sets can lead to unstable generalization across conditions [2].

In Scandinavian post-glacial forests, large surface rocks pose operational risks that make it harder for machines to work on their own, raise costs and downtime, and raise CO₂ emissions by making machines go the wrong way and wear out. Our research concentrates on enhancing forest harvesting operations for autonomous vehicles to minimize expenses and CO₂ emissions while maximizing raw material yield. For reliable planning, you need strong object-level vision in forests that change a lot.

In real life, the perception stack has to be able to find and identify rare but important things (like huge boulders, protected anthills, and sacred places) even when the light, weather, season, and viewpoint change. To train these models, you need hundreds of thousands of diverse, labeled photos for each class. However, there is a lack of comprehensive and publicly available datasets that provide the diverse training data from boreal forest interiors that these models need [3]. The data that is already collected only has a few examples per site with little variation in appearance. We use generative AI- and simulation-based data augmentation to create the variety we need for generalization while still being true to the way things really are in the forest.

Although UAV-based sensing is often used for georeferenced mapping, it is important to note that the goals of the current study do not include georeferencing, spatial coordinate alignment, or quantitative GIS evaluation. The LiDAR and GNSS modules onboard the UAV were used operationally for flight navigation and safety and were logged as metadata, but they were not used to produce or assess geospatial coordinates in the experiments [4]. Accordingly, all reported results are based on non-georeferenced RGB imagery and 2D annotations, and the primary objective is to improve image-level object detection performance in forestry scenes via generative data augmentation. We briefly discuss how image-level detections could be integrated into downstream mapping or path-planning systems in Section 7.2, but such fusion is outside the scope of this paper.

1.1. Challenges

While the UAV–RGB framework shows a lot of potential, there are several significant challenges that make accurate object detection in forest environments tricky. First off, there is a lack of annotated data. Most existing UAV forest datasets tend to focus on canopy features like trees, species, and health, with very few annotations for ground objects such as boulders. Creating large and diverse labeled datasets of forest imagery is a time-consuming and costly process. Because of this, deep detectors often miss out on many examples of rare obstacle classes, which limit their ability to generalize.

Then there is the issue of class imbalance. In forest scenes, the majority of pixels are taken up by trees, vegetation, or the background, leaving obstacles like boulders quite sparse. Standard detectors usually end up biased towards these majority classes. This “category imbalance problem” means that while a detector might do a good job of identifying trees, it often overlooks rare objects like stones, leading to skewed predictions.

Another challenge is the high variability in forest images. These images come with complex backgrounds and varying conditions. Changes in lighting, shadows, and seasonal foliage can make objects look different from one shot to the next. UAV photogrammetry relies heavily on image quality, which can be affected by altitude, angle, and environmental factors. Plus, dense vegetation and cluttered undergrowth can create noise that hides targets. In reality, forests produce highly diverse images with rotated, small-scale, and camouflaged objects that can stump even the most advanced detectors.

Lastly, there are implications for 3D modeling and planning. Poor detection can have a direct impact on 3D terrain reconstruction and autonomous planning. For instance, photogrammetric Digital Terrain Models (DTMs) created from aerial RGB data often have only moderate accuracy. Missing or misplacing obstacles can compromise the digital terrain map, and inaccurate models can undermine GIS-based planning for skidding trails and autonomous navigation routes. In short, if objects are not detected reliably, it can lead to errors in mission planning, ultimately reducing both safety and efficiency.

1.2. Defining the Low-Data Problem

We characterize our main challenge in this study as a low-data problem. We not only imply that the number of training images was small in absolute terms, but we also mean that the data was not diverse or representative enough to train a strong object detector. In Scandinavian post-glacial forests, large boulders and other interesting obstacles are not very common, and their shape, size, surface texture, and surrounding plants can make them look very different. Even if hundreds of pictures are taken, they usually only show a small number of different types of objects that were taken in similar conditions. This makes it hard for the model to apply to new situations.

Moreover, it is hard to get annotated data, which makes the problem worse. UAV surveys have to deal with thick canopies and rough ground, and things of interest are not spread out evenly across the landscape. In practice, this means that long flights may only give you a few usable examples, which makes it impossible to make a dataset of the size that is usually needed for modern deep learning. There are not enough boulder images in forest settings in any of the open datasets that are currently available, and there is not enough variety in the context to make training reliable.

In this study, “low data” refers to a condition where the available annotated images are insufficient and overly uniform to train a model capable of generalizing to unobserved conditions. The problem is not just that there are not enough images; it is also that there is not enough variation in space, time, and the way objects look. This definition is important because it shows why generative augmentation is not just a nice-to-have but a must-have: it lets us add controlled diversity to the dataset by making synthetic examples that mimic changes in lighting, weather, and occlusion that would otherwise take months or years to collect in the field.

1.3. Contributions

This paper addresses low-data UAV forestry detection by integrating and evaluating two complementary synthetic-data augmentation pipelines and quantifying their impact under cross-validation on real imagery. Our key contributions are as follows. First, we adapt a segmentation-guided diffusion inpainting pipeline for forestry scenes: SegFormer-derived forest-floor masks constrain Stable Diffusion inpainting so that synthetic boulders are inserted only into plausible ground regions, increasing realism while controlling placement. Second, we develop a photorealistic simulator-based pipeline in Unreal Engine 5 (v5.5) that generates labeled forest scenes with controlled domain randomization, enabling systematic variation in lighting, viewpoint, vegetation density, and occlusion patterns. Third, we provide a reproducible evaluation in an extremely low-data regime using ten-fold cross-validation on a publicly released real UAV dataset (64 images) and report not only mean COCO mAP but also stability indicators (cross-fold variance and worst-case fold behavior), which are critical for safety-oriented obstacle detection. Finally, all datasets and pipelines are released in COCO format to support follow-up comparisons with alternative augmentation and domain adaptation methods.

We validate the above contributions through a controlled comparative study using YOLO (You Only Look Once)-based object detection [5]. We fine-tune a fixed detector architecture under an identical training protocol across folds, changing only the training data composition (real-only vs. real + inpainting vs. real + simulation). This isolates the contribution of synthetic augmentation and allows us to characterize trade-offs between average accuracy and robustness in realistic forestry conditions.

1.4. Paper Organization

The rest of this paper is organized as follows. Section 2: “Related Work” presents a comprehensive review of related work in generative AI, data augmentation, and object detection in forestry environments. Section 3: “Methodology” describes the overall methodology, including our hypothesis and case-study overview. Section 4: “UAV-based Real-life Data Collection” details the UAV-based real-life data collection process, including the collection and annotation of forest imagery for boulder detection. Section 5: “Synthetic Data Generation Methods” presents the synthetic data generation methods, encompassing both the SegFormer-guided generative inpainting approach and the photorealistic simulator-based augmentation pipeline using Unreal Engine 5. Section 6: “Experimental Study” introduces the experimental study, including the YOLOv8x-based detector architecture, training protocol, and evaluation metrics, with particular emphasis on the integration of synthetic augmentation. This section also reports the results from baseline performance, augmented models, and ablation studies using multi-fold cross-validation. Section 7: “Discussion” provides an in-depth discussion and analysis of the findings, addressing the implications and limitations of the proposed approach. Section 8: “Conclusions” concludes the paper with a summary of key contributions and their broader impact on autonomous forestry applications. Finally, we analyze our findings, address any limitations, and wrap up with some thoughts on future work directions in Section 9: “Future Work”.

2. Related Work

Recent progress in computer vision has made generative augmentation a practical tool for improving object detection when real data is scarce. Prior work can be broadly grouped into two complementary directions: generative image editing or synthesis (GANs/diffusion) that modifies object appearance and background context, and simulator-based pipelines that generate labeled scenes via domain randomization. While these approaches are well established individually, their behavior in visually complex, under-canopy UAV imagery of rare ground-level obstacles remains insufficiently characterized, especially under extreme low-data conditions where performance variance across splits can be high. This section therefore highlights which gaps each family addresses, where synthetic-to-real domain mismatch persists, and how our two-branch augmentation design targets both realism (inpainting on real backgrounds) and diversity/control (3D simulation). See Table 1 for details.

Table 1. Summary of recent studies on generative AI and simulator-based data augmentation methods along with their key findings.

Study Name/Author	Augmentation Method	Baseline Performance	Improved Performance	Key Findings
Shivashankar et al. (2023) [6]	Generative models for semantic segmentation	Lower generalization, overfitting issues	Increased resilience, reduced overfitting	Synthetic data improved segmentation in real-world settings
Che et al. (2024) [7]	Generative augmentation with guiding mechanisms	Lower pixel-wise accuracy and boundary detection	Significant gains in accuracy and boundary detection	Enhanced model performance in low-light and occluded images
Rahat et al. (2025) [8]	Generative AI for data imbalance correction	Lower mAP and recall rates in biased datasets	Increased mAP and recall rates	AI-generated data reduced bias and improved detection performance
Li et al. (2024) [9]	Diffusion-based background augmentation	Standard object detection performance	Improved detection robustness on COCO dataset	Enhanced model generalization with diverse backgrounds
Baranchuk et al. (2021) [10]	Diffusion models for label-efficient segmentation	Limited performance with few labeled samples	Improved segmentation with fewer annotations	Diffusion models address data shortages effectively
Kupyn & Rupprecht (2024) [11]	Instance-level augmentations with diffusion models	Standard dataset variability	Improved recognition in object detection and segmentation	Augmentation enriched dataset diversity and accuracy
Carvajal et al. (2003) [12]	Simulator-generated synthetic images (CARLA)	Standard YOLOv5 performance	12% increase in mAP	Improved robustness in varying weather and lighting conditions
Song et al. (2023) [13]	Domain adaptation for simulator-generated data	Lower real-world detection accuracy	10% improvement in detection accuracy	Improved domain adaptation for synthetic-to-real transitions
Liu et al. (2019) [14]	DetectorGAN for small-data object detection	Lower detection accuracy, localization precision	Significant gains in mAP and localization accuracy	GAN-based synthetic augmentation improved detection in low-data scenarios
Yang et al. (2022) [15]	Generative AI and simulators for data augmentation	Standard object detection performance	Increased IoU and mAP	YOLO models performed better with synthetic-real data mix
Thompson et al. (2023) [16]	Stable Diffusion-based synthetic images	Lower representation of object categories	Improved feature representation and segmentation IoU	Synthetic images enhanced model generalization
Amarù et al. (2023) [17]	DALib—library for data augmentation	Standard performance in segmentation/detection	Increased mAP and IoU	Improved dataset diversity and reduced annotation costs
Clark & Cooper (2021) [18]	Stable Diffusion synthetic images for YOLO	Standard YOLO performance	Higher generalization, mAP, and IoU	Hybrid datasets improved segmentation and object detection
Liu et al. (2019) [19]	Pixel-level augmentation with GANs	Imbalanced class representation	1.3–2.1% increase in mIoU	GANs balanced label distribution and improved segmentation
Nguyen et al. (2024) [20]	FMMix—feature-level mixed-sample augmentation	Standard CNN classification performance	Improved mAP and IoU	Enhanced feature diversity improved generalization in YOLO models
Dunphy et al. (2022) [21]	GAN-generated synthetic data for detection	Lower model robustness due to limited real data	Improved robustness and mAP, but risk of over-reliance	GAN-based augmentation expanded datasets but required careful balance

We chose the studies in Table 1 by doing a structured keyword search in Scopus and Google Scholar. We looked for combinations of terms like “object detection,” “forestry,” “remote sensing,” and “low data.” Recent works and those that best show the challenges of object detection in forestry and ecological monitoring were given priority. The table is not exhaustive, but it does show the state of research that is most directly related to the goals of this study.

2.1. Generative Models for Data Augmentation

In computer vision, generative models are becoming a crucial tool for enhancing datasets, especially for tasks involving object detection and semantic segmentation. In their investigation of the effects of generative models on semantic segmentation, Shivashankar et al. [6] showed that by adding a variety of textures and structural changes to training datasets, artificially created data enhanced model generalization. According to their research, segmentation models that were trained on synthetic data showed more resilience when evaluated on real-world datasets, lowering the possibility of overfitting and improving performance in challenging settings. The researchers successfully solved problems with inadequately labeled data, a frequent constraint in supervised learning-based segmentation tasks, by utilizing generative networks [6].

In a similar vein, Che et al. [7] presented an advanced generative augmentation method that included robust guiding mechanisms to guarantee that the produced pictures preserved structural coherence and realistic characteristics. Their research aimed to increase segmentation accuracy in difficult visual circumstances, like dim lighting and occlusions. They saw significant gains in pixel-wise accuracy and object boundary detection by using synthetic data to train cutting-edge segmentation models, such as YOLO-based architectures and DeepLabV3+. The results demonstrated that generative augmentation improves semantic segmentation performance while fortifying the model’s capacity to identify intricate patterns in imperfect or low-quality pictures [7].

Rahat et al. [8] expanded on this study by examining the ways in which generative AI might address data imbalance problems in object detection. The influence of synthetic samples on models such as YOLOv5 was investigated in their study, especially when real-world datasets were biased towards certain item classes. In order to create a more balanced dataset, the researchers created fictitious examples of underrepresented categories, which raised the mean average accuracy (mAP) and recall rates. Empirical proof was presented in the paper that adding AI-generated data to training pipelines increases feature diversity, lowers bias, and raises the model’s overall detection performance in a variety of test scenarios. The idea that generative AI is a scalable and successful augmentation technique for enhancing the resilience of computer vision models is supported by these findings [8].

2.2. Diffusion Models and Background Augmentation

Diffusion models have emerged as a potent tool for enhancing data diversity in computer vision tasks, particularly in object detection and semantic segmentation. These models iteratively refine noise into coherent images, enabling the generation of high-quality synthetic data. Li et al. [9] proposed a straightforward yet effective background augmentation technique utilizing diffusion models. Their approach involved generating diverse backgrounds for existing object instances, thereby enriching the training dataset without necessitating additional annotations. This method significantly improved the robustness and generalization capabilities of object detection models, as evidenced by enhanced performance on the Common Objects in Context (COCO) dataset [9].

The use of diffusion models for semantic segmentation in situations with less labeled data was investigated by Baranchuk et al. [10]. They were able to extract rich semantic information by using intermediate activations from the reverse diffusion process, which allowed for efficient segmentation even with little supervision. This label-efficient method demonstrates how diffusion models may be used to address issues related to data shortages [10].

A technique for dataset improvement through instance-level augmentations utilizing diffusion models was presented by Kupyn and Rupprecht [11]. They increased the richness of the dataset by repainting specific items inside scenes. This augmentation approach demonstrated the effectiveness of diffusion-based augmentations in improving model accuracy by improving performance in salient object recognition, semantic segmentation, and object detection models [11].

2.3. Synthetic Data from Simulators for Object Detection

Simulators have become a vital source of synthetic data for augmenting real-world datasets in object detection tasks. These virtual environments generate large-scale, diverse, and precisely annotated images that complement real-world data, addressing limitations such as data scarcity and annotation costs. Carvajal et al. [12] explored the integration of simulator-generated synthetic images with real-world datasets to improve object detection performance using YOLOv5. Their study utilized the CARLA simulator to create photorealistic street scenes, which were then blended with real-world traffic datasets. The findings revealed that augmenting YOLOv5 training with simulated data significantly enhanced its ability to detect objects in varying weather and lighting conditions. The model exhibited a 12% increase in mean average precision (mAP) compared to training on real-world data alone, highlighting the effectiveness of simulator-based augmentation in improving detection robustness [12].

Building on these projects, Song et al. [13] investigated how domain adaptation methods may help YOLO-based object identification algorithms bridge the gap between simulated and real-world pictures. In order to better match synthetic pictures with real-world datasets, their study presented a domain adaptation technique that modified the color distribution and texture complexity of the images. This method improved detection accuracy on real-world test sets by 10% when used with YOLOv8. The study underlined that a viable strategy for improving object detection performance is to combine domain adaptation approaches with simulator-generated synthetic data, especially in situations when real-world data gathering is expensive or restricted [13].

2.4. Integration of Synthetic Data from Simulators

In the research paper, “Generative Modeling for Small-Data Object Detection” Liu et al. [14] investigate how enhancing object detection performance, especially in low-data circumstances, can be accomplished through supplementing real-world data with artificial data via generative AI and simulators. In order to optimize synthetic images for object detection rather than merely realism, their suggested model, DetectorGAN, incorporates a mechanism for detection within the generative adversarial network (GAN) pipeline. By enhancing the accuracy of detection, localization precision, and general robustness, this method helps models such as YOLO. Incorporating synthetic data into object detection models boosts generalization, minimizes the need for annotation, and raises data diversity—all of which are beneficial for medical imaging and rare-item detection. A potential approach to improving semantic segmentation and detection in data-scarce contexts, the study shows that mixing real and produced data significantly improves mean Average Precision (mAP) and the localization of objects’ accuracy [14].

Additionally, Yang et al. [15] in their study “Image Data Augmentation for Deep Learning: A Survey”, emphasize how important generated data is for improving the accuracy of computer vision models by adding synthetic data from generative AI and simulators to real-world datasets. This method improves model generalization, lowers annotation costs, and increases data diversity, which greatly helps operations like detection of objects and semantic segmentation. In order to improve learning features in models like YOLO, deep generative models like GANs and CycleGANs are very good at producing realistic synthetic images that go well with real data. The study shows that merging generated and real data increases the Intersection over Union (IoU) in tasks related to segmentation and mean Average Precision (mAP) in object detection. In situations where data is scarce, YOLO-based models perform better overall in terms of detection, localization accuracy, and robustness by methodically incorporating generative data augmentation [15].

2.5. Libraries and Tools for Data Augmentation

While the libraries and tools for data augmentation are of great use, Thompson et al. [16], in their study “Evaluating Representation in Synthetic Images: An Analysis of the Stable Diffusion Model,” examine how adding false data to real-world datasets utilizing Stable Diffusion can improve the accuracy of computer vision models. By diversifying datasets and lowering dependency on pricey labeled real-world data, this method is especially advantageous for object detection and semantic segmentation operations, including YOLO-based models. The study emphasizes how generated images enhance feature representation, resilience, and model generalization when combined with actual data. There are also difficulties in guaranteeing high-fidelity representations across various item kinds, even though synthetic images aid in filling in gaps in underdeveloped object categories. According to the results, diffusion-based generative models can be a useful addition to conventional datasets, increasing segmentation tasks’ Intersection over Union (IoU) and object identification tasks’ mean Average Precision (mAP), which would ultimately boost the model’s overall performance [16].

Moreover, Amarù et al. [17], in their study “DALib: A Curated Repository of Libraries for Data Augmentation in Computer Vision,” analyze how computer vision models can be made more accurate by using generated data, such as synthetic examples from simulators and generative AI. The study shows gains in object detection and semantic segmentation, especially in models like YOLO, by methodically adding synthetic photos to real-world datasets. The study emphasizes how data augmentation can increase training datasets, enhance model generalization, and lower annotation costs. In order to improve mean Average Precision (mAP) and Intersection over Union (IoU) scores, generative models—such as GANs and diffusion-based techniques—effectively introduce variations that aid YOLO-based models in recognizing objects in a variety of settings. The paper also classifies different augmentation methods, showing that combining generated and real data leads to improved accuracy and reliability of object detection systems, particularly in situations where real-world data collecting is expensive or scarce [17].

To add further, Clark and Cooper [18], in their study “Synthetic Image Datasets with Stable Diffusion and Data Augmentation,” look at how generated data affects the accuracy of computer vision models by adding synthetic images from Stable Diffusion to real-world datasets. The work demonstrates how this kind of augmentation increases data diversity and lessens reliance on expensive, annotated real-world photographs, which enhances object detection and semantic segmentation, especially in models like YOLO. This method increases model generalization, mean Average Precision (mAP), and Intersection over Union (IoU) scores by utilizing generative AI and simulators. According to the results, integrating artificial and real data results in detection models that are more resilient, particularly when training data is scarce. This enhances feature representation and boosts overall performance [18].

2.6. Addressing and Positioning Challenges in Data Augmentation

Despite advancements, challenges persist in data augmentation. Liu et al. [19], in their study “Pixel Level Data Augmentation for Semantic Image Segmentation Using Generative Adversarial Networks,” study how GAN-generated artificial data might supplement real-world datasets to increase the precision of computer vision models. The study focuses on using GAN-generated synthetic images to balance the distribution of semantic labels, which improves semantic segmentation efficiency by correcting class imbalances and boosting diversity. With increases of 1.3% to 2.1% in mean Intersection over Union (mIoU), their approach, which incorporates GANs into the data augmentation pipeline, improves segmentation accuracy, especially for underrepresented classes. For deep learning models like YOLO, this method is especially helpful because it lessens the need for manually labeled data while enhancing generalization and robustness, increasing the accuracy of object recognition and segmentation in settings with little data [19].

However, Nguyen et al. [20], in their study “FMMix: A Novel Augmentation Strategy for Enhancing Feature Diversity to Boost Model Generalization for Image Data,” analyze how improving data representation and diversity learning through feature-level mixed-sample augmentations affects the accuracy of computer vision models. In order to provide cohesive and semantically significant synthetic samples, their proposed FMMix technique offers four novel augmentation algorithms that change CNN feature maps. Although the main focus of their research is classification problems, comparable methods can be used to enhance object identification and semantic segmentation by supplementing real-world datasets with data generated from simulators and generative AI. By diversifying training samples, this method can improve performance, decrease overfitting, and increase generalization for YOLO-based models. Object recognition and segmentation models can learn spatial and contextual data more effectively by utilizing feature-based augmentation, which improves mean Average Precision (mAP) and Intersection over Union (IoU). This study demonstrates how sophisticated augmentation techniques can greatly improve model robustness in situations with little data when paired with synthetic data [20].

Lastly, Dunphy et al. [21], in their study “Data Augmentation for Deep-Learning-Based Multiclass Structural Damage Detection Using Limited Information,” examine how computer vision model accuracy is affected by synthetic data produced by Generative Adversarial Networks (GANs) that supplement real-world datasets. Their study demonstrates how well-generated data may be used to increase training datasets, especially for situations where genuine data is hard to come by. Similar augmentation techniques can improve object detection and semantic segmentation in models like YOLO, even though their study focuses on structural damage detection. Models gain from enhanced data diversity, improved generalization, and improved class balance by combining synthetic data from GANs and simulators with real-world images. These factors are essential for accurate segmentation and detection. Although the study demonstrates that employing a hybrid dataset—which combines produced and actual images—improves model robustness, it also cautions that performance might occasionally be harmed by an over-reliance on synthetic data. Using these augmentation methods in YOLO-based models can increase Intersection over Union (IoU) and mean Average Precision (mAP), which will result in more precise object recognition and segmentation in challenging real-world situations [21].

In this paper, we explore the exciting crossroads of cutting-edge generative AI techniques and practical dataset enhancement for object detection, particularly in the realm of forestry. Unlike traditional generative augmentation methods, our approach creatively merges precise spatial control using segmentation-guided generative models [22] with realistic data generation through gaming engines [23]. This innovative strategy aims to address the unique challenge of detecting boulders in the intricate environments of forests.

Current cutting-edge methods mainly depend on either general-purpose generative models or synthetic simulators to boost detection models. For example, diffusion models and GAN-based techniques have shown remarkable advancements in tasks like semantic segmentation and object detection by enhancing the diversity of training data and minimizing overfitting [9,14]. At the same time, simulation environments have effectively generated large-scale, annotated synthetic data that can complement real-world datasets, significantly improving the performance of YOLO-based models across different environmental conditions [15].

Our study stands out by cleverly combining two specialized generative techniques. First, we use an InPaint generative model that focuses on specific areas defined by SegFormer, a segmentation transformer [22]. This approach allows us to enhance our data while keeping the structural integrity and environmental realism that are so important for forestry applications. Next, we tap into a game engine to create highly realistic, diverse, and contextually accurate synthetic forestry data. This strategy tackles the shortcomings noted in previous research regarding the realism of synthetic datasets and the faithful representation of complex natural objects [17,18].

By combining the segmentation-focused spatial accuracy of the SegFormer-restricted InPaint model with the high-quality, scalable synthetic generation capabilities of a game engine, we present a powerful augmentation strategy that is perfectly designed to enhance the accuracy and reliability of YOLO-based detection models in real-world forestry settings. This innovative generative framework not only boosts the realism and precision of the augmented dataset but also significantly lessens the need for expensive manual annotations [21]. As a result, our approach provides a practical and effective answer to the challenges posed by data scarcity and the complexities of annotation, paving the way for more accurate, dependable, and generalized object detection models tailored for environmental and ecological research.

3. Methodology

This study explores how the lack of annotated forestry datasets and their limited diversity can be addressed by generative AI and simulation-based data augmentation. Instead of using standard top-down aerial views, the study uses a custom UAV platform assembled for the project that can navigate within dense forest setups and capture close-range RGB imagery beneath the canopy using an RGB camera module (Survey 3W RGB, MAPIR, Inc., San Diego, CA, USA). Rare forest-floor features that are usually missed in high-altitude UAV surveys, such as large boulders, stumps, and other obstacles, are observable in these near-ground views. LiDAR (OS0-64, Ouster, Inc., San Francisco, CA, USA) and RTK GNSS (Here+ Base RTK and Here 3+ GNSS, CubePilot Global Pty Ltd, Norlane, Victoria, Australia) data are recorded by the UAV system and used operationally for navigation and safety, but spatial georeferencing, localization, and object measurement are outside the scope of this research. Using synthetic data produced by generative inpainting and a 3D simulation pipeline (Unreal Engine, v5.7; CARLA simulator, v0.9.16), the key objective is to enhance image-level object detection of uncommon forestry obstacles using the YOLO framework (Ultralytics YOLOv8/ultralytics, v8.0.33); semantic segmentation is used only as a guiding signal for mask generation in the inpainting pipeline, not as an evaluated output of the detector.

3.1. Hypothesis

Our hypothesis is that generative AI- and simulation-based data augmentation improve object detection accuracy and robustness in low-data scenarios within Scandinavian post-glacial forests, in contrast to training solely on real imagery. This expectation is closely linked to the operational requirements of forestry: autonomous harvesters and mission planners cannot afford to overlook or misclassify obstacles, even when only a small amount of annotated data is available.

Boulders were selected as the target object class because they have direct operational relevance in forestry, where heavy harvesting machinery operates under constrained spatial conditions. Large boulders pose a tangible risk of equipment damage, safety hazards, and costly mission interruptions. This makes their detection not only scientifically interesting but also economically and practically significant for forest management. While mountains, deserts, or farming landscapes also contain rocks, these settings do not present the same operational hazards, as agricultural and off-road vehicles typically encounter fewer immovable natural obstacles of comparable risk. Importantly, the methodology presented here is not limited to boulders: the same data augmentation framework can be applied to other forestry-relevant objects, such as anthills, cultural heritage sites, or protected ecological features, ensuring broader applicability beyond this specific case study.

3.2. Case-Study Overview

The study took place in Vindeln, Sweden, a typical post-glacial boreal forest with lots of boulders on the surface, thick tree cover, and very uneven ground. These conditions are typical of many Scandinavian harvesting environments where autonomous machinery is likely to be used.

Data collection in Vindeln occurred within a restricted timeframe, resulting in the available images failing to represent the complete range of seasonal and lighting conditions. To make up for this time limit, two augmentation methods, which are described in detail in Section 5 that follows, were used to make it look like there was variation. Inpainting made new versions of existing images with different lighting, textures, and obstructions, while 3D simulation made forest images at different times of day and in different weather, from bright sunlight to low-light conditions. These methods combined made a dataset that was richer and more varied than what could have been collected in the field alone. In order to obtain near-field object photography rather than aerial photos, all UAV flights were carried out inside the forest below canopy level while preserving line-of-sight with the ground layer.

Figure 1 illustrates the end-to-end pipeline for reliable obstacle detection in Scandinavian post-glacial forests. A UAV collects RGB imagery (and logs onboard sensors such as LiDAR/GNSS during flight), while object masks enable two augmentation branches: GenAI In-Paint and a 3D Simulator. A YOLO detector is trained on real data with optional synthetic augmentation. While detections could be fused with LiDAR/GNSS in downstream mapping, GIS, or path-planning workflows, such geospatial fusion is not performed or evaluated in this paper.

Figure 1. End-to-end pipeline.

4. UAV-Based Real-Life Data Collection

The real-life data (Figure 2) collection happened in the Vindeln municipality, located in northern Sweden, as part of the Agrarsense project, where Komatsu runs active harvesting sites. The forested landscape in Vindeln in general features a mix of coniferous and deciduous trees, with the most common species being Scots pine (Pinus sylvestris), Norway spruce (Picea abies), and various types of birches (Betula spp.). The test site we chose focused on spruces. The terrain ranges from gently rolling hills to steeper slopes, dotted with swampy hollows and rocky outcrops. We were particularly interested in large boulders, those rock masses that exceed one cubic meter in volume, because they can pose risks to harvesting machinery during timber extraction. All the aerial imagery and sensor data were collected from these areas, providing representative snapshots.

Figure 2. Samples of real data.

Data collection was carried out using a specially constructed UAV platform that can tackle the challenges posed by dense forest canopies and uneven terrain. The airframe was based on a Tarot 650 Sport quadcopter with a foldable frame, selected for its stability and durability in tough conditions. It was powered by four Tarot 6S 4008 330 KV MT motors (TL2955) paired with four Hobbywing XRotor-40A ESCs. A 22.2 V 6S 7000 mAh LiPo battery provided extended flight times of up to twenty minutes under normal payload conditions. The flight control system was anchored by a Cube Orange+ autopilot, enhanced by an NVIDIA companion computer to handle demanding onboard processing tasks. For precise positioning, an RTK GPS system featuring Here+ Base and Here 3+ modules was used, delivering accuracy for georeferenced data logging. Telemetry and remote data transfer were managed through an RFD868x-EU link, allowing for real-time monitoring of the UAV’s status and sensor outputs.

The sensor suite was equipped with an Ouster OS0 LiDAR unit, which allowed for detailed three-dimensional point cloud mapping, along with a Survey3N camera for capturing RGB images. All these components were seamlessly integrated into an ArduPilot-based pilot system.

During the initial test flights in Vindeln, we encountered a few challenges, such as signal interference from the thick canopy and occasional GPS jitter in the deep ravines. However, these flights provided us with valuable datasets that helped refine both our hardware setups and flight planning strategies.

During data collection, LiDAR and GPS were used operationally for navigation and collision avoidance and logged as metadata, whereas only the RGB imagery was processed and analyzed in this study.

Given the scope of this work, GPS and LiDAR sensors are not employed for georeferencing or spatial localization, even though the UAV platform is equipped with them for positional stability and depth collection. Rather, depth-aware object recognition and scene segmentation use the LiDAR data as supplementary context. Coordinate transformation and georeferenced mapping were purposefully left out of this study’s design.

It is worth pointing out that boulders vary a lot in terms of color, shape, and size. Some of them appear alone, and some in aggregation. Some of them are easy to spot, whereas the others are covered by vegetation. In order to obtain a dataset that would be a good representation of the true boulder diversity, we decided to split the collection of boulder images among three people during our visit in September 2024 at the Komatsu tree felling site near Vindeln in Sweden. Each person used a different camera (i.e., a phone camera) and decided to go their own boulder-seeking path. This approach allowed for obtaining a better variety of boulder images, as there would be a variation in terms of resolution, view angle, light, boulder selection, etc.

We gathered 112 images of boulders during our visit in September 2024 at the Komatsu tree felling site near Vindeln in Sweden. We achieved quite a diverse dataset in terms of color representation and resolution (4624 px × 2604 px, 5712 px × 4284 px, and 4032 px × 3024 px). In addition, we received from our project partner and host, Komatsu Forest AB, a further 13 photos of boulders. They were also quite diverse in terms of weather/light conditions, but also in terms of resolution (3840 px × 2160 px, 4000 px × 2252 px, and 2252 px × 4000 px).

Out of the 125 boulder images, several were identified as visually similar; these were excluded, resulting in a final dataset of 64 distinct images. In addition, we decided to focus only on the foreground objects, i.e., boulders that are not too far away from the camera, during the labeling process. To annotate the images, we have used the CVAT [24], the industry-leading data engine for machine learning. We decided to use semi-automatic labeling, which means using the Segment Anything 2.0 model to get the initial boulder mask and then manually adjusting the mask further to obtain the optimal boulder mask.

The annotated UAV-based real-world dataset, comprising images captured inside forest interiors and labeled for tree trunks, boulders, and stumps, is publicly available at https://zenodo.org/records/17195469 (accessed on 24 September 2025). Note that the repository contains a mix of images from various sources and orientations; for our experiments, we restricted the data to horizontally oriented shots taken by the UAV.

Challenges in Data Collection

One problem with this method is that UAV-based data collection cannot be done when it is too dark. Dense forest canopies, seasonal foliage, and weather conditions like cloud cover can make aerial images less clear and obscure the objects we want to see. The generative inpainting and simulation pipeline helps with these problems by making a wide range of training samples, but it cannot completely make up for the lack of clear sensor input in tough situations. This restriction underscores the necessity of integrating multimodal sensing, including LiDAR, with UAV imagery in future research.

Additionally, this limitation aligns with broader difficulties in the application of deep learning to remote sensing data, where fluctuations in visibility, occlusion, and environmental conditions can markedly diminish model robustness. As emphasized in seminal texts on machine learning and computer vision [25,26,27], the diversity of training data is a crucial factor influencing model generalization. Our method follows these rules by using generative augmentation to artificially increase the variety of training samples.

5. Synthetic Data Generation Methods

To address the lack of annotated training data in forestry environments, we developed two complementary synthetic data generation pipelines that augment real-world datasets with photorealistic, labeled images depicting boulders embedded in diverse forest scenes. The first method uses a generative inpainting approach that inserts synthetic boulders into real forest photographs in a context-aware manner, guided by semantic segmentation masks. The second method employs a simulator-based strategy that reconstructs 3D forest environments from LiDAR and geospatial metadata, enabling full-scene generation with fine control over object placement and environmental conditions. Both methods focus on generating RGB data tailored for training object detection models, while differing in the degree of realism control, automation, and diversity they provide.

The samples of real data and the methods describing how the data were obtained are presented in the remainder of this chapter.

5.1. InPaint + SegFormer Augmentation

We began with a diverse set of forest photographs, both horizontal and vertical orientations (described in Section 4), and aimed to generate synthetic images (example in Figure 3) in which boulders were as seamlessly blended into the woodland environment as possible. To that end, we combined semantic segmentation (SegFormer) with image inpainting (Stable Diffusion InPainter), and we experimented extensively with text-to-image prompts to identify the formulations that yielded the highest realism.

Figure 3. Samples of InPaint-based data.

First, we used SegFormer, a hierarchical Transformer-based model that was pretrained on ADE20K [28], to carry out semantic segmentation on each real-life forestry photograph. This model labeled every object semantically, and we pulled out those labeled as “ground,” “soil,” or similar categories to create a binary mask representing the flat areas of the forest “floor”. These masks usually match up with clearings, wide paths, or spots with leaf litter. By focusing on these regions, we made sure that boulders were placed only in spots where they would realistically sit in a natural forest setting, steering clear of dense foliage or vertical tree trunks.

Next, for the image augmentation phase, we used the Stable Diffusion Inpainting model. Each candidate mask was matched with its source image, and we provided a text prompt that described how we wanted the boulder to look. We carefully experimented with a variety of prompts by changing the number of objects, using different descriptive words (like “moss-covered” or “weathered”), and adjusting lighting cues, to find the ones that produced the most photorealistic results. After pinpointing the best prompts (e.g., “exactly one realistic rock-like boulder covered with forest mulch, seamlessly blended with the ground”), our Python script created images with the masked area filled in by realistic boulders. We did not tweak the diffusion weights; we just trusted the pretrained model to accurately depict rock textures that fit the forest lighting and surroundings. Through iterations of prompt engineering, we refined the descriptive cues (e.g., texture, scale, illumination) until the generated boulders appeared naturally integrated.

In total, this procedure produced 2677 candidate images. We then applied an automated, pixel-based quality filter that removed images with artifacts and gross segmentation errors, reducing the set to 1039 images. Two successive rounds of human review followed. The first round discarded clearly unrealistic outputs, retaining 743 images, and the second round eliminated images with improbable geometry, unlikely to appear, or those too difficult to annotate, resulting in a final corpus of 674 images. These 674 images were manually annotated in CVAT and employed in all downstream experiments. The data is publicly available at https://zenodo.org/records/15525807 (accessed on 27 May 2025). In addition to the dataset, we provide the full implementation of the Generative InPainting method as a ready-to-use Google Collaboratory notebook at https://zenodo.org/records/17171598 (accessed on 21 September 2025). This notebook allows anyone to reproduce the generative augmentation pipeline without requiring specialized hardware or software installations.

5.2. Photorealistic Simulator-Based Augmentation

To complement our generative inpainting approach, we also developed a simulation-based pipeline to create synthetic forestry scenes containing boulders (Figure 4). This method provided precise control over 3D scene composition, allowing us to systematically vary environmental factors and incorporate photorealistic boulders into richly detailed forest settings. We used the Unreal Engine 5 due to its photorealistic rendering capabilities and mature support for automated data generation. A 200 m × 200 m geo-referenced forest region in Vindeln, Sweden, was selected as the simulation area. We began by reconstructing the forest environment from extensive real-world data, which includes high-resolution photographs, video recordings, and laser scanning of the selected area using a Leica BLK360 scanner and geospatial metadata, including terrain elevation, tree population density maps, and species distribution data. These inputs were used to generate detailed digital representations of forested areas through a combination of automated preprocessing and manual refinement. Parametrized tree assets were then placed according to species-specific rules and spatial distributions, replicating natural patterns observed in Vindeln forest. Boulders and undergrowth elements were programmatically inserted using placement constraints that prevented overlapping with other objects. The resulting virtual forest served as a high-fidelity base for synthetic scene generation.

Figure 4. Samples of Simulator-based data.

To systematically capture visual diversity, the simulation area was subdivided into a regular grid, with each cell guiding a unique camera viewpoint to ensure even spatial coverage. At each grid cell, the camera images were captured at a fixed height of 2 m above the local terrain surface with a random yaw drawn from 16 discrete bins spaced uniformly over 360 degrees, emulating the viewpoint of a low-flying UAV traversing the forest. Each render cycle produced high-resolution RGB images along with automatically exported bounding boxes and instance segmentation masks through custom modifications to Unreal Engine 5 [29].

To promote model generalization while retaining photorealism, we employed controlled domain randomization across several environmental and visual parameters. Lighting conditions were varied by altering the sun’s elevation and azimuth, based on randomly simulated times of the day. Cloud cover was also randomized to emulate diffuse lighting conditions such as overcast skies or shaded forest canopies. Vegetation density, species composition, and seasonal leaf coloration were procedurally randomized, while terrain surface textures were slightly varied to introduce diversity in forest floor appearance. These variations helped prevent overfitting to simulator-specific render artifacts while exposing the object detection model to a broader distribution of scene appearances. The careful balance between randomization and visual coherence ensured that the rendered scenes maintained realism critical for domain transfer, especially in visually complex environments like forests, while still introducing enough diversity to improve model robustness.

This simulator-based pipeline resulted in the creation of the SimForest dataset [30], comprising over 5000 high-resolution synthetic forest images. Of these, 192 images include annotated boulders, each with a bounding box larger than a 0.2 m cube, captured in diverse forest environments to support object detection training under low-data conditions. A detailed description of the dataset, along with a performance evaluation of YOLO-based object detection and instance segmentation models on a related tree log detection task, is available in [31].

6. Experimental Study

This chapter delves into the experimental study of fine-tuning and validating YOLOv8x for forest obstacle detection. We first trained the detector on real forest imagery and then retrained it with additional synthetic boulder images to assess the impact of augmentation. Detection performance was then evaluated with COCO-style mAP, comparing baseline results from real data with models trained on the augmented set. All experiments were carried out in a GPU-equipped Colab environment using a consistent training protocol. The findings show that synthetic-image augmentation improves both accuracy and consistency, likely because the added background diversity enhances generalization [9].

6.1. Data Overview

• The experimental study uses the following three distinct, COCO-formatted subsets: Real UAV imagery: 64 horizontal RGB images selected from an original pool of 125, described in Section 4 after deduplication and quality filtering. Images are annotated for boulders, logs, and stumps.
• Generative inpainting: 674 synthetic images produced by diffusion-based inpainting, described in Section 5.1, augmenting the variability of rare objects.
• SimForest renderings: 192 of the 5000 simulated forest images repository, described in Section 5.2, that contain annotated boulders.

Each subset is stored in its own subdirectory, containing .jpg image files and .json annotation files. The JSON files follow the COCO schema, encoding segmentation polygons, bounding-box coordinates, class names, and image references. Because one annotation file can describe multiple objects in a single image, the number of JSON entries exceeds the number of image files. All three subsets share identical class labels and the COCO format, enabling straightforward training and validation across the combined data pool.

6.2. YOLO Variants and Hyperparameters

In our experiments, we fine-tuned YOLOv8x (Ultralytics version 8.0.77) using a fixed, out-of-the-box configuration to avoid hyperparameter overfitting in a low-data regime. Each run used the default input resolution of 640 × 640 pixels and pretrained weights. The batch size was automatically selected by the Colab environment (NVIDIA L4 GPU) to maximize utilization without exceeding memory limits. We retained the default optimizer settings and the standard cosine-annealing schedule with warm-up, and we did not perform per-fold tuning. We intentionally used a single detector variant across all experiments (real-only and augmented) so that any observed performance differences can be attributed to the training data composition rather than architecture or optimization changes. While smaller YOLOv8 variants may be preferable for deployment or edge inference, using YOLOv8x provides a strong transfer-learning baseline and yields conservative evidence: improvements from augmentation cannot be explained by a weak backbone.

6.3. Training Protocol

Before running the experiments, we created ten random folds from the real UAV image pool. For each fold, we sampled a training set of 44 real images and a validation set of 20 real images. For the baseline in each fold, we fine-tuned YOLOv8x for 250 epochs using only the 44 real training images and evaluated on the fixed set of 20 real validation images. We then repeated training for another 250 epochs after expanding the training set with synthetic images (either diffusion-inpainted images or simulator-generated images, depending on the experiment), while keeping the validation set unchanged and purely real. No additional custom augmentation was introduced beyond the default augmentations already present in the YOLO training pipeline. Therefore, within each fold, the only experimental factor was the presence or absence of synthetic images in the training set, which isolates the effect of synthetic augmentation on real-image generalization.

6.4. Implementation Details

All the training and validation happened in Google Colaboratory using an NVIDIA L4 GPU instance. The notebook ran on Python 3.10 with PyTorch 2.1.0, and it utilized the YOLO repository at commit 8.0.77 because of the GPL license. Each fine-tuning run for the folds, spanning from epoch 1 to epoch 250, was carried out end-to-end on this single GPU, with model checkpoints saved directly to Google Drive. While the training times varied a bit based on the presence of synthetic images, every fold was still capable of running in a single Colab session. Thanks to the combination of Colab’s L4 GPU, PyTorch, and the YOLO implementation, all ten folds (including both baseline and synthetic-augmented runs) are completely reproducible in the same software environment.

6.5. Evaluation Metrics

We evaluate detection performance using standard COCO-style mean Average Precision (mAP) metrics. mAP50 reports average precision at a single Intersection over Union (IoU) threshold of 0.50, counting a prediction as correct if it overlaps a ground-truth box by at least 50%. mAP50-95 (mAP@[0.5:0.95]) averages mAP across IoU thresholds from 0.50 to 0.95, providing a stricter measure of localization quality. Because the study targets extremely low-data conditions, we report not only the mean mAP across folds but also the cross-fold standard deviation, the minimum/maximum fold performance (as a robustness indicator), and fold-wise comparisons. These stability measures partially address practical concerns that are not visible from a single averaged score. Precision–Recall curves and explicit false-positive/false-negative breakdowns are valuable complementary analyses; however, given the constraints of the current study, we focus on mAP-based evaluation while outlining PR/FP/FN analysis as a priority direction for follow-up work [9].

6.6. Baseline Performance on Real Data Only

The YOLOv8x baseline trained solely on real images (no synthetic augmentation) exhibits substantial variability across the 10 folds, which is expected in an extreme low-data regime. Across folds, mAP50 is 0.579 ± 0.107 (range [0.301, 0.666]), and mAP50-95 is 0.350 ± 0.092 (range [0.148, 0.460]). This corresponds to a wide 95% confidence interval for the mean performance (approximately 0.502–0.655 for mAP50 and 0.285–0.416 for mAP50-95), indicating that average performance estimates from such a small real dataset are uncertain and sensitive to the particular training split. Importantly, the worst-case fold (mAP50 ≈ 0.301, mAP50-95 ≈ 0.148) suggests that real-only training can fail severely under certain conditions, which motivates the use of synthetic augmentation to reduce failure risk and improve stability when real data collection is limited [9]. The results are presented in Table 2 and visualized in Figure 5.

Table 2. Baseline experiments on real data only.

Fold	mAP50	mAP50-95
0	0.6313	0.4601
1	0.6114	0.3062
2	0.6496	0.4048
3	0.6201	0.3678
4	0.6629	0.4517
5	0.4908	0.2732
6	0.5301	0.2831
7	0.3011	0.1479
8	0.6219	0.4153
9	0.6661	0.3933

Figure 5. Baseline experiments on real data only.

6.7. Impact of InPaint Augmented Data

Training with real images plus GenAI-inpainted synthetic images improves both average accuracy and cross-fold stability. Across the 10 folds, mAP50 increases to 0.647 ± 0.013 (range [0.619, 0.667]) and mAP50-95 increases to 0.435 ± 0.013 (range [0.406, 0.456]). Compared to the real-only baseline, this corresponds to a mean improvement of +0.068 in mAP50 and +0.085 in mAP50-95, while reducing the variability across folds by an order of magnitude. Notably, the worst-case mAP50 improves from 0.301 (baseline) to 0.619 with inpainting, which supports the practical value of the approach in reducing failure risk under limited real data. These results are consistent with prior findings that GAN- and diffusion-based augmentation can improve detector robustness by exposing the model to a broader distribution of object–background combinations, even when the fraction of synthetic data is relatively small [32,33]. The results are presented in Table 3.

Table 3. Results with Real + Inpaint Augmented data.

Fold	mAP50	mAP50-95
0	0.6451	0.4320
1	0.6503	0.4385
2	0.6550	0.4432
3	0.6587	0.4471
4	0.6405	0.4288
5	0.6408	0.4283
6	0.6349	0.4247
7	0.6188	0.4062
8	0.6672	0.4565
9	0.6544	0.4450

6.8. Impact of Simulator-Generated Data

Adding simulator-generated images leads to a robustness–accuracy trade-off. Mean performance changes only slightly (mAP50 decreases from 0.579 to 0.546 and mAP50-95 from 0.350 to 0.344), but cross-fold stability improves substantially: the standard deviation drops from 0.107 to 0.033 for mAP50 and from 0.092 to 0.034 for mAP50-95. Most importantly, the worst-case fold improves markedly (minimum mAP50 rises from 0.301 to 0.496 and minimum mAP50-95 from 0.148 to 0.306), indicating that simulation reduces the risk of catastrophic failure on difficult splits. The slightly lower mean accuracy is consistent with a remaining synthetic-to-real domain gap: despite photorealistic rendering, simulator statistics (textures, micro-structure of forest floor, and sensor characteristics) can still differ from real imagery. This observation motivates hybrid approaches that combine the large-scale coverage and controllability of simulation with the realism of compositing/inpainting on real backgrounds. The results are presented in Table 4.

Table 4. Results with Real + Simulator-Generated Data.

Fold	mAP50	mAP50-95
0	0.5251	0.3704
1	0.5381	0.3204
2	0.5224	0.3418
3	0.5667	0.3062
4	0.5370	0.3538
5	0.5689	0.3160
6	0.5574	0.3267
7	0.5377	0.3168
8	0.4962	0.3961
9	0.6058	0.3902

6.9. Ablation Study

Our ablation is a controlled data-composition comparison in which we keep the detector architecture and training protocol fixed and vary only the presence and type of synthetic augmentation (real-only vs. real + inpainting vs. real + simulation). This design isolates the contribution of synthetic data under cross-validation and supports causal attribution of performance changes to the added images. We do not explicitly decompose object-only versus background-only effects. However, the inpainting pipeline modifies object appearance and context in a coupled manner by inserting boulders into diverse real forest-floor regions, which is consistent with prior evidence that background/context augmentation can be a strong driver of robustness [9].

We acknowledge that additional baselines would further strengthen the study, including classical augmentations (e.g., color jitter, geometric transforms) and recently proposed diffusion-based background/scene augmentation methods. Due to the current resource constraints, we focus on the two representative synthetic-data families (inpainting on real backgrounds and simulator-based generation) and position the released datasets and protocol as a benchmark for future direct comparisons.

6.10. Qualitative Results

While the main evaluation in this paper is quantitative (mAP across folds), qualitative inspection is essential for understanding synthetic realism and potential domain artifacts. Figure 3 and Figure 4 provide representative examples of the two augmentation streams. The inpainting approach typically preserves real forest-floor statistics and sensor characteristics because synthetic objects are composited into real backgrounds under a mask constraint, whereas simulator-generated scenes provide broader diversity in viewpoint and illumination through domain randomization.

A dedicated qualitative error taxonomy for detector outputs (false positives/false negatives and typical failure cases) and full Precision–Recall curves would further strengthen interpretability; we therefore include these as priority follow-up analyses enabled by the released datasets and reproducible training protocol. In the current submission, we emphasize robustness indicators visible in the fold-wise results (variance and worst-case fold behavior) as practical proxies for failure risk under low data.

7. Discussion

Because this study focuses on image-level obstacle detection rather than spatial mapping, all analyses are conducted on non-georeferenced RGB imagery. LiDAR/GNSS are used operationally during flight and logged as metadata, but they are not used to produce or evaluate geospatial outputs in the experiments. This choice isolates the computer-vision question of rare-object detection under low data; downstream fusion with mapping or planning systems is discussed as an application pathway rather than an evaluated component.

Across folds, adding synthetic images (either inpainted or simulator-generated) improves generalization stability relative to real-only training by reducing cross-fold variance and lifting worst-case performance. Given the very small real dataset (64 images), we avoid over-claiming full real-world generalization; however, the consistent reduction in variance and the mitigation of catastrophic low-performance folds provide evidence that augmentation reduces failure risk in low-data conditions. The observed trade-off between inpainting (higher mean accuracy) and simulation (strong robustness gains despite a remaining domain gap) further clarifies when each synthetic-data family is most beneficial.

Although the study’s experiments were specifically designed to identify large boulders in post-glacial forests in Scandinavia, the methodology is not restricted to this particular object class. Anthills, culturally protected sacred sites, or even remaining tree stumps that endanger machinery can all be addressed with the same generative augmentation pipeline. Low data availability is a persistent issue in each scenario, and inpainting and 3D simulation offer a workable solution for generating the diversity required for model training.

Additionally, all datasets are supplied in COCO format, which ensures complete experimental reproducibility and eliminates the need for external ISO-compliant descriptions by encapsulating both annotation structure and metadata.

7.1. Analysis of Results

The findings clearly show that generative augmentation significantly boosts both accuracy and stability. So, why does GenAI enhance generalization? For starters, it dramatically broadens the diversity of training data. Real forest scenes are quite limited, which causes the baseline model to overfit those few examples, leading to high variance. Generative models can produce a wide range of realistic new images: as previous studies have pointed out, GANs and diffusion models “offer highly diverse and realistic representations while maintaining natural variations” [33]. By introducing the YOLO model to these variations, it picks up features that are not tied to any specific scene. This supports the idea that data augmentation helps reduce dataset bias; without it, “a lack of data diversity will hinder model performance” [9].

The detailed results are presented in Table 5 and visualization for each fold before and after augmentation is presented in Figure 6 (mAP50) and Figure 7 (mAP50-95).

Table 5. Comparison of results, real only data (P) vs. real + inpaint augmented data (G).

Fold	mAP50_P	mAP50_G	mAP50-95_P	mAP50-95_G	∆ mAP50	∆ mAP50-95	%∆ mAP50	%∆ mAP50-95
0	0.6313	0.6451	0.4601	0.4320	0.0138	−0.0281	2.19	−6.11
1	0.6114	0.6503	0.3062	0.4385	0.0389	0.1323	6.36	43.21
2	0.6496	0.6550	0.4048	0.4432	0.0054	0.0384	0.83	9.49
3	0.6201	0.6587	0.3678	0.4471	0.0386	0.0793	6.22	21.56
4	0.6629	0.6405	0.4517	0.4288	−0.0224	−0.0229	−3.38	−5.07
5	0.4908	0.6408	0.2732	0.4283	0.1500	0.1551	30.56	56.77
6	0.5301	0.6349	0.2831	0.4247	0.1048	0.1416	19.77	50.02
7	0.3011	0.6188	0.1479	0.4062	0.3177	0.2583	105.51	174.65
8	0.6219	0.6672	0.4153	0.4565	0.0453	0.0412	7.28	9.92
9	0.6661	0.6544	0.3933	0.4450	−0.0117	0.0517	−1.76	13.15

Figure 6. Comparison of results, real only data vs. real + inpaint augmented data (mAP50).

Figure 7. Comparison of results, real only data vs. real + inpaint augmented data (mAP50-95).

Let us dive into the second point: augmented images tend to present tougher challenges. By using inpainting, we can place boulders in fresh backgrounds or tweak their looks, which pushes the model to identify the object in a wider range of contexts. This can be thought of as a kind of implicit “hard example mining,” where the model has to tackle scenarios it wouldn’t typically face. Research indicates that data augmentation using GANs often results in better detection accuracy compared to just training on the original dataset [32]. In our experience, we noticed a significant boost in mAP50-95, which is a pretty strict metric. This improvement hints that the augmented model has honed its ability to localize or detect objects under tougher IoU standards—likely because it encountered a broader array of object-background combinations. To sum it up, generative data enhances the feature distribution and smooths out the quirks of each fold, ultimately leading to a more resilient classifier.

Finally, these findings resonate with previous research on model stability. For example, Li et al. [9] discovered that using background augmentation through text-to-image inpainting “boosts the model performance” and significantly enhances robustness. In our situation, the inpainted images probably served a similar purpose. Likewise, earlier studies in a different field demonstrated that even a small percentage of synthetic images (just 10%) can greatly improve YOLO detection [33]. This supports the notion that a bit of synthetic augmentation can make a big difference. To sum it up, GenAI augmentation enhances generalization by diversifying the training data and highlighting a wider range of object contexts, which helps reduce overfitting and leads to stable, high-performing detection across all folds.

Similarly, findings using 3D simulated data really shine a light on a classic dilemma we face in low-data situations: aiming for the highest score on familiar datasets can lead to models that are quite fragile and may fail dramatically when faced with tougher challenges. The bar charts in Figure 8 and Figure 9, along with the detailed breakdown in Table 6, clearly show how data generated by simulators can transform detector performance.

Figure 8. Comparison of results, real only data vs. real + simulator augmented data (mAP50).

Figure 9. Comparison of results, real only data vs. real + simulator augmented data (mAP50-95).

Table 6. Comparison of results, real only data (P) vs. real + simulator augmented data (G).

Fold	mAP50_P	mAP50_G	mAP50-95_P	mAP50-95_G	∆ mAP50	∆ mAP50-95	%∆ mAP50	%∆ mAP50-95
0	0.63	0.53	0.46	0.37	−0.11	−0.09	−16.82	−19.50
1	0.61	0.54	0.31	0.32	−0.07	−0.01	−11.99	4.64
2	0.65	0.52	0.40	0.34	−0.13	−0.06	−19.58	−15.56
3	0.62	0.57	0.37	0.31	−0.05	−0.06	−8.61	−16.75
4	0.66	0.54	0.45	0.35	−0.13	−0.10	−18.99	−21.67
5	0.49	0.57	0.27	0.32	0.08	0.04	15.91	15.67
6	0.53	0.56	0.28	0.33	0.03	0.04	5.15	15.40
7	0.30	0.54	0.15	0.32	0.24	0.17	78.58	114.20
8	0.62	0.50	0.42	0.40	−0.13	−0.02	−20.21	−4.62
9	0.67	0.61	0.39	0.39	−0.06	−0.00	−9.05	−0.79

In Figure 8, which displays the per-experiment mAP50 for the baseline compared to the simulator-enhanced model, you will notice that most of the orange bars are just a notch below the yellow bars, and the significant dip at experiment 7 is nearly gone. Figure 9, which illustrates mAP50-95, reflects this trend as well: the simulator model keeps a more consistent performance across all splits, while the baseline experiences sharp drops in folds 1 and 7.

Table 6 backs up these observations: the “∆ mAP50” and “∆ mAP50-95” columns reveal that most folds only show slight absolute declines (rarely going below −10 percentage points), while the “% ∆” columns indicate that the minimum mAP50 rises from 0.301 to 0.496 and the minimum mAP50-95 increases from 0.148 to 0.316. By providing the detector with a richer mix of viewpoints, lighting conditions, and occlusions from the simulator, we effectively help it avoid overfitting to our limited real-image sample—significantly reducing cross-fold variance (by over 70 percent for mAP50 and more than 65 percent for mAP50-95) and enhancing worst-case performance. However, the fact that inpainting outshines pure simulation suggests that bridging the domain gap requires not just diversity but also a touch of realism—integrating synthetic objects into real backgrounds more seamlessly. Future research should explore hybrid approaches that merge the extensive coverage of simulated environments with the lifelike quality of advanced compositing techniques.

7.2. Practical Implications

The improvements in stability and accuracy bring real-world advantages for tasks such as 3D reconstruction and mission planning. For instance, being able to reliably detect boulders is essential for creating precise 3D maps of a forest scene. If the detector overlooks or misplaces rocks, the resulting geometry will be off. The enhanced model’s consistent performance allows it to spot boulders across a broader range of conditions, like varying lighting and obstructions, resulting in more comprehensive 3D models. Similarly, in autonomous mission planning, such as a rover navigating through a forest, planners depend on object detection to steer clear of obstacles. A model that shows high variability could lead to unexpected failures, but the consistency of the GenAI-trained model means that planners can have confidence in its outputs. In the field of robotics, the value of synthetic training data is already well-established; for example, ANYbotics has found that generating thousands of synthetic images helps a robot learn inspection tasks from various angles and lighting conditions, all while using significantly less real data. By comparison, our inpainted images prepare the detector for the variability it will encounter in the real world. Therefore, these findings indicate that integrating GenAI-augmented imagery enhances the detection system’s robustness and reliability in actual deployments.

Furthermore, this research’s practical value is found in its ability to support more sustainable and effective forestry operations. Reliable obstacle detection lowers the chance of equipment damage and downtime, facilitates more precise mission planning, and aids in harvesting route optimization to cut CO₂ emissions and operating expenses. The suggested approach directly supports the twin objectives of economic productivity and environmental sustainability in contemporary forestry by facilitating safer and more effective operation of autonomous harvesters. Additionally, the experiment is not limited to the Swedish test site. While boreal regions in Canada pose comparable scale and accessibility challenges, forests in Finland and Norway share similar glacially shaped landscapes and low-diversity obstacle patterns. Because the pipeline is built to function without relying on region-specific metadata, the method can be implemented in a variety of forestry contexts with minimal changes.

This research has implications for autonomous navigation in other unstructured environments outside of forestry. Rare but important objects must be found in situations where data is scarce, such as disaster areas, agricultural fields, and construction sites. Thus, generative augmentation could be used as a general method for building strong models in fields where collecting a lot of data is either impossible or impractical.

7.3. Limitations

While our GenAI augmentation offers some clear advantages, there are still a few important considerations to keep in mind. For starters, inpainted images can sometimes come with subtle flaws—like odd textures, unexpected color changes, or blending seams—that could unintentionally skew the model if we are not careful. To tackle this, we implemented a rigorous two-step filtration process: first, an automated pixel-based filter to catch any obvious issues, and then two separate human reviews to weed out any images that still had artifacts. Secondly, even though our enhanced dataset includes a variety of forest scenes, it only covers a fraction of the world’s forest biomes. This means we cannot guarantee that it will generalize well to completely different types of forests, such as tropical rainforests or boreal woodlands. Lastly, since inpainting masks often pick up extra background elements along with boulders, we had to manually redraw all the bounding-box annotations, which added quite a bit of extra work. These measures help us ensure high-quality training data, but they also remind us that careful curation of the dataset is crucial when using GenAI augmentation in real-world scenarios.

8. Conclusions

This work is motivated by the lack of annotated training data for rare but operationally relevant forestry obstacles. We evaluated a synthetic-data augmentation framework for low-data UAV RGB object detection using two complementary pipelines. The first pipeline performs segmentation-guided diffusion inpainting: SegFormer-derived forest-floor masks constrain Stable Diffusion inpainting so that synthetic boulders are inserted only into plausible ground regions of real images. The second pipeline generates labeled scenes in a photorealistic 3D simulator (Unreal Engine 5) with controlled domain randomization across viewpoint, illumination, vegetation, and occlusion patterns. We then fine-tuned a fixed YOLOv8x detector under an identical training protocol and quantified performance using ten-fold cross-validation on a real UAV dataset of 64 images.

Quantitatively, inpainting improves mean detection accuracy and substantially reduces instability: mean mAP50 increases from approximately 0.579 (real-only) to 0.647 and mean mAP50-95 from 0.350 to 0.435, while the worst-case fold improves from 0.301 to 0.619 in mAP50. Simulator augmentation yields slightly lower mean accuracy (mAP50 ≈ 0.546; mAP50-95 ≈ 0.344) but provides strong robustness gains by mitigating collapse on difficult splits (minimum mAP50 0.496 vs. 0.301). These results support the conclusion that carefully curated generative augmentation can reduce failure risk and improve generalization stability in extremely data-limited forestry detection settings.

Finally, while this paper does not evaluate georeferenced mapping, the improved image-level detector can serve as a component in downstream systems such as hazard mapping, autonomous navigation, or decision support, where detections may later be fused with 3D reconstruction or GIS workflows. The released datasets and reproducible protocol are intended to facilitate direct comparisons against alternative augmentation and domain adaptation strategies in future work.

This study shows that generative augmentation, which combines inpainting and simulation, can effectively solve the problem of having too little data for forestry object detection. Our method enhances the performance and stability of YOLO-based models when utilized with UAV and LiDAR data from Scandinavian post-glacial forests by generating varied synthetic training samples. The findings validate that, even in fields with limited annotated imagery, AI models can be trained to attain reliable detection performance when strengthened by carefully constructed augmentation strategies.

The results have direct effects on forestry operations in addition to their technical value. Better detection of big rocks and other obstacles lowers the risk of damage to machinery, makes mission planning safer and more efficient, and helps save money and lower CO₂ emissions. Boulders were selected as a case study due to their operational significance within the project context; however, the same methodology is applicable to other object categories, including anthills, cultural sites, or ecological features necessitating protection.

The work also shows what is wrong with the current situation. UAV-based data collection is still limited by thick canopy cover and changing weather, which even generative methods cannot fully fix. Future research should investigate multimodal data fusion, integrating UAV imagery with LiDAR and potentially satellite or terrestrial sensing, to enhance detection efficacy in more challenging environments.

In conclusion, this research provides a reproducible, FAIR-compliant methodology for low-data object detection in forestry. By connecting new methods with real-world effects, it shows how AI can help with more sustainable land management and opens up possibilities for using these methods in other areas where rare but important objects need to be reliably identified.

9. Future Work

Expanding the synthetic data pipeline to mimic different weather conditions—like rain, fog, and snow—as well as changes in lighting for forest imagery can really boost detection performance in tough situations [34]. We can introduce weather effects using physically based rendering in a game engine, tweaking particle systems, volumetric clouds, and wet surfaces. Alternatively, we could use generative diffusion models to transform clear-weather images into rainy or foggy versions [35]. Training with these weather-enhanced images has been proven to improve a detector’s ability to adapt to real-world changes in precipitation and low-visibility conditions, which helps cut down on false negatives when the weather gets rough. Future efforts could also look at seasonal changes in the canopy, like leaf-on or leaf-off and autumn colors, to make sure the model stays strong all year round [36].

Deploying the YOLOv8x model on UAV-embedded hardware for real-time detection during flight is essential for avoiding obstacles and adapting to mission needs. To make this work, we will need to optimize the network using methods like structured pruning, weight quantization, or knowledge distillation, all while keeping in mind the computational and power limits of the onboard systems [37]. Additionally, looking into lightweight variant architectures, such as YOLO–Nano or YOLO–Tiny, can help achieve fast inference speeds without sacrificing too much accuracy [38]. Future efforts should also focus on creating efficient model compression pipelines that utilize edge offloading. For example, we could stream lower-resolution data to a companion processor for initial inference, while keeping critical detections on the UAV for quick decision-making. By integrating real-time detection with flight control logic, the UAV can adjust its path dynamically around obstacles, rather than just sticking to preplanned routes.

Using a synthetic-augmented model as a pretrained backbone can really speed up the fine-tuning process for various detection tasks in forestry. For instance, a model that was originally trained to detect boulders could easily be tweaked to spot fallen logs, tree stumps, or even animal dens, all with just a few new real annotations. Plus, techniques like adversarial feature alignment or meta-learning can help transfer knowledge from one forest biome to another, making it possible to generalize across different sites without having to completely rebuild the dataset [39]. Fine-tuning with just a handful of labeled examples from a new environment (known as few-shot learning) allows the detector to quickly adjust to changes in tree species, ground cover, or rock shapes. Expanding the generative augmentation framework to create synthetic examples of these new object classes can further lighten the annotation load and speed up model generalization.

Blending detection outputs with UAV-based mapping and navigation systems is set to boost both the usefulness of data and the safety of operations. Once boulders are detected and pinpointed in 2D images, they can be transformed into a 3D environmental model—created through photogrammetric reconstruction or LiDAR point clouds—to accurately highlight hazardous objects on the ground [40]. Previous studies have demonstrated that merging object segmentation with route planning algorithms can lead to collision-free UAV flights in structured settings; applying this to forest areas means integrating segmentation results with 3D occupancy grids and real-time path planners (like RRT*, A*). As the UAV navigates through the forest, the detection system can refresh the map on the fly and modify flight paths around newly spotted obstacles, paving the way for fully autonomous navigation in dense forest environments. Future research should also look into combining RGB detections with multispectral and LiDAR data to enhance obstacle localization in three dimensions, ultimately improving the accuracy of the environmental model and the dependability of future mission planning.

One exciting direction to explore is the addition of various sensor types—like near-infrared (NIR), thermal, or LiDAR—into the generative augmentation process [41]. By training diffusion models or GANs to create synthetic NIR or depth images alongside RGB images [42], we can enhance the training dataset with multimodal pairs. This approach helps to boost the detector’s features across different spectral conditions. Additionally, synthetic datasets created in a game engine can feature simulated LiDAR point clouds and depth maps, which can be used to pretrain models in a self-supervised way, ultimately enhancing performance on real RGB images derived from LiDAR. Cross-modal translation—where one type of data (like thermal) is generated from another (like RGB)—could also improve reliability, especially when a specific sensor struggles or does not perform well in dense foliage. By integrating these multimodal enhancements, we expect to see a significant boost in detection accuracy in situations where RGB alone falls short due to low contrast or heavy obstruction [37].

To cut down on the time spent on manual annotation, future research will explore self-supervised and weakly supervised learning techniques specifically designed for forestry imagery. We can take advantage of large sets of unlabeled UAV images to pretrain feature extractors using methods like contrastive learning or masked-image modeling. This approach helps create representations that need fewer labels when it comes to fine-tuning. Likewise, weakly supervised techniques—such as multiple-instance learning or class activation mapping—can help produce approximate bounding boxes from image-level labels or rough geospatial hints (like known boulder geotags) [43]. By continuously refining pseudo-labels and using active learning strategies (where the model asks for annotations on the most informative samples), we can significantly lessen the need for exhaustive manual bounding-box annotation. In the case of boulder detection, this might mean utilizing existing forestry maps or logging records to provide weak labels, which can then be iteratively improved through model predictions and a few user corrections.

The future of UAV-based object detection in forestry is looking bright! These upcoming directions are all about building a stronger, more flexible, and efficient framework. By weaving in weather and seasonal simulations, allowing for real-time on-board analysis, supporting cross-domain transfer learning, and exploring label-efficient strategies, the next wave of models will be much better at tackling the unique challenges and variations found in natural forest environments.