AgrySET: A Synthetic Data Generation Framework for Agricultural Semantic Segmentation and Defect Recognition

Abstract

While existing synthetic generation methods, such as Generative Adversarial Networks (GANs) and Diffusion Models, typically require large pre-existing datasets and substantial computational resources, this paper presents AgrySET: a novel hardware–software framework that bypasses these limitations. The system utilizes a four-camera high-resolution array and a setup that replicates industrial environmental conditions. Unlike purely algorithmic approaches, by employing Gaussian models for data generation on physically acquired samples, the framework actively addresses class imbalance issues and synthesizes realistic agricultural imagery alongside exact, pixel-perfect segmentation masks without manual pixel-level annotation after initial sample acquisition. As a case study, the segmentation of healthy coffee beans and seven defects as categorized by the National Federation of Coffee Growers of Colombia was performed. Models trained exclusively on this synthetic dataset achieved a Mean Intersection over Union (mIoU) of 92.9% in a 3-class configuration and 61.9% in a detailed 9-class scenario. When evaluated against real-world datasets, the model yielded accuracies of 90% and 58%, respectively. Furthermore, the automated pipeline accelerated the data generation and segmentation workflow by an estimated factor of 250 compared to manual execution. These quantitative results demonstrate the feasibility of AgrySET as a baseline synthetic framework for coffee bean inspection, highlighting its potential for future adaptation to other granular agricultural commodities.

1. Introduction

The global agricultural sector is facing an unprecedented crisis: the challenge of feeding a growing population under the pressures of climate change and biodiversity loss [1]. In this context, precision agriculture serves as a vital solution, utilizing cost-effective, intelligent technologies to optimize input usage. This approach facilitates a shift from conventional methods toward highly automated management systems that maximize efficiency per square meter [2]. Within this technological evolution, coffee cultivation has emerged as a primary focus of research, where precision agriculture techniques are being intensively explored to enhance sustainability and productivity [3]. The integration of Cyber-Physical Systems (CPS) and the Internet of Things (IoT) has further accelerated this transition, enabling real-time monitoring of crop health and environmental variables. However, the true transformative power of these systems lies in their ability to process vast amounts of visual data through advanced computational frameworks.

The primary drivers of this transformation are Artificial Intelligence (AI) and Deep Learning (DL), which specialize in developing mathematical models that learn directly from data [4]. These fields employ diverse architectures, ranging from statistical methods like Logistic Regression (LR) [5] and Expectation–Maximization (EM) [6] to complex structures such as Convolutional Neural Networks (CNN) [7] and Support Vector Machines (SVM) [8]. While traditional machine learning relies on handcrafted features, DL architectures autonomously extract hierarchical representations, allowing for a more nuanced understanding of complex biological patterns in agricultural products.

Deep Learning has played a pivotal role in precision agriculture [9], particularly in object detection [10] and semantic segmentation [11], due to its ability to extract pixel-level information with high precision. For instance, in greenhouse farming [12], semantic segmentation enables detailed crop monitoring, including canopy area estimation, plant height measurement, and fruit maturity analysis. Furthermore, pest segmentation [13] allows for targeted pesticide application. In coffee production specifically, multispectral imaging combined with machine learning has proven effective for both estimating plant yield [14] and the early detection of critical diseases such as coffee leaf rust (Hemileia vastatrix) [15,16]. These applications demonstrate that the granularity of semantic segmentation is essential for tasks where the mere presence of an object is insufficient, and its exact shape, size, and texture determine the subsequent agricultural intervention.

Despite the power of DL, the availability of high-quality data often limits model robustness, especially for tasks requiring pixel-precise annotations [17]. Manual annotation is time-consuming, expensive, and susceptible to human error and inter-observer variability. Consequently, obtaining large-scale annotated datasets remains a significant bottleneck for supervised learning. This “data hunger” is particularly problematic in specialized domains where expert knowledge is required to distinguish between subtle morphological features. While synthetic generation methods like Denoising Diffusion Probabilistic Models (DDPM), Generative Adversarial Networks (GANs) [18], and Variational Autoencoders (VAEs) [19] have made strides, they are often restricted to inpainting [20], super-resolution, or text-to-image generation [21]. Furthermore, these generative models frequently require substantial computational resources and a pre-existing large dataset to train effectively, creating a paradoxical barrier for low-resource agricultural applications.

In the specific case of the coffee industry, quality assessment is a critical factor for international market competitiveness. The classification of green coffee beans traditionally relies on manual inspection by certified “catadores,” a process that is difficult to scale and standardize across large production volumes. Semantic segmentation offers a path toward industrial automation; however, the scarcity of publicly available, high-resolution datasets of diverse coffee defects hinders the development of deployable AI solutions.

Therefore, this paper proposes AgrySET, a holistic hardware–software framework designed to bridge the sim-to-real gap in agricultural semantic segmentation. Unlike conventional data augmentation techniques or computationally heavy generative models (such as GANs and diffusion models) that often introduce artifacts or demand extensive preliminary data, AgrySET employs a deterministic, procedural synthesis approach. By combining a synchronized high-resolution multi-camera array with Gaussian probability modeling derived from industrial milling statistics, the framework autonomously generates high-fidelity imagery and exact pixel-level ground truth masks while natively solving class imbalance. The framework is evaluated within the agricultural domain, focusing on the segmentation of high-quality coffee beans and seven defects categorized by the National Federation of Coffee Growers of Colombia. Specifically, the performance of a segmentation model trained solely on generated synthetic data is externally validated using real-world images from the local industry. The contributions of this work include: (i) a novel hardware–software pipeline for high-resolution synthetic data synthesis, (ii) a methodology to mitigate class imbalance in defect detection, and (iii) a comparative analysis demonstrating the transferability of synthetic-to-real knowledge in agricultural vision tasks.

2. Materials and Methods

This section details the hardware and software components integrated into the development of the framework for generating synthetic datasets tailored for semantic segmentation in precision agriculture. The proposed system, AgrySET, comprises a high-resolution acquisition module and a specialized processing pipeline designed to ensure data fidelity and annotation accuracy.

2.1. Hardware Components

The synthetic database generation system begins with the construction of a stable platform designed to house the samples and high-resolution cameras at a constant distance of 25 cm. To ensure total reproducibility and eliminate ambient light interference, the platform features a closed protective enclosure. Inside, a dedicated diffused LED lighting source (operating at a standardized color temperature of 6500 K/Daylight) is distributed uniformly to maintain homogeneous, shadow-free luminous conditions across the entire Field of View (FoV). The platform design is illustrated in Figure 1.

Figure 1. Schematic diagram of the image acquisition platform.

The platform measures 80 cm in height, 70 cm in width, and 70 cm in depth. Internally, it features an acquisition area of 50 cm by 50 cm and an adjustable focal distance of 25 cm for four cameras. The platform is engineered to simulate the passage of a conveyor belt through its interior accounting for potential elevation changes while acquiring high-resolution imagery. Internally, four Intel RealSense D435 cameras [22] are positioned on an adjustable mount to control the working distance from the samples, as shown in Figure 2.

Figure 2. Spatial configuration of the Intel RealSense D435 cameras.

The Intel RealSense D435 sensors were selected for their hardware-level capability to acquire synchronized RGB and Infrared (IR) streams. While the physical capture framework records both modal streams to build a multimodal repository for extended research, the segmentation benchmark in this study relies exclusively on high-resolution RGB data. This design choice prioritizes the precise chromatic fidelity and surface morphological features required by the National Federation of Coffee Growers’ (FNC) visual standards. Following hardware integration, a rigorous calibration process was conducted to ensure simultaneous acquisition and the spatial continuity of the segmentation masks. This process is divided into two critical phases:

• Intrinsic Characterization

Each sensor was individually calibrated using a $9\times 6$ checkerboard pattern. This procedure determines the camera matrix (K), which maps 3D world coordinates to 2D image plane coordinates:

$$K=\left[\begin{array}{ccc}{f}_x& s& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]$$

(1)

where $f_x$ and $f_y$ represent focal lengths, and $(c_x,c_y)$ denotes the principal point. Additionally, radial distortion coefficients ($k_1,k_2,p_1,p_2$) were estimated to correct the “barrel” effect induced by the D435’s wide-angle lenses, ensuring that straight lines within the industrial environment remain preserved in the synthetic data.

• Extrinsic Characterization

Since the AgrySET system fuses four fields of view (FoV) into a single composite image of $4320\times 1980$ pixels, transformation matrices $\left[R\right|t]$ (Rotation and Translation) were defined relative to a global coordinate system located at the center of the conveyor belt.

• Nadir Alignment: Cameras were positioned at a $0^{\circ }$ tilt angle relative to the workspace normal to minimize perspective distortion.
• Security Overlap: A 15% overlap between adjacent areas was established to facilitate the control-point-based stitching algorithm. This ensures that beans located at sensor boundaries do not exhibit morphological discontinuities that could degrade the performance of the DeepLab network.

The mean reprojection error after calibration was $0.24$ pixels. Although the D435 includes a depth sensor, AgrySET prioritized the high-resolution RGB sensor to maintain the chromatic fidelity required by the National Federation of Coffee Growers standards. Finally, rigorous image quality control procedures were implemented at the software level. Automatic white balance and auto-exposure settings were permanently disabled and manually locked across all four sensors to prevent inter-camera color variance and temporal brightness shifts. This ensures chromatic consistency across all synthesized grains regardless of the acquisition batch. Additionally, image sharpness was continuously monitored during acquisition to prevent focus degradation, ensuring that the defined metric correspondence of $21.09\times {10}^{-3}{\mathrm{mm}}^2$ per pixel remained strictly accurate for sub-millimeter defect detection.

2.2. Software Components

The system was programmed in Python 3.9 to enable the synchronous acquisition of four images in both RGB and IR formats. These are integrated into a single high-resolution output using an image stitching methodology [23], as shown in Figure 3.

Figure 3. Image stitching of individual coffee bean samples.

The images are fused in pairs to ensure precise feature matching, generating two intermediate images before repeating the process to create a final composite with four times the original dimensions. The selection of a multi-camera stitching approach over a single wide-angle sensor was crucial to maximize the Field of View (effectively covering an industrial conveyor belt) while strictly preserving the high spatial resolution ($21.09\times {10}^{-3}{\mathrm{mm}}^2$ per pixel) required to detect sub-millimeter defects like insect holes. To achieve seamless alignment, a visual overlap of up to 15% is required; consequently, the final image dimensions may vary. To standardize the output, a resampling process is performed using resolution adjustment based on diffusion models [24]. This specific method was chosen over traditional interpolation techniques (e.g., bilinear or bicubic resampling) because standard algorithms tend to smooth and blur high-frequency details during resizing. Diffusion-based resampling, conversely, standardizes the image dimensions while actively preserving edge sharpness and textural fidelity. This ensures that the subtle morphological characteristics of the defective beans remain crisp and uncompromised for the subsequent DeepLabV3+ segmentation.

2.3. Sample Acquisition and Processing

The database creation process begins with the individual acquisition of samples for each defined class. This approach is strategically implemented to mitigate class imbalance, a recurrent challenge that significantly impacts the performance of semantic segmentation frameworks [25]. For this case study, a dataset for coffee bean defect detection was developed using samples provided by the automated milling plant at the Center for Agricultural Innovation and Productivity (CINGTAGRO). Images were captured for each specific type of defect, and these defects are described in Table 1.

Table 1. Classification of physical defects in green coffee, their main origin in the value chain, and their sensory impact on cup quality [26].

ID	Physical Defect	Main Origin in the Chain	Cup Quality Impact (Flavor/Aroma)
1	Full or partial black	Nutritional deficiencies, severe drought, poor harvesting, or over-fermentation.	Dirty cup, phenolic, sour, earthy, or medicinal flavors.
2	Sour/Cardenillo	Fungal growth due to excess moisture or delays in starting the drying process.	Moldy, earthy, or dirty cup taste.
3	Vinegar or partially vinegar	Over-fermentation during wet processing or use of contaminated water.	Sour, acetic, fermented, or onion-like flavors.
4	Crystallized	High temperatures during mechanical drying (excessive heat exposure).	Flat cup, loss of acidity, and woody notes.
5	Bleached veiny	Re-wetting of the beans after drying due to inadequate storage.	Stale, straw-like, or woody flavors.
6	Bleached old/aged	Prolonged storage or storage under high temperature and humidity conditions.	Old, stale, cardboard, or woody flavors.
7	Bleached amber, buttery o discolored	Soil nutritional deficiencies (especially lack of iron or extreme soil acidity).	Loss of acidity, flat cup, or thin body.
8	Bleached over-dried	Excessive moisture loss due to overexposure during the drying phase.	Woody, straw-like, or empty/flat cup.
9	Bitten or cut	Improperly calibrated pulper or harvesting overripe cherries (mechanical damage).	Uneven roasting, high susceptibility to fungal contamination.
10	Insect-damaged	Pest attack in the field (mainly the coffee berry borer, Hypothenemus hampei).	Bitter, earthy, moldy flavors, or dirty cup.
11	Shriveled or withered	Lack of water during fruit development (prolonged droughts) or weak trees.	Grassy, herbal, or pronounced bitter flavors.
12	Immature and/or sticks	Harvesting unripe (green) cherries or severe drought halting bean development.	Astringent, grassy, or straw-like flavor.
13	Crushed	Mechanical damage during wet milling or hulling with uncalibrated equipment.	Uneven roasting, high susceptibility to mold, and earthy flavors.
14	Spongy/Flojo	Insufficient drying or storing in highly humid environments.	Moldy, earthy, or underdeveloped flavor.

Source: Adapted from the official visual defect guide by Almacafé and the National Federation of Coffee Growers of Colombia (FNC).

Although the National Federation of Coffee Growers of Colombia (FNC) defines a comprehensive classification standard covering 14 categories of physical defects, this study selects a representative subset of seven major defect classes (black, sour, immature, severe borer damage, cut/bite, discolored, and peaberry) alongside healthy Excelso-grade beans and the background material for experimental evaluation. This selection is justified by both their empirical prevalence and their suitability for optical inspection: firstly, an empirical audit of commercial coffee samples analyzed at CINTAGRO confirmed that these seven defect categories account for the vast majority of physical defects observed in regional coffee processing lines; secondly, these specific categories exhibit distinctive surface chromatic and morphological anomalies that can be detected using 2D RGB imaging from a top-down view.

Once the classes were defined, acquisition was carried out under standardized conditions. Regarding the coffee samples, multiple images were captured for each bean, varying the orientation (face) and angle to record the maximum number of morphological features. Subsequently, each image underwent a preprocessing stage utilizing Canny edge detection [27] to extract clean contours. This workflow is detailed in Figure 4.

Figure 4. Class acquisition and preprocessing. (Left): Original image. (Center): Image mask. (Right): Processed image.

The Canny algorithm accurately identifies the grain contours due to the high contrast between the background and the samples. This facilitates the generation of new images, as the lighting conditions remain invariant throughout the database creation process.

2.4. Synthetic Image Database Generation

Following acquisition and preprocessing, synthetic images are generated by overlaying the target classes onto a background. In this specific implementation, defective coffee bean images are superimposed onto a background of high-quality (Excelso) grains, emulating the visual conditions of the CINGTAGRO conveyor belts, as shown in Figure 5.

Figure 5. Overlaying defective coffee beans onto healthy grain backgrounds. (A): Original image. (B): Superimposed defects.

The synthetic image generation pipeline operates in a two-stage procedural architecture: asset library creation and full-scene procedural assembly. First, a Source Asset Library containing 300 individual cropped images per defect class (along with their corresponding Canny-derived binary masks) was physically acquired and extracted from samples provided by CINTAGRO. Utilizing the data from Table 2, a Gaussian probability model was implemented. The assumption of a Gaussian distribution was not arbitrary; it was statistically validated using the 1400 physical samples analyzed at CINTAGRO. A Kolmogorov-Smirnov test confirmed that the natural frequency of defect occurrences across 300 g batches statistically approximates a normal distribution ($p>0.05$).

Table 2. Number and type of defective grains present in a 300-g parchment sample.

Defect [qty]	Mean	Std. Dev.
Excelso	1150	10.01
Black Beans	6.2	9.2
Sour (Vinegar)	12.5	16.4
Severe Broca	27	23.7
Immature	7	9.4
Cut/Bitten	33.6	25.6
Slight Broca (Point)	4.3	11.6
Discolored	1.3	3.7

By empirically validating this distribution, we ensure that the synthetic class imbalance natively mirrors the real-world stochastic probabilities encountered during industrial milling. Each defective bean is overlaid at a random location and rotation (ranging from $-{45}^{\circ }$ to ${45}^{\circ }$) on the background, ensuring high stochastic variability. Finally, automated pixel-level masks are generated to serve as the ground truth for semantic segmentation, as depicted in Figure 6.

Figure 6. Automated mask generation for semantic segmentation. (Left): Original image. (Right): Generated ground truth mask.

To ensure the high fidelity and realism of the generated synthetic datasets prior to deep learning evaluation, the image quality was strictly controlled and evaluated based on three physical and statistical criteria rather than standard generative metrics:

• Geometric and Morphological Fidelity: Evaluated through the strict metric correspondence of $21.09\times {10}^{-3}{\mathrm{mm}}^2$ per pixel and a low reprojection error (0.24 pixels) during calibration. This ensures that sub-millimeter morphological traits, such as the 0.5 mm entry holes of the Broca insect, are preserved without perspective distortion when superimposed.
• Photometric and Chromatic Consistency: Assessed by the seamless blending of the foreground (defects) and background (Excelso beans). By disabling automatic white balance and maintaining a standardized LED enclosure during acquisition, the superimposed beans exhibit zero illumination variance or edge artifacts, achieving natural chromatic integration.
• Contextual and Statistical Realism: Validated against the empirical distribution derived from the 1400 real-world samples from CINTAGRO. The occurrence, spatial density, and random rotation ($-{45}^{\circ }$ to ${45}^{\circ }$) of the generated defects precisely replicate the stochastic behavior observed on physical industrial conveyor belts.

Each image is accompanied by three distinct masks: (i) RGB encoding, (ii) integer-labeled classes (background, Excelso, and 7 defect types), and (iii) a generalized 3-class mask (background, Excelso, and general defect). The color palette used for the inference and visualization process is detailed in Table 3.

Table 3. Color palette and RGB encoding assigned per class for semantic segmentation.

Class	Color	RGB Value
Background	Black	(0, 0, 0)
Excelso	White	(255, 255, 255)
Black	Blue	(0, 0, 255)
Severe Broca	Red	(255, 0, 0)
Cut/Bitten	Green	(0, 255, 0)
Slight Broca (Point)	Dark Green	(0, 128, 128)
Sour (Vinegar)	Cyan	(0, 255, 255)
Immature	Yellow	(255, 255, 0)
Discolored	Magenta	(255, 0, 255)

The dataset generation is fully automated; in this case study, over 6000 high-resolution images ($4320\times 1980$ pixels) were synthesized in 7 h of processing.

2.5. Deep Convolutional Neural Networks

To validate the synthetic dataset quality, experiments were conducted using the DeepLab architecture, a family of convolutional neural networks (CNN) specifically designed for semantic segmentation [28]. DeepLab models leverage Atrous (dilated) convolutions to encode deep features at high resolution without increasing parameter count. The framework utilizes an Atrous Spatial Pyramid Pooling (ASPP) block to resample features at multiple scales, providing high-quality multi-scale information. DeepLabV3+ was selected for this task due to its ability to recover refined edge details by concatenating low-level and high-level features. A ResNet-152 backbone, pre-trained on ImageNet, was employed to handle the high resolution of the AgrySET images ($4320\times 1980$ pixels). This depth is essential for extracting complex semantic features while avoiding the vanishing gradient problem through residual connections. Finally, transfer learning was utilized to accelerate convergence and improve the discrimination of visually similar classes.

3. Results

The proposed synthetic image methodology provides a faithful representation of the acquisition conditions on the CINTAGRO conveyor belts. A sample composite image and its corresponding mask are presented in Figure 7.

Figure 7. Synthetic image and coffee bean mask using the color palette of Table 3. (Left): Original image. (Right): Segmentation mask.

Pior to evaluating neural network performance, the visual and structural quality of the generated synthetic images was quantitatively assessed against real-world benchmark images captured at CINTAGRO. Chromatic consistency was verified through RGB/HSV histogram correlation, achieving an average Pearson coefficient of $R=0.96\pm 0.02$, confirming that locked camera parameters and standardized LED lighting eliminated color drift during image fusion. Furthermore, structural similarity along superimposition contours yielded an average SSIM of $0.92$, demonstrating seamless blending without boundary artifacts while preserving edge sharpness for sub-millimeter defects at the target spatial resolution. Now, training was performed using an SGD optimizer with a learning rate of 0.02 and a weight decay of 0.00001. A crop size of $512\times 512$, a batch size of 8, and a duration of 250 epochs were specified. The dataset was split into 6000 training images and 600 validation images. Performance was evaluated using the Mean Intersection over Union (mIoU) metric on an NVIDIA RTX 2000 (16 GB) GPU. Initial experiments with three classes (background, excelso, and defective) yielded the results shown in Table 4.

Table 4. mIoU for 3-class classification (background, excelso, and defective).

Category	mIoU [%]
Background	92.7
Excelso	96.4
Defective	89.5
Mean	92.9

The DeepLabV3+ architecture effectively recognized the majority of the grains, exceeding 96% mIoU for the “Excelso” (high-quality) class and nearly 90% for the “Defective” (bad) grains in the simplified scenario. This demonstrates that the selected architecture is highly effective and adaptable to the high-resolution imagery generated by AgrySET. The stability and convergence of the training process for these categories can be observed in Figure 8.

Figure 8. DeepLabMean IoU progression on the synthetic coffee defect dataset (3 class).

Once the effectiveness of AgrySET for identifying defective grains was verified in the simplified scenario, further experiments were conducted to evaluate the system’s performance in identifying specific defect categories. This transition from a general binary classification to a multiclass approach aims to assess the model’s ability to discriminate between the subtle morphological and chromatic variations of the seven defects defined by the National Federation of Coffee Growers. While the initial results demonstrate successful pixel identification across the broader categories, the performance varied significantly when decomposing the “Defective” class into seven specific types, as detailed in Table 5.

Table 5. mIoU for 9-class classification (background, excelso, and seven defects).

Category	mIoU [%]
Background	89.8
Excelso	93.2
Black	84.2
Severe Broca	73.1
Cut/Bitten	27.5
Slight Broca (Point)	13.4
Sour (Vinegar)	63.8
Immature	34.5
Discolored	78.2
Mean	61.9

The lower mIoU values for “Slight Broca” and “Cut/Bitten” grains are attributed to the lack of representative visual features in 2D nadir views. For example, the “Slight Broca” defect consists of a sub-millimeter hole; if this hole is positioned on the underside or side of the grain, the visual signature vanishes, making it chromatically identical to an “Excelso” grain. This introduces label noise, degrading the model’s generalization. Similarly, “Cut” grains lose their ellipsoidal structure, and initial convolution layers may confuse fractures with natural grain overlaps.

Training progression is visualized in Figure 9. Both training and validation mIoU maintain a similar trend, converging at approximately 62%, which indicates learning stability and minimal overfitting. The sawtooth behavior in the validation curve is a direct consequence of the small batch size (8) necessitated by the memory footprint of high-resolution images ($4320\times 1980$ pixels), with training stabilizing steadily towards convergence.

Figure 9. DeepLab Mean IoU progression on the synthetic coffee defect dataset (9 class).

The loss decay (from 25% to 4% in training) suggests that while the CNN captures global features (edges and shapes), a gap remains in recognizing “noisy” sub-millimeter details. Inference results across multiple images are shown in Figure 10.

Figure 10. Inference results.(A): RGB image. (B): Original masks. (C): Predicted masks obtained during inference.

To validate the system’s performance in real-world scenarios, an external test set of 150 high-resolution real-world images ($4320\times 1980$ pixels) was manually acquired, annotated, and segmented using the models trained exclusively on synthetic data. The evaluation yielded an accuracy of 90% for the 3-class configuration and 58% for the detailed 9-class scenario, corroborating the proposed framework as a viable solution for defect classification in agricultural supply chains. Furthermore, the operational efficiency was benchmarked: manual preparation, acquisition, and pixel-wise segmentation of a single image required an average of 18 min (totalling 15 h of expert labor for the 150 validation images). In contrast, the AgrySET automated pipeline executes this workflow 250 times faster, demonstrating significant computational and operational scalability.

Finally, a critical validation aspect of AgrySET is the metric correspondence: each pixel represents $21.09\times {10}^{-3}{\mathrm{mm}}^2$. This density allows the model to discern stenopeic features, such as the entry hole of the Broca (0.5 mm to 1.5 mm), ensuring that the mIoU results correlate directly with the physical quality required for coffee exportation.

4. Discussion

The results obtained from the AgrySET framework demonstrate that automated synthetic data generation is a viable and efficient alternative to manual annotation in specialized agricultural tasks. The generation of over 6000 high-resolution images in just 7 h represents a significant advancement over traditional methods, which are often hindered by the “data hunger” and human error associated with pixel-precise labeling. One of the most relevant findings is the high performance achieved in the simplified 3-class classification, with a mean IoU (mIoU) of 92.9%. This indicates that the DeepLabV3+ architecture, combined with a ResNet-152 backbone, successfully extracted stable structural features from synthetic images that remain consistent in real-world scenarios. The use of Atrous Spatial Pyramid Pooling (ASPP) and deep residual feature hierarchies allowed the model to maintain high spatial resolution while discriminating fine-grained surface features, which is critical for identifying subtle defective grains in dense industrial environments. However, the decomposition into 9 classes revealed a performance drop for specific defects such as “Slight Broca” (13.4%) and “Cut/Bitten” grains (27.5%). This behavior aligns with the analysis of “label noise” in 2D nadir projections; as discussed in the hardware characterization, the occlusion of morphological traits in a monocular top-down view makes these grains chromatically identical to healthy “Excelso” beans. This suggests that while synthetic data solves the availability problem, certain micro-defects require multi-view acquisition or additional spectral information to be fully distinguishable. Achieving a 61.9% mIoU in the complex 9-class scenario represents a solid baseline for fine-grained agricultural segmentation. Direct comparisons with general-purpose benchmark datasets like Cityscapes must be contextualized: while urban scene datasets evaluate macro-objects with distinct structural boundaries, agricultural defect inspection faces severe intra-class visual overlap and subtle morphological variations. Categories such as healthy ‘Excelso’ beans and ‘Slight Broca’ or ‘Cut/Bitten’ defects share nearly identical color profiles and surface textures, differing only by sub-millimeter features. Considering these domain-specific constraints, the ability of DeepLabV3+ to discriminate between subtle defect types using exclusively synthetic training data demonstrates strong learning capacity under extreme inter-class similarity. To contextualize this contribution within the broader landscape of agricultural computer vision, AgrySET addresses key limitations inherent to alternative synthetic generation paradigms. Generative models (GANs and Diffusion Models) require massive pre-existing annotated datasets to train and frequently suffer from mode collapse or hallucination artifacts that obliterate sub-millimeter defect boundaries ($21.09\times {10}^{-3}{\mathrm{mm}}^2/\mathrm{px}$). Conversely, 3D physics engine simulations (e.g., Blender, Unreal Engine) rely on labor-intensive CAD modeling of non-rigid organic tissues and heavy rendering overhead, while standard domain randomization applies unconstrained spatial variations that break real-world agricultural co-occurrence statistics. AgrySET overcomes these barriers through a deterministic, physically grounded pipeline: by combining multi-camera physical sample acquisition with an empirical Gaussian probability model derived from 1400 commercial coffee samples, our framework eliminates generative hallucination, guarantees 100% pixel-exact ground-truth mask alignment, and preserves true operational distributions at negligible computational cost. Beyond theoretical metrics, the practical viability of AgrySET was corroborated through an external validation test using 100 manually acquired and annotated real-world images (encompassing over 24,000 individually labeled bean instances). The models evaluated on this real-world set yielded accuracies of 90% and 58% for the 3-class and 9-class configurations, respectively, closely mirroring the synthetic validation results. This high sim-to-real transferability proves that training exclusively on AgrySET synthetic data eliminates the necessity of compiling massive real-world training sets. Indeed, manually acquiring and annotating a real dataset of equivalent scale (6000 images) would require over 1800 h (225 workdays) of expert labor, rendering a ‘real-only’ training regime at scale operationally unfeasible in industrial settings and confirming the standalone value of the proposed synthetic framework. Crucially, this experiment highlighted a profound operational bottleneck in traditional methods: the manual preparation, acquisition, and pixel-wise segmentation of a single image required an average of 18 min (30 h for the 100 validation images). In contrast, the AgrySET automated pipeline mitigates this impracticality, accelerating the workflow by an estimated factor of 250 (based on a standard 8-h workday). This dramatic reduction in labor costs highlights the operational efficiency of the framework for coffee inspection, while its broader industrial scalability remains to be evaluated across additional agricultural products. Finally, the spatial resolution achieved ($21.09\times {10}^{-3}{\mathrm{mm}}^2/\mathrm{px}$) provides a metric correspondence that bridges the gap between digital simulation and the physical quality standards required by the National Federation of Coffee Growers. Nevertheless, key limitations of the current framework must be noted. First, the experimental validation was conducted on a single crop (Coffea arabica). While coffee beans provide a challenging benchmark with 9 defect classes, extending AgrySET to other produce geometries (such as larger fruits or leaves) requires recalibrating the camera distance, lighting intensity, and statistical models. Second, distinguishing visually similar defect classes such as ‘Slight Broca’ (13.4% mIoU) versus mechanical ‘Cut/Bitten’ grains (27.5% mIoU) remains challenging due to overlapping chromatic properties and subtle morphological traits under standard RGB illumination. Future work will focus on integrating NIR sensors to address internal defects and multi-angle optics to capture hidden lateral faces.

5. Conclusions

This research presented AgrySET, a procedural framework for synthetic data generation tailored for semantic segmentation in agricultural defect inspection. The system eliminates the bottleneck of manual annotation by procedurally creating high-resolution scenes alongside exact ground-truth masks. Furthermore, integrating empirical Gaussian probabilistic models provides parametric control over class distributions, ensuring a robust representational balance of minority defect categories prior to neural network training. Semantic segmentation evaluations validated the physical and photometric fidelity of the procedurally synthesized compositions. Deep learning architectures trained exclusively on synthetic data successfully generalized to real industrial inspection environments, confirming that physical photometric consistency and precise spatial alignment suffice to achieve strong sim-to-real knowledge transfer without compiling massive real-world training sets. The study identified fundamental physical limitations inherent to monocular 2D top-down vision systems. While broad defect discrimination is highly effective, isolating morphologically subtle micro-defects—such as slight insect entry holes or fine mechanical cuts—is constrained by feature occlusion in nadir projections. This finding demonstrates that detecting certain sub-millimeter anomalies is bounded by viewing angles rather than model capacity, establishing a clear physical baseline for future multi-view optical research. Operationally, AgrySET drastically reduces the temporal and human labor overhead associated with pixel-wise mask generation, establishing procedural composition as an efficient and practical alternative for agricultural vision systems. While validated on coffee bean defect recognition as a primary case study, the framework demonstrates strong potential as a proof-of-concept for automated sorting, offering a promising direction for future adaptation to other granular agricultural commodities. Future work will pursue both hardware expansion and advanced algorithmic paradigms. On the sensing front, we will integrate Near-Infrared (NIR) channels with RGB data to capture subsurface spectral signatures imperceptible in the visible spectrum. On the algorithmic front, future research will explore Vision Transformers (ViTs), SAM-based foundation models, diffusion generative refinement, self-supervised learning, and domain adaptation techniques to advance toward real-time digital twins for industrial agricultural inspection.