Comparative Analysis of Data Augmentation Strategies Based on YOLOv12 and MCDM for Sustainable Mobility Safety: Multi-Model Ensemble Approach

Abstract

The transportation sector is an important stakeholder in greenhouse gas emissions. Sustainable transportation systems come to the forefront against this problem, with the solutions within the scope of micro-mobility especially attracting attention for their environmentally friendly structures. While micro-mobility vehicles reduce the carbon footprint in transportation, their widespread use remains limited due to various security concerns. In this paper, an image processing-based process was carried out on vehicle and safety equipment usage to provide solutions to the security concerns of micro-mobility users. The effectiveness of frequently used data augmentation techniques was also examined to detect the presence of micro-mobility users and equipment usage with higher accuracy. In this direction, two different datasets (D1_Micro-mobility and D2_Helmet detection) and a total of 46 models were established and the effects of data augmentation techniques on YOLOv12 model performance outputs were evaluated with Preference Ranking Organization Method for Enrichment Evaluations (PROMETHEE), one of the Multi-Criteria Decision-Making (MCDM) methods. In addition, the determination of Multiple Model Ensemble (MME), consisting of multiple data augmentation techniques, was also carried out through the K-means clustering–Elbow method. For D1_Micro-mobility datasets, it is observed that MME improves the model performance by 19.7% in F1-Score and 18.54% in mAP performance metric. For D2_Helmet detection datasets, it is observed that MME improves the model performance by 2.36% only in the Precision metric. The results show that, in general, data augmentation techniques increase model performance in a multidimensional manner.

1. Introduction

Climate change is one of the most important threats of today, which leads to an increase in the frequency of extreme weather events, rising global temperature values, and sea levels due to increased greenhouse gas emissions and the deterioration of air quality. Whatever the reason, it is an inescapable fact that climate change has countless devastating effects on all living things [1,2]. Decision makers have focused on sectors such as transportation, which can be regulated or improved to reduce the effects of climate change. In particular, fine particulate matter (PM2.5 and PM10), which is produced as a result of the burning of fossil fuels, and is frequently used as energy sources in the transportation sector, absorbs sunlight, triggers global warming, and causes imbalances in the climate [3,4]. According to data from the United Nations Framework Convention on Climate Change, the transportation sector is responsible for 28% of greenhouse gas emissions in the USA in 2022 among economic sectors [5]. According to another climate change inventory report, this rate was 29% in the European Union region [5]. To achieve these goals and mitigate the effects of climate change, dependence on fossil fuels must be reduced [6]. This requires the development of sustainable strategies that include radical technological, structural, and behavioral changes in transportation modes, vehicle technologies, and energy sources [7,8]. Sustainable transport plays a vital role in the fight against climate change in cities, allowing for the existence and maintenance of more economically and socially livable urban areas in the long term. One way to increase the sustainability of the transportation sector is to increase and promote new types of transportation vehicles and fuels [8,9]. The main objectives of sustainable transportation systems include preferring renewable energy-fueled vehicles for long distances, encouraging pedestrian transportation for short distances, and preferring micro-mobility vehicles (bicycles, e-bikes, e-scooters, hoverboards, skates, etc.) for medium distances. It is known that micro-mobility sharing systems, especially bicycles and scooters, make significant contributions to the dissemination of the concept of sustainable transportation [10,11]. Individuals who use micro-mobility vehicles for transportation purposes can complete the remaining distance on foot after using these vehicles [12,13]. It has been emphasized that cycling habits are a fundamental solution to reduce air pollutants originating from the transportation sector [9,14,15]. However, micro-mobility users, along with pedestrians, are considered to be the most fragile and vulnerable group among road users [13]. Micro-mobility vehicles are less likely to be used for transportation purposes due to the perceived safety concerns of their users and the deep conflicts between traffic components [16,17,18,19]. The study conducted by [20] in Germany highlighted the importance of addressing the safety of use of electric micro-mobility vehicles, suggesting that the incidence of serious injuries in e-scooter and e-bike users was 13.2% and 17.7%, respectively, while this rate was 5.3% in human-powered bicycles. In a study conducted by [21], when George Washington Hospital Emergency Department data were examined, it was determined that e-scooter and bicycle users had an injury rate of 12.9% and 3.42% per million kilometers, respectively. In a study conducted by [22] in Singapore, it was determined that users of electric motorized devices (such as e-scooters and e-bikes) had a three times higher risk of serious injury compared to users of non-motorized devices (such as kick-scooters and skateboards) between 2015 and 2017, according to data from the Singapore National Trauma Registry. Generally speaking, researchers have conducted various studies on user safety of micro-mobility vehicles and have revealed significant challenges in terms of increasing user safety [23,24,25,26,27,28,29,30].

Studies are being conducted by researchers using various techniques to reduce or eliminate the safety concerns of micro-mobility vehicles. Studies involving the use of machine learning, deep learning and statistical methods to investigate and predict micro-mobility vehicle and other vehicle conflicts [31,32,33,34,35] and the use of computer vision methods to detect micro-mobility vehicle recognition and protective equipment to examine the safety of micro-mobility vehicles under physical conditions [36,37,38,39,40,41,42,43,44,45] are frequently carried out by researchers. Computer vision applications, which allow instant interventions by taking into account the current physical conditions due to the dynamic nature of traffic, and which enable violations to be detected very quickly, can relatively alleviate the security concerns of micro-mobility users. However, there are serious challenges faced by computer vision or object detection applications, which are sub-application areas [46,47,48]. Multi-scale training, detection of relatively smaller objects, necessity of large datasets, smaller-sized datasets etc., are challenges that can trigger stringent performance constraints for object recognition algorithms [49]. To increase the success levels of algorithms that can perform real-time object recognition, the dataset used must be quite large and of high quality [48,50].

To autonomously identify micro-mobility vehicles and instantly detect the use of protective equipment, the datasets used in the object recognition algorithms must have overcome many limitations. Because traffic accidents that threaten human life are irreversible, it is imperative that the success rate of object recognition studies, especially in this area, be at the highest possible level. In this context, data augmentation techniques are a frequently used approach to eliminate the limitations in the dataset in object recognition algorithms [51,52,53,54,55,56]. Many studies in the literature include data augmentation techniques, and there are data augmentation methods developed using complex software [52,57,58,59,60]. However, in most multidisciplinary studies, the use of traditional data augmentation techniques is still an accepted approach [61,62,63,64,65]. Traditional data augmentation techniques may be more advantageous than complex data augmentation techniques in terms of ease and speed of processing the dataset. Various studies have been conducted on the effect of a single complex data augmentation technique on model performance [66,67,68,69]. In the studies conducted by [70,71,72,73,74,75,76,77,78,79,80,81,82], it was observed that the application of data augmentation techniques in object detection algorithms carried out in various fields (health, environment, agriculture, transportation, etc.) improved the model performance. However, traditional data augmentation techniques were not examined one by one in the studies conducted, and it was not clearly revealed which approach had a greater effect on model performance or how the Multi-Model Ensemble (MME), obtained by determining the most effective data augmentation techniques, contributed. This leaves researchers wondering “which data augmentation technique should I use?” and forces them to manually augment their datasets, increasing the time and workload spent on research, and even possibly leading to a loss of motivation for research.

Traditional data augmentation techniques are categorized under two main headings: Image Level Augmentation (ILA) and Bounding Box Level Augmentation (BBLA) and have 22 data augmentation approaches (Flip, 90’Rotate, Crop, Rotation, Shear, GrayScale, Hue, Saturation, Brightness, Exposure, Blur, Noise, Cutout, Mosaic, etc.). Examining each traditional data augmentation technique by considering model performance outputs (mAP, F1 Score, Recall, Precision, Inference Speed, GFLOPs, IoU, etc.) is challenging for researchers because it progresses under the influence of multi-faceted dynamic variables. Evaluating traditional data augmentation techniques based on a scientific infrastructure according to more than one performance metric is a multi-dimensional and complex analysis process that requires the solution of a multi-criteria problem. In addition, the process of determining the MME, consisting of the most effective data augmentation techniques, includes stages that require deep analysis.

This study addresses the critical role of data augmentation techniques in increasing the usage and user safety of micro-mobility vehicles. In the paper, a You Only Look Once (YOLO) model that can detect micro-mobility vehicles and protective equipment has been developed, and 22 data augmentation approaches from ILA and BBLA data augmentation techniques have been applied one by one in order to improve the performance of the model and reduce the workload in the dataset creation and dataset labeling processes. In order to scientifically determine the data augmentation approach that affects or improves the model performance the most, the Preference Ranking Organization Method for Enrichment Evaluations (PROMETHEE) approach from the Multi-Criteria Decision-Making (MCDM) methods has been used. The data augmentation techniques with the highest level of impact on the model performance have been determined by the K-means clustering–Elbow method, and the MME has been determined. Thus, it will be determined which data augmentation MME will be used in object detection studies conducted with concern for the safety of micro-mobility vehicles and using the YOLO algorithm. The main motivation of the study is that, despite the increasing use of micro-mobility vehicles in urban transportation due to changing travel habits, the difficulties encountered in vehicle identification and user safety continue to increase. Another motivation source of the study is that current object detection algorithms cannot detect commonly used micro-mobility vehicles and protective equipment use with a holistic approach. This study aims to contribute to the safer and more effective use of micro-mobility vehicles in urban transportation.

2. Materials and Methods

2.1. Overview

Figure 1 includes three sections: data acquisition and preparation, training model (YOLOv12), ranking and MME, and shows the general outline of the proposed approach. First, visual data for micro-mobility and equipment detection published as open access were collected. Then, ILA and BBLA data augmentation techniques were applied one by one for both datasets, and the datasets were prepared for the model. With this approach, the original and small dataset was effectively expanded, and its diversity was increased. Each dataset was trained separately with YOLOv12, and the model was ranked according to its performance metrics to obtain the most effective data augmentation technique. According to the ranking results, the K-means clustering–Elbow method was applied to determine the optimal number of clusters for MME.

2.2. Data Acquisition and Preparation

The images accessed as open access were collected by the authors, and the original datasets were obtained. Within the scope of the study, two different datasets were created as D1_Micro-Mobility and D2_Helmet_Detection to detect both micro-mobility vehicles and equipment usage. The dataset referred to as D1 and used for the detection of micro-mobility vehicles includes bicycle, e-bike, e-scooter, hoverboard, and skate vehicles. The D2 dataset used for the detection of equipment usage includes the use of helmet equipment. Information about the datasets and their distributions is presented in

Table 1. Description of train, test and valid datasets (no augmentation).

Dataset Name	Dataset Description	Number of Images	Train Images	Test Images	Valid Images
D1	Micro-mobility detection	1029	823	103	103
	Bicycle
	E-bike
	E-scooter
	Hoverboard
	Skate
D2	Helmet detection	336	269	34	33

.

In line with the motivation to train the model realistically, different lighting conditions, weather conditions, and environmental conditions were taken into account. In this context, day and night scenarios, changing light levels, clean surface, rainy surface, snowy surface, and various urban environments are included in the dataset. Thus, the model’s adaptability to different operating conditions has been increased. In order to measure the effect of data augmentation techniques on model performance, the data augmentation techniques detailed in

Table 2. Data augmentation techniques and train, test, and validation datasets.

Category	Data Augmentation Approach	D1-Train Images	D1-Test Images	D1-Valid Images	D2-Train Images	D2-Test Images	D2-Valid Images
	No Augmentation	823	103	103	269	34	33
ILA BBLA	Shear	1646	103	103	538	34	33
ILA BBLA	Rotation
ILA BBLA	Noise
ILA BBLA	Exposure
ILA BBLA	Crop
ILA BBLA	Brightness
ILA BBLA	Blur
ILA BBLA	90° Rotate
ILA BBLA	Flip	1243	103	103	411	34	33
		1646	103	103	538	34	33
ILA	Saturation	1646	103	103	538	34	33
	Mosaic
	Hue
	Cutout
	Grayscale	943	103	103	314	34	33

were applied. Within the scope of this study, similar processes were carried out for D1 and D2 datasets.

Data was augmented using a batch processing tool in the Roboflow environment running on an Intel^® Core™ i5-7400 CPU @ 3.00 GHz Windows 11 64-bit machine. To create the augmentation datasets, 15 different filters were used: shear (±10° horizontal, ±10° vertical), rotation (between −15° and +15°), noise (up to 1.8% of pixels), exposure (between −15% and +15%), crop (0% minimum zoom, 20% maximum zoom), brightness (between −25% and +25%), blur (up to 2.5 px), 90° rotate (clockwise, counter-clockwise), flip (horizontal), saturation (between −30% and +30%), mosaic (applied), hue (between −25° and +25°), cutout (5 boxes with 15% size each), grayscale (15%).

2.3. Object Detection with You Only Look Once

This paper considers the YOLO structure in the object detection process to analyze the effectiveness of data augmentation techniques. YOLO is a real-time object detection algorithm that has recently been frequently used due to its accuracy and speed [83]. The main motivation of YOLO is that it treats the object detection problem as a single regression problem rather than running separate processes for localization and classification as in other object detection approaches [84].

The working principle of the YOLO algorithm is different from other object detection programs. First, the input image is set to a fixed size, such as 448 × 448 or 640 × 640, and then a convolutional neural network with multiple convolutional layers and pooling layers is used for feature extraction [85]. In the resulting feature map, the input image is divided into $S \times S$ grid cells and the confidence scores for each grid cell predict the $C$ bounding boxes along with the $B$ class probabilities. Each bounding box contains predicted values such as confidence score, height, width, and center coordinates. Given an input image of size $W \times H$, the YOLO approach outputs a tensor of size $S \times S \times (B x 5+C)$. Here, 5 represents the four bounding box coordinates and the confidence score. The final output tensor is obtained by processing the input image in multiple convolutional and pooling layers. As a result of all these processes, the object planned to be detected is successfully detected. As a result of completing all methodological steps, the target object will be successfully detected. As in other object detection methods, the performance of the model in the YOLO algorithm is evaluated using main metrics such as Precision, Recall, mAP, and F1-Score [70]. True positive (TP) is the number of correct detections of the classes bicycle, e-bike, e-scooter, hoverboard, skate, helmet. False positive (FP) is the number of incorrect detections of these classes. False negative (FN) is the number of times the model could not detect these classes. Precision is expressed as the ratio of the true detections of the target boundaries to the defined detections, and the relevant calculations are based on Equation (1). Recall is the ratio of true detections of target boundaries among all true object detections, and this metric is calculated according to the formulation given in Equation (2). F1-Score is the harmonic mean of Precision and Recall values and is calculated using Equation (3). mAP is the average of the mean Precision value of all target classes, and Equation (4) is taken into account in the calculations. The intersection over union (IoU) value deals with whether the detection of micro-mobility and equipment classes are a true positive or a false positive and is used in Equation (5).

$$Precision={\displaystyle \frac{TP}{TP+FP}}\times 100$$

(1)

$$Recall={\displaystyle \frac{TP}{TP+FN}}\times 100$$

(2)

$$F1-Score={\displaystyle \frac{2\times Precision\ast Recall}{Precision+Recall}}\times 100$$

(3)

$$AP={\displaystyle \frac{1}{11}}\sum Re\in \{0, 0.1, \dots , 1\}\underset{\widetilde{\mathit{Re}}: \widetilde{\mathit{Re}}\ge \mathit{Re}}{\mathit{max}}Pr\left(\widetilde{Re}\right)$$

(4)

$$IoU={\displaystyle \frac{B_1\cap B_2}{B_1\cup B_2}}$$

(5)

2.4. Preference Ranking Organization Method for Enrichment Evaluations

Among MCDM methods, overmatching approaches have made rapid progress due to their flexibility to most real decision situations. PROMETHEE method is the most common and well-known overmatching method that allows pairwise comparison of alternatives for each of the criteria [86,87]. PROMETHEE was developed in 1982, providing great advantages in terms of ease of application and understandability for decision makers [88,89]. This method has significant advantages regarding the structuring of the problem, the amount of data that can be processed, the conversion of qualitative data into quantitative data, strong software support and the presentation of results [90,91,92]. The method is widely used in different fields such as energy [93,94,95], environment [96,97,98], transportation [99,100,101,102], and agriculture [103,104,105]. Different versions have been developed such as PROMETHEE I, PROMETHEE II, PROMETHEE III, and PROMETHEE IV [106]. Decision makers have always stated that it is better to obtain full ranking results, and therefore PROMETHEE II method was chosen in this study [88]. This method starts with the formulation of a set of criteria using the performance values of the alternatives, and the decision matrix is created. The process steps of the PROMETHEE II method are as follows [107]:

Step 1. Determining the appropriate preference function for the problem: For each evaluation criterion, the preference function P (a, b) is applied to decide how much the outcome a is preferred over b.

Step 2. The preference index is calculated using Equation (6).

$$\pi (a,b)={\sum }_{j=1}^k{w}_j x P_j(a,b)$$

(6)

Step 3. Calculating alternative advantages

$${\phi }^{+}\left(a\right)={\sum }_{x\in A}\pi \left(a,x\right) \mathit{for} each a$$

(7)

$${\phi }^{-}\left(a\right)={\sum }_{x\in A}\pi \left(x,a\right) \mathit{for} each a$$

(8)

Step 4. Calculate net flow values

$$\phi \left(a\right)={\phi }^{+}\left(a\right)-{\phi }^{-}\left(a\right) \mathit{for} each a$$

(9)

Using net flow values, decision alternatives are ranked from best to worst. The alternative with the highest net flow value will be accepted as the model that gives the best result in terms of ranking.

2.5. K-Means Clustering Algorithm–Elbow Method

K-means clustering offers a new data-driven unsupervised data classification approach to stratification that can provide an optimal strategy to separate populations into statistically significant subsets called strata [108,109]. The center points are determined by the user for a predetermined number of clusters. The distances of other samples to the center point of each cluster are then calculated using various approaches, and this process can be repeated to separate the samples into different clusters [110]. After the initial cluster centers are selected, the distance between each cluster center and each sample point is calculated and assigned to the nearest cluster center (Equation (10)).

$$\mathit{arg} \underset{j}{\min }{‖x_i-c_j‖}^2,\hspace{1em}j\in \{1,2,\dots ,k\}$$

(10)

$$c_j={\displaystyle \frac{1}{N_j}}{\sum }_{x_i\in C_j}{x}_i,\hspace{1em}j\in \{1,2,\dots ,k\}$$

(11)

The cluster center, calculated as the average of the samples within the cluster, is updated according to the formulation given in Equation (11). All processing steps are executed iteratively until the cluster center reaches a predefined error threshold or remains unchanged, aiming to determine the optimal number of clusters [111].

The Elbow method is a heuristic approach used to determine the optimal k value for K-means clustering based on Sum of Squared Errors (SSE) and the formulation is presented in Equation (12) [112].

$$SSE={\sum }_{p\in A_i}^n{(p-a_i)}^2$$

(12)

In the Elbow method, different k values are initially entered into the K-means clustering algorithm to obtain the clustering results. Then, the SSE value representing the clustering result is calculated for each k value and a line graph is drawn to show the relationship between the k values and the SSE values. The point on the graph where the distortion/inertia starts to decrease linearly or the point where the largest linear decrease occurs (elbow point) represents the optimum number of clusters.

The PROMETHEE method was used to rank the effects of data augmentation techniques on model performance. The K-means clustering–Elbow method presented determined the optimum number of clusters according to the obtained ranking results and determined which data augmentation techniques were most effective on model performance. Thus, it was possible to establish a model consisting of effective data augmentation techniques.

3. Results

This section presents the performance analysis of the YOLOv12 model, which represents the latest development in the YOLO family, offering state-of-the-art object detection capabilities through various architectural improvements. A total of 46 models (1 no augmentation, 22 data augmentation techniques for D1_Micro-mobility and D2_Helmet_Detection) were evaluated using basic metrics such as Precision, Recall, F1-Score, mAP@0.5, and mAP@0.5:0.95. In order to determine the effectiveness of data augmentation techniques on model performance, the results were evaluated with the PROMETHEE method, and the outputs were presented. According to the ranking results obtained, the optimal number of clusters was determined with the K-means clustering–Elbow method and the MME effective on the YOLOv12 approach was determined.

3.1. Evaluation of Models

All YOLOv12 model analyses were performed in the Google Colab environment using the Python 3 runtime type and the freely available hardware accelerator T4 GPU. The epoch number for model training was determined as 100, and the batch size as 8. Adaptive Moment Estimation optimization was used in the model, and lr = 0.001, momentum = 0.937, and decay = 0.0005 were determined. The models were evaluated at 0.5 IoU and 0.25 confidence threshold, and the model results are presented in

Table 3. Model performance outputs for dataset D1_Micro-mobility.

Dataset	Precision	Recall	mAP@0.5	mAP@0.5:0.95	F1-Score
D1_No Augmentation	0.804	0.671	0.712	0.468	0.66
D1_ILA_Shear	0.839	0.751	0.791	0.529	0.74
D1_ILA_Saturation	0.825	0.726	0.774	0.510	0.72
D1_ILA_Rotation	0.809	0.715	0.738	0.477	0.69
D1_ILA_Noise	0.879	0.776	0.802	0.565	0.75
D1_ILA_Mosaic	0.821	0.755	0.815	0.551	0.73
D1_ILA_Hue	0.893	0.756	0.808	0.550	0.77
D1_ILA_GrayScale	0.819	0.699	0.761	0.516	0.69
D1_ILA_Flip	0.783	0.736	0.783	0.528	0.74
D1_ILA_Exposure	0.810	0.767	0.798	0.529	0.74
D1_ILA_Cutout	0.875	0.756	0.812	0.555	0.73
D1_ILA_Crop	0.818	0.716	0.738	0.517	0.71
D1_ILA_Brightness	0.843	0.736	0.799	0.537	0.74
D1_ILA_Blur	0.881	0.744	0.792	0.543	0.76
D1_ILA_90’Rotate	0.766	0.719	0.736	0.499	0.69
D1_BBLA_Rotation	0.783	0.757	0.777	0.520	0.74
D1_BBLA_Noise	0.753	0.689	0.743	0.511	0.71
D1_BBLA_Flip	0.835	0.745	0.787	0.517	0.74
D1_BBLA_Exposure	0.879	0.793	0.834	0.552	0.77
D1_BBLA_Crop	0.851	0.685	0.742	0.477	0.71
D1_BBLA_Brightness	0.853	0.723	0.773	0.534	0.7
D1_BBLA_Blur	0.821	0.777	0.799	0.555	0.76
D1_BBLA_90’Rotate	0.837	0.727	0.780	0.551	0.72

and

Table 4. Model performance outputs for dataset D2_Helmet detection.

Dataset	Precision	Recall	mAP@0.5	mAP@0.5:0.95	F1-Score
D2_No Augmentation	0.956	0.913	0.918	0.529	0.81
D2_ILA_Shear	0.952	0.893	0.916	0.495	0.89
D2_ILA_Saturation	0.959	0.913	0.964	0.561	0.89
D2_ILA_Rotation	1.000	0.858	0.878	0.483	0.84
D2_ILA_Noise	0.967	0.938	0.941	0.540	0.84
D2_ILA_Mosaic	0.982	0.796	0.877	0.503	0.81
D2_ILA_Hue	0.956	0.860	0.887	0.547	0.80
D2_ILA_GrayScale	0.983	0.875	0.936	0.509	0.91
D2_ILA_Flip	0.984	0.953	0.941	0.519	0.85
D2_ILA_Exposure	0.944	0.877	0.916	0.533	0.84
D2_ILA_Cutout	0.941	0.938	0.922	0.527	0.90
D2_ILA_Crop	0.950	0.898	0.947	0.512	0.78
D2_ILA_Brightness	0.967	0.898	0.923	0.510	0.85
D2_ILA_Blur	0.963	0.837	0.850	0.455	0.75
D2_ILA_90’Rotate	0.947	0.778	0.816	0.483	0.80
D2_BBLA_Rotation	0.947	0.858	0.864	0.486	0.77
D2_BBLA_Noise	0.962	0.908	0.919	0.505	0.90
D2_BBLA_Flip	0.968	0.950	0.941	0.519	0.89
D2_BBLA_Exposure	0.954	0.938	0.868	0.503	0.80
D2_BBLA_Crop	0.963	0.953	0.918	0.513	0.85
D2_BBLA_Brightness	0.944	0.898	0.901	0.515	0.87
D2_BBLA_Blur	0.961	0.890	0.932	0.576	0.85
D2_BBLA_90’Rotate	0.945	0.842	0.891	0.500	0.85

.

When the model performance outputs presented in Table 3 and Table 4 are examined separately based on datasets, the best performance for the Precision metric was provided by the D1_ILA_Hue dataset, created with the Hue technique, and the D2_ILA_Rotation dataset, created with the Rotation technique, while the worst performance was provided by the D1_BBLA_Noise dataset, created with the Noise technique, and the D2_ILA_Cutout dataset, created with the Cutout technique. For the Recall performance metric, the best performance belongs to D1_BBLA_Exposure and D2_ILA_Flip datasets, while the worst performance value is D1_No Augmentation and D2_ILA_90’Rotate datasets. According to the mAP@0.5 performance metric, the best-performing datasets are D1_BBLA_Noise and D2_ILA_Saturation, respectively, while the worst performing datasets are D1_No Augmentation and D2_ILA_90’Rotate, respectively. According to the mAP@0.5:0.95 performance metric, the best performances belong to D1_ILA_Noise and D2_BBLA_Blur, while the worst performances belong to D1_No Augmentation and D2_ILA_Blur datasets. According to the F1-Score performance metric, the best performances belong to D1_ILA_Hue and D1_BBLA_Exposure datasets for D1_Micro-mobility, and the worst performance value belongs to D1_No Augmentation dataset. According to the D2_Helmet detection dataset, the best performance belongs to D2_ILA_GrayScale dataset and the worst performance belongs to D2_ILA_Blur dataset.

The analysis of the results obtained must be evaluated in a holistic manner, and it is important to consider all performance metrics together in this process. This situation makes determining the best performing dataset a Multi-Criteria Decision-Making problem. In this context, the evaluation of all performance results was carried out with the PROMETHEE method within the scope of the study.

3.2. Comparison of YOLO Detection Results

The evaluation of model performances according to datasets was carried out with the PROMETHEE method and Precision, Recall, F1-Score, mAP@0.5, and mAP@0.5:0.95 metrics were used to rank the alternatives. All metrics were defined as maximization oriented. The PROMETHEE II method was used to obtain a complete ranking. Preference functions were determined as p = 0.025–0.030; q = 0.025–0.0.13 for the Precision metric, p = 0.062–0.094; q = 0.025–0.040 for the Recall metric, p = 0.053–0.071; q = 0.022–0.031 for the mAP@0.5 metric, p = 0.053–0.053; q = 0.022–0.023 for the mAP@0.5:0.95 metric, and p = 0.056–0.056; q = N/A for the F1-Score metric, respectively. The preference function of the F1-Score metric is determined as V-shape and the preference function of all other metrics is determined as Linear. The determination processes of the preference functions, q and p values, are made by taking into account the data distribution of alternative model performances. The ranking results according to net flow values are presented in

Table 5. Net flow values and rankings for datasets.

Rank	Dataset	$\mathit{\phi }$	Dataset	$\mathit{\phi }$
1	D1_BBLA_Exposure	0.4641	D2_ILA_Saturation	0.3840
2	D1_ILA_Noise	0.3306	D2_ILA_GrayScale	0.3119
3	D1_BBLA_Blur	0.3288	D2_ILA_Flip	0.3056
4	D1_ILA_Hue	0.3081	D2_BBLA_Flip	0.3036
5	D1_ILA_Blur	0.2149	D2_BBLA_Blur	0.2362
6	D1_ILA_Cutout	0.2015	D2_ILA_Noise	0.2025
7	D1_ILA_Mosaic	0.1922	D2_ILA_Cutout	0.1890
8	D1_ILA_Exposure	0.1897	D2_BBLA_Noise	0.1795
9	D1_ILA_Shear	0.1436	D2_BBLA_Crop	0.1311
10	D1_ILA_Brightness	0.1383	D2_ILA_Shear	0.0823
11	D1_BBLA_Rotation	0.1018	D2_ILA_Brightness	0.0814
12	D1_ILA_Flip	0.0966	D2_BBLA_Brightness	0.0244
13	D1_BBLA_Flip	0.0885	D2_ILA_Rotation	−0.0050
14	D1_BBLA_90’Rotate	0.0367	D2_ILA_Exposure	−0.0217
15	D1_ILA_Saturation	−0.0756	D2_No Augmentation	−0.0249
16	D1_BBLA_Brightness	−0.0897	D2_ILA_Crop	−0.1156
17	D1_ILA_Crop	−0.2311	D2_ILA_Hue	−0.1212
18	D1_ILA_GrayScale	−0.2672	D2_BBLA_90’Rotate	−0.1560
19	D1_BBLA_Noise	−0.3051	D2_BBLA_Exposure	−0.1934
20	D1_ILA_90’Rotate	−0.3553	D2_ILA_Mosaic	−0.2116
21	D1_ILA_Rotation	−0.4247	D2_BBLA_Rotation	−0.4206
22	D1_BBLA_Crop	−0.4309	D2_ILA_Blur	−0.5741
23	D1_No Augmentation	−0.6557	D2_ILA_90’Rotate	−0.5876

.

According to the ranking results, the dataset that improved the model performance the most for the D1_Micro-mobility dataset was determined as D1_BBLA_Exposure, and for the D2_Helmet detection dataset as D2_ILA_Saturation. The dataset with the least impact on performance for the D1_Micro-mobility dataset was determined as D1_BBLA_Crop. For the D2_Helmet detection dataset, datasets created with some data augmentation techniques worsened the model performance compared to the D2_No Augmentation dataset. For D1_Micro-mobility, it was determined that the model created with the D1_BBLA_Exposure dataset provided 9.23% improvement in the Precision metric, 18.26% in Recall metric, 17.18% in mAP@0.5 metric, 17.84% in mAP@0.5:0.95 metric, and 16.67% in F1-Score metric compared to the model created with the D1_No Augmentation dataset. For D2_Helmet detection, it was observed that the model created with the D2_ILA_Saturation dataset provided 0.3% improvement in the Precision metric, 5.05% in mAP@0.5 metric, 5.99% in mAP@0.5:0.95 metric, and 9.88% in F1-Score metric compared to the model created with the D2_No Augmentation dataset. It was observed that no improvement was provided in the Recall value.

The effect of the dataset obtained with a single data augmentation technique may remain limited. Therefore, the most effective dataset should be created by determining the number of clusters of datasets that have a high effect on model performance. In this context, the optimal number of clusters of data augmentation techniques that have the most effect on model performance was determined within the scope of the study.

3.3. Determining the MME of Data Augmentation Techniques That Improve YOLO Model Performance

Based on the YOLOv12 model, the net flow values obtained by the PROMETHEE method were analyzed in detail to evaluate the relative advantages of 25 datasets and to select the dataset that increased the model performance the most. The K-means clustering–Elbow method was applied to determine the optimal number of clusters of data augmentation techniques that increased the performance of the YOLOv12 model the most for determining the MME. By analyzing the relationship between the total error rate and the number of data augmentation techniques, the “elbow” point where the decreasing trend in the error rate slowed down significantly was determined, and thus the optimal number of data augmentation technique groups was determined. The error rate graph obtained as a result of the Elbow method is presented in Figure 2.

When Figure 2 is examined for D1_Micro-mobility, D1_BBLA_Exposure, D1_ILA_Noise, and D1_BBLA_Blur, datasets were determined as MME. For D2_Helmet detection, D1_ILA_Saturation, and D1_ILA_GrayScale datasets were determined as MME. In addition, the dataset created with the determined MME was re-trained using the same indicators in the YOLOv12 approach. The performance outputs of the model trained with the MME dataset are presented in

Table 6. Model performance outputs of MME datasets.

Dataset	Precision	Recall	mAP@0.5	mAP@0.5:0.95	F1-Score
D1_Micro-mobility_MME	0.927	0.782	0.844	0.593	0.79
D2_Helmet detection_MME	0.978	0.862	0.871	0.509	0.81

.

Upon review of Figure 3, it is observed that the D1_Micro-mobility_MME dataset is superior to the D1_No Augmentation dataset in all performance metrics. Accordingly, it is seen that performance increases are achieved in the Precision (+15.29%), Recall (+16.52%), mAP@0.5 (+18.54%), mAP@0.5:0.95 (+26.50%), and F1-Score (+19.7%) metrics. When Figure 4 is examined together, it is seen that the D2_Helmet detection_MME dataset provides a 2.36% improvement only in the Precision metric. Performance decreases are observed in Recall (−5.59%), mAP@0.5 (−5.15%), and mAP@0.5:0.95 (−3.68%) metrics. There is no change in the F1-Score value. In this study, after data augmentation, Precision increased by 15.29% for the D1 dataset and 2.36% for the D2 dataset, while Recall decreased by 5.59% for the D2 dataset. This shows that in real applications, sensitivity should be carefully adjusted according to the intended use of the system. Especially in security-oriented scenarios, high Recall priority should be given, but the false alarm rate should also be managed.

4. Discussion

In recent years, the use of YOLO object detection models in the fields of transportation, agriculture, environment, health, etc., has been increasing and has become an important tool in the field of computer vision on an international scale [113,114]. Before using any object detection approach, it is extremely important to determine the data augmentation techniques to be applied to the datasets. For this purpose, the main motivation of this study is to list the data augmentation techniques to be used in the YOLO family, which is shown as the state-of-the-art in object detection approaches, and to determine the MME. For this purpose, the effects of a total of 22 different data augmentation approaches on object detection performance were systematically investigated on two different datasets (D1-Micro-mobility detection and D2-Helmet detection) using the YOLOv12 model. The obtained model outputs were analyzed through performance metrics widely used in the literature, such as Precision, Recall, F1-Score, and mAP. Multi-dimensional performance data were evaluated holistically with the PROMETHEE method; then, the most appropriate data augmentation MME was determined with the K-means clustering–Elbow method.

The analysis results show that data augmentation techniques generally create significant and class-specific differential effects on model performance. While some data augmentation techniques provide consistent and positive contributions to all performance metrics, some techniques provide contributions only to some performance metrics. This shows that the effects of data augmentation approaches are not limited to increasing the number of images; they include features such as the ability to distinguish between classes, focusing on specific classes, and generalization. The ranking performed using the PROMETHEE method allows multi-dimensional performance analysis, unlike the traditional single-metric evaluation approach. Thus, it has been demonstrated in a computable way which data augmentation approach contributes to the overall success. The K-means clustering–Elbow method provides powerful information for MME to researchers or decision makers who want to use more than one data augmentation technique.

The results show that data augmentation techniques generally improve model performance. In the ranking of MCDM according to the considered performance metrics, it was determined that the data augmentation techniques that increased the model performance the most were D1_BBLA_Exposure and D1_ILA_Saturation. Similarly, in the study conducted by [115], the effectiveness of Exposure and Saturation techniques was shown among the considered data augmentation techniques. In this paper, it was determined that each data class was not detected with the same success rate in both D1_Micro-mobility and D2_Helmet detection datasets. The reasons for this difference may be the imbalance in the dataset, the difficulty of visual features, the similarity between classes, the labeling quality, the capacity of the model, the weight sharing and the training parameters.

Limitations of the study include the fact that the datasets have certain visual limitations, the evaluation was made through a single object detection approach, and the data augmentation techniques were obtained with fixed parameters. The dataset used in this study specifically addresses micro-mobility vehicles and safety devices, which limits comparison with public datasets (e.g., Cityscapes, BDD100K). Such a comparison would not yield fair or meaningful results, as these datasets do not adequately represent the target audiences of our study. However, it is suggested as a research direction for comparative analysis, particularly with the development of more comprehensive public datasets on micro-mobility. In addition, this study analyzes in detail the advantages and disadvantages of the data augmentation methods used. The main advantages of data augmentation include increasing the generalization capacity of the model, increasing robustness to changes in the environment, and reducing the risk of overstressing. However, some disadvantages should also be taken into account, such as increased training time, the possibility of making unrealistic changes to the model, and limited effectiveness on already large or very diverse datasets.

Data imbalance can have a significant impact on model performance, especially in limited and heterogeneous datasets such as micro-mobility vehicles. However, scientific literature shows that data resampling methods (oversampling, under sampling) and advanced loss functions such as class-weighted loss or focal loss can improve model performance even in imbalanced datasets. Integrating these techniques into future studies can further increase the generality of the model and its success in real-world conditions.

5. Conclusions

In this study, the effects of a total of 22 data augmentation techniques (number of models: 46—no data augmentation for D1_Micro-mobility, 22 data augmentation techniques; no data augmentation for D2_Helmet_Detection, 22 data augmentation techniques) on model performance were analyzed using a multi-criteria approach on two different datasets using the YOLOv12 approach. Model outputs were evaluated using common metrics such as Precision, Recall, F1-Score, and mAP. Net flow values were obtained and ranked using the model performance values of data augmentation techniques with the PROMETHEE method. MME, which consists of techniques with a high level of effect according to net flow values, was determined by the K-means clustering–Elbow method.

As a result of the analysis, it was observed that each data augmentation technique did not provide homogeneous success for all performance metrics and training classes. The most successful model for D1_Micro-mobility was obtained in the D1_BBLA_Exposure dataset. The least successful model for the same dataset was obtained in the D1_No Augmentation dataset. Similarly, the most successful model for the D2_Micro-mobility dataset was obtained in the D2_ILA_Saturation dataset, and the least successful model was obtained in the D2_ILA_90’Rotate dataset. This situation revealed that data augmentation techniques vary depending on the dataset and training class.

In the model analysis performed with the dataset D1_BBLA_Exposure, D1_ILA_Noise and D1_BBLA_Blur, which consist of data augmentation techniques included in the MME optimal set determined using the net flow values obtained as a result of the PROMETHEE method for the D1_Micro-mobility dataset, the F1-Score value was obtained as 0.79 and the mAP value as 0.844. This shows that MME improves the model performance by 19.7% in the F1-Score metric and 18.54% in the mAP metric. Similarly, for the D2_Helmet_Detection dataset, the F1-score value was 0.81 and the mAP value was 0.871 in the model analysis performed with the D1_ILA_Saturation and D1_ILA_GrayScale datasets, which consist of data augmentation techniques in the MME optimal set. However, a 2.36% performance success increase was observed only in the Precision metric. In this context, the study goes beyond an experimental success evaluation and provides a strong informatics framework for decision makers and researchers by structuring the data augmentation technique strategy in the YOLOv12 object detection algorithm and ensuring the integrity of the performance metric.

In future studies, it is planned to increase the diversity of the dataset, evaluate different object detection approaches in an integrated manner, and analyze parametric data augmentation techniques in order to eliminate the limitations.