Adaptive K-Fold Siamese Neural Network Classifier for Automatic Seatbelt Monitoring

Abstract

A seatbelt is an essential aspect of safety in road traffic accidents. Although most traffic regulations enforce drivers and passengers to wear and fasten the seatbelt manually, AI-based techniques have been introduced for monitoring to improve safety standards. In this study, a new approach is proposed to address the monitoring problem of seatbelts. Deep learning (DL) classification based on adaptive Siamese Neural Network (SNN) has been developed utilizing the K-fold method for feature verification. The proposed adaptive K-Fold-based SNN approach utilizes a binary seatbelt dataset, with positive and negative classes, to verify the status of the seatbelt. The network involves sharing a convolutional feature extractor, followed by a distinct-based similarity function. To enhance model reliability, 5-fold cross validation is applied (k = 5), splitting the dataset into 5 subsets, where the model is trained on four sets and validated on the fifth one. The model was trained using binary cross entropy loss, Adam optimization, and performance metrics such as accuracy, precision, recall, and F1 score. The seatbelt dataset is basically designed for object detection models. In this work, we used a dataset in the verification model and achieved high-performance metrics. The model is implemented using a Python-based Jupyter Notebook 7.5.1. It achieved a high performance in seatbelt verification with an average Accuracy = 0.9989, average Precision = 0.9988, average Recall = 0.9990, and average F1 Score = 0.9989. The proposed adaptive K-Fold SNN model can ensure reliability and reduce the risk of over fitting.

1. Introduction

Seatbelt monitoring systems are crucial equipment in most modern vehicle safety systems, which have been designed to ensure that vehicle passengers are restrained properly in their seats, thereby reducing the risk of injury or fatality during traffic accidents. The last five decades showed an increasing demand in sensor technology, data processing, and vehicle safety with vehicle network scenarios [1,2]. Traffic policemen try their best to ensure the correct execution of traffic rules manually on highway roads, traffic intersections, and other places. However, it is very difficult, inefficient, and complicated to attain the required goal considering the limited number of police officers [3]. Wearing a seatbelt can easily save the lives of the driver and other passengers. Otherwise, they can be injured and may die.

In order to support traffic officers, automated monitoring and detection techniques are mandatory. Therefore, researchers in the institutions that are responsible for traffic safety are trying to propose automated methods for monitoring [4].

Seatbelts were first used early in the 20th century. Actually, they were launched in 1959 by the Volvo Company (Swedish) as stated by Nils Bohlin, a safety engineer at the Volvo Company [5]. Moreover, in 1970, the recorders marked the first trend in actual seatbelt monitoring steps via simple sensors in order to ensure that the seatbelt was actually fastened. Unfortunately, these systems were limited in providing detailed data on the seatbelt status.

Most technological methods used in seatbelt monitoring systems are: sensor-based integration systems for modern seatbelt monitoring based on advanced hall effect sensors, pressure sensors, computer vision-based systems and inertial measurement units (IMUs), machine learning and AI, and IoT-based smart systems. Several studies used AI to detect proper seatbelt usage, the integration of seatbelt monitoring systems with vehicle-to-everything (V2X) communication technologies for enhancing safety, and an occupant classification system (OCS) to monitor the connectivity of seatbelts and differentiate between elders, adults, children, and even empty seats [6].

In various fields of study, artificial intelligence methods, especially deep learning algorithms, play a vital role in beneficial outcomes and solving various image detection issues. However, most algorithms utilized in deep learning consume a lot of time for training and predicting a huge number of parameters and their associated weights. Nowadays, deep learning has been shown to be the most successful tool for achieving high-quality results across various practical domains [7,8,9,10,11].

The main goal of this study is to build a Siamese Neural Network to develop an automatic seatbelt monitoring system. The Siamese Neural Network is a special type of neural network structure that consists of at least two or more identical sub-networks. These sub-networks participate with the same weights and other networks, acting as a close partner to each other. The main intention of a Siamese network is to compare between two inputs and process them through these identical sub-networks and evaluate the degree of similarities between both inputs. However, the main contributions of this work are summarized in the following points:

• To enhance model reliability, the proposed network model involved a convolutional feature extractor supported by distinct similarity functions.
• The Siamese Neural Network requires paired input data for training. The dataset has been split into training and test sets before generating image pairs to avoid data leakage.
• The proposed Siamese Neural Network is designed for multi-class classification rather than verification.
• A comparison study to other exiting models in the literature was conducted to verify the effectiveness of proposed model.

Other sections of this paper are arranged as follows: Section 2 discusses previous related works on seatbelt monitoring; Section 3 explains the Siamese Neural Network, Section 4 describes the methodology in seatbelt monitoring and the utilized dataset; Section 5 explains the output obtained from experimental results; and finally, Section 6 discusses the conclusion and recommendation for future work.

2. Related Works

A study presented by Lwando et al. [12] suggested a seatbelt monitoring system utilizing pressure sensors that are embedded in the seatbelt buckle. The proposed monitoring system perceives whether the seatbelt is fastened correctly and also measures the tension of the seatbelt to guarantee proper usage. On the other hand, Ahmad et al. [13] presented a wearable measurement sensor-based system combined with the seatbelt. The proposed system utilizes accelerometers and gyroscope measurements in order to detect the usage of the seatbelt in real time.

In addition to the sensor-based systems discussed previously, computer vision-based systems have been adopted by several researchers. Rupal et al. [14] developed a deep learning algorithm in order to investigate various images captured from in-car cameras. The proposed system ensures whether the passengers are wearing seatbelts and evaluate the correctness of their usage. Moreover, Warusia et al. [15] developed this integration using You Only Look Once (YOLOv5) in real time, and they obtained an elevated precision in various conditions. However, obstruction and divergent parameters conditions were significant hurdles.

Most modern vehicles contain traditional belt fastness detection methods that can be easily tricked. Therefore, Kashevnik et al. [16] proposed a detection method by installing a camera inside the driver’s cabin to identify the seatbelt status. The proposed model was based on a YOLO neural network, which detects both the seatbelt and the corner detection, using a tiny-YOLO neural network to detect three cases, including the seatbelt is fastened, not fastened, or fastened behind the driver’s back.

A low-cost Internet of Things (IoT) monitoring system was employed by Yeqiang et al. [6]. In this model, a wireless sensor was adopted to collect and transmit the seatbelt status to the central unit to keep an eye on compliance in real-time. Not far from a Wireless Sensor Network (WSN), a well-tuned seatbelt system was incorporated with smart phones or Radio Frequency Identification (RFID) tags and vehicle far away communication to send alerts and reports on the specified seatbelt usage. Moreover, Wang and Ma [17] suggested a traffic management system to detect and recognize the safety seatbelts of drivers through smart IoT. Both researchers utilized YOLOv3 to detect and recognize stationary vehicles using the smart IoT management system.

Balci et al. [18] suggested a mobile seatbelt system that can be installed on the roof of a police car. A deep learning algorithm was utilized to detect the most traffic abuse cases at night utilizing a dataset of 2600 existent real images from the installed camera through a single shot multi-box detector (SSD). The results obtained an overall accuracy of 83% and 97% for the sensitivity rate. However, the decision-making process for the type of driver violation was not included. On the other hand, Elihos et al. [19] proposed a classification and object detection method for the driver’s seatbelt. A deep learning-based SSD algorithm was adopted utilizing 13,444 actual images captured in day and night situations, and the method achieved an accuracy rate of 91.9% with 94.5% precision in all situations.

A Convolutional Neural Network (CNN) model for two dimensions, driver and passengers, has been used in order to detect their positions. A proposed model by Chun et al. [20] was examined on a new database that consists of more than 100 video clips for drivers in 50 driving tests. In addition to the above database, the proposed model was evaluated again utilizing the MS COCO database by testing various model situations. Moreover, Hosam [21] suggested a deep convolutional neural network (CNN) for possible seatbelt detection in all weather conditions such as rain, night, clear, or hybrid. General weather condition detection was based on the sensor AlexNet (S-AlexNet) method. The proposed model achieved higher than 90% accuracy.

Seema and Sandeep [22] suggested a Machine Learning (ML) method to forecast the seatbelt usage information depending on driver activities and dynamic environmental aspects using historical data for elevating the monitoring precision. In addition to machine learning, a reinforcement learning model was proposed by Sara and Abdolhossein [23] to improve the seatbelt monitoring system in vehicles by taking into consideration various seat configurations and driver’s activities.

An algorithm for detecting passenger’s seatbelts was proposed by Yu et al. [24], utilizing a specific type of feature based on gradient orientation. To achieve this goal, narrowing the front window location and the driver face is considered. Then, the gradient orientation was calculated from pixel to pixel in the specified region. Moreover, the researchers reduced the interference using the sifting and template matching technique. In their work, 213 high-degree images from traffic intersections were considered to validate the proposed method, and hence, the achieved accuracy was 86%.

Zhang [25] studied the position forms for drivers and other passengers who are wearing the belt in various traffic intersections in three different phases: vehicle position, driver’s and passengers’ position, and seatbelt driver or passenger behavior. This behavior was detected based on both probability and statistical approaches via a specified roadmap method. The behavior decision was conducted using linear target recognition utilizing the perceptual grouping mechanism to enhance the driver’s seatbelt detection. This method may complicate the overall procedure of detection and prohibit real-time implementation.

Quick review of previous works indicates that most of them focused on seatbelt monitoring. However, the previous works revolve around three basic approaches:

• Object detection: Using You Only Look Once Versionx (YOLOx), Single Shot Detector (SSD), Fast Recurrent Neural Network (FRCNN).
• Classification: Using Convolutional Neural Network (CNN), and Fisher Vector (FV), and Support Vector Machine (SVM).
• Transfer Learning; Using Residual Network (ResNet), Xception.

However, no previous study has used a Siamese Neural Network (SNN). In this study, we proposed a modified version of SNN for classification instead of verification. The K-fold approach is involved within the proposed SNN model for reliable evaluation. In the methodology section, more details about the adapted mode have been presented.

Table 1 summarizes the past studies related to this work:

Table 1. Past studies summary.

Ref.	Method Type	Algorithm/Technique	Input Data	Target Detection
[10]	Deep learning vision	YOLO-based object recognition	Aerial UAV imagery	Safety harness usage
[11]	Deep learning (transfer learning)	Pretrained CNN models (transfer learning)	Traffic surveillance images	Seat belt usage
[12]	Computer vision (DL)	YOLOv7 model	RGB cabin camera feeds	Seat belt condition
[13]	Wearable sensing	Motion sensors with ML	Accelerometer & gyroscope data	Human activity patterns
[14]	Vision-based learning	Convolutional neural network	Vehicle interior images	Belt fastening state
[15]	Intelligent vision	Enhanced YOLOv5	Roadside surveillance footage	Seat belt compliance
[16]	Hybrid vision system	Classical image processing + ML	Cabin-mounted cameras	Belt fastening accuracy
[17]	Vision + IoT	Deep learning integrated with IoT	Smart traffic cameras	Vehicle and belt monitoring
[18]	Infrared vision (DL)	CNN using NIR inputs	Near-infrared imagery	Seat belt violations
[19]	Comparative vision study	CNN classifiers vs. YOLO detectors	RGB and NIR images	Belt misuse detection
[20]	Lightweight DL model	Custom compact CNN (NADS-Net)	In-vehicle images	Driver presence & belt usage
[21]	Robust vision model	Weather-tolerant CNN	Outdoor traffic images	Belt status classification
[24]	Classical vision	Gradient-based feature extraction	Intersection camera images	Belt usage detection
[25]	Vision + ML	Statistical image analysis	Road traffic imagery	Seat belt wearing behavior
[26]	Image-based CV	Classical image processing (color & edge features)	RGB images	Driver seat belt presence
[27]	Image-based CV	Feature extraction + rule-based detection (real-time optimized)	RGB images/video frames	Driver seat belt usage
[28]	Deep learning (object detection)	FCOS (anchor-free detector)	Thermal images	Seat belt on driver
[29]	Deep learning (CNN + attention)	CNN with multi-attribute self-attention	RGB images	Vehicle local regions (incl. seat belt area)
[30]	Deep learning (object detection)	YOLO-based real-time object detection (inferred)	Traffic camera images/video	Traffic violations (seat belt, helmet, etc.)
[31]	Deep learning (CNN + attention)	Improved Dense Residual Network + two-level attention	Road surveillance images	Driver seat belt usage
[32]	Deep learning (object detection)	CNN-based object detection (YOLO/SSD inferred)	Traffic surveillance images	Seat belt violation
[33]	Deep learning (CNN)	Convolutional Neural Network	Driver images	Driver seat belt presence
[34]	Deep learning (robust detection)	CNN-based seat belt detection with robustness enhancements	RGB images/video	Seat belt usage recognition
[35]	Hybrid ML–DL	CNN feature extractor + SVM classifier	Driver images	Driver seat belt usage
[36]	Deep learning (object detection)	YOLOv8	RGB images	Driver & passenger without seat belt
[37]	Deep learning (spatiotemporal)	YOLOv3 + CNN + LSTM	Video sequences	Seat belt usage over time
[38]	Lightweight deep learning	Lightweight CNN + attention mechanism	RGB images	Seat belt detection

3. Siamese Neural Network Structure

A Siamese Neural Network (SNN) is a special type of neural network architecture that consists of a twin pair of neural networks that share the same parameters and weights [39,40]. Its main objective is to identify whether a pair of data or images are identical or not. Its layers and functions will be discussed later, and its main block diagram is shown in Figure 1.

Figure 1. Siamese Neural Network block diagram [40].

Input Layer

Each input layer consists of two grayscale images of size (100 × 100 × 1)

$$X_1\in R^{\left(100\times 100\times 1\right)},X_2\in R^{(100\times 100\times 1)}$$

Features Extraction

Both inputs pass through the same features extractor (shared weights), which involves the following: Convolutional Neural Layer, Pooling Layer, and Dense Fully Connected Layers. The features extractor output is the features vector of each image:

$$F_1=f\left(X_1\right),F_2=f\left(X_2\right)$$

(1)

Computing the Distance between Two Feature Vectors

To compare two features vectors F1 and F2, calculate the distance between them:

$$D=\left|F_1-F_2\right|$$

(2)

Fully Connected Layer and Sigmoid Activation

The distance vector D is passed through the fully connected layers, followed by the activation layer to evaluate the output similarity:

$$y=\sigma (W^{T}D+b)$$

(3)

where W and b are learned parameters, σ(.) is the sigmoid function.

$$\sigma \left(x\right)={\displaystyle \frac{1}{1+e^{-x}}}$$

(4)

where y is the final prediction: y ≈ 1: images belong to the same class. y ≈ 0: images belong to different classes.

3.1. K-Fold Validation

It is a technique used to evaluate the performance of a model by dividing the dataset into K portions (folds). Then, the model is trained and validated multiple times to obtain a more reliable performance estimate [41,42,43]. When k = 5, the dataset is divided into 5 equal parts (folds). The model is trained 5 times, and each time a different fold is used for validation (testing), the remaining 4 folds are used for training, as denoted in Table 2.

Table 2. The split of datasets when K-Fold = 5.

Folds Number	Training Data (80%)	Validation Data (20%)
Fold 1	Folds 2, 3, 4, 5	Fold 1
Fold 2	Folds 1, 3, 4, 5	Fold 2
Fold 3	Folds 1, 2, 4, 5	Fold 3
Fold 4	Folds 1, 2, 3, 5	Fold 4
Fold 5	Folds 1, 2, 3, 4	Fold 5

Since the model is tested five times on different validations sets, the results might vary slightly. Therefore, taking the average performance (Accuracy, Precision, Recall, and F1 Score) gives a more stable and realistic measure of how well the model will perform on a new dataset. By averaging the results across all folds, a fairer estimate of the model’s performance is obtained.

3.2. Model Evaluation

The model is trained and evaluated using the following performance metrics:

Accuracy: Measures the proportion of the correctly classified pairs.

Precision: Measures how well the model identifies all actual seatbelt users.

F1 Score: Represents the harmonic mean of precision and recall.

Confusion Matrix: Provides a detailed breakdown of model predictions.

Table 3 shows the confusion matrix elements.

Table 3. Confusion matrix elements.

Actual/Predicted	Positive	Negative
Positive	True Positive (TP)	False Negative (FN)
Negative	False Positive (FP)	True Negative (TN)

4. Methodology

The Siamese Neural Network (SNN) is considered one of the most powerful kinds of neural networks, and it typically consists of two or more identical sub-networks [41,44] that have the same architecture, parameters and even weights. These two subnetworks are related to each other by a loss function, which computes a similarity score between the two input samples depending on the feature extracted from the two subnetworks. Moreover, all network parameter updates are reflected in both subnetworks. In other words, all updates in one network weights will be automatically updated on the other network. Particularly, the Siamese Neural Network does not produce a multi-class classification. Instead, it produces a binary output between two input images (i.e., ‘1’ for similar and ‘0’ for dissimilar). SNN is not working under one-shot or few-shot learning. The SNN is fully supervised model using many labels data pairs of images, The SNN model based on supervision pairs similarity along multiple epochs.

The main objective of this paper is to develop a Siamese Neural Network for automatic seatbelt monitoring system with reduced parameters and low computational efforts while maintaining a superior capability and robustness to realize a rapid and precise monitoring. In this section, the outline of our approach is precisely described for each portion of the proposed model.

4.1. Method Overview

The block diagram of the proposed algorithm is shown in Figure 2. The algorithm begins by preparing the seatbelt dataset for both the positive and negative samples. Subsequently, the proposed algorithm conducts the recognition of the driver’s seatbelt by employing a Siamese network.

Figure 2. Block diagram of the proposed multi-modal Siamese Neural Network for seatbelt classification.

4.2. Data Collection and Preprocessing

The basic dataset consists of four classes (seatbelt, helmet, gloves, and vest), and this dataset is designed for object detection. However, in this work, only the seatbelt class is taken. Table 4 summarizes the dataset properties [45].

Table 4. Seatbelt dataset summary [45].

Parameter	Description
Classes	Positive and negative
Number of total Positive (Before Augmentation)	4118
Number of total Negative (Before Augmentation)	4500
Use Augmentation	Two cases
Number of total Positive (After Augmentation)	12,351
Number of total Negative (After Augmentation)	13,500
Colored	Yes
Training set	80%
Validation set	20%

Figure 3 shows the samples for the two classes. Each raw image undergoes the following preprocessing operations:

Figure 3. Seatbelt class samples [45].

Grayscale conversion: To reduce the calculation cost and eliminate the color channel from 3 channels to 1 channel, this step allows focusing on features.

Resizing: Unify the size of the input image into 100 × 100.

Normalization: Pixel values are scaled between 0 and 1 to improve training stability. Creating Image Pairs

The Siamese Neural Network requires paired input for training. The dataset was split into training (80%) and testing (20%) sets before generating the image pairs, where two types of training pairs are created:

Positive pairs (Label = 1): Two images from the same class (both with a seatbelt or neither with a seatbelt).

Negative Pairs (Label = 0): Two images from different classes (one with a seatbelt and the other with no seatbelt).

The main standards used for augmentation:

• Rotation: Random rotation within ±20°.
• Horizontal Translation: Width shift up to 10%.
• Vertical Translation: Height shift up to 10%.
• Zooming: Random zoom within ±20%.
• Horizontal Flipping: Random left-right flip.
• Brightness Adjustment: Random brightens in range (0.7–1.3).
• Fill mode: Nearest neighbor filling for empty pixels.

The augmentation is performed offline before training; furthermore, an identical augmentation is applied to both classes to balance the expansion, where the dataset is expanded from 8168 to 25,851.

4.3. The Proposed Model

A Siamese Neural Network is implemented for seatbelt verification. The model consists of two identical convolutional branches for feature extraction. The base features an extraction network that is shared among input images consisting of specific layers, and these layers are denoted in Figure 4:

Figure 4. Basic feature extraction network [44].

Similarity computation: This is the absolute difference between the two embedding features, which is calculated using the Lambda layer:

$$\mathit{d}\mathit{i}\mathit{s}\mathit{t}\mathit{a}\mathit{n}\mathit{c}\mathit{e}=\left|\mathit{f}{(\mathit{x}}_1)-{\mathit{f}(\mathit{x}}_2)\right|$$

(5)

where $f\left(xn\right)$ is the features extracted for each image

Final Classification Layer: This is the fully connected layer that consists of one neuron layer and sigmoid activation, as shown in Figure 5. It is used to classify image pairs as similar “1” or different “0”. Algorithm 1 denotes the proposed model in detail, while Algorithm 2 shows the procedure of splitting data and pair generating.

Algorithm 1. Shows the proposed model

Inputs: Seatbelt dataset

Step 1: Load and preprocess images                  Set Dataset path, Load dataset, Resize into (100 × 100)

Step 2: Create image pairs for Siamese                  Positive (Same Class), Negative (Different Classes)

Step 3: Define Siamese Neural Network                  Base model and Siamese Network

Step 4: Train and evaluate with K-fold                  The dataset is split into 5 folds, for each fold:                            Train the model on 4 folds, test on 1 fold.                            Compute evaluation metrics (Accuracy, Loss, F1, Precision) and Confusion matrix.                            Save Score for average

Step 5: Compute final results:                  Average the metrics over all the folds                  Print the final performance results

Outputs: Average of Accuracy, Loss, F1, Precision, and Confusion matrix.

Figure 5. Final layer [44].

Algorithm 2. Shows the procedure of splitting data and pair generating

Inputs: Seatbelt dataset

Step 1: Load dataset               positive_images = load_dataset(positive_folder)               negative_images = load_dataset(negative_folder)

Step 2: Split images into:               80% training, 20% test               train_test_split

Step 3: Generate image separately               Positive (Same Class), Negative (Different Classes)

Step 4: Reinitialized at the beginning of each K-fold:               model = create_siamses_model()

Outputs: Splitting dataset

The SNN model in our work is reinitialized at the beginning of each K-fold, model = create_siamses_model(), which ensures that each fold is trained from scratch using random initial weights to avoid the leakage of data.

5. Results and Discussion

The model is designed and implemented by the training of various Siamese networks, which was carried out utilizing the programming language Python 3-based Jupyter Notebook via MSI laptop with 12th generation Intel (R) Core (TM) i5-12500H@ 232 GHz, RAM 16.0 GB. By using K-fold = 5, the properties of the proposed model are summarized in Table 5.

Table 5. Model summary.

Component	Description
Dataset	Seatbelt dataset (Positive, Negative)
Augmentation	Two cases (Without, With)
Image Preprocessing	Convert to grayscale, resize into (100 × 100), Normalize the pixel value (0–1)
Data Pairing	Create positive pairs (both images have seatbelts), and negative pairs (one with, one without)
Neural Network Type	Siamese Neural Network
Base Network Architecture	Conv2DLayers (64,128,128 filters), Maxpooling2D, Flatten, Dense (256 neurons)
Similarity Function	Absolute difference between two embeddings (using Lambda layer)
Final Classification	Fully connected Dense (1, Sigmoid Activation)
Optimizer	Adam
Learning Rate	0.001
Training Method	K-fold Cross Validation (5 fold) to improve generalization
Metrics Evaluated	Accuracy, Precision, Recall, F1 Score, Confusion Matrix, ROC, AUC
Visualization	Confusion matrix Heatmaps (using Seaborn)
Loss Function	binary_crossentropy
Batch Size	32
Epochs	10
K-fold	5
Random seed	42
Conv2D #1	Filters = 64, kernal_size = (10,10), activation = ‘relu’, padding = ‘same’
MaxPooling2D #1	Default (2,2)
Conv2D #2	Filters = 128, kernal_size = (7,7), activation = ‘relu’, padding = ‘same’
MaxPooling2D #2	Default (2,2)
Conv2D #3	Filters = 128, kernal_size = (4,4), activation = ‘relu’, padding = ‘same’
MaxPooling2D #3	Default (2,2)
Flatten	Converts features matrix into 1D Vector
Dense (embedding)	Units = 256, activation = ‘relu’
Lambda Layer	X1, X2
Dense (output)	Units = 1, activation = ‘sigmoid’
Total Parameters	53,893,77
Inference Time per Frame	0.109296 s
Model Size	~21.6 MB
FLOPs	4.95 × 109
Stride Size	(1,1)
Drop Out	No (because we used cross-validation)

The suggested method for seatbelt monitoring using the Siamese Neural Network utilized both positive and negative datasets with a total of 8618 images divided into 4118 and 4500 as positive and negative samples, respectively. The Siamese network is trained and tested utilizing the standard seatbelt dataset. Of the total data, 80 percent are used for training while the remaining 20 percent are used for the testing process.

Basically, the model is trained using gray images; hence, the time of processing is decreased because gray images use one layer. On the other hand, the colored images have more details of edges and features, but the processing time is multiplied by 3. So, the gray images are used in this investigation, and we compared it with colored as will be shown later in the results. As denoted the metrics are closed, and hence the application is real-time, so using the model with less processing time is better). The model presented the performance metrics shown in Table 6.

Table 6. The performance metrics of the proposed model.

Without Augmentation	Folds	Accuracy	Precision	Recall	F1 Score	AUC	Processing Time (s)
	Fold #1	0.9964	0.9976	0.9951	0.9963	0.9999	2211
	Fold #2	0.9982	0.9964	1.0000	0.9982	0.9993	2140
	Fold #3	1.0000	1.0000	1.0000	1.0000	0.9999	2201
	Fold #4	1.0000	1.0000	1.0000	1.0000	0.9998	2183
	Fold #5	1.0000	1.0000	1.0000	1.0000	0.9993	2159
	Average Scores	0.9989	0.9988	0.9990	0.9989	0.9996	2179
With Augmentation	Folds	Accuracy	Precision	Recall	F1 Score	AUC	Processing Time (s)
	Fold #1	0.9848	0.9730	0.9976	0.9852	0.9970	13,993
	Fold #2	0.9792	0.9608	0.9988	0.9794	0.9951	13,890
	Fold #3	0.9881	0.9888	0.9876	0.9882	0.9979	13,636
	Fold #4	0.9785	0.9728	0.9846	0.9787	0.9960	13,045
	Fold #5	0.9814	0.9643	0.9992	0.9814	0.9939	12,123
	Average Scores	0.9824	0.9719	0.9936	0.9826	0.9959	13,338

The model runs for 5 folds, for the two cases (without augmentation and with augmentation). For the two cases, the metrics (Accuracy, Precision, Recall, F1 Score, ROC, and AUC) are balanced and converged among folds, which mean the model is stable across folds. These results prove that the utilization of the Siamese Neural Network in seatbelt verification is more efficient than detection. In other words:

High Accuracy means the model classifies most samples correctly;

High Precision means the model predicted the samples as positive;

High Recall means the model identifies all actual positive samples;

High F1 Score means the model has strong balance between Recall and Precision;

High AUC means the model has strong distinguish between classes.

Figure 6A,B below shows the confusion matrix with Augmentation (Figure 6A) and without (Figure 6B)

Figure 6. (A) Confusion matrix for each fold without augmentation. (B) Confusion matrix for each fold with augmentation.

By referring to Figure 6A, it can be concluded that the model is accurate, and it correctly classifies most samples. The highest number of samples predicted are true positive and true negative in folds = 1 and 2, while the model predicted 100% in fold = 3, 4, and 5. This outcome is an achievement.

The same thing is repeated for Figure 6B. The confusion matrix is stable across all folds.

By having another look at Table 6 and comparing between the two cases (without augmentation and with augmentation), all metrics are close in the two cases. This is because of the well training without overfitting, but the processing time in the case of augmentation is multiplied by 6. This is too much time. For this reason, in this work, the case without augmentation is used.

The Siamese Neural Network based on the K-fold strategy is applied in the suggested monitoring approach. The elevated accuracy values of 99.75% in training and 99.73% in testing were quite reasonable while the loss function is 0.0313 for both training and testing datasets, as shown in Figure 7 and Figure 8 for both Accuracy and Loss Function of the proposed model for each fold.

Figure 7. Training and validation accuracy for each fold.

Figure 8. Training and validation loss function for each fold.

By examining Figure 7 and Figure 8, it can be noted that the overall Accuracy and Loss are satisfactory, which means that the average of the fifth fold is acceptable, Hence, in the previous figures of Accuracy and Loss Functions (Figure 7 and Figure 8), the scale used approximately 96–100% and 0–10%. Figure 9 and Figure 10 show the Accuracy and Loss Function with limits from 0 to 1.2 for accuracy and 0 to 1 for Loss.

Figure 9. The Accuracy for each epoch using gray images.

Figure 10. The Loss Function for each epoch using gray images.

A new sight to Figure 9 and Figure 10, the Accuracy and Loss Functions are showing stable training and satisfactory validation.

The big deal of feature extraction is what one should use. When using gray image, absolutely, the time of processing will be gained, but it affects training and accuracy, and when using colored images, the processing time increases, but the metrics improve.

In this model, the gray image is dependent because the results were accepted for a real-time model. At the same time, the results between the use of gray or colored images are closed. So, to obtain the benefits of time processing the gray images are used.

The following Figure 11 and Figure 12 show the Accuracy, Loss Function, and Confusion Matrix for each fold by using a color image in the training of the model.

Figure 11. The Accuracy Function for each epoch depending on the colorful training images.

Figure 12. The Loss Function for each epoch depending on the colorful training images.

Figure 13 denotes the average Accuracy and Loss across folds, depending on the colorful training images.

Figure 13. The average Accuracy and Loss across folds, depending on the colorful training images.

Figure 13 shows that the Accuracy and Loss Function is close to that when using gray, resulting from the proficiency of the proposed model, which succeeded in this work. Return to Table 5. The training Accuracy is raised to 99.28% from the second epoch and to 99.37% for the validation value. The Precision obtained is 0.9978. The superiority of the network performance from the second epoch is achieved as a result of applying the K-fold approach in the right case; however, the original domain matches the target domain.

Figure 14 shows the ROC curves for each fold, and as denoted in each fold, the AUC is approximately 0.9996, which means an excellent distinction, where the model correctly distinguishes between positive and negative classes.

Figure 14. The ROC curve for each fold and the average.

The suggested model has attained the best evaluation metrics by comparing the obtained results with other those of previous works. Most of the state-of-the-art methods in this field are based on detection approaches, while in the proposed work, the method is based on classifying the seatbelt datasets.

Moreover, to ensure the model does not enter into overfitting:

• The model proposed consistent achievement across each fold of validation, which means the model is stable rather than memorized
• The training and validation curves for accuracy and loss converged, which is obvious from obtained Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14.
• In addition, when data augmentation was applied it clearly shows that the obtained result is very close to the output of the model without augmentation.

Finally, a comparison study between the Siamese network model and previous studies is explained in Table 7.

Table 7. Performance comparison with previous works.

Reference	Method	Accuracy	Year
Guo et al. [45]	Edge Detection and HSV	81%	2011
Elihos et al. [18]	Multi-box object detector (SSD), Convolutional Neural Network (CNN), and Fisher Vector (FV)	91.9%	2018
Zhang [24]	Multi-box object detector (SSD)	83%	2019
Chun et al. [19]	Multi-box object detector (SSD)	91.9%	2019
Yang et al. [26]	SSD MobileNet V2 net work	95.21%	2020
Kannadaguli et al. [27]	Fully Connected One Shot (FCOS)	93.6%	2020
Kashevnik et al. [15]	YOLO neural network	93.20%	2020
Hosam [20]	S-AlexNet	90%	2020
Chen et al. [28]	YOLOv3	92.1%	2020
Saad-Bin et al. [29]	YOLOV4 or AlexNet LeNet	93.6%	2021
Luo et al. [30]	ARD-Net	97.08%	2021
Roopa Ravish et al. [31]	ITVD	92.80%	2021
DS Bhupal Naik et al. [32]	CNN	91.4%	2021
DS Bhupal Naik et al. [32]	SVM	87.17%	2021
Wang and Ma [16]	Tiny YOLO	95%	2022
Sara and Abdolhossein [22]	YOLOv5s and ResNet34	98.9%	2022
Rupal et al. [13]	Deep learning	89.1%	2022
Feng Hu [33]	Shape modeling	97.5%	2022
Singh L. et al. [34]	Hybrid CNN-SVM	98.5%	2023
Sutikno et al. [35]	YOLOv8	88.46%	2023
Udayanti et al. [36]	YOLOv3 + LSTM	89%	2024
Qiu et al. [37]	ResNet-TA	98.25%	2024
Nibras et al. [11]	Transfer learning	99.42%	2024
Proposed method	K-Fold Siamese Network	99.89%	2025

Moreover, Figure 15 denotes the previous studies as a flowchart.

Figure 15. Flowchart of a comparison with a previous study [11,13,15,16,18,19,20,22,24,26,27,28,29,30,31,32,33,34,35,36,37,45].

Traditionally, The SNN model is matched and achieved good performance compared to object detection models. Regarding experimental rigor, the model used K-fold to prevent the bias obtained from data. Furthermore, all models that we compared with aimed to check whether the seatbelt was put on or not. The models (YOLO, SSD, and ResNet) are widely adopted for detection and used for seatbelts.

However, the object detection approach is more complex in binary classification; it is based on localization and classification. The intention is not to compare between classifiers and detectors, but the target is to contextualize model size deployment within the model’s characteristics.

By comparing our proposed model with the standard YOLOv8 and ResNet-50, Table 8 shows the comparison.

Table 8. Comparison of the proposed model cost with YOLOv8 and ResNet-50.

Model	Parameters	Model Size	FLOPs
YOLOv8	~11–68 million (Depending on versions)	40–250 MB	8.7 GFLOPs
ResNet-50	~25.6 million	98 MB	4.1 GFLOPs
Proposed method	~5.39 million	21.6 MB	4.9 GFLOPs

6. Conclusions

Automatic seatbelt monitoring systems (ASMs) have shown an increasing trend in the last 10 years regarding the innovation in computer vision techniques, wearable sensors, and artificial intelligence. Challenges keep the previous systems from enhancing road safety and saving lives. In this study, a Siamese Neural Network was developed and evaluated for seatbelt verification using a dataset of positive and negative seatbelt images. The model was trained and validated using 5-fold cross-validation, ensuring a robust assessment of its performance. The model is implemented, trained, and evaluated using Python. The SNN is not working under one-shot or few-shot learning. The SNN is a fully supervised model using many label data pairs of images. The SNN model based on supervision pairs shows similarity along multiple epochs. In this work, SNN learned on two classes (positive and negative). The model used 5 folds for validation training: average Accuracy for 5 folds (99.89%), average Precision (99.88%), average Recall (99.90%), and average F1 Score = (99.89%). The high Accuracy refers to the ability of the model to distinguish between wearing and not wearing the seatbelt. The high Precision means the model failed to classify the negative image as positive, while the high Recall shows that the true images are classified correctly. These metrics are exceptional, and this means the model is safe and efficient. Additionally, the model is run twice, one time for colored images and the other for the gray image. The results showed close evaluation metrics for the two times; however, the model is based on real time. Therefore, the gray images are dependent for the decreased processing time. Given the outstanding results, this approach has strong potential for real-time seatbelt compliance monitoring in vehicles. Future work can focus on deploying the model in real world scenarios, optimizing it for different lighting conditions, angles, and motion blur and integrating with edge computing devices or traffic surveillance systems for practical implementation.