ICK-PANet: A Multiscale Driver Distraction Detection Network Based on Attention and Pyramid Convolution

Abstract

In recent years, the number of deaths caused by traffic accidents has continued to rise. According to investigations, approximately one-fifth of accidents are caused by drivers being distracted. With the rapid development of convolutional neural networks (CNNs) in the field of computer vision, many researchers have developed CNN-based network models to recognize distracted driving actions. However, many models have too many parameters, making them unsuitable for deployment in actual vehicles. To address this issue, we propose a multiscale driver distraction detection network called ICK-PANet, which combines attention, lightweight incremental convolution kernels, and lightweight pyramid convolution to quickly and accurately identify driver distraction actions. First, ICK-PANet uses lightweight incremental convolution kernels to capture global information and driving action details effectively. Then, it introduces lightweight pyramid convolution and attention modules to extract multistage features, thereby expanding the network’s receptive field to improve the recognition ability of key features. Finally, it fuses multistage features to predict the results. ICK-PANet was experimentally evaluated on two public datasets: the American University in Cairo Distracted Driver (AUC) dataset and the StateFarms dataset (SFD) provided by the Kaggle competition platform. The AUC and SFD accuracies are 95.66% and 99.84%, respectively, which are higher than those achieved by many other state-of-the-art methods. ICK-PANet requires only 0.4M parameters, making it one of the most lightweight models currently available.

1. Introduction

According to statistics provided by the National Highway Traffic Safety Administration (NHTSA), traffic accidents caused by distracted driving resulted in 391,000 injuries and 3477 fatalities. Among these accidents, one-tenth of the number of deaths were caused by distracted driving [1]. Distracted driving behavior refers to any activity that may divert a driver’s attention, such as operating a phone, texting, drinking, eating, and applying makeup [2]. An advanced driver assistance system (ADAS) achieves the early detection of potential accidents during driving by comprehensively analyzing data from various sensors, including cameras, ultrasonic sensors, LiDAR, and the global positioning system (GPS). However, if a driver excessively trusts auxiliary driving functions and becomes distracted by activities unrelated to driving, significant traffic safety risks may result.

There are two lines of research in distraction detection: the computer vision branch and the inattention monitoring branch. The computer vision branch aims to extract facial features, head pose, gaze direction, or non-driving-related activities such as mobile phone use from video streams or images. The output from the computer vision branch is then utilized by the inattention monitoring branch, which estimates whether the driver is attentive enough or should be classified as distracted by taking time history and context into account. Researchers have used cameras to capture driver actions in real time, and by using these live video or image data, CNNs can recognize and analyze the operational behaviors of drivers. In deep learning research, a series of mainstream network architectures, such as VGG [3], GoogLeNet [4], ResNet [5], InceptionNet [6], MobileNet [7] and SqueezeNet [8], have been shown to be highly effective in the field of computer vision. However, owing to the large number of network parameters in these deep learning models and the fact that they are not specifically designed for detecting distracted driving, there is still considerable room for improvement in terms of enhancing detection accuracy and reducing the number of parameters.

Pyramidal convolution (Pyconv) [9], an innovative convolutional architecture, was proposed by the Inception Institute of Artificial Intelligence (IIAI) in 2020. This architecture achieves multiscale feature extraction by stacking convolutional kernels of different sizes, demonstrating outstanding application results in various fields such as image classification, object detection, and image segmentation. Although Pyconv is efficient in image feature extraction, it has shortcomings when processing local image features. To address this issue, we integrate Pyconv with attention mechanisms to achieve fine-grained focus on key areas of images, thereby obtaining more effective feature information. Overall, PyConv is substantially enhanced in this study. The main contributions are as follows:

•

We propose a network structure called ICK-PANet (incremental convolutional kernel–pyramid attention network). The network consists of four layers of incremental convolutional kernels, which demonstrate efficient performance in extracting feature information. The multistage output features of ICK-PANet are subsequently fed into an improved lightweight pyramid structure, which integrates the convolutional block attention module (CBAM) attention mechanism to precisely capture subtle differences between different categories.

•

Compared with a traditional 3×3 standard convolutional kernel, the lightweight incremental convolutional kernel structure in ICK-PANet not only significantly reduces the number of parameters but also generates more effective image features.

•

ICK-PANet achieves the rapid localization of key features by introducing channel attention and spatial attention. The attention module focuses on key areas of the image rather than the entire image, thereby improving network efficiency when recognizing key features.

This study enhances pyramidal convolution (Pyconv) by integrating it with attention mechanisms, proposing the ICK-PANet structure which features incremental convolutional kernels and an improved lightweight pyramid structure with CBAM attention, effectively reducing network parameters, improving feature extraction efficiency, and enhancing the accuracy and efficiency of distracted driving detection. The remainder is organized as follows: Section 3 introduces related work. Section 4 provides detailed introduction to the ICK-PANet model. Section 5 introduces the experimental environment and datasets. Section 6 presents the experimental results. Finally, the work content is summarized.

2. Ethics and Privacy Considerations

2.1. Data Privacy and Protection

The datasets used in this study (AUC, SFD and 100-Driver dataset) contain images of individuals performing various driving behaviors, which raises important privacy considerations. While these datasets are publicly available for research purposes, the deployment of driver distraction detection systems in real-world vehicles requires careful attention to data privacy protection. In practical applications, driver images should be processed locally on vehicle-mounted devices rather than being transmitted to external servers, minimizing the risk of data breaches and unauthorized access. Additionally, implementing robust encryption protocols for any data storage or transmission is essential to protect driver privacy.

2.2. Human Subject Considerations

This study involve human subject data, as the algorithms are trained and tested on images of real individuals performing driving behaviors. While this study does not involve direct interaction with human participants or interventional research, the use of human images carries ethical responsibilities. Researchers must ensure that the use of these datasets complies with relevant ethical guidelines and regulations, and that the potential benefits of the research in improving road safety justify the use of human data.

2.3. Monitoring and Surveillance Concerns

The implementation of driver distraction detection systems represents a form of workplace surveillance that may raise concerns about privacy intrusion and psychological impact on drivers. Constant monitoring could potentially create stress or anxiety among drivers, particularly if they feel that every minor deviation from perfect driving behavior is being recorded and evaluated. It is important to establish clear guidelines about the appropriate use of monitoring data, ensuring that systems are used primarily for safety improvement rather than punitive measures. The balance between safety benefits and privacy rights must be carefully considered in the design and deployment of these systems.

3. Related Work

In recent years, the rapid advancement of convolutional neural networks (CNNs) in computer vision has driven extensive research on deep learning-based distracted driving detection [10,11,12]. The core paradigm of existing studies involves inputting camera-captured images or videos into detection models to recognize driving actions and judge driver distraction status. However, a critical review of these works reveals prominent limitations in model efficiency, deployment feasibility, detection accuracy, or practical adaptability—gaps that the present study aims to address. Below is a systematic comparison of existing methods, followed by explicit identification of research gaps and clarification of how the proposed model advances the field. To improve feature extraction capability, some studies have integrated attention mechanisms or specialized network structures, but at the cost of increased computational complexity. Mittal et al. [13] proposed the CAT-CapsNet model by combining capsule networks with attention mechanisms, enabling focused extraction of key image regions. However, the model’s high computational complexity and large parameter count (8.5 M) make it impractical for on-vehicle deployment, where hardware resources are limited. Similarly, Liu et al. [14] developed an innovative knowledge distillation-based model, constructing a high-performance teacher network by enhancing CNN robustness to lighting changes from shallow to deep layers, then guiding the student network through knowledge distillation. Despite its innovation, the teacher network’s excessive parameter count (44.62 M) severely hinders its real-world application.

Pretrained networks have been widely adopted to reduce training costs, but many such studies suffer from insufficient accuracy or excessive parameters. Goel et al. [15] utilized pretrained MobileNetV2 and DenseNet for distracted driving detection, but their model only achieved a maximum accuracy of 93.8% with a large number of parameters, failing to balance accuracy and efficiency. Hossain et al. [16] fused pretrained VGG16, ResNet50, and MobileNetV2 networks for multi-category distracted driving detection, but the network fusion led to excessive parameters, making it unsuitable for on-vehicle deployment. Arefin et al. [17] improved AlexNet by integrating HOG features, reducing parameters from 62.3 M to 9.7 M, but this optimization sacrificed accuracy (dropping from 93.65% to 93.15%), indicating an unresolved trade-off between parameter quantity and detection performance.

Other studies have explored specialized feature fusion or auxiliary tasks, but they introduce additional complexity or limitations in practicality. Wu et al. [18] proposed a multi-feature fusion network based on pose estimation, emphasizing hands as key indicators of distraction and fusing global, hand, and pose features. However, the need for additional processing steps to extract hand and pose information increases computational overhead and reduces adaptability in real-world driving scenarios. Ma et al. [19] incorporated driver emotion recognition as supplementary information via a pseudolabeled multitask learning algorithm, but this approach relies on high-quality emotion labels, which are difficult to obtain in real-time and may introduce noise. Hari et al. [20] and Mase et al. [21] focused on head posture, eye state, and posture spectral-spatial features, respectively, but their single-feature focus limits generalization to complex distracted behaviors (e.g., mobile phone use without obvious posture changes).

A few studies have achieved high accuracy, but they either lack efficiency or fail to generalize. Baheti et al. [22] improved VGG16 by modifying fully connected layers to convolutional layers, adding regularization and batch normalization, and achieved 95.54% accuracy. However, the model still retains the inherent parameter redundancy of VGG16, limiting its deployment. Abouelnaga et al. [10] created the AUC dataset and achieved 95.98% accuracy by finetuning pretrained models on multi-modal images and fusing outputs via genetic algorithms, but the model’s complexity makes it difficult to apply to real-time detection. Alotaibi et al. [23] combined ResNet, Inception, and HRNNs to achieve 99.30% accuracy on the SFD dataset and 92.36% on the AUC dataset, while Ke et al. [24] proposed SFANet with 99.87% accuracy on the SFD dataset. However, both models rely on complex network structures with high parameter counts, and their performance on edge devices (critical for on-vehicle deployment) has not been validated, leaving a gap in efficient high-accuracy detection.

In summary, the existing research on distracted driving detection faces three core unresolved gaps: (1) The trade-off between model accuracy and efficiency remains unbalanced—high-accuracy models often have excessive parameters and computational complexity, while lightweight models sacrifice accuracy; (2) Practical adaptability is insufficient, with many models relying on additional feature extraction steps, complex data preprocessing, or hard-to-obtain auxiliary labels, making them unsuitable for real-time on-vehicle deployment; (3) Generalization to diverse real-world scenarios (e.g., varying lighting, complex driving postures) is limited, as most models are validated on specific datasets without sufficient robustness testing.

This paper presents a systematic analysis of current driver distraction detection approaches and identifies three fundamental limitations that impede their practical deployment in vehicle safety systems. First, there exists an unresolved tension between model accuracy and computational efficiency—high-accuracy models are typically too heavy for real-time inference, while lightweight alternatives suffer from significant performance degradation. Second, most existing methods rely on generic image classification backbones such as VGG16 or ResNet50, which are not tailored to the visual characteristics of driver behaviors and thus waste capacity on irrelevant cues like interior textures and lighting variations. Third, these approaches lack an effective mechanism for capturing multi-scale behavioral patterns, ranging from subtle finger movements to large arm gestures, either due to fixed receptive fields or excessive depth that inflates model size. To overcome these challenges, we introduce ICK-PANet, a task-oriented lightweight architecture featuring three key components. It begins with Sobel edge preprocessing to eliminate background distractions and emphasize structural motion cues. It then employs a sequence of progressively enlarged convolutional kernels, from 3×3 to 9×9, enabling rich multi-scale feature extraction within shallow layers. Finally, it integrates a compact pyramid convolution module with a channel-spatial attention mechanism to adaptively allocate representational capacity to the most discriminative spatial regions and feature channels. With only 0.40 million parameters, ICK-PANet achieves state-of-the-art performance, demonstrating a principled resolution to the accuracy–efficiency dilemma and establishing a new paradigm for behavior-aware model design in embedded vision systems.

4. Details of the Proposed Approach

4.1. Sobel Edge Detection

We attempts to find an optimized network to complete the detection in real time. Therefore, three contradictory requirements must be addressed, namely, a small number of parameters, high accuracy, and high speed. ICK-PANet includes three features: (i) Sobel Edge Detection contour images, (ii) lightweight incremental convolution kernels, and (iii) PyConv attention modules. The origin of the Sobel operator is sparsely documented in academic circles. However, according to a study by Heath et al. [25], the initial proposal of the Sobel operator can be traced back to Sobel’s doctoral dissertation published in 1970 [26]. Among various edge detection algorithms, the Sobel operator demonstrates outstanding performance when dealing with images that have significant grayscale gradients and noise, and it can locate edges quite accurately. Owing to its intuitive computation process and ease of implementation, the Sobel algorithm is particularly suitable for efficient operation on various hardware platforms. It also has strong noise resistance and can suppress noise to a certain extent. The Sobel operator effectively identifies edges in an image by applying two 3×3 convolution kernels to calculate the gradient of the image in the horizontal and vertical directions, thereby increasing edge detection accuracy. Therefore, the Sobel operator has been widely applied in the fields of computer vision and image processing. In this way, we were able to filter out irrelevant background information from the images, such as the color of clothing or changes in interior lighting, as shown in Figure 1.

Figure 1. Raw images are preprocessed using the Sobel Edge Detection algorithm to remove background noise.

This image processing method enables the network to focus more on learning different actions performed by drivers.

4.2. Lightweight Incremental Convolutional Kernel

In the literature [27], we observed that a model using standard incremental convolution kernels achieved an accuracy of 95.85%, which was 0.26% higher than that of a model using decremental convolution kernels. However, the number of parameters in the standard incremental convolution kernel model reached 5.46 M, approximately seven times that of the decremental convolution kernel model. This is disadvantageous for deploying models on resource-constrained mobile vehicle-mounted devices. Our method applies depthwise separable convolution to reduce the model parameters while still maintaining good predictive performance. Depthwise separable convolution is often used in lightweight convolution kernels, which divides the regular convolution into two steps: depthwise convolution and pointwise convolution, reducing the coupling and the amount of convolution computation. A depthwise separable convolution structure is shown in Figure 2. Depthwise separable convolution reduces computations around 8 times compared to standard 3×3 convolution kernels without much loss in accuracy [28]. Therefore, our ICK-PANet achieves high classification accuracy while significantly reducing the number of network parameters. In addition, as we noted in Ref. [27], more convolutional layers lead to more computation time. Consequently, the network structure we employed has only four layers, and the convolutional kernels are designed as an incremental sequence, with sizes of 3×3, 5×5, 7×7, and 9×9.

Figure 2. Depthwise separable convolution structure.

To further verify the effectiveness of the lightweight incremental convolutional kernels in feature extraction, we visually display the output feature maps of each network layer at a 1:1 ratio. The visualizations of the standard 3×3 convolutional kernel, 3×3 depthwise separable convolution kernel, standard incremental convolutional kernel, and incremental 3×3 depthwise separable convolution kernel are shown in Figure 3. The first layer feature map is used as the baseline layer, and the four methods use the standard 3×3 convolution kernel to extract the underlying visual features, providing a unified reference for subsequent comparison. From the second layer, the performance differences of different convolution strategies gradually appear. The feature images generated by the standard 3×3 convolution kernel and 3×3 deep separable convolution kernel show high similarity in texture detail and edge definition, which indicates that deep separable convolution has significant parameter compression advantages while maintaining feature representation. When the network enters the third and fourth layers, the two methods using the incremental convolution architecture, namely, the standard incremental convolution kernel and the lightweight incremental 3×3 depth separable convolution kernel, begin to show a different feature extraction mode from the fixed-scale convolution kernel. The feature maps of these two incremental structures show a certain degree of fuzziness. However, this fuzziness actually reflects that the convolutional neural network extracts higher-level semantic abstract features in the deep stage. Although these features are not clear enough visually, they may have a stronger ability to distinguish drivers’ different types of distractions. The comparative analysis in Figure 3 reveals three key technology findings. Firstly, the incremental convolution structure can effectively capture the visual information of different spatial frequencies through the multi-scale incremental design of 3×3, 5×5, 7×7 and 9×9, which is theoretically superior to the single-scale convolution kernel in the richness of feature representation. Secondly, the deep separable convolution technology realizes the order of magnitude compression of model parameters on the premise of ensuring the quality of feature extraction, which provides a feasible technical path for lightweight network design. Third, the lightweight incremental convolution kernel successfully combines the dual advantages of incremental structure and deep separable convolution and significantly reduces the number of model parameters while maintaining the same feature extraction ability as the standard incremental convolution kernel.

Figure 3. Visualization feature maps of the different convolution kernels from the first level to the fourth level.

4.3. PyConv Attention Module

Although the standard PyConv model demonstrates excellent performance, it still has certain limitations in extracting key features from images. To more effectively capture key features, the multiscale feature extraction capability of PyConv is combined with attention mechanisms. A lightweight residual structure is introduced to mitigate potential performance degradation during network training and to optimize PyConv. The Convolutional Block Attention Module (CBAM)—comprising a Channel Attention Module (CAM) and a Spatial Attention Module (SAM)—is integrated to enhance feature representation.

Channel attention focuses on identifying effective feature information, whereas spatial attention, while enhancing channel attention, is responsible for constructing the spatial relationships of attention features. The final design of the lightweight PyConv attention module is shown in Figure 4.

Figure 4. PyConv attention module structure.

4.4. ICK-PANet Network Structure

The proposed approach is a multiscale driver distraction detection network. As shown in Figure 5, the system processes image streams captured by cameras mounted above the passenger seat. A trained ICK-PANet model classifies various types of driver distraction, and the corresponding training methodology is detailed in this article.

Figure 5. The overall architecture diagram of the proposed approach.

The overall network structure of ICK-PANet is shown in Figure 6. The original image is first denoised via Gaussian filtering, then the driver’s driving contour information is obtained via the Sobel Edge Detection algorithm. The processed data are then input into ICK-PANet, which consists of four convolutional layers, with the number and size of the convolutional kernels increasing incrementally. The outputs of the third and fourth layers are then extracted and input into the PyConv attention module. The output results and the output features of the fourth convolutional layer are fused together after global max pooling to form a fully connected feature layer. Finally, the Softmax layer is used to obtain the driver’s distracted driving categories. The overall algorithm of building the teacher network is given in Algorithm 1.

Figure 6. Simplified visualization of ICK-PANet where different colors signify different types of operations.

Algorithm 1 Building the ICK-PANet Computation Process

Require: Given a dataset $D={\left\{({\mathrm{input}}^i,p_{\mathrm{truth}}^i)\right\}}_{i=1}^T$ (T is the total number of images in D), and py_block represents the Pyramid Convolution Module. Let $\{x_1, x_2, x_3, x_4\}$ denote the output feature maps. SC is defined as SeparableConv. We use cross entropy loss ${\mathsf{\ell }}_{\mathrm{cls}}$ to minimize the distance between ground truth label $p_{\mathrm{truth}}$ and each prediction probability distribution.

1: for epoch $\in [1, \mathrm{num}\text{\_}\mathrm{of}\text{\_}\mathrm{epoch}]$ do

2:     for (input, $p_{\mathrm{truth}}$) in D do

3:         for n $\in [1, N]$ do

4:               $x_1=\mathrm{SC}(3,64)\left(\mathrm{input}\right)$

5:               $x_2=\mathrm{SC}(5,128)\left(f^{bn}\left(f^{\max \text{\_}\mathrm{pool}}\left(x_1\right)\right)\right)$

6:               $x_3=\mathrm{SC}(5,256)\left(f^{bn}\left(f^{\max \text{\_}\mathrm{pool}}\left(x_2\right)\right)\right)$

7:               $x_4=\mathrm{SC}(9,512)\left(f^{bn}\left(f^{\max \text{\_}\mathrm{pool}}\left(x_3\right)\right)\right)$

8:               $x_5=\mathrm{py}\text{\_}\mathrm{block}\left(x_3\right)$

9:               $x_6=\mathrm{py}\text{\_}\mathrm{block}\left(x_4\right)$

10:             $x_7=f^{\mathrm{concat}}\left(f^{\mathrm{GPA}}\left(x_4\right),f^{\mathrm{GPA}}\left(x_5\right),f^{\mathrm{GPA}}\left(x_6\right)\right)$

11:             $p=f^{\mathrm{softmax}}\left(\mathrm{Dense}\left(x_7\right)\right)$

12:             $\mathsf{\ell }={\mathsf{\ell }}_{cls}\left(p,p_{\mathrm{truth}}\right)$

13:             BACKPROP(ℓ)

14:        end for

15:     end for

16: end for

5. Experimental Environment and Dataset

5.1. Experimental Environment

The experiment is based on Python 3.7 and uses TensorFlow 2.5 to build a deep learning network environment. The training and testing system is Windows 10 version 1903, with the following hardware specifications: the CPU is an i5-13400F with a base frequency of 2.5 GHz; the memory is 32 GB; and the GPU is an NVIDIA GTX 4060Ti.

5.2. Dataset Introduction

Currently, there are two publicly available datasets on driver distraction. The first is the State Farm Distracted Driver Detection competition dataset on the Kaggle competition platform, known as SFD [29]. The SFD dataset has images with a resolution of 640×480, including 22,424 labeled training images and 79,727 unlabeled test images. Since the labels for the test images are not accessible, the experiments are limited to the training set images. These images show drivers in a safe driving state and in nine different states of distracted driving. Some sample data are shown in Figure 7.

Figure 7. Sample images of distracted drivers in the SFD dataset.

The second dataset is the AUC Distracted Driver dataset created by Abouelnaga [10]. It also contains images of drivers in a safe driving state and nine states of distracted driving. Some sample data are shown in Figure 8. The AUC dataset was captured by 31 drivers from different countries using four different vehicles, and various driving conditions, such as sunlight and shadows, were considered during capture. The dataset contains a total of 17,308 images, which are divided into 12,977 training images and 4331 test images according to the partitioning method used by Abouelnaga. Notably, the training and test sets are not randomly partitioned. For comparison with other studies, we also adopts the same partitioning method.

Figure 8. Sample images of distracted drivers in the AUC dataset.

5.3. Dataset Preprocessing

First, we preprocess the Sobel image by subtracting its mean values. On the one hand, this normalizes the data distribution, as image pixel values typically span a wide range (such as 0–255). After subtracting the mean values, the data distribution centers around zero, making the update directions of all dimensions more consistent during the gradient descent process, thereby accelerating model convergence. On the other hand, it eliminates illumination biases. The mean value of an image often reflects overall brightness or color biases (such as differences between indoor and outdoor lighting). By subtracting the mean, the model focuses more on structural information in the image (like edges and textures) rather than brightness differences, enhancing generalization ability.

To ensure that the Sobel Edge Detection does not filter out useful information, it is imperative to strike a balance among noise suppression, effective edge preservation through image preprocessing, and fine-tuning of Sobel operator parameters. The detailed technical approach is outlined below:

(1) Preprocessing optimization utilizing Gaussian Filtering. Gaussian filtering is introduced to smooth the original images, thereby suppressing the interference of high-frequency noise on gradient computation. This study employs a 3×3 Gaussian kernel to achieve a balance between noise reduction effectiveness and detail preservation. If the kernel size is too small (e.g., 1×1), noise cannot be effectively filtered out, making the subsequent edge extraction susceptible to interference. Conversely, if the kernel size is too large, while noise reduction capability can be enhanced, it will excessively smooth the image, causing critical fine structures such as subtle facial movements or hand posture changes of drivers to be blurred, thereby weakening the discriminative capability of the downstream neural network model for distracted behaviors. As shown in Figure 9, the 3×3 Gaussian kernel can effectively suppress noise while better preserving task-relevant details, providing more robust input for subsequent edge feature extraction.

Figure 9. Visualization of different Gaussian filtering sizes.

(2) Fine-tuning of Sobel Edge Detection parameters. We employ a smaller 3×3 convolution kernel size for Sobel Edge Detection, which is particularly suitable for preserving image details, thereby effectively avoiding the loss of key image information. In the driver distraction detection task, subtle features of palm contours are crucial for distinguishing different behavior types. If these details are lost during the edge detection stage, subsequent neural network models will struggle to make accurate classification judgments. To scientifically verify the rationality of this parameter selection, we conducted systematic experiments and visualized the detection performance of different Sobel Edge Detection kernel size parameters, as shown in Figure 10. Through comparing the performance of various kernel sizes such as 1×1, 3×3, and 5×5, it can be clearly observed that the 1×1 kernel size cannot effectively suppress noise interference, while 5×5 and larger kernel sizes cause excessive smoothing leading to detail blurring. Only when the kernel size is set to 3×3 does the system achieve the optimal balance between noise suppression capability and detail preservation effect.

Figure 10. Visualization of different Sobel Edge Detection kernel size parameters.

6. Experimental Results

6.1. Results and Discussion

(1) AUC Dataset Results and Discussion: The algorithm model is optimized using the momentum optimizer, with a total of 200 training batches and a batch size of 32 per input image. The momentum value is set to 0.95, and the learning rate is initialized at 0.001. Figure 11 presents the performance diagrams, including training and test accuracy, training and test loss, and the confusion matrix. A comparison of the experimental results between ICK-PANet and other state-of-the-art methods is provided in Table 1. The data split settings for the experiments are consistent with those used in the baseline models.

Figure 11. Performance diagrams for ICK-PANet using the AUC dataset: training and test accuracy (a), training and test loss (b), confusion matrix (c).

Table 1. Comparison of results with state-of-art methods on AUC dataset.

Model	Image Source	Approx. No. of Parameters (Million)	Accuracy (%)
Majority Voting ensemble of all 5 [10]	Original	140	95.77
GA weighted ensemble of all 5 [10]	Original	140	95.98
Orginal VGG [22]	Original	140	94.44
VGG with Regularization [22]	Original	140	96.31
Modified VGG [22]	Original	15	95.54
DenseNet + Latent + Pose [30]	Original	8.06	94.20
MobileNet [7]	Original	4.2	94.67
MobileNetV2 [31]	Original	3.5	94.74
NasNetMobile [32]	Original	5.3	94.69
SqueezeNet [8]	Original	1.25	93.21
MobileVGG [28]	Original	2.2	95.24
VGG-one-attention [33]	Original	>140	84.82
VGG-two-way-att [33]	Original	>140	87.74
CNN-H [34]	Original	3.6	95.15
Grabcut+CNN [11]	Original	0.5	95.64
D-HCNN [27]	Original	0.76	95.59
ICK-PANet (Ours)	Original	0.45	95.41
ICK-PANet (Ours)	Sobel image	0.40	95.66

After multiple experimental tests, we achieved an accuracy rate of 95.66% on the AUC dataset, which is highly accurate while also significantly reducing the model’s network parameters. The ICK-PANet model has the fewest parameters. We also compared our method with classic and state-of-the-art networks. Table 1 shows that compared with traditional classic networks such as VGG, the ICK-PANet model significantly reduces the number of parameters. Compared with mainstream lightweight networks such as SqueezeNet, the ICK-PANet model has approximately one-third the number of parameters, but the accuracy rate is increased by 2.45%. Compared with the recently proposed Student Network [14], the ICK-PANet model has 0.02 M fewer parameters and an accuracy rate increase of 0.02%. Compared with D-HCNN [27], the parameter count is reduced by approximately 48%, but the accuracy rate is increased by 0.07%. Although the detection model designed by Abouelnaga, the proposer of the AUC dataset, achieved an accuracy rate of 95.98%, the network parameter count reached 120 M, which does not meet the requirements for deployment in practical scenarios.

Ablation experiments on different input image types show that ICK-PANet achieves an accuracy of 95.41% with 0.45 M parameters when using original images. Although this setting seems superficially superior to the Sobel input scheme, it actually increases the number of parameters by 12.5% and reduces accuracy by 0.25%. This indicates that Sobel edge preprocessing can effectively extract key contour and motion features of driving behaviors, reducing redundant information while achieving an optimal trade-off between parameter efficiency and detection performance. Although the ensemble method proposed by Abouelnaga et al. [10] achieves an accuracy of 95.98%, its parameter size of 120 M is 300 times that of ICK-PANet, making it difficult to meet the real-time deployment requirements of in-vehicle edge devices.

(2) SFD Dataset Results and Discussion: For the experimental parameter configuration on the SFD dataset, consistency is maintained with the settings used for the AUC dataset. Following a common practice in the literature, the training set of the SFD dataset is randomly split into an 8:2 ratio for training and testing, respectively. Figure 12 presents the performance diagrams, including training and test accuracy, training and test loss, and the confusion matrix. A comparison of the experimental results between ICK-PANet and other state-of-the-art methods is provided in Table 2. To ensure a fair comparison, all networks are evaluated under identical parameter settings, including the number of epochs, optimizer, and learning rate.

Figure 12. Performance diagrams for ICK-PANet using the SFD dataset: training and test accuracy (a), training and test loss (b), confusion matrix (c).

Table 2. Comparison of results with state-of-art methods on SFD dataset randomly split by 8:2.

Model	Approx. No. of Parameters (Million)	Accuracy (%)
VGG-16 [35]	140	91.3
VGG-GAP [35]	140	98.7
Ensemble VGG-16 and VGG-16 [35]	>140	92.6
MLP [36]	-	82.0
RNN [36]	-	91.7
Driver-Net [36]	-	95.0
Vanilla CNN with Data Augmentation [37]	26.05	97.05
CNN with Transfer Learning [37]	>3.5	71.72
Student Network [14]	0.42	99.89
HCF [38]	>72.3	96.74
CNN-H [34]	3.6	99.84
OLCMNet [39]	2.78	89.83
D-HCNN [27]	0.76	99.86
MobileVGG [28]	2.20	99.75
ISTNet [40]	-	98.03
Ensemble-Faster-RCNN [41]	-	97.70
Efficientdet-D0 [42]	-	95.56
Efficientdet-D1 [42]	-	98.74
Efficientdet-D2 [42]	-	98.86
Efficientdet-D3 [42]	-	99.16
Efficientdet-D4 [42]	-	98.89
MobileViT [43]	5.58	99.73
MobileViT-CA [43]	5.58	99.89
ICK-PANet (Ours) Original Image	0.40	99.71
ICK-PANet (Ours) Sobel Image	0.40	99.84

We ultimately achieved an accuracy rate of 99.84%, as shown in Table 2. Against classic networks like VGG-16 and VGG-GAP, ICK-PANet delivers far higher accuracy while slashing the parameter count by more than 99.7%. When compared to the lightweight CNN-H model, our method matches its 99.84% accuracy but uses only around one-ninth of its parameters, making it much more efficient for edge deployment. Compared to D-HCNN [27], ICK-PANet cuts the parameter size by nearly half, with only a 0.02% drop in accuracy, striking an excellent balance between model size and performance. Even against state-of-the-art lightweight models like MobileViT-CA and Student Network, our approach maintains competitive accuracy with a smaller footprint. Across both the AUC and SFD public datasets, ICK-PANet consistently achieves the lowest parameter count among all compared methods, minimizing memory usage and computational load. The pyramid convolution and attention modules in our model effectively extract key discriminative features of driver behaviors, validating the design’s effectiveness.

(3) ICK-PANet and the existing ViT and Transformer Results and Discussion: We compared our method with existing approaches based on Vision Transformers (ViTs), ensuring that the dataset partitioning strategy matched that of the baseline models. In all comparative experiments, we maintained identical model hyperparameters without any modifications. The results, summarized in Table 2 and Table 3, demonstrate that ViT-based methods [43,44,45], Swin Transformer [40], CNN architectures [41,42], and knowledge distillation frameworks [14] all achieve competitive performance. Furthermore, we evaluated several state-of-the-art ViT and Transformer models implemented in the keras_cv_attention_models library [46], with results reported in Table 4. Our proposed ICK-PANet consistently outperforms these baselines in specific driver distraction recognition tasks, highlighting its enhanced capability in this domain.

Table 3. Comparison of results with state-of-art ViT and Transformer on SFD dataset.

Model	Accuracy (%)
VIT-B [44]	87.5
VIT-L [44]	89.5
ViTConv-Ti [45]	92.40
ViTConv-S [45]	94.00
ViTConv-B [45]	94.70
ViTConv-Ti+KNN [45]	96.20
ViTConv-S+KNN [45]	96.80
ViTConv-B+KNN [45]	97.90
ICK-PANet (Ours)	99.80

Table 4. Comparison of results with state-of-art ViT and Transformer on AUC dataset.

Model	Approx. No. of Parameters (Million)	Accuracy (%)
CMTTinY [47]	8.25	93.47
TinyViT [48]	5.08	93.84
GCViT [49]	11.49	92.96
RepViT [50]	4.77	93.49
DaViT [51]	27.60	92.03
FlexiViT [52]	85.81	88.57
MobileViT [53]	4.96	94.32
ConvNeXtV2 [54]	27.87	94.16
UniFormer [55]	21.04	91.43
EfficientFormer [56]	27.59	90.23
EfficientFormerV2 [57]	3.30	91.43
EfficientNetV2 [58]	5.93	94.27
ResNeXt50 [59]	23.07	94.20
LCNet [60]	1.70	93.63
FBNetV3D [61]	8.40	94.37
FBNetV3G [61]	14.75	94.90
FBNetV3B [61]	6.68	93.88
ICK-PANet (Ours)	0.40	95.66

(4) ICK-PANet in Occlusions, Varying Lighting Conditions and Driver Positioning Variations Results and Discussion: To further verify the efficiency and classification accuracy of the improved algorithm for driver distraction detection, we assess the model’s generalizability under adverse conditions such as occlusions, varying lighting conditions, and driver positioning variations. Despite the interference of varying lighting conditions and driver positioning variations, ICK-PANet can pay attention to the driver’s distracted actions and make correct distraction predictions. This statement is proved with the Grad-cam visualization in Figure 13. Additionally, the 100-Driver dataset contains occlusions, varying lighting conditions and driver positioning variations; classification accuracy results demonstrate the robustness of ICK-PANet (Table 5). Meanwhile, in order to evaluate the recognition performance of the model under different distracted behaviors, we visualize and analyze the recognition confidence of the model using several types of behavior samples from the test set. Figure 14 shows a boxplot of the recognition confidence of different models for each distracted behavior. The confidence distribution of ICK-PANet is concentrated above 0.99. Therefore, ICK-PANet has high recognition accuracy and stability for different categories of distracted behaviors.

Figure 13. Original images and grad-cam visualization images.

Table 5. Comparison of the ICK-PANet on 100-Driver dataset randomly split by 8:2.

Model	Image Source	Approx. No. of Parameters (Million)	Accuracy (%)
EfficientNetB7 [62]	Original	64.12	99.67
DenseNet201 [63]	Original	20.24	99.71
ResNet50 [5]	Original	23.59	99.62
NASNetMobile [64]	Original	4.28	99.59
MobileNet [7]	Original	3.24	99.65
ICK-PANet (Ours)	Original	0.45	99.71
ICK-PANet (Ours)	Sobal image	0.40	99.80

Figure 14. Visualization of boxplots with different distraction behavior recognition confidence levels on AUC (a) and SFD dataset (b).

6.2. Real-Time Testing

Considering the GPU’s capability for batch processing, we tested the model’s GFLOPS, the time required to process a single photo, and the time for batch processing on both the GPU and the CPU. Owing to the potential randomness in the processing, we took the average processing time as the evaluation result. To demonstrate the effectiveness of our lightweight structure in real-time processing, we also conducted experimental comparisons with other mainstream pre-trained lightweight networks, such as EfficientNetB7, DenseNet201, and NASNetMobile. The test results are shown in Table 6. Processing a single image in the GPU environment takes only 0.036, and processing 32 images at once requires merely 0.06. This processing speed is quite impressive. When comparing with EfficientNetB7, the accuracy of both models is basically consistent, with ICK-PANet reaching 95.66% and EfficientNetB7 at 95.54%, but the difference in parameter quantity is very large. ICK-PANet has only 0.40 M parameters, while EfficientNetB7 has as many as 64.12 M, a difference of approximately 160 times. Moreover, the processing speed of ICK-PANet is significantly faster, being nearly 5 times faster than EfficientNetB7. However, the performance in the CPU environment is more valuable as a reference, because this is closer to the actual operating conditions of vehicle-mounted devices. ICK-PANet takes 0.043 to process a single image on the CPU, and processing 32 images in batch takes 0.394. In comparison, ResNet152 requires 0.185 and 2.641, respectively, while DenseNet201 needs 0.149 and 2.024. Even when compared with the lightweight model NASNetMobile, ICK-PANet has fewer parameters but faster processing speed. From the perspective of computational complexity, the GFLOPS of ICK-PANet is 0.558, which is equivalent to NASNetMobile’s 0.567 but far lower than EfficientNetB7’s 5.202 and ResNet152’s 11.29. This data indicates that ICK-PANet significantly reduces the demand for computational resources while maintaining relatively high accuracy, which is very important for vehicle-mounted devices with limited hardware resources. Overall, ICK-PANet achieves a good balance across the three key indicators of accuracy, parameter quantity, and real-time performance. Its accuracy reaches 95.66%, surpassing many complex models whose parameter quantities are tens or even hundreds of times greater. At the same time, having only 0.40 M parameters and 0.558 GFLOPS of computational complexity enables it to achieve real-time processing under limited hardware resources.

Table 6. Comparison of Ick-Panet with existing lightweight networks in terms of GFLOPS and time consumption.

Model	Inference Time (ms)	Accuracy (%)	Approx. No. of Parameters (Million)	GFLOPS	GPU Time Cost (Second)		CPU Time Cost (Second)
					Single Image	A Batch of 32 Images	Single Image	A Batch of 32 Images
EfficientNetB7 [62]	151.71	95.54	64.12	5.202	0.057	0.268	0.302	3.771
ResNet152 [5]	85.59	95.22	58.39	11.29	0.045	0.135	0.185	2.641
ResNet50 [5]	29.15	95.47	23.59	3.86	0.041	0.079	0.085	1.053
Xception [65]	28.16	94.11	20.88	4.55	0.041	0.087	0.068	0.974
DenseNet201 [63]	132.46	95.50	20.24	4.290	0.052	0.114	0.149	2.024
NASNetMobile [64]	122.73	94.94	4.28	0.567	0.049	0.067	0.065	0.495
ICK-PANet (Ours)	4.12	95.66	0.40	0.558	0.036	0.060	0.043	0.394

6.3. Ablation Experiment

(1) Test of Different ICK-PANet Model Combinations: Specifically, “ICK-PANet with a standard convolution kernel and PyConv” employs standard incremental convolution kernels together with the lightweight PyConv attention module; “ICK-PANet with a lightweight convolution kernel and pyramid” uses 3×3 lightweight incremental convolution kernels along with the same attention module; and the “lightweight ICK-PANet without PyConv” variant utilizes lightweight incremental convolution kernels but omits the PyConv attention module entirely, the results as shown in Table 7.

Table 7. Comparison of ablation results with lightweight standard convolution kernels, lightweight incremental convolution kernels and lightweight PyConv attention modules.

Model	Approx. No. of Parameters (Million)	Accuracy (%)
with standard convolution kernel and Pyconv	12.63	95.68
with 3×3 lightweight convolution kernel and Pyconv	0.38	95.06
with lightweight and No Pyconv	0.21	95.29
ICK-PANet (Ours)	0.40	95.66

The standard incremental convolution kernel, when integrated with the lightweight Pyconv attention module with 12.63 M parameters, improved the accuracy by only 0.02% compared with that of ICK-PANet, demonstrating that lightweighting maintains a high level of accuracy while significantly reducing the number of parameters. Moreover, using the incremental structure with lightweight convolution kernels improved the accuracy by 0.6% compared with the 3×3 structure, with the incremental structure’s parameters only increasing by 0.02M, indicating that incremental convolution kernels can extract more effective features. Without the lightweight Pyconv attention module, although the number of parameters can be reduced by approximately 50%, the accuracy decreases by 0.35%. Additionally, it is evident that the ICK-PANet model, while leading other models in terms of parameter quantity, accuracy, and GFLOPS, still boasts real-time processing capabilities that surpass those of other mainstream models. Furthermore, it demands less storage and memory space, facilitating its deployment on vehicle-mounted devices.

(2) Test of Different Incremental Convolution Kernels: There are various sizes of incremental convolution kernels, each having an impact on parameters, accuracy, and real-time performance on the model, as shown in Table 8. We chose sizes 3, 5, 7, and 9 mainly based on the following two considerations:

Table 8. Comparison of the Ick-Panet with different incremental convolution kernels in terms of GFLOPS and time consumption.

Filter Size	Accuracy (%)	Approx. No. of Parameters (Million)	GFLOPS	GPU Time Cost (Second)		CPU Time Cost (Second)
				Single Image	A Batch of 32 Images	Single Image	A Batch of 32 Images
5, 7, 9, 11	95.61	0.42	0.752	0.036	0.062	0.047	0.422
5, 9, 13, 17	95.41	0.47	0.847	0.037	0.068	0.052	0.579
7, 9, 11, 13	95.57	0.44	1.035	0.038	0.067	0.051	0.503
3, 7, 11, 15	95.45	0.45	0.635	0.037	0.064	0.050	0.485
ICK-PANet (Ours)	95.66	0.40	0.558	0.036	0.060	0.043	0.394

(1) In terms of convolution kernel values, odd numbers are typically chosen for convolution kernels, primarily to facilitate processing and preserve locational information. An odd-sized convolution kernel ensures symmetrical padding on both sides of the image and guarantees that the anchor point (the center of the kernel) is precisely in the middle. This helps to perform sliding convolution based on the center of the module, thereby avoiding positional information deviation.

(2) The input image size of the model is set to 224×224, and after processing through four incremental convolution layers of ICK-PANet, the feature map is reduced to 28×28. Therefore, an excessively large convolutional kernel it is not advisable.

To sum up, convolution kernels of 3, 5, 7, and 9 have the fewest parameters. We can also find that when the step size of the incremental convolution kernel is 4, the accuracy is not better than a step size of 2 in ablation studies.

6.4. Cross-Dataset Testing

To further evaluate the performance, we integrated the AUC and SFD datasets and randomly selected 1000 images from each category within both datasets to create a new dataset comprising a total of 20,000 images. This new dataset was then randomly split into training and testing sets in an 8:2 ratio for model evaluation. Compared with classic network structures such as ResNet50, the ICK-PANet model significantly improved the accuracy rate. Although DenseNet201 improved by 0.02% compared to ICK-PANet, the number of parameters was about 50 times that of ICK-PANet, as shown in Table 9.

Table 9. Comparison of Ick-Panet on integrated AUC and SFD datasets randomly split by 8:2.

Model	Image Source	Approx. No. of Parameters (Million)	Accuracy (%)
EfficientNetB7 [62]	Original	64.12	97.65
DenseNet201 [63]	Original	20.24	98.20
ResNet50 [5]	Original	23.59	97.87
NASNetMobile [64]	Original	4.28	98.00
MobileNet [7]	Original	3.24	98.15
ICK-PANet (Ours)	Original	0.45	98.10
ICK-PANet (Ours)	Sobal image	0.40	98.18

Additionally, we conducted experiments on the 100-Driver dataset [66]. Specifically, we selected 32,843 images from 10 distraction categories in the 100-Driver dataset, which were the same as those in the AUC and SFD datasets. These images were randomly divided into training and testing sets in an 8:2 ratio for model evaluation. The results are presented in Table 5. Compared with classic network structures such as MobileNet, the ICK-PANet model significantly improved the accuracy.

7. Conclusions

A novel ICK-PANet model is proposed, employing lightweight incremental convolutional kernels and integrating multistage features—including a lightweight PyConv attention module—to enhance the utilization of key visual cues. With only 0.4 M parameters, the model achieves high accuracy and strong real-time detection performance. On the two public datasets AUC and SFD, it attains accuracies of 95.66% and 99.84%, respectively—surpassing many state-of-the-art methods while maintaining the lowest parameter count reported to date. Ablation studies further validate the rationality of the network design and explore its applicable scenarios.