2. Materials and Methods
2.1. Participants
Eleven healthy male volunteers (26 ± 3 years old) consented to participate in the experiment. The study adhered to the principles outlined in the Declaration of Helsinki of 1975, as revised in 2000. Approval for the research protocol was obtained from the Ethics Committee of Sapienza University of Rome (Prot. CE/PROG.604 dated 5 April 2017). Participation was limited to healthy, normal individuals who volunteered for the study. Each participant provided written informed consent after receiving a thorough explanation of the study. Careful consideration was given to participant selection to ensure sample homogeneity, with recruitment focused exclusively on male participants for the assessment of cognitive control behaviour.
2.2. Experimental Tasks
The experimental task and conditions were designed in accordance with Rasmussen’s original S-R-K model description [
1]. Specifically, six conditions were designed to elicit Skill, Rule, and Knowledge behaviours with two conditions for each cognitive control level. These conditions were labelled as Skill 1 (S1), Skill 2 (S2), Rule 1 (R1), Rule 2 (R2), Knowledge 1 (K1), and Knowledge 2 (K2). The S1 and S2 conditions (
Figure 2) entailed straightforward tracking tasks where participants maneuverer a cursor from the starting point (green box) to the finish line (checkered flag) using a joystick. S1 and S2 tasks required a mere automated sensorimotor sequence.
In the R1 and R2 conditions (
Figure 3), obstacles were introduced requiring participants to apply specific rules to navigate throughout the path to the arrival. In particular, the obstacles and the corresponding rules to be applied were as follows:
- -
Triangle
![Brainsci 15 00981 i001 Brainsci 15 00981 i001]()
: raise the altitude by moving the lever upwards.
- -
Circle:
![Brainsci 15 00981 i002 Brainsci 15 00981 i002]()
: turn right the circle with respect to the moving direction.
- -
Star:
![Brainsci 15 00981 i003 Brainsci 15 00981 i003]()
: lower the altitude by moving the lever downward.
- -
Asterix
![Brainsci 15 00981 i004 Brainsci 15 00981 i004]()
: turn left the asterisks with respect to the moving direction.
In the R1 condition, both the triangle and the circle were introduced (
Figure 3a), whereas in the R2 condition, only the star and asterisk were added (
Figure 3b). This adjustment aimed to enable participants to effortlessly retain the rules. R1 and R2 tasks thus required participants to recognise the condition, correlate it with the rule to be applied, and finally apply the corresponding rule.
The K1 and K2 conditions were designed to make participants face unfamiliar and challenging situations where prior experiences or knowledge could not offer direct solutions (
Figure 4). In the K1 condition, participants encountered a strong wind area upon entering an air sector, causing the cursor to move randomly without any control. When this started, a warning message flashed on the interface, indicating “Strong wind share on this altitude” to offer a hint (
Figure 4a). To resolve the situation, participants needed to adjust the altitude level to exit the windy area and then guide the cursor to the finish line figuring out how to control it. Understanding the rotation of the joystick axes by 90 degrees was essential for successful navigation. In the K2 condition, during the initial segment of the track, the cursor suddenly frozen, and the red light on the right panel began flashing (
Figure 4b). To regain control of the cursor, participants had to follow a specific sequence representing the following steps:
Acknowledge the emergency (press button 2).
Notice that the engine and fuel buttons are off, and consequently, activate the auxiliary ones, starting with the fuel (press button 6).
Lastly, activate the auxiliary engine (press button 4).
After applying the correct sequence, the emergency light ceased flashing and turned green meaning that the emergency had been solved. Subsequently, the cursor could be taken to the finish line without encountering any further command failures. K1 and K2 tasks required the application of the “Identification, Task Selection, and Strategizing” sequence to solve the ongoing conditions.
2.3. Experimental Protocol
The experimental protocol consisted of asking the participants to deal with the different experimental conditions. These six conditions, two for each cognitive control behaviour level, were randomized to mitigate habituation and expectation effects. Prior to the formal experiment, all participants also completed a practice session to mitigate potential learning-related biases. The total duration of the experimental protocol was approximately 30 min.
2.4. Behavioural Data
For evaluating participants’ performance, we recorded the time required to complete each experimental condition. Subsequently, distinct accuracy indicators were computed separately for each cognitive control behaviour level. For each
Skill condition, the participant’s accuracy was compared with the ideal track in terms of length (
L) and error (
err) using the following metrics (Equation (1)):
where
Lopt: length of the ideal track;
L: length of the track completed by the participant;
errMAX: maximum distance of the cursor from the ideal track during the entire run
err: average of these distance values for all runs.
In the
Rule condition, a comparison was made between the number of successfully applied rules and the total number of rules to be applied as follows:
The equation compares the number of successfully applied rules to the total number of Rules for each R1 and R2 condition. The denominator, 14, represents the total number of rules for each condition. The parameters turn and alt can take values of 1 or 0 depending on whether the corresponding rule (to turn or change altitude) has been executed correctly or not, respectively. Additionally, the alt parameter is weighted by a factor w, which accounts for instances where the participant maintains the cursor up or down continuously for consecutive targets.
In the K1 condition, the comparison was made between the ideal time necessary to complete the circuit and the time required by the participant, using the following equation:
The equation compares the ideal time necessary to complete the circuit
Topt with the time spent by the participant to complete the event (increasing or decreasing the altitude) T
ev and the time spent by the participant to complete the whole circuit (
Tcir). The accuracy for the
Knowledge2 condition was evaluated by giving equal weight to two parameters: the number of attempts (
natt) made to reach the correct button sequence, and the percentage of time used (
T) relative to the available time to solve the failure (
Tav set to 20 min), using the following equation:
To establish a level of accuracy relating to the performance of the experimental task, each of these metrics was converted into percentages. This enabled the aggregation of performance across the different experimental conditions. For instance, the average accuracy of Skills was computed by combining the accuracy of S1 and S2, and the same approach was applied to the other conditions (Rule and Knowledge).
2.5. Subjective Data
An ad hoc questionnaire was created to evaluate how participants perceived their performance in terms of cognitive control behaviour. The questionnaire was developed based on Rasmussen’s three levels of cognitive control behaviour, focusing on aspects where participants could provide ratings:
- -
Level of attention;
- -
Task automation level;
- -
Familiarity with the task.
Participants completed this questionnaire at the end of each experimental condition and an average score was estimated for cognitive control level and then compared statistically.
2.6. EEG Data Recording and Analysis
Throughout the experiment, the electroencephalogram (EEG) and electrooculogram (EOG, utilized solely to eliminate eye-related artifacts from EEG data) signals were recorded using a high-resolution (BrainAmp, Brain Products, Gilching, Germany) 61-channel system (Fp
1, Fp
z, Fp
2, AF
7, AF
3, AF
z, AF
4, AF
8, F
7, F
5, F
3, F
1, F
z, F
2, F
4, F
6, F
8, FT
7, FC
5, FC
3, FC
1, FC
z, FC
2, FC
4, FC
6, FT
8, T
7, C
5, C
3, C
1, C
z, C
2, C
4, C
6, T
8, TP
7, CP
5, CP
3, CP
1, CP
z, CP
2, CP
4, CP
6, TP
8, P
7, P
5, P
3, P
1, P
z, P
2, P
4, P
6, P
8, PO
7, PO
3, PO
z, PO
4, PO
8, O
1, O
z, O
2) with a sampling frequency of 250 (Hz). Participants were seated 60 cm away from the monitor (
Figure 5). Before starting the experimental tasks, a one-minute baseline period of data collection was conducted. During this baseline period, participants were instructed to observe the cursor moving toward the practice track without any active participation (REF condition).
All EEG electrodes were referenced to both earlobes and grounded to the left mastoid, with electrode impedances maintained below 10 kΩ. Bipolar electrodes for the EOG were positioned vertically above the left eye. The EEG signal was first band-pass filtered with a 5th-order Butterworth filter in the interval 2–45 (Hz). The eye blink artefacts were detected and corrected online by employing the o-CLEAN method [
11]. For further sources of artefacts, specific algorithms of the EEGLAB toolbox [
12] were applied. Specifically, the pre-processed EEG signal has been divided into 1 s long epochs. A Threshold criterion has been applied to recognize artifactual data automatically. EEG epochs with signal amplitudes exceeding ±80 μV were marked as “artifacts”. In the end, EEG epochs marked as “artifacts” were removed from the EEG dataset with the aim of having a clean EEG signal to perform the analyses. The average number of rejected epochs due to artifacts was very low (0.36 ± 0.92 epochs, corresponding to 0.03 ± 0.08%), indicating high data quality and an effective artifact rejection procedure.
From the artifact-free EEG, the Global Field Power (GFP) was calculated for the EEG frequency bands using a Hanning window of 1 s (which means 1 Hz of frequency resolution). The EEG frequency bands (Delta, Theta, Alpha High, Beta1, Beta2, Beta3, and Gamma) were defined according to the Individual Alpha Frequency (IAF) value [
13] computed for each participant. Since the Alpha peak is mainly prominent during rest conditions, the participants were asked to keep their eyes closed for one minute before starting the experiment. Such a condition was then used to estimate the IAF value specifically for each team member.
The Brodmann reference model [
14] was then used to identify the cortical areas and corresponding cognitive functions significantly activated under the Skill, Rule, and Knowledge conditions. The cortical area analysis was conducted using a 66 (Brodmann areas—BAs) × 6 (frequency bands) × 11 (participants) matrix. The coordinates of the BAs were derived from [
15,
16]. A key step was the resolution of the inverse problem, which involves identifying neural sources from surface EEG signals [
17]. This provides an accurate map of underlying brain cortical activity. The estimation of the current density strength for each dipole was achieved by solving the electromagnetic linear inverse problem [
18,
19]. Specifically, the solution to the linear system
at a specific time
t provides an estimate of the dipole source configuration
x that produces the measured EEG potential distribution
b at that same time. The model also incorporates measurement noise
n, which is assumed to be normally distributed. The matrix
A represents the lead field matrix, with each
j-th column indicating the potential distribution created on the scalp electrodes by the
j-th unit dipole [
20].
Data were normalized using the z-score formula. For each participant, each condition, and each of the 66 BAs, the mean and standard deviation over time were calculated. The z-score was computed by subtracting the mean and dividing by the standard deviation for each second of data, using the REF condition as a reference. This reduced individual differences, allowing for a more accurate comparison of brain activity across participants and experimental conditions. Statistical analyses were performed separately for each EEG band and cortical region, rather than across all bands, regions, and time points, to avoid multiple comparison issues. For each experimental condition and brain area, a vector was created for each participant, and a Kolmogorov–Smirnov normality test was applied. If the data were normally distributed, a Student’s t-test was used, otherwise Kruskal–Wallis test was used to identify brain areas which did differ among the Skill, Rule and Knowledge conditions. This process generated 66 ROIs × 6 EEG-bands matrices for cortical areas, representing neural activity across experimental conditions. In addition, we selected only those cortical areas commonly activated between the two repetitions of each cognitive control behaviour level (S1 + S2 = S; R1 + R2 = R; K1 + K2 = K) to avoid those linked to potential familiarization or learning phenomena and reduce potential false positives.
The cortical data were graphically represented using the sLORETA software (version 2008-November-04) [
21]. EEG cortical data of each participant were estimated and localized in the whole grey matter volume by applying the source localization algorithm standardized LOw-REsolution brain electromagnetic Tomography (sLORETA) [
21] implemented in sLORETA software [
22,
23]. This approach allowed to estimate the current density distribution of cortical and subcortical brain sources from the scalp electric potentials with zero localization error. As a solution of the forward model reproducing the electromagnetic propagation from the active sources to the EEG sensors, we used a lead field matrix obtained from the application of the Boundary Element Method [
24] to the MNI152 realistic head model [
25]. We used a 3D lead field matrix modelling the propagation of 6239 active sources distributed in the whole grey matter at 5 mm spatial resolution towards the EEG sensors. The regularization parameter λ used for sLORETA solution was computed by means of a cross-validation approach [
26]. The solution of the source localization problem for each participant and experimental condition consisted of 6239 waveforms (one for each dipole) used to model the grey matter. Since we wanted to characterize the S, R and K levels considering only pure cognitive functions, the Fp, FC, C, CP, T, and TP EEG channels were excluded from the analyses as already did in previous studies [
20].
The process of channels selection followed the same methodological approach as the selection of ROIs. In both cases, statistical analyses were conducted to identify those brain areas significantly activated with respect to the REF condition. While the selection of ROIs pertains to a specific set of brain regions, the scalp data analysis involves 61 channels. Despite this difference in scale, the procedure remains consistent, ensuring a rigorous and comparable approach.
2.7. Machine Learning-Based SRK Index
The study explored the use of a supervised machine learning model to classify the cognitive control behaviour levels (S, R and K). In particular, given the novelty of this study, there were no documented models in the literature that could be used for this purpose. Some of the authors of this study previously demonstrated the possibility to discriminate the S, R and K levels [
3,
27], but the work was performed in realistic settings, hence with some limitations. On the contrary, this study aimed at characterising the S, R and K levels under controlled settings to avoid biases due to uncontrolled variables.
To identify the most suitable approach for this classification task, we tested several supervised machine learning algorithms, including Decision Tree (DT), Random Forest (RF), K-Nearest Neighbour’s (KNN), and Gradient Boosting (XGBoost). All analyses presented were conducted in Python (version 3.11) using the Scikit-learn library [
28].
The choice of algorithms and hyperparameter ranges was not arbitrary. We adopted the set of models and tuning grids implemented in the Mindtooth tool, which is specifically designed for EEG data analysis. This approach ensured methodological consistency with established practices rather than relying on an exploratory model search. Similar models and parameter grids have also been employed in previous EEG classification studies [
29,
30], further supporting the validity of this choice.
Decision Tree (DT): The Decision Tree algorithm creates a hierarchical structure of conditional nodes to classify observations. Each internal node represents a decision based on feature values, while terminal nodes provide the final classification. This method offers high interpretability, making it valuable for understanding which features are most important in distinguishing between cognitive control levels. However, individual decision trees can be prone to overfitting, especially with limited data (
Table 2).
Random Forest (RF): Random Forest is an ensemble method that combines multiple decision trees to improve classification performance. Each tree is trained on a random subset of features and data, and the final classification is determined by majority voting among all trees. This approach reduces overfitting compared to individual decision trees and provides better generalization, while still maintaining reasonable interpretability through feature importance rankings (
Table 3).
2.8. Random Forest Easy (RF-Easy)
We developed a lighter version of the Random Forest model, in which only the number of estimators and the maximum tree depth were optimized—using the same values as in the previous model. This approach was aimed at reducing computational cost during training.
K-Nearest Neighbors (KNN): KNN is a non-parametric algorithm that classifies observations based on the majority class of their k nearest neighbors in the feature space. It is considered a “lazy learner” because it requires no explicit training phase, simply storing all training instances. The algorithm is particularly suitable for datasets with limited records and high-dimensional feature spaces, as it does not make assumptions about the underlying data distribution (
Table 4).
Gradient Boosting (XGBoost): XGBoost employs an iterative boosting approach where weak learners (typically decision trees) are sequentially added to correct the errors of previous models. The algorithm uses gradient optimization to minimize classification errors at each step while incorporating regularization techniques to prevent overfitting. This method is particularly effective for complex pattern recognition tasks and can handle high-dimensional data well (
Table 5).
Given the large number of features (61 EEG channels × 6 EEG frequency bands) compared to the limited number of records per session (30–256 records), we reduced the dataset size by applying Principal Component Analysis (PCA) [
31]. Since it was not possible to select a fixed number of principal components that would be suitable for each subject, we decided to select a different number of principal components for each session based on the amount of variance explained by the data. The method trained on the dataset (session) was then used to transform both training and testing datasets before running the classification model.
The accuracy of the models was evaluated using a confusion matrix, which provided a basis for calculating key metrics such as overall accuracy, specificity, recall, precision, and the F1-score. Specifically, the analysis considered the effect of overlapping windows of variable sizes, calculating the mode for each window to identify trends in the models’ performance.
The machine learning classifier provides an objective and quantitative tool to distinguish between Skill, Rule and Knowledge levels of cognitive control based on EEG data. While sLORETA representations enable the visualization of cortical activations associated with each cognitive state, they require expert interpretation and do not offer a direct method for classification. In contrast, the supervised classifier translates complex neurophysiological patterns into measurable outcomes, allowing for automatic identification of higher-order cognitive engagement (Knowledge-based behavior). This approach reduces the subjectivity inherent in visual inspection and supports scalability, cross-subject generalization, and the potential for real-time monitoring of cognitive control strategies in applied or operational contexts.
4. Discussion
The subjective measures did not show any significant differences among the S, R, and K conditions. This lack of differentiation may be due to the limited sensitivity of the questionnaire items, participants’ difficulty in introspectively assessing their cognitive strategies, or the relatively short duration of exposure to each condition, which might not have been sufficient to elicit clearly distinguishable subjective experiences.
Interestingly, behavioural data revealed significant performance reductions under knowledge-based conditions (
Figure 6). As the level of cognitive control behaviour increased, moving from Skill to Rule, and finally to Knowledge, the performance decreased as well (
Figure 6).
The first hypothesis (H1) posited that the three cognitive control levels—Skill, Rule, and Knowledge—would be associated with distinct patterns of brain activation, both in terms of the specific brain regions engaged and the frequency bands involved.
The analysis of EEG data revealed an increasing number of brain areas engaged as participants progressed from Skill-based to Knowledge-based behavior. This finding aligns well with Rasmussen’s model (
Figure 1), which posits that Skill-based behavior involves minimal cognitive effort, while Rule- and Knowledge-based levels require progressively greater cognitive engagement and activation of more cortical regions (
Appendix A). This consistency supports a coherent neurophysiological characterization of Skill, Rule, and Knowledge levels in terms of brain areas (BAs) and their related cognitive functions.
Specifically, the Skill-based level is characterized by combined involvement in visual processing, decision-making, and attention. Visual processing engaged the entire beta frequency band in BAs 17, 18, and 19. Previous studies [
33] show that beta activity increases during visual attention, suggesting the visual cortex contributes to both basic processing and higher cognitive functions like attention and awareness. Gamma-band activity also involves these areas during Skill-based behavior. According to [
34], Gamma synchronization in the visual cortex is linked to attentional stimulus selection. BA 37, corresponding to the fusiform gyrus, was similarly active in the Gamma band during Skill behavior, reflecting enhanced neural responses to visual stimuli driven by attentional mechanisms [
35]. The fusiform gyrus also participates in the beta band, supporting advanced visual processing such as object identification [
36].
In terms of attention, a high alpha band desynchronization was found in BA 8 (frontal eye field, FEF). The dorsal frontoparietal network, including FEF and intraparietal sulcus (IPS), manages spatial attention distribution, partly through alpha rhythm desynchronization that prepares the brain for visual targets [
37]. Decision-making functions involved BA 23 and 24 in the high alpha band. Recent computational models in social neuroscience have highlighted these areas’ roles beyond traditional ‘social brain’ regions, particularly in acquiring self-knowledge during interactions [
38]. Memory retrieval, critical for Skill-based behavior through self-learning, was associated with beta oscillations in the ACC (BA 23-24), which predict reward-related bias and optimize automatic responses [
36]. BA 30 in the cingulate gyrus was active in beta and Beta 3 bands, while BA 31 showed high alpha desynchronization linked to episodic memory retrieval [
39]. Gamma-band activation in BA 23 (ventral posterior cingulate cortex) and BA 30 (angular gyrus) also supports the role of memory retrieval in Skill-based behavior [
40]. Coordination between BA 8 and 24 (ACC), reflected by Theta and beta synchronization, further suggests ACC’s key role in cognitive control during tasks involving saccades [
41].
Rule-based behavior engages cognitive functions including decision making, memory retrieval, complex processing, and visual attention. Like Skill behavior, beta activity spans BAs 17, 18, and 19 in the visual cortex [
33], with Gamma-band activity also present. Theta band activation in BAs 17 and 18 aligns with saccadic rhythms observed during natural scene observation [
42]. High-frequency activation of BA 37 again underscores fusiform gyrus involvement, enhancing stimulus-driven Gamma oscillations and visual attention [
35]. BA 21 in the Gamma band supports motion perception within the visual field [
43]. Decision-making-related Theta activation occurs in BAs 23 and 24, with additional involvement of BAs 30, 31, and 32, reflecting increased planning time and cognitive demands in Rule and Knowledge tasks [
44]. Beta and high alpha bands also implicate ACC-related regions (BAs 23-24, 32) in reward monitoring and adaptive behavior [
36,
38]. Memory-related activations in BAs 23, 30, and 31 reinforce the importance of memory retrieval in applying learned Rules [
39,
40]. Beta bands engage BA 36 (perirhinal cortex), critical for recognition memory [
45], and BA 39, also involved in complex cognitive scenarios [
46]. Gamma band involvement of BA 32 highlights its role in target detection, response inhibition, and performance monitoring [
47]. Synchronization between BA 8 (FEF) and 24 (ACC) in the Theta band, noted during Skill tasks, similarly supports cognitive control during Rule-based behaviors [
41].
Knowledge-based behavior encompasses a broader set of cognitive functions including visual processing, decision making, memory retrieval, problem-solving, and complex cognitive operations. Theta band activation in BAs 17, 18, and 19 supports saccadic visual attention [
42], and Gamma activity within these areas emphasizes attentional stimulus selection [
34]. BA 21, involved in motion perception, participates in both Rule and Knowledge behavior [
43]. The beta band consistently engages visual areas (BAs 17–19) across all behavior types, underlying visual attention’s central role [
33]. Fusiform gyrus (BA 37) activation in Gamma band remains consistent across Skill, Rule, and Knowledge behaviors, facilitating advanced visual processing and attentional modulation [
35]. BA 39 (Wernicke’s area) also shows activity in Beta 3, Theta, and Gamma bands, reflecting language comprehension and speech production involvement during tasks [
48]. Additional areas unique to Knowledge behavior include BA 41 (auditory cortex, Beta 3) and BA 44 (Broca’s area, Beta 3), as well as BA 47 (pars orbitalis) in the alpha band, related to social communication and facial expression. Decision-making in Knowledge behavior specifically involves BA 13 (insula) in the Theta band, a region linked to emotional processing, risk assessment, self-awareness, and complex social functions like empathy [
49]. BA 21’s Theta activation highlights its role within the Default Mode Network (DMN) for self-referential thought and memory retrieval during complex decision-making [
50]. Consistent involvement of cingulate regions (BAs 23, 24, 30, 31, 32) in Theta and Gamma bands reflects their role in plan elaboration and cognitive control under higher cognitive demands [
44]. BA 36 again contributes to recognition memory and contextual evaluation critical for Knowledge behavior [
45]. High alpha activity in BA 24 supports social cue processing relevant for informed decision-making [
38]. Problem-solving engages BA 10 (anterior prefrontal cortex) in Beta and Gamma bands, a region essential for abstract thinking and strategic planning. DLPFC areas (BAs 9 and 46) activate in alpha bands, mediating cognitive control and emotional regulation during complex tasks [
51]. Finally, Theta synchronization between BA 8 (FEF) and 24 (ACC) persists across all behavior types, reinforcing its importance for cognitive coordination during saccadic tasks [
41].
The observed patterns of activation across multiple BAs suggest coordinated engagement of large-scale functional networks rather than isolated regional activity. Interactions between frontal, cingulate, and parietal areas are consistent with the recruitment of the Frontoparietal Control Network, while activity in BA 21, BA 39, and cingulate regions aligns with the Default Mode Network. These findings indicate that Skill, Rule, and Knowledge-based behaviors involve not only specific cortical regions but also the dynamic interplay of distributed networks, supporting the view that cognitive control emerges from large-scale functional connectivity rather than single-region activations [
52].
The second hypothesis (H2) stated that a supervised classification model would be able to significantly distinguish between the different levels of cognitive control behavior.
Among all the analysed conditions, the one that produced the best results was the binary condition. In this condition, the Skill and Rule behaviours were merged and compared to the Knowledge level. This approach yielded the most promising results (
Figure 7), as the tasks performed by participants in the Skill and Rule conditions were highly similar, both from a practical perspective and according to the existing literature, which identifies these two conditions as the most comparable.
The accuracy trend of the KNN model was plotted against the varying time window (
Figure 8). The time window represents the duration used to collect data, and the goal was to identify a configuration capable of achieving good results even with reduced windows, indicating greater efficiency. According to the literature [
53,
54], an accuracy value is considered good when it exceeds 0.70. This threshold was surpassed after 7 s, and a positive trend in accuracy was observed over time. The results (
Figure 9,
Table 7) show that 7 out of 11 participants surpassed the accuracy threshold.
Table 8 underscores the robustness of the KNN model for S-R-K classification using a window size of 35 sec. The per-subject performance metrics for the binary classification (Knowledge vs. Skill + Rule), including Precision, Recall, Specificity, F1-score, and Accuracy for each class, reported as mean ± 95% confidence interval across cross-validation folds, as well as confusion matrix entries (TN, FP, FN, TP), can be found in
Table A1 in
Appendix A.
Despite the innovative results, it is important to note some limitations of the study. Firstly, the sample size of the experimental group was relatively small. The next step will aim therefore at enlarging the number of participants. Secondly, the characterization and measure of Skill, Rule and Knowledge levels were performed after the execution of tasks, not while dealing with tasks. Thirdly, the machine learning model used, despite its simplicity, demonstrated a moderate level of accuracy in class classification, even in the presence of class imbalance (
Table 8, average F1-score = 0.72). However, it is important to acknowledge that the imbalance among classes S, R, and K—especially for class K—may have affected the model’s reliability and should be carefully addressed in future research. Additionally, the results indicate that for some participants, the model did not reach an adequate level of accuracy. This may be due to challenges in feature selection and, more broadly, to the difficulty of designing a model that effectively utilizes a limited set of features given the dataset’s characteristics. Since the length of the runs cannot be determined a priori—being dependent on the time each participant takes to complete the task—it was not possible to predefine a representative set of features for the phenomenon. Future studies could explore two possible directions: (1) analyzing the dataset collected from existing subjects to identify common feature patterns across participants and (2) implementing an ensemble classification model trained on feature subsets, with the final prediction obtained by combining their outputs.
Based on our results, we may assume a similarity between the SRK model and the Kahneman’s Systems 1 and 2 model [
55]. In fact, we can assume that Skill and Rule in the SRK model can be associated with System 1, which is fast, automatic, and intuitive. These behaviours are in fact quick and require minimal effort, relying on learned patterns and predefined rules. They enable efficient task handling without deep reflection. On the other hand, Knowledge can be associated with System 2, which is slower, reflective, and rational. It requires a deeper understanding, analysis, and problem-solving, involving conscious, deliberate thinking for more complex tasks [
56].