Next Article in Journal
A New Approach to Assess Quality of Motion in Functional Task of Upper Limb in Duchenne Muscular Dystrophy
Previous Article in Journal
Green Enzymatic Recovery of Functional Bioactive Compounds from Unsold Vegetables: Storability and Potential Health Benefits
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 12
Issue 23
10.3390/app122312248
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Method Detecting Student’s Flow Construct during School Tests through Electroencephalograms (EEGs): Factors of Cognitive Load, Self-Efficacy, Difficulty, and Performance

by
Shu-Fen Wu
1,
Chieh-Hsin Kao
2,
Yu-Ling Lu
1,* and
Chi-Jui Lien
1
1
Department of Science Education, National Taipei University of Education, Taipei City 10671, Taiwan
2
School of Medicine, Chung Shan Medical University, Taichung City 40201, Taiwan
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(23), 12248; https://doi.org/10.3390/app122312248
Submission received: 28 October 2022 / Revised: 20 November 2022 / Accepted: 24 November 2022 / Published: 30 November 2022
(This article belongs to the Section Applied Biosciences and Bioengineering)
Browse Figures
Versions Notes

Abstract

:
This study gathers and examines information about the flow state’s emergence during tests and its factors using an electroencephalogram (EEG) to establish a method and reveal an individual student’s flow construct. Through a single-case experimental design and 766 test items, multiple measurements were performed on a 14-year-old junior high school science-gifted student. During the test, self-efficacy, item difficulty, cognitive load, and test performance (long-term test performance [LT-tp] and short-term test performance [ST-tp]) were examined to establish the construct of EEG-detected, real-time flow states (EEG-Fs). Based on the chi-square test of independence results, the EEG-F had a significant correlation with the student’s cognitive load, self-efficacy, LT-tp, and item difficulty. Furthermore, a J48 decision tree analysis and logistic regression revealed four inhibiting and two inducing conditions affecting the emergence of EEG-Fs. The two inducing conditions included (1) high self-efficacy with a low cognitive load (odds ratio (OR) = 3.7) and (2) high cognitive load when combined with high self-efficacy and LT-tp for low-difficulty items (OR = 3.5). The established method and findings may help teaching designers or automated teaching applications detect the individual student’s flow construct to select appropriate test tasks accordingly, resulting in an optimal experience and better achievements.

1. Introduction

Educational psychologists have regarded the flow state as a self-reward mental state that stimulates high immersion and attention during an activity, thereby stretching the mind’s limits [1]). After decades worth of studies, these flow experiences have been associated with various factors, such as students’ cognitive load [2], self-efficacy [3,4,5,6], performance [7,8], and task difficulty [9]. However, the method for measuring an individual student’s flow state and its construct involving these factors has yet to be sufficiently explored.
A major obstacle restricting additional research on the flow construct and its related factors is the absence of a measuring technique that instantly detects the real-time status of the dynamic and interactive flow state and its related factors [2,10]. Given the emerging development and applications of electroencephalograms (EEGs) in education and cognitive sciences [11,12,13,14], the authors established a method to measure a group of students’ flow construct [15,16] and found it to be applicable and robust. In previous studies, however, the flow construct was derived from the student group’s data, an “idiographic” student’s flow construct and its related factors, which are crucial and urgently needed for individual diagnosis and can thus be used to guide adequate individualized intervention for sustaining an idiographic student’s flow state, which has never been studied before. Moreover, previous studies [15,16] focused primarily on self-efficacy, long-term and short-term test performance (LT-tp and ST-tp, respectively), and item difficulty instead of the factor of cognitive load [17], which significantly affects an individual student’s ability to reach the flow state. As such, this exploratory study intended to adopt a continuous detection method that uses EEGs to examine an individual student’s flow states [15,16] and cognitive load [17,18,19] and explores interactions between the flow state and its associated factors, namely, cognitive load, self-efficacy, LT-tp, ST-tp, and item difficulty. This exploration can establish the construct of an individual student’s EEG-detected, real-time flow states (EEG-Fs) concerning these particular factors, addressing this research area’s gaps. In contrast to previous experimental research group approaches, this study adopted a single-case experimental design research approach [20] to develop a method that uses EEGs to analyze an idiographic student’s flow construct during school tests and answer the following research questions (RQs).
RQ1: Based on the detecting method, do cognitive load, self-efficacy, LT-tp, ST-tp, and item difficulty significantly affect a student’s EEG-Fs?
RQ2: Based on the detecting method, how and to what extent do cognitive load, self-efficacy, LT-tp, ST-tp, and item difficulty contribute to the construct of a student’s EEG-Fs?

2. Theoretical Background

2.1. Self-Efficacy and Test Performance Related to Flow State

A flow experience, also called an “optimal experience” [1], refers to the highest state of personal engaging actions resulting in a high probability of increased performance [21]. Previous flow studies indicated that a learner’s academic (test) performance in a learning environment is closely related to their sense of control (SC) [8].
Furthermore, a higher capacity for a particular activity results in greater likelihoods of good performance [7] and inducing the flow state [15,16]. Accordingly, highly competent students tend to achieve flow states when working on their favorite subjects [22]. However, even a high-achieving student may encounter temporary difficulties during learning [23].
Flow experiences are also closely related to factors such as a person’s intrinsic motivation and perception [6]. According to a study [24], individuals’ motivations and achievements have been significantly influenced by self-efficacy, defined as a person’s belief in his or her ability to succeed in specific situations or accomplish tasks. In education, this factor has been shown to affect students’ choices of activities, efforts expended, persistence, interest, and achievement [25]. Students with high self-efficacy report flow experiences associated with learning activities more often [4], consequently impacting their performance [26]. In line with this, a study [3] revealed that self-efficacy has direct positive effects on flow and learning engagement. Learners with high academic self-efficacy display more positive attitudes toward learning subjects [5].
In relation to EEG signal analysis, it was claimed that such a method effectively monitors new learners’ patterns for academic self-efficacy evaluation [27]. However, this monitoring method has yet to be used in examining self-efficacy’s direct effect on EEG-Fs.

2.2. EEG Measurement of Cognitive Load and Flow State

EEGs have been commonly used in learning technology and educational studies [11,12,14]. In particular, several research studies, such as [28], have attempted to measure cognitive load. The EEG measurement of cognitive load may be conducted using working memory [29]. In one study [19], (θ + α)/(10 × low_γ) was found to be an effective index for measuring the extent of cognitive load through portable EEG devices. In relation to task difficulty, EEG signal analyses reveal a high degree of discrimination between the dynamic changes in mental workload [18,30,31,32]. The main effect of task difficulty on EEG-measured cognitive load estimates was correlated with learning performance or intelligence [33].
For the flow state, two studies [15,16] also used an EEG for real-time flow measurement in students and the determination of its indicators, namely, high levels of attention and engagement. In addition to these indicators, the said study also observed the positive effect of several factors, such as the balance between challenges and skills (BCS), overall test performance (i.e., the present study’s LT-tp), and momentary test performance (i.e., the present study’s ST-tp), on flow experiences. Although cognitive load was not considered in the abovementioned study, other works suggested a possible connection between the flow state and cognitive load. One study [34] indicated that the flow state could alleviate negative experiences, while another study [35] stated that the reduction of cognitive load could promote engagement, possibly leading to the flow state. In line with this, a study [2] revealed that cognitive load negatively impacts flow. However, the EEG measurement of cognitive load in relation to the flow state and its implications remains unexplored, hence the present study’s focus.

3. Materials and Methods

This research was designed based on the context of the questions studied, considering the feasibility of student recruitment, equipment, and environments. Such considerations led the researchers to employ a within-subject research design, in which the same subject tests all the conditions to minimize random outcomes. It was adopted to eliminate possible errors resulting from differences in the subjects, equipment measurements, and so on [36]. With this consideration, this study adopted a single-case experimental design research approach [20]. It took advantage of the carefully selected subject who closely met the set criteria for the controlled variables, avoiding individual differences that are recurrent and overlooked in a more extensive class sampling. Moreover, this carefully selected case allowed the researchers to administer multiple and longer tests to collect sufficient data, which could prove difficult and problematic in a larger group experiment design [37,38]. It is not uncommon for brain wave studies to adopt a case approach while still acquiring useful findings [39,40,41]. Further details are explained below.

3.1. Participant

This study adopted a single-case experimental design research approach [20]. Only one student participated in this experiment to eliminate problems associated with group studies. In particular, a gifted student was selected to ensure that her self-efficacy and performance in different disciplines would correspond to each other. The participant is a 14-year-old junior high school student recognized as gifted in science following the Special Education Act promulgated by Taiwan’s Ministry of Education. In the identification of such students, the Special Education Students Diagnosis and Placement Counseling Committee (DPCC) uses standardized assessment tools to confirm that the participant’s score in the mathematical and scientific academic aptitude or achievement tests is positive by two standard deviations or more than the percentile rank (PR) of 97 compared to those of same-aged students, proving her academic excellence in mathematics and science. Furthermore, in the Comprehensive Assessment Program (CAP) for junior high school students, which evaluates students’ academic competencies [42], she demonstrated her proficiency in the science subject (PR 99), but her performance in the nonscience subject (PR 96) was not as high. The difference in her performance was also observed in her semestral grades. For example, she performed better in the natural science subject than in the social science subject. When she enrolled in high school later, she was invited to enter the mathematics and science gifted class to support her scientific capacity through her school’s mathematics and science giftedness assessment.
This study was vetted by the Institutional Review Board (IRB), and the participant, with her guardians’ consent, signed understanding and awareness agreements for this study’s execution. Considering the complexity of equipment and experimental environments, the researchers conducted multiple measurements to garner sufficient EEG data, and 766 test items were given.

3.2. Experimental Procedure

The effect of cognitive load and self-efficacy, the study’s primary focus, on the participant’s flow states was examined through different achievement tests in a classroom setting. Given the participant’s designation as gifted, she made use of her proficiency in science in answering the assessments, and this was considered in the observed conditions, particularly science and nonscience items, as well as difficult and easy items. The flow experience was represented by the EEG data collected through a portable brain wave device, which used an algorithm to convert electronic information showing the status of the participant’s attention, engagement, and cognitive load. Further details regarding the use of an EEG will be explained in the following EEG-detected, real-time flow states section.

3.3. Instruments, Groupings, and Data Collection

3.3.1. Self-Efficacy

The New Multiple Aptitude Test (NMAT), which has split-half reliability of 0.81–0.96 [43], measured the participant’s self-efficacy or self-reported skill sense in science and nonscience subjects. The measurements showed that the participant’s self-efficacy levels for the two subjects were noticeably different. In the science subject, the participant demonstrated a high self-efficacy (PR 97), while in the nonscience subject, the participant had a relatively low self-efficacy (PR 70).

3.3.2. Item Difficulty

The subject achievement tests in this study included 23 test sheets and 766 items, with 276 and 490 belonging to science and nonscience subjects, respectively. Each item had a different difficulty level, similar to those for CAP. Afterward, the classical test theory was used to count the number of examinees that responded correctly, and the number of all examinees (N = 30) was divided for the difficulty index of the items. In particular, the lower the difficulty index, the higher the item difficulty. Then, the average difficulty index was used to classify all 766 test items into high- and low-difficulty groups.

3.3.3. Test Performance

In line with previous studies [8,15,16], this study examined the participant’s test performance and used that as an indicator of her SC when accomplishing a task. Two types of test performances were recorded in this study. After the test, her answer for each item was recorded as correct or incorrect and labeled as her ST-tp (i.e., the student’s short-term SC). In contrast, the score for each test, which usually consists of 30–40 items, was designated as the LT-tp (i.e., the student’s long-term SC). Then, her performance on each test was classified as either high or low LT-tp based on the average passing score for all 766 items from the 23 sheets.

3.3.4. Cognitive Load

This study adopted the use of the portable brain wave device with a working memory index represented by (θ + α)/(10 × low_γ) [19]. The data were then normalized to a T-score as cognitive load. Afterward, the average T-score was used to divide all 766 items into high- and low-cognitive load groups.

3.3.5. EEG-Detected, Real-Time Flow States (EEG-Fs)

During each test, the participant wore a NeuroSky headset with the single dry sensor (Fp1) that had been used in other studies [14,44], as seen in Figure 1. However, through eSense, the apparatus measured attention but did not provide a ready metric for engagement [14,45]. As such, the present study used [15,16] flow measurement technique, which combined the directly measured attention index and engagement index calculated from the α, β, and θ wave readings [46,47]. The combination of high attention and high engagement was defined as “high flow experiences by EEG” (high EEG-Fs), and the remaining combinations were “low flow experiences by EEG” (low EEG-Fs). From 10,240,000 EEG measurements (the EEG was sampled 512 times/s × 20,000 s), the student’s brain waves data (the α, β, and θ wave readings) and the attention indexes were grouped by the student’s time stamp for each test item. This yielded in 766 records representing the brain waves and attention states as the student responded to each test item.

3.4. Data Analyses

Given that the outcome variables, flow state, and several key predictor variables (self-efficacy and ST-tp) were ordinal numbers, the study transformed all other numeric variables (cognitive load and LT-tp) into ordinal ones for better examination.
For RQ1, the chi-square (χ2) test of independence for association was used to examine the relationships between this student’s flow states and other predictor variables (i.e., difficulty, ST-tp, LT-tp, self-efficacy, and cognitive load). In the test, smaller p-values indicated that the variable was likely correlated with the difference in distribution. Conversely, for RQ2, only those predictor variables shown to be significant in chi-square analyses were combined with EEG-F, then input into the J48 decision tree analysis. Logistic regression, which may provide odds ratios (ORs), were subsequently used. In particular, the OR is the ratio of the odds of an event occurring in one group to the odds of it occurring in another. Afterward, the decision tree’s results were evaluated using 10-fold cross-validation, and measures (i.e., true positive (TP) rate, false positive (FP) rate, precision, recall, F-measure, Matthews correlation coefficient, receiver operating characteristic (ROC) area, and the area under the precision-recall curve (PRC)) were reported [48]. For the results analysis, Statistical Package for the Social Sciences (SPSS) Statistics 20.0, Microsoft Excel statistics software, and Waikato Environment for Knowledge Analysis (WEKA) ver. 3.8 were used.

4. Results

4.1. Flow State Factors

To answer RQ1, the frequency and percentage for EEG-F by predictor variable (i.e., difficulty, LT-tp, ST-tp, self-efficacy, and cognitive load) were listed in the contingency table (Table 1). Among 766 test items, 423 (55.2%) were identified as low–EEG-F items, and 343 (44.8%) were high–EEG-F items. While answering the high–EEG-F items, the EEG detected the participant’s high attention and engagement levels. Although the participant’s ST-tp did not have a significant correlation with the flow state (χ2 [1, N = 766] = 1.654, p = 0.198 > 0.05), as revealed by the chi-square test of independence for association, there were significant correlations between the participant’s flow state and item difficulty (χ2 [1, N = 766] = 10.965, p = 0.001 < 0.05), LT-tp (χ2 [1, N = 766] = 13.566, p < 0.001), self-efficacy (χ2 [1, N = 766] = 70.334, p < 0.001), and cognitive load (χ2 [1, N = 766] = 102.999, p < 0.001).
As seen in the chi-square test of independence, item difficulty, LT-tp, self-efficacy, and cognitive load had a significant correlation with EEG-F. Therefore, such significant factors can be included in establishing the construct of the student’s flow states.

4.2. Flow State Construct: J48 Decision Tree and Logistic Regression Analysis

4.2.1. J48 Decision Tree

To answer RQ2, Weka’s classifiers.trees.J48 was used to data-mine the relationship between variables. As the results of chi-square analyses indicated that ST-tp was not significant, only item difficulty, LT-tp, self-efficacy, and cognitive load were input into the classifiers with the outcome variable (EEG-F). The classifier mode for EEG-F (high vs. low EEG-F) was presented, and the cognitive load serves as the parent node (Node 1), as shown in Figure 2.
In Node 1, the sample is divided into high (Node 2) and low cognitive load (Node 3), which then classified the sample according to the levels of self-efficacy: high and low self-efficacy. In particular, Leaf 1 showed that when this student had a high cognitive load and low self-efficacy, a low EEG-F was induced (total instances: 167.0; incorrect instances: 10.0). In contrast, Leaf 2 showed that a high EEG-F emerged when the student had a low cognitive load and high self-efficacy (176.0/40.0). These relationships demonstrated that the interaction between the participant’s cognitive load and self-efficacy influenced her flow experience.
For Node 4 (high cognitive load and high self-efficacy) and Node 5 (low cognitive load and low self-efficacy), the sample was divided into high and low LT-tp. In particular, Leaf 3 showed that a student with a high cognitive load, high self-efficacy, and low LT-tp induced a low EEG-F (40.0/7.0). Similarly, Leaf 4 showed that a student with a low cognitive load, low self-efficacy, and high LT-tp had a low EEG-F (136.0/56.0). Ultimately, these findings showed that the effect of a student’s LT-tp on flow experiences might vary.
Ultimately, for Node 6 (high cognitive load and high self-efficacy with high LT-tp) and Node 7 (low cognitive load and low self-efficacy with low LT-tp), the sample was divided into high and low difficulty. Leaf 5 showed that when the student had a high cognitive load and high self-efficacy with high LT-tp for low difficulty, she had a high EEG-F (40.0/12.0). In contrast, Leaf 6 showed that when she had a high cognitive load and high self-efficacy with high LT-tp for high difficulty, she had a low EEG-F (20.0/8.0). Meanwhile, Leaf 7 suggested that the student’s low cognitive load and low self-efficacy with low LT-tp for low difficulty induced a high EEG-F (99.0/42.0), but Leaf 8 showed that her low cognitive load and low self-efficacy with low LT-tp for high difficulty resulted in a low EEG-F (88.0/41.0). This demonstrated that item difficulty affected a student’s flow experiences. In particular, a student working on a low-difficulty item was more likely to induce a high EEG-F.
Through 10-fold cross-validation, the classification and classifiers were evaluated to have adequate validity, and the model was deemed acceptable (TP rate: 0.704; FP rate: 0.306; precision: 0.703; recall: 0.704; F-measure: 0.703; Matthews correlation coefficient: 0.399; ROC area: 0.770; PRC area: 0.744).

4.2.2. Logistic Regression

In line with answering RQ2, the study utilized logistic regression for each of the seven nodes to obtain the OR of contrasting conditions represented in the leaves, which helped identify each factor’s contribution. Table 2 displays the results of the logistic regression analysis. In particular, the factor in Node 1 (cognitive load) illustrates that the OR of the low-load group vs. the high-load group was 5.6, showing a significant difference (p < 0.001). This suggests that the low-load group was more likely to reach a high EEG-F than the high-load group, with a 5.6 OR. Among all the nodes, only the factor in Node 7 (difficulty), with the conditions of a low cognitive load and low self-efficacy with low LT-tp, appeared insignificant (p = 0.133), leading to its exclusion from the final construct. In the following section, Figure 3 presents the final construct of the flow state.

5. Discussion

This study developed a method for exploring the EEGs’ measured flow construct and factors of an idiographic student’s cognitive load, self-efficacy, test performance, and test difficulty, and answered the two RQs. For RQ1, results showed that an EEG analysis revealed a significant correlation between the student’s flow states with cognitive load, self-efficacy, test performance, and difficulty factors. These findings are in line with the flow theory [1] and other studies that reported the influence of cognitive load [2], self-efficacy [3,4,5,6], performance [7,8], and difficulty [9] on the flow state and the possible prediction of such a state through the said factors.
For RQ2, the study synthesized the findings from the J48 decision tree classification and ORs from the logistic regression analysis (Figure 3), which presented the construct of this student’s flow states during tests through a flow construct the size of 13 with 6 nodes, 7 leaves, and 12 branches. This construct was supported by two different statistical analyses.
In Level 1, the student’s flow experience is first decided by the cognitive load (Level 1, Node 1), and lower cognitive loads often induced a high EEG-F with a 5.6 OR compared to the 1.0 OR of high cognitive load, supporting the claim that cognitive load heavily influences flow experiences [2]. Next, Level 2 indicated the commencement of self-efficacy (Nodes 2 and 3). The OR reached 11.8 when this student had a high cognitive load with a high self-efficacy (Node 2), while the OR reached 3.7 when this student had a low cognitive load and a high self-efficacy (Node 3). Given these observations, those with high self-efficacy are more likely to reach the flow state [3,4,5,6]. In addition, comparing the OR of Nodes 2 and 3, it was found that a high self-efficacy was more conducive to the emergence of a high EEG-F in a high cognitive load (OR = 11.8) than in a low cognitive load (OR = 3.8). This reminds us that when a student faces more complex test items, it is necessary to provide guidance to build her skills and self-confidence.
After cognitive load and self-efficacy, Levels 3 and 4 corresponded to the factors of test performance and item difficulty, respectively. In particular, Level 3 indicated that the student’s flow experience was decided by test performance in certain situations (Nodes 4 and 5). When the student had both a high cognitive load and high self-efficacy (Node 4), the high LT-tp group was more likely to reach a high EEG-F with a 7.1 OR than the low LT-tp group. This result showed that a high sense of long-term control with both high cognitive load and high self-efficacy was more powerful for high EEG-F. However, when the student had both low cognitive load and low self-efficacy (Node 5), the high LT-tp group was not more likely to reach a high EEG-F, with only a 0.6 OR. This result showed that cognitive load and self-efficacy are more powerful for EEG-F than SC, especially low EEG-F caused by low self-efficacy.
Lastly, Level 4 indicated that the student’s flow experience also depended on item difficulty (Node 6). The OR reached 3.5 when the participant faced low-difficulty items while having high LT-tp with high self-efficacy and high cognitive load (Node 6). However, the difficulty omnibus was not significant when the participant had low LT-tp with low self-efficacy and low cognitive load in the decision tree, as seen in Figure 2 (Node 7), so it was not included in Figure 3.
Ultimately, two patterns for the emergence of higher EEG-Fs (Leaf 2 and 5) and four patterns that may have had the emergence of lower EEG-Fs (Leaf 1, 3, 6, 4) were identified. First, as in Leaf 2 (in Figure 3), a high EEG-F was frequently detected when the student had a high self-efficacy and low cognitive load, as observed in a study’s findings [35]. Second, as in Leaf 5 (in Figure 3), when the student had a high cognitive load, and it was combined with a high self-efficacy and high performance, a high flow state may have emerged when facing low-difficulty items, in line with a study’s results [34]. Similar to previous research [8,15,16], this study used the participant’s test performances to correspond to her SC in the flow state. Notably, the previous study [15] identified both LT-tp and ST-tp as factors affecting the students’ flow construct, supporting previous studies that claimed that SC is essential to flow experiences [1,7,8]. In addition, one study’s final construct of flow during tests in a technological interactive learning environment included ST-tp [16]. However, in the present study’s chi-square analysis (Table 1), ST-tp did not have a significant correlation with the participant’s flow state, and the final construct of flow in tests did not include ST-tp. As such, this result was not entirely in line with previous studies. This may be attributed to the fact that the participant did not receive immediate feedback after answering each question, rendering the effect of the ST-tp insignificant. In addition, there are too few cases of incorrect answers (only 67, as seen in Table 1) in this study, so the short-term SC’s effect requires further exploration.
In addition, following flow theory [1], self-efficacy was used to indicate the participant’s self-reported skill sense, and item difficulty signified the challenges the participant faced. Leaf 5’s result showed that a student with a high skill sense may reach a high EEG-F when facing low challenges under a high cognitive load. Notably, previous studies [15,16] identified that when high-skill students were matched with a high-challenge task, BCS positively affected students’ flow experiences measured with an EEG. This was consistent with the claim that BCS is a key to flow experiences [1]. In contrast with previous studies, the present study’s results were more subtle. Leaf 5’s result may have been caused by low-difficulty items reducing the stress caused by a high cognitive load, allowing the participant to concentrate and engage in tests while reducing threats to her performance. Eventually, this led the participant into a flow state. Furthermore, even though the idiographic student had both low cognitive load and a high sense of long-term control, low self-efficacy significantly impacted the emergence of lower EEG-Fs (Leaf 4). This reflects the findings that people with low perceived abilities experienced a low flow level [6].
This study had theoretical and practical values. Taking this ideographic student’s flow construct (Table 1 and Figure 3) as an example, the four factors can be categorized into two dimensions (student-related and test-related), with which strategies for improving a student’s flow experience in learning can be implemented and are explained as the following:
  • To student-related factors (self-efficacy and performance): the results indicated that enhancing this student’s self-efficacy and learning performance also improved her flow experience. Thus, teachers may adopt strategies, such as encouraging positive metacognition (e.g., self-regulation), motivation (e.g., the value of learning, belief in her capacity, etc.), and behavior (e.g., attending and concentrating, using effective learning strategies) in her learning [9,49].
  • To test-related factors (cognitive load and difficulty): although the results showed that this student tended to get a higher flow in lower cognitive load and difficulty items (OR = 5.6 and 3.5, respectively; Figure 3), this should not be interpreted as a reason to give her only “easy” tests to induce the flow experience. Instead, learning theories have determined that learning in a student’s zone of proximal development can induce meaningful learning, and educators must arrange tests (challenges) that are of optimal loading and difficulty so that the learning progression always provides enough learning challenges while still allowing the student to maintain sufficient flow and motivation.

6. Conclusions

Flow experiences affect learning sustainability and give students the ability and motivation to perform their tasks, resulting in performance improvements [50,51]. This research established a method to explore an idiographic student’s flow construct. The study findings, which answered the two RQs, revealed that the factors (cognitive load, self-efficacy, difficulty, and performance) are largely consistent with previous flow studies [3,8,15,16,22], suggesting the reliability of the proposed method, its findings, and application values. Teachers and instructional system designers may use the present study’s findings as a basis for their future efforts to enhance students’ flow states for student-centered learning which may lead them to an optimal experience and attain better achievements. While this study yielded significant findings, its limitations must be addressed. As the study was exploratory and had limited samples, the results must not be overinterpreted. Future studies may address this by increasing the sample size and refining the research design. Moreover, although this study adopted a within-subject analysis that avoided possible errors related to varying individual skills, further research on the topic using a larger sample size with experimental control can elaborate on this aspect. Moreover, as the objective of this study is mainly on establishing a student’s flow construct, this study did not split the data into training, and test data sets. In future studies, when a larger sample is available, further evaluation with splitting data sets should be conducted. It is also worth noting that future studies may further explore the applicability of other devices (such as eye movement tracking devices) and the comparative reliability and validity of human–computer interaction systems and conventional measurement methods, which will continue to enrich our understanding of real-time detection regarding human’s flow state.

Author Contributions

Conceptualization, S.-F.W., Y.-L.L. and C.-J.L.; methodology, S.-F.W., Y.-L.L. and C.-J.L.; software, S.-F.W.; formal analysis, S.-F.W.; investigation, S.-F.W. and C.-H.K.; resources, S.-F.W., C.-H.K. and Y.-L.L.; data curation, S.-F.W., C.-H.K. and C.-J.L.; writing—original draft preparation, S.-F.W. and C.-H.K.; writing—review and editing, Y.-L.L. and C.-J.L.; visualization, S.-F.W. and C.-H.K.; supervision, Y.-L.L.; project administration, Y.-L.L.; funding acquisition, Y.-L.L. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Ministry of Science and Technology, Taiwan (MOST 106-2511-S-152-005-MY2 and MOST 108-2511-H-152-013-MY3).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of the University of Taipei (protocol code IRB-2017-011 and date of approval 5 October 2017).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

All the data generated or analyzed during this study are included in this published article.

Acknowledgments

We would like to express our gratitude to the study’s student participant, as well as the staff and teachers, for kindly supporting our work.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Csikszentmihalyi, M. Flow: The Psychology of Optimal Performance; Cambridge University Press: Cambridge, UK, 1990. [Google Scholar]
  2. Chang, C.-C.; Warden, C.A.; Liang, C.; Lin, G.-Y. Effects of digital game-based learning on achievement, flow and overall cognitive load. Australas. J. Educ. Technol. 2018, 34. [Google Scholar] [CrossRef] [Green Version]
  3. Adil, A.; Ameer, S.; Ghayas, S. Mediating role of flow and study engagement between academic psychological capital and perceived academic stress among university students. Pak. J. Soc. Clin. Psychol. 2019, 17, 12–18. Available online: https://www.researchgate.net/publication/341913295 (accessed on 27 October 2022).
  4. Bassi, M.; Steca, P.; Fave, A.D.; Caprara, G.V. Academic self-efficacy beliefs and quality of experience in learning. J. Youth Adolesc. 2007, 36, 301–312. [Google Scholar] [CrossRef] [PubMed]
  5. Chen, M.J.; Lee, C.Y.; Hsu, W.C. Influence of mathematical representation and mathematics self-efficacy on the learning effectiveness of fifth graders in pattern reasoning. Int. J. Learn. Teach. Educ. Res. 2015, 13. Available online: http://www.ijlter.net/index.php/ijlter/article/view/1038 (accessed on 27 October 2022).
  6. Schiepe-Tiska, A.; Schattke, K.; Seeliger, J.; Kehr, H.M. Distal and proximal motivational processes related to flow experience: Investigating the role of implicit motives, affective and cognitive preferences, and perceived abilities. Curr. Psychol. 2021. [Google Scholar] [CrossRef]
  7. Rodríguez-Ardura, I.; Meseguer-Artola, A. Flow in e-learning: What drives it and why it matters. Br. J. Educ. Technol. 2016, 48, 899–915. [Google Scholar] [CrossRef]
  8. Skinner, E.; Greene, T. Chapter 13: Perceived control: Engagement, coping, and engagement. In 21st Century Education: A Reference Handbook; Good, T.L., Ed.; SAGE Publications Inc.: Newbury Park, CA, USA, 2008; pp. I-121–I-130. [Google Scholar] [CrossRef]
  9. Tse, D.; Nakamura, J.; Csikszentmihalyi, M. Beyond challenge-seeking and skill-building: Toward the lifespan developmental perspective on flow theory. J. Posit. Psychol. 2019, 15, 171–182. [Google Scholar] [CrossRef]
  10. Katahira, K.; Yamazaki, Y.; Yamaoka, C.; Ozaki, H.; Nakagawa, S.; Nagata, N. EEG correlates of the flow state: A combination of increased frontal theta and moderate frontocentral alpha rhythm in the mental arithmetic task. Front. Psychol. 2018, 9, 300. [Google Scholar] [CrossRef] [Green Version]
  11. Al-Samarraie, H.; Sarsam, S.M.; Alzahrani, A.I.; Alalwan, N. Reading text with and without diacritics alters brain activation: The case of Arabic. Curr. Psychol. 2020, 39, 1189–1198. [Google Scholar] [CrossRef]
  12. Katona, J.; Ujbanyi, T.; Sziladi, G.; Kovari, A. Examine the effect of different web-based media on human brain waves. In Proceedings of the 2017 8th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), Debrecen, Hungary, 11–14 September 2017; pp. 407–412. [Google Scholar]
  13. Katona, J. A review of human–computer interaction and virtual reality research fields in cognitive InfoCommunications. Appl. Sci. 2021, 11, 2646. [Google Scholar] [CrossRef]
  14. Yang, X.; Cheng, P.-Y.; Lin, L.; Huang, Y.M.; Ren, Y. Can an integrated system of electroencephalography and virtual reality further the understanding of relationships between attention, meditation, flow state, and creativity? J. Educ. Comput. Res. 2019, 57, 846–876. [Google Scholar] [CrossRef]
  15. Wu, S.-F.; Lu, Y.-L.; Lien, C.-J. Detecting students’ flow states and their construct through electroencephalogram: Reflective flow experiences, balance of challenge and skill, and sense of control. J. Educ. Comput. Res. 2021, 58, 1515–1540. [Google Scholar] [CrossRef]
  16. Wu, S.F.; Lu, Y.L.; Lien, C.J. Measuring effects of technological interactivity levels on flow with electroencephalogram. IEEE Access 2021, 9, 85813–85822. [Google Scholar] [CrossRef]
  17. Charles, R.L.; Nixon, J. Measuring mental workload using physiological measures: A systematic review. Appl. Ergon. 2019, 74, 221–232. [Google Scholar] [CrossRef] [PubMed]
  18. Tsai, Y.H.; Wu, S.K.; Yu, S.S.; Tsai, M.H. Analyzing Brain Waves of Table Tennis Players with Machine Learning for Stress Classification. Appl. Sci. 2022, 12, 8052. [Google Scholar] [CrossRef]
  19. Yoshida, K.; Hirai, F.; Miyaji, I. Learning system using simple electroencephalograph feedback effect during memory work. Procedia Comput. Sci. 2014, 35, 1596–1604. [Google Scholar] [CrossRef] [Green Version]
  20. Gast, D.L.; Ledford, J.R. Research approaches in applied settings. In Single Case Research Methodology: Applications in Special Education and Behavioral Sciences; Ledford, J.R., Gast, D.L., Eds.; Routledge: London, UK, 2018; pp. 1–26. [Google Scholar] [CrossRef]
  21. Özhan, Ş.Ç.; Kocadere, S.A. The effects of flow, emotional engagement, and motivation on success in a gamified online learning environment. J. Educ. Comput. Res. 2020, 57, 2006–2031. [Google Scholar] [CrossRef]
  22. Borovay, L.A.; Shore, B.M.; Caccese, C.; Yang, E.; Hua, O. Flow, achievement level, and inquiry-based learning. J. Adv. Acad. 2018, 30, 74–106. [Google Scholar] [CrossRef]
  23. Joseph, N.; Nallapati, A.; Machado, M.X.; Nair, V.; Matele, S.; Muthusamy, N.; Sinha, A. Assessment of academic stress and its coping mechanisms among medical undergraduate students in a large Midwestern university. Curr. Psychol. 2021, 40, 2599–2609. [Google Scholar] [CrossRef]
  24. Bandura, A. Self-efficacy: Toward a unifying theory of behavioral change. Psychol. Rev. 1977, 84, 191–215. [Google Scholar] [CrossRef]
  25. Britner, S.L.; Pajares, F. Sources of science self-efficacy beliefs of middle school students. J. Res. Sci. Teach. 2006, 43, 485–499. [Google Scholar] [CrossRef]
  26. MacLaren, R.; Tran, V.H.; Chiappe, D. Effects of motivation orientation on schoolwork enjoyment and achievement and study habits. Think. Ski. Creat. 2017, 24, 199–227. [Google Scholar] [CrossRef]
  27. Yu, Z.; Zhao, H.; Guo, C.; Guo, J.; Zhang, S.; Hu, K.; Chen, Z. A LSTM network-based learners’ monitoring model for academic self-efficacy evaluation using EEG signal analysis [Conference session]. In Proceedings of the 2020 5th IEEE International Conference on Big Data Analytics (ICBDA), Xiamen, China, 8–11 May 2020. [Google Scholar] [CrossRef]
  28. Cabañero, L.; Hervás, R.; González, I.; Fontecha, J.; Mondéjar, T.; Bravo, J. Analysis of cognitive load using EEG when interacting with mobile devices. Proceedings 2019, 31, 70. [Google Scholar] [CrossRef] [Green Version]
  29. Xiong, R.; Kong, F.; Yang, X.; Liu, G.; Wen, W. Pattern recognition of cognitive load using EEG and ECG signals. Sensors 2020, 20, 5122. [Google Scholar] [CrossRef] [PubMed]
  30. Fatimah, B.; Pramanick, D.; Shivashankaran, P. Automatic detection of mental arithmetic task and its difficulty level using EEG signals [Conference session]. In Proceedings of the 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kharagpur, India, 1–3 July 2020; pp. 1–6. [Google Scholar] [CrossRef]
  31. Lin, F.-R.; Kao, C.-M. Mental effort detection using EEG data in E-learning contexts. Comput. Educ. 2018, 122, 63–79. [Google Scholar] [CrossRef]
  32. So, W.K.; Wong, S.W.; Mak, J.N.; Chan, R.H. An evaluation of mental workload with frontal EEG. PLoS ONE 2017, 12, e0174949. [Google Scholar] [CrossRef] [Green Version]
  33. Friedman, N.; Fekete, T.; Gal, K.; Shriki, O. EEG-based prediction of cognitive load in intelligence tests. Front. Hum. Neurosci. 2019, 13, 191. [Google Scholar] [CrossRef]
  34. Mao, Y.; Yang, R.; Bonaiuto, M.; Ma, J.; Harmat, L. Can flow alleviate anxiety? The roles of academic self-efficacy and self-esteem in building psychological sustainability and resilience. Sustainability 2020, 12, 2987. [Google Scholar] [CrossRef]
  35. Martin, A.J.; Ginns, P.; Burns, E.C.; Kennett, R.; Pearson, J. Load reduction instruction in science and students’ science engagement and science achievement. J. Educ. Psychol. 2020, 113, 1126–1142. [Google Scholar] [CrossRef]
  36. Charness, G.; Gneezy, U.; Kuhn, M.A. Experimental methods: Between-subject and within-subject design. J. Econ. Behav. Organ. 2012, 81, 1–8. [Google Scholar] [CrossRef]
  37. Kazdin, A.E. Single-case experimental designs: Characteristics, changes, and challenges. J. Exp. Anal. Behav. 2021, 115, 56–85. [Google Scholar] [CrossRef] [PubMed]
  38. Vuković, V.; Koropanovski, N.; Marković, S.; Kos, A.; Dopsaj, M.; Umek, A. Specific Test Design for the In-Depth Technique Analysis of Elite Karate Competitors with the Application of Kinematic Sensors. Appl. Sci. 2022, 12, 8048. [Google Scholar] [CrossRef]
  39. Bornas, X.; Noguera, M.; Tortella-Feliu, M.; Llabres, J.; Montoya, P.; Sitges, C.; Inma, T. Exposure induced changes in EEG phase synchrony and entropy: A snake phobia case report. Int. J. Clin. Health Psychol. 2010, 10, 167–179. Available online: https://www.researchgate.net/publication/41805663 (accessed on 27 October 2022).
  40. Delorme, A.; Grandchamp, R.; Curot, J.; Barragan-Jason, G.; Denuelle, M.; Sol, J.C.; Valton, L. Effect of meditation on intracerebral EEG in a patient with temporal lobe epilepsy: A case report. EXPLORE 2021, 17, 197–202. [Google Scholar] [CrossRef]
  41. Formica, C.; De Salvo, S.; Micchìa, K.; La Foresta, F.; Dattola, S.; Mammone, N.; Corallo, F.; Ciavola, A.; Arcadi, F.A.; Marino, S.; et al. Cortical Reorganization after Rehabilitation in a Patient with Conduction Aphasia Using High-Density EEG. Appl. Sci. 2020, 10, 5281. [Google Scholar] [CrossRef]
  42. Hwang, J.-J. Problems and perspectives for implementing the entrance system of senior high and vocational high schools in Taiwan’s 12-year national basic education program. Taiwan Educ. Rev. Mon. 2014, 3, 102–132. [Google Scholar] [CrossRef]
  43. Chen, R.-H.; Wu, M.-H.; Chen, H.-Y. 新編多元性向測驗指導手冊 [Instruction Manual for New Multiple Aptitude Test Battery]; Chinese Behavioral Science Corporation: Taipei, Taiwan, 2010. [Google Scholar]
  44. Liao, C.-Y.; Chen, R.-C.; Tai, S.-K. Evaluating attention level on MOOCS learning based on brainwaves signals analysis. Int. J. Innov. Comput. Inf. Control 2019, 15, 39–51. [Google Scholar] [CrossRef]
  45. Bos, A.S.; Herpich, F.; Kuhn, I.; Guarese, R.L.; Tarouco, L.M.; Zaro, M.A.; Pizzato, M.; Wives, L. Educational technology and its contributions in students’ focus and attention regarding augmented reality environments and the use of sensors. J. Educ. Comput. Res. 2019, 57, 1832–1848. [Google Scholar] [CrossRef]
  46. Eldenfria, A.; Al-Samarraie, H. Towards an online continuous adaptation mechanism (OCAM) for enhanced engagement: An EEG study. Int. J. Hum.–Comput. Interact. 2019, 35, 1960–1974. [Google Scholar] [CrossRef]
  47. Pope, A.T.; Bogart, E.H.; Bartolome, D.S. Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 1995, 40, 187–195. [Google Scholar] [CrossRef]
  48. Powers, D.M.W. Evaluation: From precision, recall and F-factor to ROC, informedness, markedness & correlation. J. Mach. Learn. Technol. 2011, 2, 37–63. Available online: https://dspace.flinders.edu.au/xmlui/handle/2328/27165 (accessed on 27 October 2022).
  49. Schunk, D.H.; Ertmer, P.A. Self-regulation and academic learning: Self-efficacy enhancing interventions. In Handbook of Self-Regulation; Boekaerts, M., Pintrich, P.R., Zeidner, M., Eds.; Academic Press: New York, NY, USA, 2000; pp. 631–649. [Google Scholar] [CrossRef]
  50. Hotulainen, R.; Vinni-Laakso, J.; Kupiainen, S. Development of learning to learn competence across secondary education and its association with attainment in Finnish/Swedish high-stake exit exam. Think. Ski. Creat. 2020, 38, 100738. [Google Scholar] [CrossRef]
  51. Turan, N. A general literature review on flow experience. J. Soc. Sci. 2019, 37, 181–199. [Google Scholar] [CrossRef]
Applsci 12 12248 g001 550
Figure 1. Collection of the participant’s electroencephalogram (EEG) wave bands.
Figure 1. Collection of the participant’s electroencephalogram (EEG) wave bands.
Applsci 12 12248 g001
Applsci 12 12248 g002 550
Figure 2. J48 decision tree for electroencephalogram (EEG)–detected, real-time flow state (EEG-F) factors.
Figure 2. J48 decision tree for electroencephalogram (EEG)–detected, real-time flow state (EEG-F) factors.
Applsci 12 12248 g002
Applsci 12 12248 g003 550
Figure 3. Construct for predicting student’s electroencephalogram (EEG)–detected, real-time flow states (EEG-Fs). Note. LT-tp: long-term test performance; OR: odds ratio; SC: sense of control.
Figure 3. Construct for predicting student’s electroencephalogram (EEG)–detected, real-time flow states (EEG-Fs). Note. LT-tp: long-term test performance; OR: odds ratio; SC: sense of control.
Applsci 12 12248 g003
Table
Table 1. Contingency table for EEG-detected, real-time flow states (EEG-Fs) by predictor.
Table 1. Contingency table for EEG-detected, real-time flow states (EEG-Fs) by predictor.
Predictor
Variables		GroupsEEG-F
N (%)		Total
N (%)
Low EEG-F 1High EEG-F 2
Cognitive loadLow load209 (27.3%)290 (37.9%) ***499 (65.1%)
High load214 (27.9%) ***53 (6.9%)267 (34.9%)
Self-efficacyLow self-efficacy326 (42.6%) ***164 (21.4%)490 (64.0%)
High self-efficacy97 (12.7%)179 (23.4%) ***276 (36.0%)
LT-tpLow LT-tp223 (29.1%) ***126 (16.4%)349 (45.6%)
High LT-tp200 (26.1%)217 (28.3%) ***417 (54.4%)
ST-tpIncorrect42 (5.5%)25 (3.3%)67 (8.7%)
Correct381 (49.7%)318 (41.5%)699 (91.3%)
DifficultyLow difficulty247 (32.2%)240 (31.3%) ***487 (63.6%)
High difficulty176 (23.0%) ***103 (13.4%)279 (36.4%)
Total423 (55.2%)343 (44.8%)766 (100%)
Note. * p < 0.05, ** p < 0.01, *** p < 0.001; dependent variable = high EEG-F vs. low EEG-F; LT-tp: long-term test performance; H: high; L: low; OR: odds ratio. 1 A low EEG-F means that the student is in a low flow state, as detected by an EEG. 2 A high EEG-F means that the student is in a high flow state, as detected by an EEG.
Table
Table 2. Summary of logistic regression analysis for electroencephalogram (EEG)-detected, real-time flow states (EEG-Fs).
Table 2. Summary of logistic regression analysis for electroencephalogram (EEG)-detected, real-time flow states (EEG-Fs).
NodeVariablesOmnibus χ2Sig.GroupWald’s χ2Sig.OR
1 Cognitive load108.88 ***<0.001L vs. H93.46 ***<0.0015.6
H load (OR = 1.0)
2Self-efficacy53.74 ***<0.001H vs. L41.52 ***<0.00111.8
4(H efficacy) LT-tp18.80 ***<0.001H vs. L15.77 ***<0.0017.1
6(H LT-tp) Difficulty4.97 *0.026L vs. H4.79 *0.0293.5
L load (OR = 5.6)
3Self-efficacy42.82 ***<0.001H vs. L38.73 ***<0.0013.7
5( L efficacy) LT-tp4.00 *0.046H vs. L4.00 *0.0470.6
7(H LT-tp) Difficulty2.260.133L vs. H2.250.1341.6
Note. * p < 0.05, ** p < 0.01, *** p < 0.001; dependent variable = high EEG-F vs. low EEG-F; LT-tp: long-term test performance; H: high; L: low; OR: odds ratio.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Wu, S.-F.; Kao, C.-H.; Lu, Y.-L.; Lien, C.-J. A Method Detecting Student’s Flow Construct during School Tests through Electroencephalograms (EEGs): Factors of Cognitive Load, Self-Efficacy, Difficulty, and Performance. Appl. Sci. 2022, 12, 12248. https://doi.org/10.3390/app122312248

AMA Style

Wu S-F, Kao C-H, Lu Y-L, Lien C-J. A Method Detecting Student’s Flow Construct during School Tests through Electroencephalograms (EEGs): Factors of Cognitive Load, Self-Efficacy, Difficulty, and Performance. Applied Sciences. 2022; 12(23):12248. https://doi.org/10.3390/app122312248

Chicago/Turabian Style

Wu, Shu-Fen, Chieh-Hsin Kao, Yu-Ling Lu, and Chi-Jui Lien. 2022. "A Method Detecting Student’s Flow Construct during School Tests through Electroencephalograms (EEGs): Factors of Cognitive Load, Self-Efficacy, Difficulty, and Performance" Applied Sciences 12, no. 23: 12248. https://doi.org/10.3390/app122312248

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop