Next Article in Journal
A LoRa-Based Communication System for Coordinated Response in an Earthquake Aftermath
Previous Article in Journal
Redesign and Evaluation of AffectiVest, a Wearable Tool to Convey Affection Gestures between Geographically-Separated Grandparents and Grandchildren †
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Proceedings
Volume 31
Issue 1
10.3390/proceedings2019031070
proceedings-logo
Submit to this Journal
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Proceeding Paper

Analysis of Cognitive Load Using EEG when Interacting with Mobile Devices †

by
Luis Cabañero
1,*,
Ramón Hervás
1,
Iván González
1,
Jesús Fontecha
1,
Tania Mondéjar
2 and
José Bravo
1
1
Department of Information Systems and Technologies, University of Castilla–La Mancha, 13071 Ciudad Real, Spain
2
Department of Psychology, University of Castilla–La Mancha, 13071 Ciudad Real, Spain
*
Author to whom correspondence should be addressed.
†
Presented at the 13th International Conference on Ubiquitous Computing and Ambient Intelligence UCAmI 2019, Toledo, Spain, 2–5 December 2019.
Proceedings 2019, 31(1), 70; https://doi.org/10.3390/proceedings2019031070
Published: 21 November 2019
(This article belongs to the Proceedings of 13th International Conference on Ubiquitous Computing and Ambient ‪Intelligence UCAmI 2019‬)
Browse Figures
Versions Notes

Abstract

:
The study of cognitive responses and processes while using applications is a critical field in human–computer interaction. This paper aims to determine the mental effort required for different typical tasks with smartphones. Mental effort is typically associated with the concept of cognitive load, and has been studied by analyzing electroencephalography (EEG) signals. Thus, this paper shows the results of analyzing the cognitive load of a set of characteristic tasks on smartphones. To determine the set of tasks to analyze, this paper proposes a taxonomy of smartphone-based actions defined after considering the related proposals in the literature and identifying the significant characteristics of the tasks to classify them. The EEG data was obtained through an experiment with real users doing tasks from the aforementioned taxonomy. The results show significant differences in the cognitive load of each task category and identify those tasks that involve a higher degree of mental effort. The results will be the starting point of the M4S project that aims to contribute to the early diagnosis of mild cognitive impairment through monitoring everyday dual-tasking in terms of interaction with smartphones.

1. Introduction

The widespread use of smartphones and mobile devices today allows us to provide applications and services to address many issues, such as those in healthcare. The smartphone is a great source of information; its sensors provide us with data about location, movement, voice, battery, and application use, among others; this is being used to assess the behavioral aspects of users in their daily life [1]. Concerning health, for example, the analysis of smartphone use allows us to track GPS locations and trajectories followed by users and can be used, for example, to measure anxiety levels as a prelude to mental health problems [2]. Additionally, data coming from the interaction between users and their own mobile devices provide us with valuable information related to human behavior. In fact, the smartphone can be an ally for diagnostic purposes [3], although it should serve a complementary role in the doctor–patient relationship. Particularly, in dementia, Blanka Klimova [4] evidenced the potential of mobile apps for diagnosis support, minimizing examiner bias, increasing patient independence, cutting hospitalization costs, and improving the overall quality of life of elders. As we can see, analyses of interactions (explicit or implicit) with smartphones can be invaluable in the field of human–computer interactions, as well as in the field of health, with diagnosis and treatment purposes.
This paper is part of the project “Mobile computing-based Multitasking for Mild cognitive impairment Monitoring and early Screening (M4S)” that aims to contribute to the early diagnosis of mild cognitive impairment through monitoring everyday dual-tasking in terms of interactions with smartphones on the move. An initial stage of this project is to determine the cognitive load required for various, typical tasks performed on a mobile device, i.e., the main objective of this paper. The findings of this paper not only contribute to the aforementioned project, but also to the research community in the field, giving rise to a better understanding of cognitive processes which are associated with the use of mobile devices.
In order to analyze the cognitive load (i.e., the amount of working memory resources or “mental effort” associated with a specific task, concepts explored in depth in Section 2.1), we analyzed the EEG activity of users performing a set of typical tasks with a smartphone. The fundamentals of the EEG-based cognitive load analysis are also described in Section 2.1. To determine the set of tasks to analyze, this paper proposes a taxonomy of smartphone-based actions. We considered the related proposals in the literature (Section 2.2), and identified the significant characteristics of the tasks to classify them, in order to propose the HuSBIT-10 taxonomy: Human-Smartphone Basic Interactions Taxonomy for 10-s tasks (Section 3). An experiment with real users was conducted with the dual objective of (i) studying the cognitive load of different typical tasks with the smartphone, and (ii) validating the classification made in the taxonomy in terms of the mental effort associated with the identified task categories. The protocol, material, and methods of the experiment are explained in Section 4, and the obtained results are presented in Section 5. Finally, Section 6 concludes the paper.

2. Fundamentals and Background

2.1. Cognitive Load

2.1.1. Cognitive Load Fundamentals

Each person has a specific way of responding to external stimulus, but the brain processes this information following a common pattern for most people. One similar point is usually the mental effort and cognitive load, which are closely-related but different concepts; the first occurs in interactions between the characteristics of a task and the characteristics of the subject. Each task has a different load, i.e., it can be more complex or simple (depending on the steps or the level of precision required in order to perform the action), and each subject processes it differently, according to their skills and aptitudes [5]. On the other hand, mental effort is related to the cognitive resources we use to undertake a specific task.
With these basic concepts in mind, we can mention the Sweller’s [6] theory of cognitive load, which focuses on working memory, and specifically, on Mayer’s [7] theory of multimedia learning. These theories are part of the cognitive sciences that seek to improve multimedia environments [8] within the information processing paradigm, taking it as a “natural information processing system” [9].
In our work, tasks with smartphones have different mental burdens and are related to some external stimulus, therefore the tasks are translated into cognitive information. For this, we use several sensitive channels. In our case, the auditory and visual channels trigger working memory. The processing of information in working memory is related to the activity we are consciously carrying out [10]. In addition, recent research has shown that working memory is divided into three processors or channels [8]. The information processed in this memory is distributed between two partially independent processors, i.e., the auditory and the visual, which manipulate verbal and pictorial information, respectively. Additionally, there is a third processor known as the central-executive, which is responsible for coordinating the processing of information that enters and leaves the working memory.
For this reason, we must consider the presentation of information to avoid overloading these channels. In addition, it is critical to consider whether the information is new, so that it can be acquired only if the subject’s mental activity can relate it to mental schemes previously stored in long-term memory [7,11]. The difference here is that if a person has done a task repeatedly, their processing is different because they have response patterns associated with that task and the execution is faster or easier to do. This is achieved with practice time and depends on the intuitiveness of the tasks, which has been considered in the development of our work.
All the previous fundamentals are considered in this paper to define the taxonomy tasks with smartphone and to define the experiment protocol, as we describe in next sections.

2.1.2. EEG-Based Cognitive Load Analysis

Attending to the EEG activity, four main areas of the brain have been discussed in the literature to study neurological activity: parietal, occipital, temporal and frontal [4,12]. This neurological activity has been observed to produce a range of electrical waves per second at different frequencies with greater or lesser level of coverage that depends on the task being performed.
A clear example of the differences that occur in the electrical response of the brain associated with neurological activity can be seen in the electrical oscillations emitted during sleep compared to those made when awake. The brain produces very low-frequency electrical waves (<1 Hz) in the electroencephalogram (EEG) of sleep stages, between the 0.55–0.95 Hz range and with peaks at 0.7–0.8 Hz in the frequency band known as delta [13]. On the other hand, higher frequencies and faster waves predominate in waking conditions, where bands oscillate between 0.5–40 Hz. The intervals that correspond to each band are as follows: 0.5–4 Hz (delta band), 4–8 Hz (theta band), 8–13 Hz (alpha band), 13–30 (beta band), and finally 30–40 Hz (gamma band). As stated before, the composition of the electrical response strongly depends on the cognitive task.
EEG techniques can capture the electrical response of the brain by means of electrodes placed on the scalp. These electrical signals are generated by ionic movements in and around neurons during the activation and deactivation of neurons involved in a cognitive task. EEG measures the fluctuating voltages in these electrical signals. There is not a straightforward way to estimate cognitive load from EEG electrical signals, however, some approaches can be found in the literature. The three most commonly used analysis techniques are: (i) event-related desynchronization (ERD), (ii) theta-alpha ratio (TAR) and (iii) those techniques based on machine learning. In relation to detecting changes in cognitive load using the ERD technique, Klismech found that the spectral power in the theta band increases while the spectral power in the alpha band decreases [14]. Further relevant contributions have studied the use of ERD from alpha and theta bands to measure cognitive load. For example, Antonenko et al. have applied ERD technique in two different case studies related to the learning context [15].
On the other hand, some recent studies have explored the use of TAR technique as a measure of cognitive load [16,17,18]. In particular, Trammell at el. have found associations between age and estimated cognitive load by using this technique. TAR is obtained by dividing the spectral power of theta band in the middle frontal area (Fz) by the spectral power of alpha band in central parietal area (Pz).
Other novel and powerful approaches to estimate cognitive load are those based on machine learning. There are many research works using these techniques for this particular purpose [19] which uses Naïve-Bayes, and [20], which uses deep convolutional neural networks. Through machine learning models, robust and useful metrics can be extracted from EEG signals, although they have some problems related to the sample size and data gathered from the acquisition trials. Specifically, (i) it takes a large number of participants to adequately train a classifier or fit a regression model that can be able to work properly on EEG data from anyone; (ii) the studies found are mostly based on supervised learning, therefore, a big labeled dataset is required to train the model. Such considerations have made us discard machine learning techniques as a method for this work.

2.2. Mobile-Device Interaction Background

Interaction with mobile devices has been widely studied in the literature. Today, many projects and research apply a user-centered design and development, highlighting the role of usability and user experience in terms of mobile device interactions. According to Hoober [21], users interact with their mobile devices in three different ways: (1) using only one hand, (2) using both hands, and (3) in a passive manner. Likewise, this study also indicates three types of human-device interactions: active use passive use, and talking, as well as the body posture that the users have when they interact with their mobile phones: walking, standing and sitting.
In 2005, Karam and Shraefel carried out a broad study leading to the creation of a general taxonomy of gestures in human computer interactions [22]. In this work, authors also presented a review of possible interactions with any device, not just mobile. Focusing on mobile phones, the most common inputs were: camera, touch surface and sensors-on-body (e.g., accelerometer, GPS). The last input is considered as a pervasive or implicit way of interaction with the mobile device. In case of interaction with touch screens, Wroblewski [23] proposed a reference guide very popular as standardized guide about gestures in these kind of displays.
Furthermore, there is a vast amount of works which are focused on analyzing user-Smartphone interactions in different domains, using a variety of measurement mechanisms, and facing multiple purposes. Today, works such as the one presented by Hinckley et al. [3] show new ways to detect interaction with smartphone screens before it happens. which is referred to as “pre-touch sensing”. Cameras and vision-based systems are also useful to analyze interactions with mobile applications. Authors such as Souza [4] and Chang [5] highlight the importance of eye-tracking data for usability studies, comparing them with traditional techniques.
The analysis of population behaviors can also be studied by observing interactions between users and their mobile phones. In this regard, new usage and behavioral patterns can be found [6], as well as different types of smartphone users [7]. In the field of psychology, Harari et al. [8] determine that the use of smartphones is an important observation tool in psychological science, considering all data provided by these mobile devices. Measuring and analyzing patterns for smartphone addiction is also possible by interacting with the mobile phone [9,10].
All interactions with mobile phones occur at different levels. We can study these interactions with the operating system, built-in sensors, and physical device buttons, as well as with the installed applications. Given our objectives, and considering the related work, in which we did not find research regarding the classification of smartphone tasts on a cognitive level, we propose a specific taxonomy of basic tasks related to the most common types of interaction with mobile phones in the following section.

3. Proposed Taxonomy: HuSBIT-10

According to the objectives of our study, we need to define a set of usual tasks focused on the user-smartphone interaction. Thesetasks are quick and simple tasks that require less than 10 s. The name of the taxonomy is HuSBIT-10: Human-Smartphone Basic Interactions Taxonomy for 10-s tasks.
First, we have identified four types of interactions that a user could carry out with his mobile phone: (τ) touch, (ι) look, (ς) speak, and (η) hear. All of them are closely related to human senses, critical to analyze the cognitive load and information processing [8]. In addition, considering some approaches from the literature, the interaction types can be classified in two categories: (α) active and (ρ) passive, depending on whether the user explicitly interacts with the device or not. With this, we can determine if a specific interaction type from the first four types previously mentioned is active or passive.
Moreover, any interaction task with the mobile phone could employ one or several types of interaction from the above. Hence, we have defined AMPEC-10 as a term to group the five types of tasks that a user can carry out with the smartphone in a maximum time of 10 s (limit obtained experimentally), making use of the four interaction types. According to the acronym AMPEC-10, the tasks have been grouped into the following types:
  • (A) Automated. This represents the tasks with or without a minimal cognitive effort that we typically perform automatically or unconsciously.
  • (M) psychoMotor. This kind of task requires a quick or direct interaction with the smartphone, where the main difficulty is to perform a touching interaction carefully or with proper accuracy.
  • (P) Production. It includes tasks which require basic content creation, requiring creative skills to produce new content.
  • (E) Exploration. This kind of tasks requires the analysis of a set of data to obtain specific information.
  • (C) Consumption. It defines the tasks that require content consumption.
The prevalence of interaction types (touch, look, speak, and hear) on these task types is another factor to consider. A first approach reveals that touch and look interaction types are the most common interactions between user and smartphone. Likewise, as we saw in Section 2.2, speak and hear occur less often. This fact confirms what other studies in the literature, as studied in [24,25].
With these assumptions, we have modeled a new taxonomy approach from scratch, called HuSBIT-10 to classify any task with a duration under 10 s, that users perform with their smartphones. In Table 1, an overview of identified tasks (classified by task type) and some examples are shown.
The aim of HusBIT-10 approach is to provide support to classify the AMPEC-10 tasks in terms of planning and cognitive load from a bidimensional perspective, as well as promoting replicability in other trials and experiments.

4. Experiment: Cognitive Load in Smartphone Interactions

4.1. Experiment Protocol and Method

The study was conducted in the MAmI research lab at the University of Castilla–La Mancha, a group focused on health informatics and human-computer interaction (http://mami.uclm.es). The participants were informed about the scope and goals of this research and about the collected data. The work was conducted with six participants, from 22 to 31 years old who received and signed the information sheet and consent form, which provided detailed information about the study’s objective, procedures, and types of data to be collected. All participants had the opportunity to consider their participation before making a final decision. Thereby, the preservation of the dignity and autonomy of the end-users was ensured by their voluntary participation and the fact that they could leave the study at any time without any consequences.
This study followed the empirical method for gathering evidence regarding EEG data while user interact with a smartphone. The followed protocol can be summarized as follows: (1) all participants received an instruction sheet with the actions to do with the smartphone and ensuring they fully understood it (Figure 1d). The actions were randomly sorted for each participant; (2) all participants were wearing the EEG headset (Figure 1a) and sat at a desk with the smartphone (Figure 1c); (3) participants were required to perform the EEG calibration with the Xavier TechBench SoftwareTM (www.emotiv.com/product/emotivpro/) (Figure 1b); (4) the participants, without receiving any additional instruction, performed all the tasks of the sheet. This entire process for each participant took approximately 25 min.

4.2. Material

The cognitive load is studied using a device for capturing EEG signals for scientific purposes, model Emotiv EPOC+. This device has 14 EEG channels and two references for positioning and accurate spatial resolution. These channels have eight frontal electrodes (AF3, F7, F8 and FC5 on the left hemisphere, and FC6, F4, F8 and AF4 on the right hemisphere), two temporal electrodes (T7 and T8), two occipital electrodes (O1 and O2) and two parietal electrodes (P7 and P8). The headset uses a sequential sampling method at a rate of 128 samples per second. To collect the EEG raw data, we used the Xavier TechBench SoftwareTM. For the EEG data processing, there is a specialized software developed to process EEG data, called eeglib, which can also applied to other kind of data sources [26,27]. eeglib is actually a Python-based library for EEG processing that provides some data structures to help for that purpose. This library can load CSV and EDF files that are typical formats in which EEG is stored, and allows the user to import the data from Python and NumPy data structures. It can apply three different pre-processing techniques to the signals: bandpass filtering, z-scores normalization and Independent Component Analysis. It also includes a set of processing techniques to extract features from data: FFT, Higuchi Fractal Dimension, Petrosian Fractal Dimension, Hjorth parameters, Detrended Fluctuation Analysis, Lempel-Ziv Complexity, Multiscale Sample Entropy, Synchronization Likelihood, and Pearson Cross Correlation Coefficient. The library includes a tool to generate datasets (in pandas DataFrame format) that can be used easily to apply machine learning techniques or to perform statistical analysis.
The smartphone used in the experiment was a Samsung J6 with the operating system Android 9.0 Pie. The list of specific tasks to perform is shown in Table 2. There are three tasks per category, omitting the category Automated due to the very low cognitive load associated to unconscious or mechanic tasks. Thus, the total number of evaluated tasks were 12. The design of the list of tasks follows the considerations and fundamentals about cognitive load in Section 2.1.

4.3. EEG Data Processing

Participants performed all the tasks described in Table 2, specifically, three defined tasks per category considered in the HuSBIT-10 proposed taxonomy. EEG activity was recorded during each task for a 10-s interval (EEG segment). The recorded EEG data can be found in the link in Section Supplementary Material.
The EEG segments, denoted as <Participant_Id, Task Type>, were then analyzed to estimate cognitive load in accordance with the following procedure: first (i), each EEG segment was preprocessed applying a 2–15 Hz bandpass filter to remove frequencies that were not under analysis (neither alpha nor theta band were removed); then (ii), zScore normalization was performed on the filtered EEG signals to made possible relative comparisons across EEG segments for all the participants in the next steps. After pre-processing (iii), each normalized EEG segment was split into 1-s windows with 50% overlapping; later (iv), a variation of the TAR index was computed for each analysis window within the EEG segment, as indicated below:
TAR = (thetaF3 + thetaF4)/(alphaP7 + alphaP8)
where thetaF3 and thetaF4 were the spectral power of the theta band in the F3 and F4 electrodes (frontal area in both brain hemispheres); and alphaP7 and alphaP8 the spectral power of the alpha band in the P7 and P8 electrodes (parietal area in both brain hemispheres). Once theta-alpha ratio values were obtained for each analysis window (v), the TAR index was averaged for the entire EEG segment that enclosed them all. This average TAR index (for the current EEG segment) was considered as a cognitive load estimation for a particular task performed by one participant (<Participant_Id, Task Type>). If the average TAR indices across all participants are grouped by task and again averaged, cognitive load estimations for each particular task will be obtained.

4.4. Result

The experiment results show differences among the cognitive load associated with each of the categories of smartphone tasks. These finding should be considered preliminary due to the reduced number of subjects in the experiment, the slight differences, and the wide standard deviation of the data, as shown in Table 3. This table summarizes the average cognitive load data for each category, considering all the users. It can be observed that the tasks with the highest cognitive load are those of the Exploration category with a significant difference, followed by Production. The ones that show the least cognitive load are the psychoMotor and Consumption tasks, with very similar cognitive load measures.
Observing the average value obtained in each of the 12 evaluated tasks, we see two cases of tasks that have a significantly higher cognitive load than the rest of tasks: analysis of textual contents (E3) and analysis of visual content (E4). The task regarding analysis of textual contents has been carried out by means of asking to switch off the data roaming in the device settings. This task is complex because users must navigate into the numerous categories in the smartphone setting and localize one element through understanding the setting organization. The second case regarding analysis of visual content corresponds to the search for a route in a map, a more complex exploration task than the search for an element as a whole. The lowest cognitive load is observed in the task about searching into a set of textual elements, in the experiment case, to search for a given date in the calendar. Figure 2 shows the graphical representation of the average values of each task.

5. Conclusions

The present work is the beginning of a new research line that aims to help in the evaluation of cognitive abilities and contribute to the early diagnosis of mild cognitive impairment through the continuous analysis of the interaction with smartphones. The first steps have focused on two aspects that correspond to the main contributions of this article: the taxonomy of tasks with smartphones named HuSBIT-10, and the analysis of the cognitive load of typical tasks. The taxonomy is based on similar classifications focused on other devices found in the literature, as well as on the cognitive components related to each of the tasks. This taxonomy can support researchers in human-computer interaction to have a model that classifies the types of interactions with smartphones. Secondly, the data obtained after the experiment to analyze the cognitive load of different tasks provides us with information about the associated mental effort of each one. It will serve as a starting framework to evaluate their performance over time and associate it with cognitive impairment. It is important to notice that the EEG headset used in this study has no medical or scientific purpose, so, although it is reliable, its accuracy is not very high. Also, the population used in this experiment can be insufficient to be statistically significant. Future work will focus on improving this experiment, both with a more accurate EEG measuring devices, and by increasing the population and the set of tasks to be performed. Moreover, future work will involve analyzing cognitive load when interacting with specific mobile application for people with special needs. Examples of this are augmented reality for guiding people with dementia [28,29], mobile-based biomedical signals measurement [30], and avatar-based apps for emotion management [31].

Supplementary Materials

The EEG dataset generated and analyzed for this study can be found in www.esi.uclm.es/www/mami/web/index.php/datasets.

Author Contributions

I.G., J.F. and R.H. generated the initial ideas for the study. R.H. and T.M. designed the experiment, and R.H. and L.C. led the writing of this manuscript. T.M. identified the psychological fundamentals related with this study. R.H. coordinated the experiment conduction. L.C. developed the tools for the EEG analysis, and, with the support of I.G., conducted the data analysis and its interpretation. J.F. designed the smartphone interaction taxonomy. J.B., expert on assistive technologies and human-computer interaction, helped with the specific decisions, and critically reviewed the manuscript.

Funding

This research was funded by the MINISTERIO DE CIENCIA, INNOVACIÓN Y UNIVERSIDADES, grant number RTI2018-098780-B-I00 (national research project) and the PEJ2018-003213-A (for hiring research staff). The EEG headset was purchased with the funding of the UNIVERSITY OF CASTILLA–LA MANCHA, grant number 2019-AYUDA-27430.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Harari, G.M.; Lane, N.D.; Wang, R.; Crosier, B.S.; Campbell, A.T.; Gosling, S.D. Using Smartphones to collect behavioral data in psychological science: Opportunities, practical considerations, and challenges. Perspect. Psychol. Sci. 2000, 11, 838–854. [Google Scholar] [CrossRef] [PubMed]
  2. Huang, Y.; Xiong, H.; Leach, K.; Zhang, Y.; Chow, P.; Fua, K.; Teachman, B.A.; Barnes, L.E. Assessing Social Anxiety using GPS Trajectories and Point-of-Interest Data. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany, 12–16 September 2016; pp. 898–903. [Google Scholar]
  3. Lupton, D.; Jutel, A. ‘It’s like having a physician in your pocket!’ A critical analysis of self-diagnosis smartphone apps. Soc. Sci. Med. 2015, 133, 128–135. [Google Scholar] [CrossRef]
  4. Klimova, B. Mobile Phone Apps in the Management and Assessment of Mild Cognitive Impairment and/or Mild-to-Moderate Dementia. Front. Hum. Neurosci. 2017, 11, 461. [Google Scholar] [CrossRef]
  5. Lotero, L.A.A. Teoría de la carga cognitiva, diseño multimedia y aprendizaje: Un estado del arte. Magis Rev. Int. Investig. Educ. 2012, 5, 75–92. [Google Scholar]
  6. Sweller, J. Cognitive Load Theory, Learning Difficulty, and Instructional Design. Learn. Instr. 1994, 4, 295–312. [Google Scholar] [CrossRef]
  7. Mayer, R.E. The Cambridge Handbook of Multimedia Learning; Cambridge University Press: New York, NY, USA, 2005. [Google Scholar]
  8. Chong, T.S. Recent Advances in Cognitive Load Theory Research: Implications for Instructional Designers. Malays. Online J. Instr. Technol. 2005, 2, 106–117. [Google Scholar]
  9. Paas, F.; Tuovinen, J.E.; Tabbers, H.; van Gerven, P.W.M. Cognitive Load Measurement as a Means to Advance Cognitive Load Theory. Educ. Psychol. 2003, 38, 63–71. [Google Scholar] [CrossRef]
  10. Sweller, J. Discussion of ‘Emerging Topics in Cognitive Load Research: Using Learner and Information Characteristics in the Design of Powerful Learning Environments’. Appl. Cogn. Psychol. 2006, 20, 353–357. [Google Scholar] [CrossRef]
  11. Clark, R.C.; Mayer, R.E. E-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning; John Wiley and Sons: New York, NY, USA, 2007. [Google Scholar]
  12. Kumar, N.; Kumar, J. Measurement of Cognitive Load in HCI Systems Using EEG Power Spectrum: An Experimental Study. Procedia Comput. Sci. 2016, 84, 70–78. [Google Scholar] [CrossRef]
  13. Achermann, P.; Borbély, A.A. Low-frequency (<1 Hz) oscillations in the human sleep electroencephalogram. Neuroscience 1997, 81, 213–222. [Google Scholar]
  14. Klimesch, W. EEG Alpha and Theta Oscillations Reflect Cognitive and Memory Performance: A Review and Analysis. Brain Res. Rev. 1999, 29, 169–195. [Google Scholar] [CrossRef]
  15. Antonenko, P.; Paas, F.; Grabner, R.; van Gog, T. Using Electroencephalography to Measure Cognitive Load. Educ. Psychol. Rev. 2010, 22, 425–438. [Google Scholar] [CrossRef]
  16. Holm, A.; Lukander, K.; Korpela, J.; Sallinen, M.; Müller, K. Estimating Brain Load from the EEG. Sci. World J. 2009, 9, 639–651. [Google Scholar] [CrossRef] [PubMed]
  17. Dan, A.; Reiner, M. Real Time EEG Based Measurements of Cognitive Load Indicates Mental States during Learning. J. Educ. Data Min. 2017, 9, 32–44. [Google Scholar]
  18. Trammell, J.P.; MacRae, P.G.; Davis, G.; Bergstedt, D.; Anderson, A.E. The Relationship of Cognitive Performance and the Theta-Alpha Power Ratio Is Age-Dependent: An EEG Study of Short-Term Memory and Reasoning during Task and Resting-State in Healthy Young and Old Adults. Front. Aging Neurosci. 2017, 9, 364. [Google Scholar] [CrossRef]
  19. Grimes, D.; Tan, D.S.; Hudson, S.E.; Shenoy, P.; Rao, R.P.N. Feasibility and Pragmatics of Classifying Working Memory Load with an Electroencephalograph. In Proceedings of the 2008 Conference on Human Factors in Computing Systems, Florence, Italy, 5–10 April 2008; pp. 835–844. [Google Scholar]
  20. Jiao, Z.; Gao, X.; Wang, Y.; Li, J.; Xu, H. Deep Convolutional Neural Networks for Mental Load Classification Based on EEG Data. Pattern Recognit. 2018, 76, 582–595. [Google Scholar] [CrossRef]
  21. Hoober, S. How Do Users Really Hold Mobile Devices? 2013. Mobile Matters. Available online: https://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobile-devices.php? (accessed on 9 October 2019).
  22. Karam, M.; Schraefel, M.C. A Taxonomy of Gestures in Human Computer Interactions s.n. 2005. Project Report. Available online: https://eprints.soton.ac.uk/261149/ (accessed on 9 October 2019).
  23. Wroblewski, L. Touch Gesture Reference Guide. Available online: https://www.lukew.com/ff/entry.asp?1071 (accessed on 9 October 2019).
  24. Lee, H.E.; Lee, Y.J. A Study on Content and Interaction Types on Smartphone. Adv. Sci. Technol. Lett. 2015, 120, 208–210. [Google Scholar]
  25. Love, S. Understanding Mobile Human-Computer Interaction; Butterworth-Heinemann: Oxford, UK, 2005. [Google Scholar]
  26. Cabañero, L.; Hervás, R.; Bravo, J.; Rodríguez-Benitez, L. eeglib: Computational analysis of cognitive performance during the use of video games. J. Ambient Intell. Humaniz. Comput. 2019. under review. [Google Scholar]
  27. Cabañero, L.; Hervás, R.; Bravo, J.; Rodríguez-Benitez, L. Computational EEG Analysis Techniques When Playing Video Games: A Systematic Review. Proceedings 2018, 2, 483. [Google Scholar]
  28. Luna, J.M.; Hervás, R.; Fontecha, J.; Bravo, J. A Friendly Navigation-System Based on Points of Interest, Augmented Reality and Context-Awareness. In Proceedings of the 6th Conference on Ubiquitous Computing and Ambient Intelligence (UCAmI 2012), Vitoria-Gasteiz, Spain, 3–5 December 2012; Volume 7656, ISBN 978-3-642-35377-2. [Google Scholar]
  29. Hervás, R.; Bravo, J.; Fontecha, J.; Villarreal, V. Achieving Adaptive Augmented Reality through Ontological Context-Awareness applied to AAL Scenarios. J. Univers. Comput. Sci. 2013, 19, 1334–1349. [Google Scholar]
  30. Bravo, J.; Villarreal, V.; Hervás, R.; Urzaiz, G. Using a communication model to collect measurement data through mobile devices. Sensors 2012, 12, 9253–9272. [Google Scholar] [CrossRef] [PubMed]
  31. Johnson, E.; Hervás, R.; de la Franca, C.G.L.; Mondéjar, T.; Ochoa, S.F.; Favela, J. Assessing empathy and managing emotions through interactions with an affective avatar. Health Inform. J. 2018, 24, 182–193. [Google Scholar] [CrossRef] [PubMed]
Proceedings 31 00070 g001
Figure 1. Experiment setup with the following materials: (a) Emotiv Epoc + EEG headset; (b) Laptop with the required software: Xavier TechBench SoftwareTM to collect raw data and eeglib to process and analyze; (c) Mobile phone Samsung J6 with Android 9.0 Pie; (d) task sheet with the experiment instructions and the consent form.
Figure 1. Experiment setup with the following materials: (a) Emotiv Epoc + EEG headset; (b) Laptop with the required software: Xavier TechBench SoftwareTM to collect raw data and eeglib to process and analyze; (c) Mobile phone Samsung J6 with Android 9.0 Pie; (d) task sheet with the experiment instructions and the consent form.
Proceedings 31 00070 g001
Proceedings 31 00070 g002
Figure 2. Graphical representation of averaged cognitive load for the 12 evaluated tasks.
Figure 2. Graphical representation of averaged cognitive load for the 12 evaluated tasks.
Proceedings 31 00070 g002
Table 1. AMPEC-10 tasks classification according to HuSBIT-10 approach.
Table 1. AMPEC-10 tasks classification according to HuSBIT-10 approach.
Task CategoryIdTask TypeCharacteristicsExamples
AutomatedA1Query an item(α) (τ, ι)Check time/Check if there are notifications/Check if I have WiFi
A2Action on any physical button(α) (τ)Turn on-off device/Turn up-down Volume
psychoMotorM1Pattern(α) (τ, ι)Device unlock (with unlock pattern)
M2Move(α) (τ, ι)Add and move a shortcut
M3Dismiss(α) (τ, ι)Close opened apps, Close notification preview
M4Copy & Paste(α) (τ, ι)Share information among applications
M5Select(α) (τ, ι)Select a part of a text
ProductionP1Text Production(α) (τ, ι)Add a new contact/Set an alarm/Write a message/Reminder
P2Voice Production(α) (τ, ς)Make a call/Make a voice command/Create voice message
P3Visual Production(α) (τ, ι)Take a photo
ExplorationE1Search on a textual set(α) (τ, ι)Search for a contact/Search for a song/Search for date in the calendar/Last call made to someone
E2Search on a visual set(α) (τ, ι)Search for a specific application/Browse images/Change direct-access settings (e.g., airplane mode)
E3Analysis of textual contents(α) (τ, ι)Change settings details (e.g., data roaming)/Do a search in an Internet Browser
E4Analysis of visual contents(α) (τ, ι)Search for a route/site on a map
ConsumptionC1Text Consumption(ρ) (ι)View/Read notifications, Read a text message
C2Audio Consumption(ρ) (η)Listen to an audio message/Listen to a podcast
C3Media Consumption(ρ) (ι, η)Watch a video
Table 2. List of tasks performed in the experiment according to HuSBIT-10 taxonomy.
Table 2. List of tasks performed in the experiment according to HuSBIT-10 taxonomy.
Task CategoryTask TypeSpecific Task in the Experiment
ConsumptionC1Read a message that contains a poem by Espronceda
C2Listen to a podcast from the daily news
C3Watch a video
ExplorationE1Search for a given date in the calendar
E3Switch off the data roaming in the device settings
E4Search how to reach a given place (about 500 m away) in the map from the current location
psychoMotorM2Add and move an app shortcut (2 times)
M4Copy a message into the browser search box (Google widget)
M5Select one word, then two and, finally, two and a half words in a Wikipedia article
ProductionP1Write down the places where you would go in a zombie apocalypse
P2Create a voice message with the list of objects you would collect in a zombie apocalypse
P3Take an artistic photo of one object around you
Table 3. Results of cognitive load in each task performed in the experiment according to HuSBIT-10 taxonomy.
Table 3. Results of cognitive load in each task performed in the experiment according to HuSBIT-10 taxonomy.
Task CategoryAveraged Cog. LoadStandard DeviationTask TypesAveraged Cog. Load
Consumption C11.206
1.2530.251C21.273
C31.280
Exploration E11.097
1.3940.247E31.597
E41.488
psychoMotor M21.224
1.2470.297M41.241
M51.277
Production P11.287
1.2840.255P21.240
P31.324

Share and Cite

MDPI and ACS Style

Cabañero, L.; Hervás, R.; González, I.; Fontecha, J.; Mondéjar, T.; Bravo, J. Analysis of Cognitive Load Using EEG when Interacting with Mobile Devices. Proceedings 2019, 31, 70. https://doi.org/10.3390/proceedings2019031070

AMA Style

Cabañero L, Hervás R, González I, Fontecha J, Mondéjar T, Bravo J. Analysis of Cognitive Load Using EEG when Interacting with Mobile Devices. Proceedings. 2019; 31(1):70. https://doi.org/10.3390/proceedings2019031070

Chicago/Turabian Style

Cabañero, Luis, Ramón Hervás, Iván González, Jesús Fontecha, Tania Mondéjar, and José Bravo. 2019. "Analysis of Cognitive Load Using EEG when Interacting with Mobile Devices" Proceedings 31, no. 1: 70. https://doi.org/10.3390/proceedings2019031070

APA Style

Cabañero, L., Hervás, R., González, I., Fontecha, J., Mondéjar, T., & Bravo, J. (2019). Analysis of Cognitive Load Using EEG when Interacting with Mobile Devices. Proceedings, 31(1), 70. https://doi.org/10.3390/proceedings2019031070

Article Metrics

Back to TopTop