2.1. Assessment Methods
Cognitive resources are assets used by cognition to think, remember, make decisions, solve problems, or coordinate movements, such as perception, attention, short- and long-term memory, and motor control [27
]. According to Navon et al. [29
], these resources underlying human learning and information processing are limited [30
], in his multiple resource theory, suggests that these resources can be used in parallel for multiple tasks, using several resources at once. However, when task demand is high, the resources allocated to that task are not available for another task if the same mental resources are required at the same stage of processing. Excessive use, moreover, can cause a state of overload known as cognitive resource depletion [31
]. This overload means that the brain is unable to process new information, resulting in processing and/or execution errors [32
Mental workload results from the different levels of resource demand, depending on the parallel tasks that the person is performing [8
]. Excessive resource demand can cause distraction, increase errors, generate stress and frustration, and reduce the ability to undertake mental planning, problem solving, or decision-making [34
]. One example is the distraction caused by unwelcome advertisements on a Web page while the user is browsing. In this case, the intermingling of the browsing task with the intrusion of commercial advertisements forces the user to divide attention and allocate cognitive resources to the new stimulus.
Traditionally, mental workload has been assessed in different situations using subjective methods [10
] based on surveys, auto-perception scales, or think-aloud protocols [36
]. These methods are applied after the user has already finished the task, and the assessment of the mental workload depends of the user’s final perception [39
]. Therefore, these methods are constrained by the reporting bias introduced by relying on past memories and by the problem of ecological validity based on observing responses to hypothetical scenarios rather than behaviors in a real setting [40
]. In addition, the static nature of these methods makes them unfit for real-time evaluation. The most widespread example of this method is the NASA Task Load Index, which measures the mental and physical performance, as well as the effort and frustration, of the user [41
Performance-based methods have also been used, which measure indicators generated during task execution, such as the percentage of correct responses or execution time [3
]. In this method, the user needs to be engaged in only one task. Its major restriction is the difficulty of assessing mental workload in near real time.
The attempts to find objective indicators to measure mental workload in real time are based on collecting contextual information, which can be captured mainly using psychophysiological sensors [42
]. Indeed, there is ample empirical evidence in psychophysiology showing that some physiological responses are directly related to psychological factors such as stress, mental workload, and emotions [45
]. That is, there is a correlation between the physiological responses triggered by the nervous system and psychological stimuli.
Psychophysiological responses are controlled by the autonomic nervous system (ANS), which regulates and coordinates bodily processes such as digestion, temperature, blood pressure, and many aspects of emotional behavior [48
]. These actions occur independently of the conscious control of the individual. The ANS includes the sympathetic nervous system (SNS) and parasympathetic nervous system (PNS). The SNS controls actions required in emergency situations, such as stress and movement. It can cause heart rate acceleration, pupil dilation, and increased blood flow to the muscles, sweating, and muscle tension. The PNS controls the functions related to rest, repair, and relaxation of the body. The responses elicited by this system include a decrease in heart rate and blood pressure, stimulation of the digestive system, and pupillary contraction, among others [45
2.2. Psychophysiological Measurements
There are different types of methods to measure psychophysiological responses elicited complementarily by the SNS and PNS [49
]. For instance, the device for tracking gaze is the eye tracker. It consists of a camera typically positioned below the computer screen that works according to the “corneal-reflection/pupil-center” method, which consists of recording the centre of the pupil to identify the gaze position and recording the reflection of infrared lights [50
]. It also allows the measurement of the variation of the pupil diameter. Pupillography measures changes in pupil size, which can be attributed to both parasympathetic inhibition, which explains the first dilation phase, and sympathetic activation, which explains the subsequent contraction phase [51
]. Although pupil dilation can be triggered by a light reflex caused by changes in environment illumination or by a proximity or accommodation reflex to improve visual focus, it can also be caused by a psychosensory reflex associated with the cognitive or emotional engagement of the person while exposed to any sensory stimulus [53
]. In contrast to changes in the two previous reflexes, changes in pupil size in this case are subtler, so a high-precision device or eye tracker is required for their detection [54
Nevertheless, some shortcomings with pupillometrics need to be taken into account. For instance, response delays can reach up to 1000 ms, which may invalidate the work with short time windows; pupil variations can be due to multiple factors, such as exhaustion, stimulants and gaze shifts, among others. The eye tracker is also used for tracking the eye to determine gaze position or movements within a scene, including two relevant measurements:
Fixations: moments during which the gaze is relatively fixed or focused. They occur because sharp vision is only possible within a small area in the human eye called the fovea. It is useful to determine when eye fixation occurs because, in most cases, it coincides with attention.
Saccades: rapid eye movements or jumps from one fixation point to another. Saccades follow a pattern (or trajectory) depending on several factors: what is currently being looked at, visual target tracking, experience, and emotions.
Another set of psychophysiological measurements is obtained by electroencephalography. This is based on recording the electrical activity of the brain measured on the scalp. The device used is the EEG, which measures the voltage resulting from changes in ionic current flow within the neurons of the brain, produced by the brain’s synaptic activity. The EEG signal is a blend of different subjacent frequencies, which represents different cognitive or affective states. For its capture, it is used, among others, the 10–20 distribution of electrodes located on the skull (see Figure 1
). Each electrode is named with a letter and a number. The first refers to a specific region of the brain—frontal lobe (F), temporal lobe (T), center (C), occipital lobe (O)— while the second indicates its position. If this number is even, it represents the right side, if odd, the left side.
There are five major brain waves: delta (1–4 Hz), theta (4–8 Hz), alpha (8–12 Hz), beta (12–25 Hz), and gamma (approximately 25 Hz). The alpha band is suppressed during mental or bodily activities with open eyes. The suppression of the alpha band is a sign of mental activity and commitment to the task. This means that the brain is coordinating attention resources and focusing on the task. The alpha band is generated in the occipital, parietal and posterior temporal areas of the brain.
The theta band correlates with the difficulty of mental operations, for example during periods of focused attention or information gathering, processing and learning and during memory recall. It has been found that the frequency of the theta band becomes more prominent when the difficulty of the task increases. This band can be obtained from the whole cortex, which indicates that it is generated by a wide network that involves the prefrontal, central, parietal and temporal cortices.
There is evidence that the most relevant bands when it comes to distinguishing cognitive load are the alpha and theta bands in the parietal and frontal lobes, respectively, suppressing the first and increasing the second [10
In general, these bands are used limited to the EEG channels that correspond to frontal and parietal lobes (F3, F4, F7, F8, P7 and P8). However, in [57
] it is indicated that, although the oscillations of the alpha and theta bands reflect changes in cognitive load and memory performance, it is important to define the alpha and theta band for each subject starting of the peak frequency of its alpha band, named as the Individual Alpha Frequency. That is, the cutting frequencies are not the same for each person. Despite this, there is literature that uses the standard EEG bands to classify cognitive load with good results [58
EDA is a psychophysiological response that can be assessed by measuring changes in the electrical properties of the skin. Skin conductivity varies with changes in skin moisture (sweat) and may reveal changes in the SNS. EDA is also known as galvanic skin response (GSR), and it is inexpensive to assess, easily captured, and robust. It is measured by attaching one or two electrodes usually to the fingers or toes. It is an indicator of psychological and physiological arousal. When arousal increases, there is an increase in sweat gland activity, decreasing electrical resistance, and thus increasing conductivity. In addition, it serves to identify emotional states.
EDA has two components named tonic and phasic. The tonic component or base signal varies slowly, presenting slight changes in the scale of 10–100 s and sets basic skin conductance. The rise and decay of the signal changes constantly within the same subject, depending on its hydration, dry skin or autonomic regulation. This component can differ highly between subjects. The phasic component or conductive response of the skin is above the tonic component and shows significantly faster alterations. The signal is sensitive to specific emotional stimulus events, which induce peaks that occur between 1–5 s after the start of the stimulus.
The cardiovascular system is particularly interesting for psychophysiology because it is highly sensitive to neurological processes and psychological factors such as stress. It is regulated by the ANS, which produces patterns of electrical activity that are fundamental for psychophysiological measurements [45
]. Several studies associate changes in cardiac activity with psychological phenomena, such as mental work, perception, attention, problem solving, and signal detection [63
An ECG is used to measure the electrical activity of the heart, using at least three electrodes attached to the chest. The electrodes collect the necessary data with regard to the electric waves that describe the cardiac cycle, based on which the HR or its variation (HRV) are obtained.
The human body constantly exchanges heat with the environment as part of the process of self-regulation to maintain homeostasis (internal balance of the body). Body temperature increases and decreases in relation to the energy exchanged. The regulation of blood flow to the skin and thermal radiation is considered a function of the ANS [64
]. Studies conducted in this field, according to Genno et al. (1997) [65
], suggest that skin temperature has potential as a psychophysiological measure of the individual.