- freely available
Sensors 2012, 12(5), 6075-6101; doi:10.3390/s120506075
Published: 10 May 2012
Abstract: Sometimes, one needs to control different emotional situations which can lead the person suffering them to dangerous situations, in both the medium and short term. There are studies which indicate that stress increases the risk of cardiac problems. In this study we have designed and built a stress sensor based on Galvanic Skin Response (GSR), and controlled by ZigBee. In order to check the device's performance, we have used 16 adults (eight women and eight men) who completed different tests requiring a certain degree of effort, such as mathematical operations or breathing deeply. On completion, we appreciated that GSR is able to detect the different states of each user with a success rate of 76.56%. In the future, we plan to create an algorithm which is able to differentiate between each state.
Stress, better explained in , is a response to particular events. It is the way our body prepares itself to face a difficult situation with focus, strength and heightened alertness. When we perceive a threat, our nervous system responds by releasing a flood of stress hormones, including adrenaline and cortisol. These hormones rouse the body for emergency action. In some cases it is necessary to collect feedback in order to control this symptom because it can become dangerous in certain situations. Therefore, it is necessary to build a device to detect stress.
For this objective, we have designed a Galvanic Skin Response (GSR) device in order to detect the different conductance of the skin when a person is under stress or when not . It uses just two electrodes which are placed on the fingers and act as if they were the two terminals of one resistance [3,4].
This device sends different data to a coordinator via ZigBee and, at the same time, this coordinator will send the information to a computer. The final objective is to implement this GSR into an application which controls different medical devices, [5,6]. The Figure 1 shows the communication of the final application.
The user can use the stress sensor anywhere in his home provided he is at a distance of less than 10 meters . By using a wireless communication system, the user is provided with a certain degree of freedom when using the device. The final user could manage the different devices from his television and the control center could take different action so as to change a person's stress levels. Therefore, the coordination center could use different systems to help the person relax, such as turning down the lights or changing the kind of music the user is listening to. There are two main reasons why we decided to work with ZigBee:
It's low power consumption.
It is possible to connect as many as 255 nodes.
This means of communication has been used before in healthcare applications, as can be seen in  and . In order to verify the stress sensor, we carried out different trials with 16 adult subjects. The idea was to establish one threshold for each person because there are people who are more nervous than others, so there could be cases where the results are not reliable. Despite this, there exist studies which have obtained good results establishing the same limit for all the subjects .
Therefore, there are two parts: the hardware design of the GSR and the algorithm which detects the emotional state of the user. This first study focuses on the hardware part, so the trials were done in order to verify that the device detects different changes in the person's condition, more specifically whether it detects an effort being made.
The output voltage of the designed circuit is connected to the ADC of a ZigBee board. There are two ZigBee boards: one for acquiring the data, and a second one to send it to the computer. This second board (the coordinator) also receives information about other devices and functions of the domotic application.
This paper is divided into the following sections: first, the paper describes the state of the art. Secondly, a complete methodology with all the technologies involved in this system is described, and then the system design is explained, as well as the results obtained during the tests carried out on the system. This document ends with the conclusions and the discussions arising from the topic.
2. State of the Art
There exist different studies which try to detect people's emotional states, including attempts to find out whether someone is suffering from stress or is not. Studies  and  use EEG to classify the different data acquired by brain activity. They extract different frequency features of the signals for their posterior classification with good results.
Heart Rate Variability is another parameter used to measure stress levels . To induce stress the authors in  propose using hyperventilation and talk preparation. Then, they present a method based on fuzzy logic to analyze the different data from HR and GSR. An ambulatory device is developed in  in order to evaluate stress in blind people. This device also includes the measurement of skin temperature, which is another parameter used to analyze stress .
As regards Galvanic Skin Response (GSR), there are several studies which propose different methods of detecting stress levels by measuring skin conductance . The study described in  has the objective of detecting sweat levels for the diagnosis of sudomotor dysfunction, something that can help in the diagnosis of diabetes. There are other medical applications based on skin conductance, such as epilepsy control: sweaty hands may be a warning signal of an epileptic attack ; or , as support of the diagnosis and treatment of bipolar disorder patients.
By combining the sweat of the hands with the temperature of the skin, it is possible to develop a truth meter ; as when the person is lying, his hands are colder and skin resistance is lower. In this case it is not necessary to include an ADC because the variation of skin resistance happens at odd times so, with different resistances and transistors, it is possible to build a lie detector.
In , different videos are shown to the participants in order to induce different emotions. The data acquired by GSR are classified by Immune Particle Swarm Optimization, obtaining a high average when classifying different emotions from the conductance of the skin. The study presented in  shows a method based on Support Vector Regression for recognizing emotions by combining different devices.
Continuing with the differentiation of the emotional state, literature includes other studies like , where different devices are combined so that, by means of Cross-Correlation and Fisher, it is possible to distinguish six different kinds of emotions.
In , applying a method based on Principal Component Analysis (PCA) to reduce the dimension of the GSR data is proposed, saving as much information as possible. The devices described above are also used in the analysis of different bio signals .
In order to develop GSR, it is necessary to use a mechanism to send the data via ZigBee as well as the corresponding algorithm to determine the stress level in accordance with the different tests. The different methods used in this first study can be seen below:
We use Jennic JN-5148 boards (Figure 2) for data acquisition and its subsequent submission to the computer. These were chosen because of their ease of implementing a communication protocol between the coordinator board and the sensor board and because they are part of the ZigBee Alliance. Other devices like those used in  were not appropriate for developing a global domotic system. The resolution of the Analogic to Digital Converser is appropriate for the needs of the device.
The output signal of the (Vo) device is connected to pin 34 of the sensor board, while the reference signal is connected to number 40. Through the ZigBee communication protocol, the sensor board sends the data to the coordinator board, which, by means of a USB with a JWT terminal, sends the data to a computer. The weight and the size of these boards are quite small, so they can easily be implemented in the same kit as the device.
After doing some tests, it was seen that the Analogic Digital Converser saturates at 2.35 V. It is an ADC of 12 bits, so the resolution is:
The galvanic skin response oscillates between 10 kΩ and 10 MΩ [25,26], as it can be seen in existing studies about the skin conductance obtained from different applied voltages [27,28]. After initial contact with the subjects, we established an input tension of 1.8 V. We took measurements from our circuit, using different resistances that are within the range of skin resistance (Table 1). These values were chosen in order to know the theoretical behavior of the output voltage, depending on skin conductance. The different values of Rs (Table 1) are determined by the combination of different real resistances.
As the board's ADC has a resolution of 0.573 mV and the minimum tension is 136 mV, an operational amplifier does not have to be included. We can also observe that the differences between some resistance values and others are higher than the ADC resolution. There are GSR devices which use an amplifier before the ADC .
Different tests were conducted in order to verify the behavior of the GSR device:
In a calm state, trying to feel relaxed.
Trying to be nervous: at the second stage, we asked the subjects to think about something which makes him nervous or produces anxiety.
Taking in air and expelling it forcefully: the subject is relaxed and, after one minute, he is asked to take in air and expel it, trying to push himself as hard as possible. As the result of this, the nervous system indicates to the sweat glands that an effort has been made.
Mathematical operations: the subject is relaxed and, after one minute, the computer shows him different mathematical operations. The subject is asked to feed the different results into the computer.
Reading: after 90 seconds, the screen shows some words that the user has to read as fast as possible (Figure 3).
Another test where the computer shows different images to the subject was also used. Some of these pictures are “emotional” (they should affect the subject's emotional state) and the others are “neutral” (they have no influence on the subject). It was supposed that the emotional ones would provide a response, but, after trying with some subjects, we have not included this because the results were insignificant.
Table 2 shows the places where the different studies took place. The studies have been made at the subject's home and office because GSR is intended to work in daily situations.
In order to analyze and manipulate the different data, we use Matlab.
To verify the different tests, we used the WEKA learning machine for testing the following methods: Bayesian Network, J48 and Sequential Minimal Optimization (SMO). They were chosen because there are studies that have obtained good results with them . A Cross-Validation method was used to evaluate the different results.
4. System Design
We now present different diagrams showing the application's system design.
4.1. High Level Design
The main application performs the operations on the Figure 4.
The data acquisition and subsequent sending of information to the computer is done by two different boards, which are connected to each other via ZigBee (see Figure 5).
The device built performs the operations of the Figure 6:
The GSR electrodes collect the skin resistance and, after this, the device can determine the person's stress level.
4.2. Processing Stage
At this stage (Figure 7) it is necessary to develop an algorithm which must be able to differentiate between the stress levels.
We separate the data according to stress situations and relax situations.
4.3. Low Level Design
A person's skin acts as a resistance to the passage of electrical current. By placing two electrodes on the fingers, we can calculate the Galvanic Skin Response (GSR). To find out this value, we use one resistance, as it can be seen in Figure 8, in series with the skin resistance, to form a voltage divider.where Rs is the resistance of the skin.
It can be observed that the Vo output tension is inversely proportional to the value of the skin resistance. The more stressed the person is, the more his hands will sweat, so his resistance will decrease. Therefore, we can conclude that the more stress the person is under, the higher output voltage will be.
It also includes a low-pass filter made by a capacitor and a resistance to filter the high frequencies. The resulting circuit in Figure 9.
We will use a 1.5 V battery of as supply. The device, Figure 10, will have the following form:
We have conducted several tests in order to change the emotional state of the subjects. Knowing the moments when the person should be stressed and the ones where he should not, we can analyze each kind of data separately. We have used 16 subjects aged between 23 and 56 (eight women and eight men). The sample rate is 4 Hz. Once the data have been obtained, we have smoothed them with an average size-5 window.
All the users have done the following tests:
Reading as fast as possible
Table 3 shows the output voltage averages for the different tests in those cases where the differences are better appreciated.
Following it can be seen the answers to different situations. Figures 11, 16 and 17 show the variation of the output voltage when the users are reading; Figures 12 and 15 represents the effort doing mathematical operations and Figures 13, 14 and 18 are the answer of the user breathing deeply.
It has been appreciated that when the participants were asked to relax, there was a decrease in the output voltage for that period of time. Figures 19–22 show the decreasing of the signal when the user is relaxed.
The Table 4 shows whether changes have been appreciated in these four tests for each user.
With this data, we obtained an average success rate of 76.56%. The users who had done some trials beforehand (Users 1, 2, 3 and 12), were more successful than the rest. Additionally, there are users who were asked to think of something that makes them nervous for a later comparison (Table 5):
User 4 was a special case: after doing the stage where she had to think about something that makes her nervous, she had said that she could not do so. We then told her that the next stage was mathematical operations, something which made her nervous (Figure 25). Therefore, we decided repeat the acquisitions:
After drinking coffee, User 1 presents a higher output voltage.
While User 4 was relaxed, she became nervous, which is why an increment in output voltage can be appreciated.
Figure 28 represents the following three situations:
Upon arrival at the department, done fairly quickly (blue).
After a while, much more calmly (green).
In the afternoon, thinking about something she had to do, and that made her nervous (red).
In order to verify the different tests, we introduced the different data to WEKA machine learning, using BayesNet, J48 and SMO. There separated those measurements which are supposed to represent an effort from those where the user was relaxed. The data have been processed individually for each participant due to the fact that each one has got different thresholds. Cross-Validation was used to test the different classifiers. Below, in Table 6, we present the different results:
Confusion matrix is shown in Table 7:
Below is a comparative graph (Figure 35) with the results obtained from the different subjects, showing the averages when relaxed and when in situations requiring effort:
Table 8 contains the different voltages between relaxing situations and effort situations:
We did more trials for Users 2, 3, 4, 5 and 13 in order to determine whether they are stressed or not. In these trials they were asked to feel both relaxed and nervous. After talking to them, they concluded that most of them were not able to make themselves nervous (Figures 36–40). Because of this, the measurements were not treated as they were supposed to be, but as according to how the participant felt. The results are reflected in Tables 9–13.
Table 14 shows the classification average.
We have established the following limit to differentiate being relaxed from being nervous for an initial study:
We have separated the data in windows of 10 seconds, overlapping 5 seconds. If the average is higher than the limit, the result is 1, if is lower, the result is −1 (Figures 41–46). These are the results:
The main part of this study involved the design of a device which is able to detect skin resistance in different situations. It also includes an initial threshold between being stressed and being relaxed, but it is not the algorithm that is going to be implemented in the final application.
With the different graphs, it can be observed that signals increase or decrease depending on the effort or the mental situation of the user. User 4, User 2, and User 3 had done some previous tests before these results. This may be the explanation for why they present more clarity in their graphs.
The main problem is that, for the moment, we cannot differentiate being stressed from making an effort. This is clearly seen in User 13's last graph, where a laugh presents a similar response to feeling stressed. Apart from the reflected trials, we also collected data while the user was playing different games, such as Tetris or PacMan. However, significant results were not obtained, so they are not included in this study. There are other studies  which have used different games in longer tests, and they have obtained good results.
The GSR device detects whether there has been an effort or a different situation from being relaxed with a success rate of 90.97%. It has been observed that participants who had done some trials before obtained the highest difference; so the average could be higher if the user is familiarized with the device. The next stage is to design an algorithm in order to establish the threshold between different emotional situations because this first algorithm does not distinguish between being stressed or making an effort. Two tasks lying ahead of us are:
This research work is partly funded by the Basque Government Department of Education and Research, as well as the Department of Industry, Trade and Tourism.
- Cooper, C.L.; Dewe, P.J. Stress: A Brief History; Blackwell Publishing: Oxford, UK, 2004. [Google Scholar]
- Handri, S.; Nomura, S.; Kurosawa, Y.; Yajima, K.; Ogawa, N.; Fukumura, Y. User Evaluation of Student's Physiological Response Towards E-Learning Courses Material by Using GSR Sensor. Proceedings of 9th IEEE/ACIS International Conference on Computer and Information Science, Yamagata, Japan, 18–20 August 2010.
- Zhai, J.; Barreto, A. Stress Detection in Computer Users Based on Digital Signal Processing of Noninvasive Physiological Variables. Proceedings of IEEE 28th Annual International Conference on Engineering in Medicine and Biology Society, New York, NY, USA, 30 August–3 September 2006.
- Massot, B.; Baltenneck, N.; Gehin, C.; Dittmarm, A.; McAdams, E. EmoSense: An Ambulatory Device for the Assessment of ANS Activity—Application in the Objective User Evaluation of Stress with the Blind. Sensors J. 2011, 12, 543–551. [Google Scholar]
- Burns, A.; Greene, B.R.; McGrath, M.J.; O'Shea, T.J.; Kuris, B.; Ayer, S.M.; Stroiescu, F.; Cionca, V. SHIMMER—A Wireless Sensor Platform for Noninvasive Biomedical Research. Sensors J. 2010, 10, 1527–1534. [Google Scholar]
- Huang, T.; Xiong, J.; Xuan, L. Design and Implementation of a Wireless Healthcare System Based on Galvanic Skin Response. Commun. Comput. Inform. Sci. 2011, 225, 337–343. [Google Scholar]
- Farahani, S. ZigBee Wireless Networks and Transceivers; Newness: Oxford, UK, 2008. [Google Scholar]
- Zhao, Z.; Cui, L. EasiMed: A Remote Health Care Solution. Proceedings of 27th Annual International Conference of the Engineering in Medicine and Biology Society, Shanghai, China, 17–18 January 2006.
- Jung, J.Y.; Lee, J.W. ZigBee Device Access Control and Reliable Data Transmission in ZigBee Based Health Monitoring System. Proceedings of 10th International Conference on Advanced Communication Technology, Phoenix Park, Korea, 17–20 February 2008.
- Wu, G.; Liu, G.; Hao, M. The Analysis of Emotion Recognition from GSR Based on PSO. Proceedings of International Symposium on Intelligence Information Processing and Trusted Computing, Huanggang, China, 28–29 October 2010.
- Hosseini, S.A.; Khalilzadeh, M.A.; Branch, M. Emotional Stress Recognition System Using EEG and Psychophysiological Signals: Using New Labelling Process of EEG Signals in Emotional Stress State. Proceedings of International Conference on Biomedical Engineering and Computer Science, Wuhan, China, 23–25 April 2010.
- Khosrowabadi, R.; Quek, C.; Ang, K.K.; Tung, S.W.; Heijnen, M. A Brain-Computer Interface for classifying EEG Correlates of Chronic Mental Stress. Proceedings of International Joint Conference on Neural Networks (IJCNN), San Jose, CA, USA, 31 July–5 August 2011.
- Jongyoon, C.; Gutierrez-Osuna, R. Removal of Respiratory Influences from Heart Rate Variability in Stress Monitoring. IEEE Sensor J. 2011, 11, 2649–2656. [Google Scholar]
- Santos, A.; Sánchez, C.; Guerra, J.; Bailador del Pozo, G. A Stress-Detection System Based on Physiological Signals and Fuzzy Logic. IEEE Trans. Industr. Electrs. 2011, 58, 4857–4865. [Google Scholar]
- Yamakoshi, T.; Yamakoshi, K.; Tanaka, S.; Nogawa, M.; Park, S. B.; Shibata, M.; Sawada, Y.; Rolfe, P.; Hirose, Y. Feasibility Study on Driver's Stress Detection from Differential Skin Temperature Measurement. Proceedings of 30th Annual Conference of the IEEE in Engineering in Medicine and Biology Society, Vancouver, Canada, 20–25 August 2008.
- Jing, Z; Barreto, A.B.; Chin, C.; Chao, L. Realization of stress Detection Using Psychophysiological Signals for Improvement of Human-Computer Interactions. Proceedings of the IEEE Southeastcon, Fort Lauderdale, FL, USA, 8–10 April 2005.
- Khalfallah, K.; Ayoub, H.; Calvet, J. H.; Neveu, X.; Brunswick, P.; Griveau, S.; Lair, V.; Cassir, M.; Bedioui, F. Noninvasive Galvanic Skin Sensor for Early Diagnosis Of Sudomotor Dysfunction: Application to Diabetes. IEEE Sensor J. 2010, 12, 456–463. [Google Scholar]
- Takahashi, K. Epilepsy Research Progress; Nova Science Publishers: New York, NY, USA, 2008. [Google Scholar]
- Kappeler-Setz, C.; Gravenhorst, F.; Schumm, J.; Arnrich, B.; Tröster, G. Towards Long Term Monitoring of Electrodermal Activity in Daily Life. Proceedings of 5th International Workshop on Ubiquitous Health and Wellness, Copenhagen, Denmark, 26 September 2010.
- Li, J.; Zhuo, Q.; Wang, W. A Novel Approach to Analyze the Result of Polygraph. Proceedings of IEEE International Conference on Systems, Man and Cybernetics, Nashville, TN, USA, 8–11 October 2000.
- Chang, C.-Y.; Zheng, J.-Y.; Wang, C.-J. Based on Support Vector Regression for Emotion Recognition Using Pshysiological Signals. Proceedings of 2010 International Joint Conference on Neural Networks, Barcelona, Spain, 18–23 July 2010.
- Wen, W.H.; Qiu, Y. H.; Liu, G. Y.; Cheng, N.P.; Huang, X.T. Construction and Cross-Correlation Analysis of the Affective Physiological Response Database. Sci. China Inf. Sci. 2010, 53, 1774–1784. [Google Scholar]
- Tarvainen, M.P.; Karj, P.A.; Koistinen, A.S.; Valkonen-Korhonen, M.V. Principal Component Analysis of Galvanic Skin Responses. Proceedings of 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Piscataway, NJ, USA, 23–28 July 2000.
- Ranta-aho, P.O.; Tarvainen, M.T.; Koistinen, A.S.; Karj, P.A. Software Package for Bio-Signal Analysis. Proceedings of 23rd Annual International Conference on Engineering in Medicine and Biology Society, Istanbul, Turkey, 25–28 October 2001.
- Nabours, R.E.; Fish, R.M.; Hill, P.F. Electrical Injuries: Engineering, Medical and Legal Aspects, 2nd ed.; Lawyers & Judges Publishing Company, Inc.: Tucson, AZ, USA, 2004. [Google Scholar]
- Aguado, M.J. Resistencia de la piel al paso de la corriente eléctrica en adultos trabajadores; E-Prints Complutense. Biblioteca Universidad Complutense: Madrid, Spain, 2004. [Google Scholar]
- Kallman, R. Realities of Wrist Strap Monitoring Systems. Proceedings of Electrical Overstress-Electrostatic Discharge Symposium, Las Vegas, NV, USA, 27–29 September 1994.
- Andrews, H.L. Skin Resistance Changes and Measurements of Pain Threshold. Clin. Investig. J. 1943, 22, 517–520. [Google Scholar]
- Sun, F.; Kuo, C.; Cheng, H.; Buthpitiya, S.; Collins, P.; Griss, M. Activity-Aware Mental Stress Detection Using Physiological Sensors. Proceedings of 2nd International Conference on Mobile Computing, Applications and Services (MobiCASE), Santa Clara, CA, USA, 25–28 October 2010.
- Wu, S.; Lin, T. Exploring the Use of Physiology in Adaptive Game Design. Proceedings of International Conference on Consumer Electronics Communications and Networks, Xianning, China, 16–18 April 2011.
|Table 1. Different values of the resistances.|
|10 k||1.755 V|
|49.5 k||1.677 V|
|100 k||1.587 V|
|200 k||1.434 V|
|560 k||1.054 V|
|760 k||0.923 V|
|1 M||0.813 V|
|3.3 M||0.357 V|
|9.93 M||0.136 V|
|Table 2. Places of the tests.|
|User 1||Work||User 9||Home|
|User 2||Work||User 10||Home|
|User 3||Work||User 11||Home|
|User 4||Work||User 12||Work|
|User 5||Home||User 13||Home|
|User 6||Home||User 14||Work|
|User 7||Home||User 15||Work|
|User 8||Home||User 16||Home|
|Table 3. Average of different situations.|
|Relax(V)||Operations (V)||Breathing (V)||Reading (V)|
|User 4 (25)||1.4068||1.6945||1.6476||1.6712|
|User 5 (30)||1.1123||1.1383||1.1484||1.1426|
|User 6 (27)||0.876||1.0381||0.9567||1.0609|
|User 7 (24)||1.0011||1.0868||1.0786||1.1176|
|User 8 (26)||0.8238||0.9904||0.8864||0.9975|
|User 9 (26)||1.101||1.1145||1.1105||1.1439|
|User 10 (27)||1.060||1.1197||1.0695||1.1022|
|User 11 (55)||0.7096||0.7546||0.7840||0.8408|
|User 12 (28)||1.0529||1.0685||1.0893||1.0856|
|User 13 (23)||1.3699||1.5542||1.5599||1.6468|
|User 14 (30)||0.8238||0.9904||0.8864||0.9975|
|Table 4. Success of each test by subject.|
|Table 5. Differences between trying to be relaxed and trying to be nervous.|
|User 1 (26)||1.6118||1.7396|
|User 2 (26)||1.5535||1.5379|
|User 3 (24)||1.5576||1.6153|
|User 4 (25)||1.4068||1.3839|
|User 5 (30)||1.1123||1.1266|
|User 15 (26)||1.0902||1.1388|
|Table 6. Global results.|
|Relative absolute error||15.86%||16.91%||17.17%|
|Root relative squared error||26.77%||27.20%||41.61%|
|Table 7. Confusion matrix.|
|Table 8. Difference between being relaxed and effort situations.|
|Calm (V)||Effort (V)||Diference (%)|
|Table 9. Results of the different methods for User 2.|
|Trial 1: stressed|
|Relative absolute error||0.95%||0.00%||0.00%|
|Table 10. Results of the different methods for User 3.|
|Trial 1: stressed and relaxed|
|Relative absolute error||30.29%||43.61%||21.35%|
|Trial 2: relaxed|
|Relative absolute error||38.05%||42.40%||29.87%|
|Table 11. Results of the different methods for User 4.|
|Trial 1: stressed and relaxed|
|Relative absolute error||29.43%||40.73%||24.91%|
|Trial 2: relaxed|
|Relative absolute error||0.21%||7.84%||0.00%|
|Table 12. Results of the different methods for User 5.|
|Trial 1: relaxed|
|Relative absolute error||0.00%||0.00%||0.00%|
|Trial 2: relaxed|
|Relative absolute error||0.12%||0.00%||0.00%|
|Table 13. Results of the different methods for User 13.|
|Trial 1: stressed and relaxed|
|Relative absolute error||32.50%||35.11%||40.94%|
|Trial 2: relaxed|
|Relative absolute error||42.17%||45.61%||51.85%|
|Table 14. Classification average of the different methods.|
© 2012 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).