Next Article in Journal
Implementation of a Surface Electromyography-Based Upper Extremity Exoskeleton Controller Using Learning from Demonstration
Next Article in Special Issue
Self-Tuning Threshold Method for Real-Time Gait Phase Detection Based on Ground Contact Forces Using FSRs
Previous Article in Journal
Calibration of Elasto-Magnetic Sensors on In-Service Cable-Stayed Bridges for Stress Monitoring
Previous Article in Special Issue
Estimation of Foot Plantar Center of Pressure Trajectories with Low-Cost Instrumented Insoles Using an Individual-Specific Nonlinear Model
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Flexible Piezoelectric Sensor-Based Gait Recognition

1
Center for Robotics Research, Korea Institute of Science and Technology, Seoul 02792, Korea
2
School of Electrical Engineering, Korea University, Seoul 02841, Korea
*
Author to whom correspondence should be addressed.
Sensors 2018, 18(2), 468; https://doi.org/10.3390/s18020468
Submission received: 2 January 2018 / Revised: 1 February 2018 / Accepted: 3 February 2018 / Published: 5 February 2018
(This article belongs to the Special Issue Sensors for Gait, Posture, and Health Monitoring)

Abstract

:
Most motion recognition research has required tight-fitting suits for precise sensing. However, tight-suit systems have difficulty adapting to real applications, because people normally wear loose clothes. In this paper, we propose a gait recognition system with flexible piezoelectric sensors in loose clothing. The gait recognition system does not directly sense lower-body angles. It does, however, detect the transition between standing and walking. Specifically, we use the signals from the flexible sensors attached to the knee and hip parts on loose pants. We detect the periodic motion component using the discrete time Fourier series from the signal during walking. We adapt the gait detection method to a real-time patient motion and posture monitoring system. In the monitoring system, the gait recognition operates well. Finally, we test the gait recognition system with 10 subjects, for which the proposed system successfully detects walking with a success rate over 93 % .

1. Introduction

Recent advancements in wearable sensors have promoted major scientific and technological developments in the field of human activity recognition [1,2]. Wearable sensors, for this purpose, have significant benefits over external sensors in terms of privacy, pervasiveness, and complexity [1]. Most people would be averse to being continuously monitored by external cameras [1,3]. It is also difficult to obtain full body imagery all the time. Moreover, image processing to recognize human activities is computationally complex.
Gait recognition and analysis are one of the most important research areas, because walking is a common human physical activity [2,4]. The gait recognition system can be utilized for therapeutic and diagnostic applications [5,6]. In addition, the sensing part can be combined to gait assistant system [7,8]. Various wearable sensors, such as inertial measurement units (IMU), pressure sensors, force sensitive resistors, accelerometers, and gyroscopes, are utilized for gait recognition and analysis [2,4,9]. These sensors are positioned on hips, thighs, knees, shanks, and feet in the lower body; they measure angles and contacts during walking [4]. For example, a piezoelectric gyroscope on the thigh of one leg has been used for detecting stride length and walking velocity [10]. Piezoresistive accelerometers have been studied in ambulatory monitoring [11]. A real-time gait phase detection system with a gyroscope and force sensitive resistors installed inside a shoe insole has also been presented [12]. Bending type piezoelectric accelerometers have been utilized for footstep detection [13]. A method of measuring joint angle for gait using a combination of accelerometers and gyroscopes has been presented [14]. A step counting method, using tri-axial accelerometers on the ankles, thigh, and waist has been proposed [15]. In most research, wearable sensors have been tightly attached to the body for accurate sensing. However, tight attachment can cause inconvenience for users.
In this paper, we introduce a flexible piezoelectric sensor-based gait recognition system available in loose clothing. The flexible sensor is made of a piezoelectric material: polyvinylidene fluoride (PVDF) [16,17,18]. PVDF is one of the most flexible piezoelectric materials, outputting voltage signals when it experiences strain [19,20,21]. Four flexible piezoelectric sensors are attached to the knee and hip parts of the loose clothing. We measure the sensor outputs during human walking by analyzing the raw signals, signal processed values, and fast Fourier transform (FFT) magnitudes. Furthermore, we demonstrate a gait detection system based on the outputs of the flexible piezoelectric sensors. Specifically, we add the gait detection method to the real-time patient motion and posture monitoring system reported in [22]. The integration of the gait detection to the monitoring system presents that the proposed method can be combined as other posture detections or system. In addition, the monitoring system can have an important update about gait detection of patients through the integration. The monitoring system recognizes five postures including walking and eight transitions between postures. We show the operation of the monitoring system in a demo scenario. Additionally, we conduct user tests of gait detection in the monitoring system.
This paper is organized as follows. In Section 2, we introduce the experimental setup of the flexible piezoelectric sensor, the module for collecting the sensor’s signal, and walking test on a treadmill. In Section 3, we present an investigation of the sensor outputs during the walking motion. The decision method for gait recognition using the sensor outputs during walking is described in Section 4. In Section 5, we present the demonstration of gait recognition with a real-time patient motion and posture monitoring system. The conclusions are summarized in the final section.

2. Experimental Setup

We use similar patient cloth with flexible sensors as [22] for our experiments. In this work, the four sensors are embedded in both knee and hip parts on the patient’s pants (see Figure 1). The sensors are composed of PVDF, produced by Measurement Specialties, and Mylar [17,22]. A PVDF sheet is glued on a Mylar sheet with 3M DP460 epoxy. The sensor size is ( L ) 75 × ( W ) 25 × ( T ) 0.3 mm 3 . We sense electrical signals from the flexible sensors using conductive adhesive copper tape electrodes, produced by 3M, attached on both surface of PVDF. The copper electrodes are connected to a sensing module through wires. The voltages from the sensors, carried by the wires, are digitized using the analog-digital converters in the sensing module. The signals are then transmitted by Bluetooth wireless communication. The sensing frequency is approximately 100 Hz .
A KTS 7500TS treadmill is utilized to test walking at standard speed. In the tests, a male student (age: 25 years, height: 170 cm, weight: 70 kg) wears the patient clothes with flexible sensors and walks on the treadmill. The test walking speed varies from 0.5 to 6 km / h , which, if exceeded, requires running. When the student walks on the treadmill, we record the sensor outputs via the sensing system.

3. Sensor Output for Walking

We analyze the sensor outputs during walking. Figure 2 shows the voltage outputs from the sensors during walking at 4 km / h on the treadmill. The signal amplitudes at both knee sensors are bigger than those from the hip sensors because of several peaks. However, all signal wave forms of the hip sensors are more periodic than the knee sensors. The differences can be attributed to the loose patient cloth. During walking, the knee has a larger bending angle motion than the hip [23]. With tight clothes, the electric signal from the piezoelectric sensor attached to the knee will be larger than the signal at the hip [24]. With the loose clothing of this test, the knee sensors do not perfectly capture the periodic motions of the knees during walking. Conversely, the hip signals are sensed well during walking, because the waist band of the pants fits more tightly. Thus, the hip sensors are attached more closely than the knee sensors.
The sensor voltages correlate to the angular velocities of their bending [22]. We process data to acquire the bending motions from the sensor voltages, following the methods of [22]. Briefly, the sensor voltage outputs are processed through removing their voltage offsets and integrating as the time. We sense clearer periodic signals during walking (see Figure 3). Similarly, the processed values at the hip are more periodic than at the knee. To clarify the periodicity between the processed values at the hip and knee, we conduct FFT in MATLAB using the processed values. Figure 4 displays the FFT results from the signal processed values of Figure 3. The magnitudes of the main walking frequency in all FFT results are the largest. However, the results at the knee show that the higher harmonic frequencies still have big amplitudes.

4. Gait Recognition

We decide to use only left and right hip sensors to recognize walking motion, based on the findings in Section 3. We perform additional tests at various walking speeds ( 0.5 , 1, 2, 3, 4, 5 and 6 km / h ) on the treadmill. Figure 5 displays the FFT magnitude spectra at 0.5 , 1, 3 and 6 km / h . We can detect the main walking frequency by finding the biggest magnitude in each plot. We observe that the main walking frequencies increase with the walking speeds. The walking frequency as the speed of the treadmill is presented in Table 1. Herein, we focus on normal walking speed [25], which is approximately 1.1 m / s (∼ 4 km / h ). At that walking speed, the main frequency is between 0.9 and 1 Hz .
For detecting the walking frequency in real-time system, we use discrete time Fourier series instead of FFT to reduce computing time. In the same context, we obtain the series coefficients of the frequency, 1 Hz , and the second harmonic. Specifically, the coefficients are obtained every half second. When the average of the two coefficients is over a decision value, we make a decision about the gait recognition. Herein, we select the decision value of 0.8 from preliminary tests. When we use the gait recognition method with the decision value, the detection rate is over 90 % at 3– 6 km / h .

5. System Integration

We adapt the gait detection to the patient monitoring system of [22]. The patient monitoring system was developed for motion and posture recognitions of patients; the gait detection method is also programmed for the system. Figure 6 displays the test condition for the patient motion and posture monitoring system. Specifically, the flexible sensor data is collected and processed in an external computer for monitoring; the monitoring program shows the decision about the motion and posture of the patient. In the monitoring program, we can detect a total of eight transitions and five postures. The five postures are (i) walking, (ii) standing, (iii) sitting, (iv) sitting knee extension, and (v) supine. We also predict the motions during the transitions between the postures (see Figure 7). To avoid incorrect operations, the walking is only transited from and to the standing pose. We note that the total four sensors’ data are monitored in the system, the two hip sensors’ data are utilized for the gait detection, and the two left sensors’ data are used for capturing the transitions between standing, sitting, sitting knee extension, and supine. We comment that the system follows [22] about the detailed procedures for the decision between the postures of standing, sitting, sitting knee extension, and supine.
Figure 8 displays the captured computer screen image of the patient motion and posture monitoring system. At the left of the screen, it displays the processed values from the sensors of the left knee and hip and the right knee and hip. At the right of the screen, the detected patient posture is shown with a green outline. A demonstration of the patient motion and posture monitoring system is shown in Figure 9. Therein, the patient performs the transitions: (1) standing → (2) sitting on the bed → (3) sitting knee extension on the bed → (4) supine on the bed → (5) sitting knee extension on the bed → (6) sitting on the bed→ (7) standing up → (8) walking to the wheelchair → (9) stopping in front of the wheelchair → (10) sitting on the wheelchair→ (11) moving on the wheelchair → (12) standing up. All patient motion and posture monitoring of Figure 9 are displayed in Video S1 as Supplementary Material.
Additionally, we conduct gait detection tests from the other 10 users wearing the patient clothing. Specifically, the users walk and stop 50 times for a total of 1000 tests, 100 tests per user. We give the users simple orders about walking and stopping without specifying speed. Table 2 shows the success rates from the user test of the patient motion and posture monitoring system. The success rates are over 93 % , and the total average is 97.5 % . Several failed cases missed the changes or falsely detected sitting. In addition, most errors happen at taller users than the subject with the treadmill test. These errors can be attributed to the slow walking speeds of users and the low sensor outputs of the loose patient clothing. The success rate can be improved by selecting the gait detection frequency through personal statistics. For example, when we use the monitoring system for weak patients, they can have low walking speed. In this case, we need to adjust the gait detection frequency as lower value.

6. Conclusions

In this paper, we introduced a gait recognition system with flexible piezoelectric sensors in loose clothing. The flexible sensors were attached to the knee and hip parts on the pants. We analyzed the sensor outputs of a user during walking. Specifically, we investigated raw signals from the sensors, signal processed signals from integration, and FFT magnitude spectra. From the analysis, we found that the signals of the hip sensors are more suitable for gait detection. We demonstrated the gait detection method in a real-time patient motion and posture monitoring system. The monitoring system was utilized to detect five postures including walking and eight transitions between the postures. We showed the successful operation of the monitoring system in a demo video, which consisted of motions and postures around a bed and wheelchair. Additionally, our gait detection system was tested on 10 subjects, and the success rates were over 93 % . The successful performance of our system shows the feasibility of gait detection from loose clothes using flexible sensors. We anticipate that the same methodology can be utilized to explore gait recognition using alternative flexible sensors. Moreover, it can provide an insight into motion recognition using smart suits or textiles.

Supplementary Materials

The following are available online at https://www.mdpi.com/1424-8220/18/2/468/s1.

Acknowledgments

This work was supported by the Korea Institute of Science and Technology (KIST) flagship program “Connected Active Space for X” (Project No. 2E28250).

Author Contributions

Youngsu Cha led the experiment, performed data analysis, and drafted the manuscript. Hojoon Kim conducted the experiment. Doik Kim contributed to the setup of the experiment.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Lara, O.D.; Labrador, M.A. A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutor. 2013, 15, 1192–1209. [Google Scholar] [CrossRef]
  2. Sprager, S.; Juric, M.B. Inertial sensor-based gait recognition: A review. Sensors 2015, 15, 22089–22127. [Google Scholar] [CrossRef] [PubMed]
  3. Sathyanarayana, S.; Satzoda, R.K.; Sathyanarayana, S.; Thambipillai, S. Vision-based patient monitoring: A comprehensive review of algorithms and technologies. J. Ambient Intell. Humaniz. Comput. 2015, 1–27. [Google Scholar] [CrossRef]
  4. Rueterbories, J.; Spaich, E.G.; Larsen, B.; Andersen, O.K. Methods for gait event detection and analysis in ambulatory systems. Med. Eng. Phys. 2010, 32, 545–552. [Google Scholar] [CrossRef] [PubMed]
  5. Moore, S.T.; MacDougall, H.G.; Gracies, J.; Cohen, H.S.; Ondo, W.G. Long-term monitoring of gait in Parkinson’s disease. Gait Posture 2007, 26, 200–207. [Google Scholar] [CrossRef] [PubMed]
  6. Bae, J.; Kong, K.; Byl, N.; Tomizuka, M. A mobile gait monitoring system for abnormal gait diagnosis and rehabilitation: A pilot study for Parkinson disease patients. J. Biomech. Eng. 2011, 133, 041005. [Google Scholar] [PubMed]
  7. Kloos, A.D.; Kegelmeyer, D.A.; White, S.E.; Kostyk, S.K. The impact of different types of assistive devices on gait measures and safety in Huntington’s disease. PloS ONE 2012, 7, e30903. [Google Scholar] [CrossRef] [PubMed]
  8. Quinlivan, B.T.; Lee, S.; Malcolm, P.; Rossi, D.M.; Grimmer, M.; Siviy, C.; Karavas, N.; Wagner, D.; Asbeck, A.; Galiana, I.; et al. Assistance magnitude versus metabolic cost reductions for a tethered multiarticular soft exosuit. Sci. Robot. 2017, 2, eaah4416. [Google Scholar] [CrossRef]
  9. Kavanagh, J.J.; Menz, H.B. Accelerometry: A technique for quantifying movement patterns during walking. Gait Posture 2008, 28, 1–15. [Google Scholar] [CrossRef] [PubMed]
  10. Miyazaki, S. Long-term unrestrained measurement of stride length and walking velocity utilizing a piezoelectric gyroscope. IEEE Trans. Biomed. Eng. 1997, 44, 753–759. [Google Scholar] [CrossRef] [PubMed]
  11. Foerster, F.; Smeja, M.; Fahrenberg, J. Detection of posture and motion by accelerometry: A validation study in ambulatory monitoring. Comput. Hum. Behav. 1999, 15, 571–583. [Google Scholar] [CrossRef]
  12. Pappas, I.P.I.; Keller, T.; Mangold, S.; Popovic, M.R.; Dietz, V.; Morari, M. A reliable gyroscope-based gait-phase detection sensor embedded in a shoe insole. IEEE Sens. J. 2004, 4, 268–274. [Google Scholar] [CrossRef]
  13. Mazarakis, G.P.; Avaritsiotis, J.N. A prototype sensor node for footstep detection. In Proceedings of the Second European Workshop on Wireless Sensor Networks, Istanbul, Turkey, 2 February 2005; pp. 415–418. [Google Scholar]
  14. Dejnabadi, H.; Jolles, B.M.; Aminian, K. A new approach to accurate measurement of uniaxial joint angles based on a combination of accelerometers and gyroscopes. IEEE Trans. Biomed. Eng. 2005, 52, 1478–1484. [Google Scholar] [CrossRef] [PubMed]
  15. Fortune, E.; Lugade, V.; Morrow, M.; Kaufman, K. Validity of using tri-axial accelerometers to measure human movement-Part II: Step counts at a wide range of gait velocities. Med. Eng. Phys. 2014, 36, 659–669. [Google Scholar] [CrossRef] [PubMed]
  16. Broadhurst, M.G.; Davis, G.T.; McKinney, J.E.; Collins, R.E. Piezoelectricity and pyroelectricity in polyvinylidene fluoride-A model. J. Appl. Phys. 1978, 49, 4992–4997. [Google Scholar] [CrossRef]
  17. Akaydin, H.D.; Elvin, N.; Andreopoulos, Y. Energy Harvesting from Highly Unsteady Fluid Flows using Piezoelectric Materials. J. Intell. Mater. Syst. Struct. 2010, 21, 1263–1278. [Google Scholar] [CrossRef]
  18. Cha, Y.; Hong, J.; Lee, J.; Park, J.M.; Kim, K. Flexible Piezoelectric Energy Harvesting from Mouse Click Motions. Sensors 2016, 16, 1045. [Google Scholar] [CrossRef] [PubMed]
  19. Shen, D. Piezoelectric Energy Harvesting Devices for Low Frequency Vibration Applications; ProQuest: Ann Arbor, MI, USA, 2009. [Google Scholar]
  20. Farinholt, K.M.; Pedrazas, N.A.; Schluneker, D.M.; Burt, D.W.; Farrar, C.R. An Energy Harvesting Comparison of Piezoelectric and Ionically Conductive Polymers. J. Intell. Mater. Syst. Struct. 2009, 20, 633–642. [Google Scholar] [CrossRef]
  21. Tiwari, R.; Kim, K.J. IPMC as a mechanoelectric energy harvester: tailored properties. Smart Mater. Struct. 2013, 22, 015017. [Google Scholar] [CrossRef]
  22. Cha, Y.; Nam, K.; Kim, D. Patient Posture Monitoring System Based on Flexible Sensors. Sensors 2017, 17, 584. [Google Scholar] [CrossRef] [PubMed]
  23. Riemer, R.; Shapiro, A. Biomechanical energy harvesting from human motion: theory, state of the art, design guidelines, and future directions. J. Neuroeng. Rehabilit. 2011, 8, 22. [Google Scholar] [CrossRef] [PubMed]
  24. Proto, A.; Penhaker, M.; Bibbo, D.; Vala, D.; Conforto, S.; Schmid, M. Measurements of Generated Energy/Electrical Quantities from Locomotion Activities Using Piezoelectric Wearable Sensors for Body Motion Energy Harvesting. Sensors 2016, 16, 524. [Google Scholar] [CrossRef] [PubMed]
  25. Liu, M.Q.; Anderson, F.C.; Schwartz, M.H.; Delp, S.L. Muscle contributions to support and progression over a range of walking speeds. J. Biomech. 2008, 41, 3243–3252. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Patient clothing for four flexible sensors.
Figure 1. Patient clothing for four flexible sensors.
Sensors 18 00468 g001
Figure 2. Sensor voltage outputs during walking at the (a) left hip; (b) right hip; (c) left knee; and (d) right knee. The test walking velocity is 4 km / h .
Figure 2. Sensor voltage outputs during walking at the (a) left hip; (b) right hip; (c) left knee; and (d) right knee. The test walking velocity is 4 km / h .
Sensors 18 00468 g002
Figure 3. Signal processed values from the sensor outputs during walking at the (a) left hip; (b) right hip; (c) left knee; and (d) right knee. The test walking velocity is 4 km / h .
Figure 3. Signal processed values from the sensor outputs during walking at the (a) left hip; (b) right hip; (c) left knee; and (d) right knee. The test walking velocity is 4 km / h .
Sensors 18 00468 g003
Figure 4. FFT magnitude spectra of the signal processed values from the sensor outputs during walking at the (a) left hip; (b) right hip; (c) left knee; and (d) right knee. The test walking velocity is 4 km / h .
Figure 4. FFT magnitude spectra of the signal processed values from the sensor outputs during walking at the (a) left hip; (b) right hip; (c) left knee; and (d) right knee. The test walking velocity is 4 km / h .
Sensors 18 00468 g004
Figure 5. FFT magnitude spectra during walking of (a) left and (b) right hip at 0.5 km / h , (c) left and (d) right hip at 1 km / h , (e) left and (f) right hip at 3 km / h , and (g) left and (h) right hip at 6 km / h .
Figure 5. FFT magnitude spectra during walking of (a) left and (b) right hip at 0.5 km / h , (c) left and (d) right hip at 1 km / h , (e) left and (f) right hip at 3 km / h , and (g) left and (h) right hip at 6 km / h .
Sensors 18 00468 g005
Figure 6. Experimental setup for testing the patient motion and posture monitoring system.
Figure 6. Experimental setup for testing the patient motion and posture monitoring system.
Sensors 18 00468 g006
Figure 7. Flow chart for motion and posture decisions.
Figure 7. Flow chart for motion and posture decisions.
Sensors 18 00468 g007
Figure 8. Captured computer screen showing patient walking detection and signal processed values from the four sensor outputs.
Figure 8. Captured computer screen showing patient walking detection and signal processed values from the four sensor outputs.
Sensors 18 00468 g008
Figure 9. Test of the patient motion and posture monitoring system. (17) The patient lies down on the bed, and then stands up. (79) He moves to a wheelchair. (912) He sits on the wheelchair, moves via the wheelchair, and stands up from the wheelchair.
Figure 9. Test of the patient motion and posture monitoring system. (17) The patient lies down on the bed, and then stands up. (79) He moves to a wheelchair. (912) He sits on the wheelchair, moves via the wheelchair, and stands up from the wheelchair.
Sensors 18 00468 g009
Table 1. Walking speeds, frequencies, and detection rate on the treadmill.
Table 1. Walking speeds, frequencies, and detection rate on the treadmill.
Speed [km/h]Frequency [Hz]Detection Rate [%]
0.50.2519
10.4136
20.6169
30.7998
40.92100
51.04100
61.14100
Table 2. Success rate of the gait recognition in the patient motion and posture monitoring system.
Table 2. Success rate of the gait recognition in the patient motion and posture monitoring system.
Subject NumberGenderAge [Years]Weight [kg]Height [cm]Success Rate [%]
#1M256017496
#2F295116196
#3M2670175100
#4M256317395
#5M3377166100
#6M2664170100
#7M257017793
#8M3372173100
#9M286817895
#10F2848159100

Share and Cite

MDPI and ACS Style

Cha, Y.; Kim, H.; Kim, D. Flexible Piezoelectric Sensor-Based Gait Recognition. Sensors 2018, 18, 468. https://doi.org/10.3390/s18020468

AMA Style

Cha Y, Kim H, Kim D. Flexible Piezoelectric Sensor-Based Gait Recognition. Sensors. 2018; 18(2):468. https://doi.org/10.3390/s18020468

Chicago/Turabian Style

Cha, Youngsu, Hojoon Kim, and Doik Kim. 2018. "Flexible Piezoelectric Sensor-Based Gait Recognition" Sensors 18, no. 2: 468. https://doi.org/10.3390/s18020468

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop