Eating Event Recognition Using Accelerometer, Gyroscope, Piezoelectric, and Lung Volume Sensors
Abstract
:1. Introduction
1.1. Background
1.2. Related Work
- Inertial Measurement Unit
- Piezoelectric Sensor
- Respiratory Inductance Plethysmographic Sensor
1.3. Proposed Sensor System
- An accelerometer and gyroscope to detect eating gestures in the ingestion step.
- A piezoelectric sensor to detect chewing food in the food processing step.
- An RIP sensor to detect swallowing food in the deglutition phase.
2. Materials and Methods
2.1. Experimental Setup
2.2. Data Processing
2.3. Feature Vectors
2.4. Classification
3. Results
3.1. Sensor Combinations
3.2. Feature Vector Size
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
ADM | Automatic Dietary Monitoring |
RIP | Respiratory Inductance Plethysmography |
SVM | Support Vector Machine |
NBC | Normalised Breathing Cycles |
NNBC | Non-Normalised Breathing Cycles |
Appendix A. Features
Appendix A.1. Accelerometer and Gyroscope
- Mean
- Standard deviation
- Median
- Correlation
- Energy
- Spectral entropy
- Number of zero crossings
- Mean time between zero crossings
- Standard deviation of time between zero crossings
- Mean magnitude of the gyroscope vector (gyroscope only)
- Mean magnitude of the derivative of the acceleration vector (accelerometer only)
- Covariance (accelerometer only)
- Coefficients of fourth order polynomial fit to each acceleration component with hamming window weighting (accelerometer only, five features per axis)
Appendix A.2. Piezoelectric Sensor
- Number of peaks
- Mean time difference between peaks
- Standard deviation of the time difference between peaks
- Number of zero crossings
- Mean time difference between zero crossings
- Minimum time difference between zero crossings
- Maximum time difference between zero crossings
- Median time difference between zero crossings
- Standard deviation of the time difference between zero crossings
- Shannon entropy of the time difference between zero crossings
- Ratio between the number of peaks and the number of zero crossings
- Standard deviation
- Minimum
- Maximum
- Mean
- Median
- Root mean square
- Ratio between maximum and root mean square
- Ratio between root mean square and mean
- Peak frequency of the fast Fourier transform
- Standard deviation of the fast Fourier transform
- Energy
- Spectral entropy
- Shannon entropy
- Shannon entropy of peaks
- Shannon entropy of derivative at zero crossings
- Shannon entropy of values nearest to zero crossings
- Fractal dimension
- Sample entropy
Appendix A.3. RIP Sensor
- Standard deviation
- Mean
- Energy
- Median
- Root mean square
- Ratio of the maximum to root mean square
- Number of peaks
- First ten fast Fourier coefficients (ten features, NBC only)
- Hist-60; the amplitude was divided into ten equal intervals, where each datapoint in the breathing cycles fell within one of the intervals, resulting in x values per interval; x was then summed from the bottom up until 60% of the values were included. The value for this feature was the number of the interval at which this 60% was reached. (NBC only)
- First value (NNBC only)
- Last value (NNBC only)
- Maximum (NNBC only)
- Minimum of inhalation (NNBC only)
- Minimum of exhalation (NNBC only)
- Ratio of maximum to minimum inhalation (NNBC only)
- Ratio of maximum to minimum exhalation (NNBC only)
- Ratio of minimum to maximum exhalation (NNBC only)
- Ratio of minimum exhalation to minimum inhalation (NNBC only)
- Ratio of minimum to maximum inhalation (NNBC only)
- Ratio of minimum inhalation to minimum exhalation (NNBC only)
- Duration (NNBC only)
- Breathing frequency (=1/duration, NNBC only)
- Duration of inhalation (NNBC only)
- Duration of exhalation (NNBC only)
- Ratio of duration of inhalation to duration of exhalation (NNBC only)
- Ratio of duration of exhalation to duration of inhalation (NNBC only)
- Absolute breathing cycle duration (NNBC only)
- Ratio between the duration and average duration of the two neighbouring breathing cycles (NNBC only)
References
- Maringer, M.; van’t Veer, P.; Klepacz, N.; Verain, M.C.; Normann, A.; Ekman, S.; Timotijevic, L.; Raats, M.M.; Geelen, A. User-documented food consumption data from publicly available apps: An analysis of opportunities and challenges for nutrition research. Nutr. J. 2018, 17, 59. [Google Scholar] [CrossRef] [PubMed]
- Obesity and Overweight—World Health Organization. Available online: https://www.who.int/news-room/fact-sheets/detail/obesity-and-overweight (accessed on 22 October 2020).
- Apovian, C.M. Obesity: Definition, comorbidities, causes, and burden. Am. J. Manag. Care 2016, 22, s176–s185. [Google Scholar] [PubMed]
- Martin-Rodriguez, E.; Guillen-Grima, F.; Martí, A.; Brugos-Larumbe, A. Comorbidity associated with obesity in a large population: The APNA study. Obes. Res. Clin. Pract. 2015, 9, 435–447. [Google Scholar] [CrossRef]
- Haslam, D.W.; James, W.P.T. Obesity. Lancet 2005, 366, 1197–1209. [Google Scholar] [CrossRef] [PubMed]
- Pi-Sunyer, X. The medical risks of obesity. Postgrad. Med. 2009, 121, 21–33. [Google Scholar] [CrossRef] [PubMed]
- Lung, T.; Jan, S.; Tan, E.J.; Killedar, A.; Hayes, A. Impact of overweight, obesity and severe obesity on life expectancy of Australian adults. Int. J. Obes. 2019, 43, 782–789. [Google Scholar] [CrossRef] [PubMed]
- Barness, L.A.; Opitz, J.M.; Gilbert-Barness, E. Obesity: Genetic, molecular, and environmental aspects. Am. J. Med. Genet. A 2007, 143, 3016–3034. [Google Scholar] [CrossRef]
- Wing, R.R.; Phelan, S. Long-term weight loss maintenance. Am. J. Clin. Nutr. 2005, 82, 222S–225S. [Google Scholar] [CrossRef] [PubMed]
- Freire, R. Scientific evidence of diets for weight loss: Different macronutrient composition, intermittent fasting, and popular diets. Nutrition 2020, 69, 110549. [Google Scholar] [CrossRef] [PubMed]
- Fontana, J.M.; Sazonov, E.S. A robust classification scheme for detection of food intake through non-invasive monitoring of chewing. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, San Diego, CA, USA, 28 August–1 September 2012; pp. 4891–4894. [Google Scholar] [CrossRef]
- Todd, K.; Hudes, M.; Calloway, D.H. Food intake measurement: Problems and approaches. Am. J. Clin. Nutr. 1983, 37, 139–146. [Google Scholar] [CrossRef] [PubMed]
- Rumpler, W.V.; Kramer, M.; Rhodes, D.G.; Moshfegh, A.J.; Paul, D.R. Identifying sources of reporting error using measured food intake. Eur. J. Clin. Nutr. 2008, 62, 544–552. [Google Scholar] [CrossRef] [PubMed]
- Meyers, A.; Johnston, N.; Rathod, V.; Korattikara, A.; Gorban, A.; Silberman, N.; Guadarrama, S.; Papandreou, G.; Huang, J.; Murphy, K.P. Im2Calories: Towards an automated mobile vision food diary. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 1233–1241. [Google Scholar]
- Schiboni, G.; Amft, O. Automatic dietary monitoring using wearable accessories. In Seamless Healthcare Monitoring; Springer: Berlin/Heidelberg, Germany, 2017; pp. 369–412. [Google Scholar] [CrossRef]
- Amft, O.; Tröster, G. On-body sensing solutions for automatic dietary monitoring. IEEE Pervasive Comput. 2009, 8, 62–70. [Google Scholar] [CrossRef]
- Fortuna, C.; Giraud-Carrier, C.; West, J. Hand-to-Mouth Motion Tracking in Free-Living Conditions for Improved Weight Control. In Proceedings of the 2016 IEEE International Conference on Healthcare Informatics (ICHI), Chicago, IL, USA, 4–7 October 2016; pp. 341–348. [Google Scholar] [CrossRef]
- Fallmann, S.; Kropf, J. Human Activity Recognition of Continuous Data Using Hidden Markov Models and the Aspect of Including Discrete Data. In Proceedings of the 2016 International IEEE Conference UIC, ATC, ScalCom, CBDCom, IoP and SmartWorld, Toulouse, France, 18–21 July 2016; pp. 121–126. [Google Scholar] [CrossRef]
- Kyritsis, K.; Tatli, C.L.; Diou, C.; Delopoulos, A. Automated analysis of in meal eating behavior using a commercial wristband IMU sensor. In Proceedings of the 39th International Conference IEEE EMBC, Jeju, Republic of Korea, 11–15 July 2017; pp. 2843–2846. [Google Scholar]
- Farooq, M.; Sazonov, E. A novel wearable device for food intake and physical activity recognition. Sensors 2016, 16, 1067. [Google Scholar] [CrossRef] [PubMed]
- Tatulli, E.; Fontecave-Jallon, J.; Calabrese, P.; Gumery, P.Y. Respiratory Inductance Plethysmography for Automated Swallowing Detection. IJEHMC 2020, 11, 64–77. [Google Scholar] [CrossRef]
- Dong, B.; Biswas, S. Analyzing Breathing Signals and Swallow Sequence Locality for Solid Food Intake Monitoring. J. Med. Biol. Eng. 2016, 36, 765–775. [Google Scholar] [CrossRef] [PubMed]
- Sazonov, E.S.; Schuckers, S.; Lopez-Meyer, P.; Makeyev, O.; Sazonova, N.; Melanson, E.L.; Neuman, M. Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. Physiol. Meas. 2008, 29, 525. [Google Scholar] [CrossRef] [PubMed]
- Sazonov, E.S.; Fontana, J.M. A sensor system for automatic detection of food intake through non-invasive monitoring of chewing. IEEE Sens. J. 2011, 12, 1340–1348. [Google Scholar] [CrossRef]
- Merck, C.; Maher, C.; Mirtchouk, M.; Zheng, M.; Huang, Y.; Kleinberg, S. Multimodality sensing for eating recognition. In Proceedings of the PervasiveHealth: Pervasive Computing Technologies for Healthcare, Cancun, Mexico, 16–19 May 2016; pp. 130–137. [Google Scholar] [CrossRef]
- Dong, B.; Biswas, S. Meal-time and duration monitoring using wearable sensors. Biomed. Signal Process. Control 2017, 32, 97–109. [Google Scholar] [CrossRef]
Subjects | n = 6 |
Male (%) | 50 |
Age (y) | 23.5 ± 1.9 |
Weight (kg) | 76.7 ± 9.6 |
Heigth (cm) | 179.3 ± 10.5 |
Action | Duration |
---|---|
1. Breathe in and out to minimal and maximal lung volume | 20 s. |
2. Eat a bowl of yoghurt with the right arm. The left hand can be used to hold the bowl in place. | ~5 min. |
3. Move the right arm in a random fashion above the table | 1 min. |
4. Sit still + scratch back of head every 10 s | 2 min. |
5. Read out loud from a book | 3 min. |
6. Sit still + scratch back of head every 10 s | 2 min. |
7. Eat a croissant with the right hand | ~3 min. |
8. Move the right arm in a random fashion above the table | 1 min. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mevissen, S.J.; Klaassen, R.; van Beijnum, B.-J.F.; Haarman, J.A.M. Eating Event Recognition Using Accelerometer, Gyroscope, Piezoelectric, and Lung Volume Sensors. Sensors 2024, 24, 571. https://doi.org/10.3390/s24020571
Mevissen SJ, Klaassen R, van Beijnum B-JF, Haarman JAM. Eating Event Recognition Using Accelerometer, Gyroscope, Piezoelectric, and Lung Volume Sensors. Sensors. 2024; 24(2):571. https://doi.org/10.3390/s24020571
Chicago/Turabian StyleMevissen, Sigert J., Randy Klaassen, Bert-Jan F. van Beijnum, and Juliet A. M. Haarman. 2024. "Eating Event Recognition Using Accelerometer, Gyroscope, Piezoelectric, and Lung Volume Sensors" Sensors 24, no. 2: 571. https://doi.org/10.3390/s24020571
APA StyleMevissen, S. J., Klaassen, R., van Beijnum, B.-J. F., & Haarman, J. A. M. (2024). Eating Event Recognition Using Accelerometer, Gyroscope, Piezoelectric, and Lung Volume Sensors. Sensors, 24(2), 571. https://doi.org/10.3390/s24020571