Machine Learning for Touch Localization on an Ultrasonic Lamb Wave Touchscreen
Abstract
:1. Introduction
2. Hardware Setup
3. Dataset
4. Localization
4.1. Classification Approach
4.2. Regression
4.3. Results
5. Applications
5.1. Touch Localization on a Virtual Keypad Interface
5.2. Localization of a Human Finger
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
References
- Walker, G. A review of technologies for sensing contact location on the surface of a display. J. Soc. Inf. Disp. 2012, 20, 413–440. [Google Scholar] [CrossRef]
- Dai, J.; Chung, C. Touchscreen everywhere: On transferring a normal planar surface to a touch-sensitive display. IEEE Trans. Cybern. 2013, 44, 1383–1396. [Google Scholar] [CrossRef]
- Bhalla, M.; Bhalla, A. Comparative study of various touchscreen technologies. Int. J. Comput. Appl. 2010, 6, 12–18. [Google Scholar] [CrossRef]
- Quaegebeur, N.; Masson, P.; Beaudet, N.; Sarret, P. Touchscreen surface based on interaction of ultrasonic guided waves with a contact impedance. IEEE Sens. J. 2016, 16, 3564–3571. [Google Scholar] [CrossRef]
- Yang, Z.; Liu, X.; Wu, B.; Liu, R. Adaptability of Ultrasonic Lamb Wave Touchscreen to the Variations in Touch Force and Touch Area. Sensors 2021, 21, 1736. [Google Scholar] [CrossRef]
- Adler, R.; Desmares, P. An economical touch panel using SAW absorption. IEEE Trans. Ultrason. Ferroelectr. Addit. Freq. Control. 1987, 34, 195–201. [Google Scholar] [CrossRef]
- Son, K.; Lee, C. Design and reliability of acoustic wedge transducer assemblies for outdoor touch panels. IEEE Trans. Compon. Packag. Addit. Manuf. Technol. 2011, 1, 1178–1185. [Google Scholar] [CrossRef]
- Ing, R.; Quieffin, N.; Catheline, S.; Fink, M. In solid localization of finger impacts using acoustic time-reversal process. Appl. Phys. Lett. 2005, 87, 204104. [Google Scholar] [CrossRef]
- Liu, Y.; Nikolovski, J.; Mechbal, N.; Hafez, M.; Vergé, M. Tactile objects based on an amplitude disturbed diffraction pattern method. Appl. Phys. Lett. 2009, 95, 251904. [Google Scholar] [CrossRef] [Green Version]
- Liu, Y.; Nikolovski, J.; Mechbal, N.; Hafez, M.; Vergé, M. An acoustic multi-touch sensing method using amplitude disturbed ultrasonic wave diffraction patterns. Sens. Addit. Actuators A Phys. 2010, 162, 394–399. [Google Scholar] [CrossRef] [Green Version]
- Pham, D.; Ji, Z.; Yang, M.; Wang, Z.; Al-Kutubi, M. A novel human-computer interface based on passive acoustic localisation. In Proceedings of the International Conference On Human-Computer Interaction, Beijing, China, 22–27 July 2007; pp. 901–909. [Google Scholar]
- Firouzi, K.; Nikoozadeh, A.; Khuri-Yakub, B. Numerical modeling of ultrasonic touchscreen. In Proceedings of the IEEE International Ultrasonics Symposium, Chicago, IL USA, 3–6 September 2014; pp. 753–756. [Google Scholar]
- Firouzi, K.; Nikoozadeh, A.; Carver, T.; Khuri-Yakub, B. Lamb wave multitouch ultrasonic touchscreen. IEEE Trans. Ultrason. Ferroelectr. Addit. Freq. Control 2016, 63, 2174–2186. [Google Scholar] [CrossRef] [PubMed]
- Campbell, C. Surface Acoustic Wave Devices and Their Signal Processing Applications; Elsevier: Amsterdam, The Netherlands, 2012. [Google Scholar]
- Reis, S.; Correia, V.; Martins, M.; Barbosa, G.; Sousa, R.; Minas, G.; Lanceros-Mendez, S.; Rocha, J. Touchscreen based on acoustic pulse recognition with piezoelectric polymer sensors. In Proceedings of the IEEE International Symposium On Industrial Electronics, Bari, Italy, 4–7 July 2010; pp. 516–520. [Google Scholar]
- Reju, V.; Khong, A.; Sulaiman, A. Localization of taps on solid surfaces for human-computer touch interfaces. IEEE Trans. Multimed. 2013, 15, 1365–1376. [Google Scholar] [CrossRef]
- North, K.; D Souza, H. Acoustic pulse recognition enters touch-screen market. Inf. Disp. 2006, 22, 22. [Google Scholar]
- Ing, R.; Cassereau, D.; Fink, M.; Nikolovski, J. Tactile touch plate with variable boundary conditions. J. Acoust. Soc. Am. 2008, 123, 4225–4229. [Google Scholar] [CrossRef] [Green Version]
- Masson, P.; Quaegebeur, N.; Ostiguy, P.; Beaudet, N.; Sarret, P. Active Acoustic Pressure Mapping System. U.S. Patent 9,750,451, 5 September 2017. [Google Scholar]
- Firouzi, K.; Khuri-Yakub, B. A learning method for localizing objects in reverberant domains with limited measurements. J. Acoust. Soc. Am. 2017, 141, 104–115. [Google Scholar] [CrossRef]
- Chang, C.; Lee, Y. Ultrasonic Touch Sensing System Based on Lamb Waves and Convolutional Neural Network. Sensors 2020, 20, 2619. [Google Scholar] [CrossRef]
- Li, C. Using Ray Tracking and Machine Learning to Localize a Lamb Wave Scatterer on a Plate; Hong Kong University of Science: Hong Kong, China, 2020. [Google Scholar]
- Alleyne, D.; Cawley, P. Optimization of Lamb wave inspection techniques. NDT E Int. 1992, 25, 11–22. [Google Scholar] [CrossRef]
- Feng, B.; Pasadas, D.; Ribeiro, A.; Ramos, H. Locating defects in anisotropic CFRP plates using ToF-based probability matrix and neural networks. IEEE Trans. Instrum. Addit. Meas. 2019, 68, 1252–1260. [Google Scholar] [CrossRef]
- Xu, B.; Yu, L.; Giurgiutiu, V. Advanced methods for time-of-flight estimation with application to Lamb wave structural health monitoring. In Proceedings of the International Workshop On Structural Health Monitoring, Stanford, CA, USA, 9–11 September 2009; pp. 1202–1209. [Google Scholar]
- Cantero-Chinchilla, S.; Chiachío, J.; Chiachío, M.; Chronopoulos, D.; Jones, A. A robust Bayesian methodology for damage localization in plate-like structures using ultrasonic guided-waves. Mech. Syst. Addit. Signal Process. 2019, 122, 192–205. [Google Scholar] [CrossRef] [Green Version]
- Giurgiutiu, V.; Zagrai, A.; Jing Bao, J. Piezoelectric wafer embedded active sensors for aging aircraft structural health monitoring. Struct. Health Monit. 2002, 1, 41–61. [Google Scholar] [CrossRef]
- Torkamani, S.; Roy, S.; Barkey, M.; Sazonov, E.; Burkett, S.; Kotru, S. A novel damage index for damage identification using guided waves with application in laminated composites. Smart Mater. Addit. Struct. 2014, 23, 095015. [Google Scholar] [CrossRef]
- Quaegebeur, N.; Masson, P.; Langlois-Demers, D.; Micheau, P. Dispersion-based imaging for structural health monitoring using sparse and compact arrays. Smart Mater. Addit. Struct. 2011, 20, 025005. [Google Scholar] [CrossRef]
- Quaegebeur, N.; Ostiguy, P.; Masson, P. Correlation-based imaging technique for fatigue monitoring of riveted lap-joint structure. Smart Mater. Addit. Struct. 2014, 23, 055007. [Google Scholar] [CrossRef]
- Sbarufatti, C.; Manson, G.; Worden, K. A numerically-enhanced machine learning approach to damage diagnosis using a Lamb wave sensing network. J. Sound Addit. Vib. 2014, 333, 4499–4525. [Google Scholar] [CrossRef]
- De Fenza, A.; Sorrentino, A.; Vitiello, P. Application of Artificial Neural Networks and Probability Ellipse methods for damage detection using Lamb waves. Compos. Struct. 2015, 133, 390–403. [Google Scholar] [CrossRef]
- Vitola, J.; Pozo, F.; Tibaduiza, D.; Anaya, M. A sensor data fusion system based on k-nearest neighbor pattern classification for structural health monitoring applications. Sensors 2017, 17, 417. [Google Scholar] [CrossRef]
- Zhang, Z.; Pan, H.; Wang, X.; Lin, Z. Machine learning-enriched lamb wave approaches for automated damage detection. Sensors 2020, 20, 1790. [Google Scholar] [CrossRef] [Green Version]
- Yang, L.; Xu, F. A novel acoustic emission sources localization and identification method in metallic plates based on stacked denoising autoencoders. IEEE Access 2020, 8, 141123–141142. [Google Scholar] [CrossRef]
- Grondin, F.; Glass, J. Multiple sound source localization with SVD-PHAT. In Proceedings of the Interspeech, Graz, Austria, 15–19 September 2019; pp. 2698–2702. [Google Scholar]
- Grondin, F.; Glass, J. SVD-PHAT: A fast sound source localization method. In Proceedings of the IEEE International Conference On Acoustics, Speech in addition, Signal Processing, Brighton, UK, 12–17 May 2019; pp. 4140–4144. [Google Scholar]
- Rautela, M.; Senthilnath, J.; Moll, J.; Gopalakrishnan, S. Combined two-level damage identification strategy using ultrasonic guided waves and physical knowledge assisted machine learning. Ultrasonics 2021, 115, 106451. [Google Scholar] [CrossRef]
- Ewald, V.; Groves, R.; Benedictus, R. DeepSHM: A deep learning approach for structural health monitoring based on guided Lamb wave technique. In Proceedings of the Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2019, Denver, CO, USA, 4–7 March 2019; Volume 10970, p. 109700H. [Google Scholar]
- De Oliveira, M.; Monteiro, A.; Vieira Filho, J. A new structural health monitoring strategy based on PZT sensors and convolutional neural network. Sensors 2018, 18, 2955. [Google Scholar] [CrossRef] [Green Version]
- Kim, I.; Jeon, H.; Baek, S.; Hong, W.; Jung, H. Application of crack identification techniques for an aging concrete bridge inspection using an unmanned aerial vehicle. Sensors 2018, 18, 1881. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ye, J.; Ito, S.; Toyama, N. Computerized ultrasonic imaging inspection: From shallow for deep learning. Sensors 2018, 18, 3820. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Civera, M.; Surace, C. Non-Destructive Techniques for the Condition and Structural Health Monitoring of Wind Turbines: A Literature Review of the Last 20 Years. Sensors 2022, 22, 1627. [Google Scholar] [CrossRef]
- Zonzini, F.; Bogomolov, D.; Dhamija, T.; Testoni, N.; De Marchi, L.; Marzani, A. Deep Learning Approaches for Robust Time of Arrival Estimation in Acoustic Emission Monitoring. Sensors 2022, 22, 1091. [Google Scholar] [CrossRef] [PubMed]
- Le, H.; Mayer, S.; Henze, N. Investigating the feasibility of finger identification on capacitive touchscreens using deep learning. In Proceedings of the International Conference On Intelligent User Interfaces, Marina del Ray, CA, USA, 17–20 March 2019; pp. 637–649. [Google Scholar]
Grid Resolution (N) | Number of Classes () | Class Area (cm2) |
---|---|---|
2 | 4 | 10.0 |
3 | 9 | 4.4 |
4 | 16 | 2.5 |
5 | 25 | 1.6 |
6 | 36 | 1.1 |
7 | 49 | 0.8 |
8 | 64 | 0.6 |
9 | 81 | 0.5 |
10 | 100 | 0.4 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bahrami, S.; Moriot, J.; Masson, P.; Grondin, F. Machine Learning for Touch Localization on an Ultrasonic Lamb Wave Touchscreen. Sensors 2022, 22, 3183. https://doi.org/10.3390/s22093183
Bahrami S, Moriot J, Masson P, Grondin F. Machine Learning for Touch Localization on an Ultrasonic Lamb Wave Touchscreen. Sensors. 2022; 22(9):3183. https://doi.org/10.3390/s22093183
Chicago/Turabian StyleBahrami, Sahar, Jérémy Moriot, Patrice Masson, and François Grondin. 2022. "Machine Learning for Touch Localization on an Ultrasonic Lamb Wave Touchscreen" Sensors 22, no. 9: 3183. https://doi.org/10.3390/s22093183
APA StyleBahrami, S., Moriot, J., Masson, P., & Grondin, F. (2022). Machine Learning for Touch Localization on an Ultrasonic Lamb Wave Touchscreen. Sensors, 22(9), 3183. https://doi.org/10.3390/s22093183