Mobile Typing as a Window into Sensorimotor and Cognitive Function
Abstract
1. Introduction
2. Mobile Typing Skill Acquisition and Development
2.1. The Origins of Touchscreen Competence in Infants and Influence on Fine Motor Skills
2.2. General Typing Skill Acquisition and Its Adaptation to Mobile Devices
3. Mobile Typing Modalities and Interfaces
3.1. The QWERTY Keyboard: A Legacy in Mobile Interaction
3.2. Types of Mobile Text Entry: Tap vs. Swipe and Intelligent Text Entry (ITE)
4. Methodological Approaches for Studying Mobile Typing
4.1. Key Performance Metrics
Category | Metric Name (Acronym) | Description | Key Source(s) |
---|---|---|---|
Overall/Speed | Words Per Minute (WPM) | Baseline speed measurement based on output text. | Yamada (1980) [84] |
Adjusted Words Per Minute (AdjWPM) | Incorporates error penalties to reflect the trade-off between speed and accuracy. | Matias et al. (1996) [85] | |
Keystrokes Per Second (KSPS) | Measure of data transfer rate, capturing user’s action rate regardless of textual output. | Wobbrock (2007) [80] | |
Gestures Per Second (GPS) | Comparable metric to KSPS specifically for gesture-based methods. | Wobbrock (2007) [80] | |
Learning Curves | Modelling metrics over time to reveal the learnability of a method. | Card et al. (1983) [86] | |
Accuracy | Keystrokes per Character (KSPC) | Evaluates accuracy by comparing input keystrokes to transcribed characters. | Soukoreff & Mackenzie (2001) [83]; MacKenzie (2002) [89] |
Minimum String Distance (MSD) | Quantifies accuracy after entry by measuring the edit distance between intended and produced text. | Soukoreff & Mackenzie (2001) [83]; Wagner & Fischer (1974) [87] | |
Corrected Error Rate | Proportion of errors corrected by the user. | Soukoreff & MacKenzie (2003) [83] | |
Uncorrected Error Rate | Proportion of errors remaining in the final output text. | Soukoreff & MacKenzie (2003) [83] | |
Total Error Rate | All errors made, regardless of correction status. | Soukoreff & MacKenzie (2003) [83] | |
Efficiency | Correction Efficiency | Assesses the effectiveness of correction mechanisms. | Soukoreff & MacKenzie (2003) [83] |
Participant Conscientiousness | Reflects user diligence in error correction. | Soukoreff & MacKenzie (2003) [83] | |
Utilized Bandwidth | Proportion of keystrokes contributing to correct text. | Soukoreff & MacKenzie (2003) [83] | |
Wasted Bandwidth | Proportion of keystrokes that do not contribute to correct text (e.g., errors, corrections). | Soukoreff & MacKenzie (2003) [83] | |
Cost per Correction (CPC) | Evaluates the effort associated with the error correction process. | Gong & Tarasewich (2006) [91] | |
Temporal/Character- Level | Intra-character Time | Time elapsed within the production of a single character. | Rumelhart & Norman (1982) [30] |
Intercharacter Time | Time elapsed between successive keystrokes; reflects typing rhythm and pauses. | Rumelhart & Norman (1982) [30]; Wengelin (2006) [93] | |
Uncorrected Errors at character level | Identification of specific error types (substitutions, insertions, omissions) at the character level | MacKenzie & Soukoreff (2002) [90]; Wobbrock & Myers (2006) [82] | |
Corrected Errors in Input Streams | Captures nuances of correction behavior (e.g., “corrected-and-wrong”, “corrected-but-right”). | Wobbrock & Myers (2006) [82] |
4.2. Controlled Laboratory Tasks
4.3. Real-World Data Collection
5. Mobile Typing as a Behavioral Biomarker
5.1. Smartphone Interaction Dynamics: Applications in Aging, Affective States, and Neurological Health
5.2. Neurophysiological Evidence for Sensorimotor Plasticity
6. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Gentner, D.R. The Acquisition of Typewriting Skill. Acta Psychol. 1983, 54, 233–248. [Google Scholar] [CrossRef]
- Ferrari, A.; Punie, Y.; Redecker, C. Understanding Digital Competence in the 21st Century: An Analysis of Current Frameworks. In Proceedings of the 21st Century Learning for 21st Century Skills, Saarbrücken, Germany, 18–21 September 2012; Ravenscroft, A., Lindstaedt, S., Kloos, C.D., Hernández-Leo, D., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; Volume 7563, pp. 79–92. [Google Scholar]
- We Are Social & Meltwater Digital 2024 Global Overview Report. Available online: https://datareportal.com/reports/digital-2024-global-overview-report (accessed on 14 March 2025).
- Gutiérrez, J.D.S.; de Fonseca, F.R.; Rubio, G. Cell-Phone Addiction: A Review. Front. Psychiatry 2016, 7, 175. [Google Scholar] [CrossRef] [PubMed]
- Wilmer, H.H.; Sherman, L.E.; Chein, J.M. Smartphones and Cognition: A Review of Research Exploring the Links between Mobile Technology Habits and Cognitive Functioning. Front. Psychol. 2017, 8, 605. [Google Scholar] [CrossRef] [PubMed]
- Jenkins, W.M.; Merzenich, M.M.; Ochs, M.T.; Allard, T.; Guic-Robles, E. Functional Reorganization of Primary Somatosensory Cortex in Adult Owl Monkeys after Behaviorally Controlled Tactile Stimulation. J. Neurophysiol. 1990, 63, 82–104. [Google Scholar] [CrossRef]
- Elbert, T.; Pantev, C.; Wienbruch, C.; Rockstroh, B.; Taub, E. Increased Cortical Representation of the Fingers of the Left Hand in String Players. Science 1995, 270, 305–307. [Google Scholar] [CrossRef]
- Gindrat, A.-D.; Chytiris, M.; Balerna, M.; Rouiller, E.M.; Ghosh, A. Use-Dependent Cortical Processing from Fingertips in Touchscreen Phone Users. Curr. Biol. 2015, 25, 109–116. [Google Scholar] [CrossRef]
- Hahamy, A.; Macdonald, S.N.; van den Heiligenberg, F.; Kieliba, P.; Emir, U.; Malach, R.; Johansen-Berg, H.; Brugger, P.; Culham, J.C.; Makin, T.R. Representation of Multiple Body Parts in the Missing-Hand Territory of Congenital One-Handers. Curr. Biol. 2017, 27, 1350–1355. [Google Scholar] [CrossRef]
- Anderson, M.L.; Richardson, M.J.; Chemero, A. Eroding the Boundaries of Cognition: Implications of Embodiment. Top. Cogn. Sci. 2012, 4, 717–730. [Google Scholar] [CrossRef]
- Gallese, V.; Lakoff, G. The Brain’s Concepts: The Role of the Sensory-Motor System in Conceptual Knowledge. Cogn. Neuropsychol. 2005, 22, 455–479. [Google Scholar] [CrossRef]
- Wilson, M. Six Views of Embodied Cognition. Psychon. Bull. Rev. 2002, 9, 625–636. [Google Scholar] [CrossRef]
- Becchio, C.; Sartori, L.; Castiello, U. Toward You: The Social Side of Actions. Curr. Dir. Psychol. Sci. 2010, 19, 183–188. [Google Scholar] [CrossRef]
- Sartori, L.; Becchio, C.; Castiello, U. Cues to Intention: The Role of Movement Information. Cognition 2011, 119, 242–252. [Google Scholar] [CrossRef] [PubMed]
- Soriano, M.; Cavallo, A.; D’Ausilio, A.; Becchio, C.; Fadiga, L. Movement Kinematics Drive Chain Selection toward Intention Detection. Proc. Natl. Acad. Sci. USA 2018, 115, 10452–10457. [Google Scholar] [CrossRef] [PubMed]
- Jeannerod, M. The Timing of Natural Prehension Movements. J. Mot. Behav. 1984, 16, 235–254. [Google Scholar] [CrossRef] [PubMed]
- Castiello, U. The Neuroscience of Grasping. Nat. Rev. Neurosci. 2005, 6, 726–736. [Google Scholar] [CrossRef]
- Prensky, M. Digital Natives, Digital Immigrants Part 1. Horizon 2001, 9, 1–6. [Google Scholar] [CrossRef]
- Craighero, L.; Marini, M. Implicit Associations between Adverbs of Place and Actions in the Physical and Digital Space. Brain Sci. 2021, 11, 1523. [Google Scholar] [CrossRef]
- Ziemer, C.J.; Wyss, S.; Rhinehart, K. The Origins of Touchscreen Competence: Examining Infants’ Exploration of Touchscreens. Infant. Behav. Dev. 2021, 64, 101609. [Google Scholar] [CrossRef]
- Hourcade, J.P.; Mascher, S.L.; Wu, D.; Pantoja, L. Look, My Baby Is Using an IPad! An Analysis of Youtube Videos of Infants and Toddlers Using Tablets. In Proceedings of the Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, 18–23 April 2015; Association for Computing Machinery: New York, NY, USA, 2015; Volume 2015, pp. 1915–1924. [Google Scholar]
- Bedford, R.; Saez de Urabain, I.R.; Celeste, C.H.; Karmiloff-Smith, A.; Smith, T.J. Toddlers’ Fine Motor Milestone Achievement Is Associated with Early Touchscreen Scrolling. Front. Psychol. 2016, 7, 1108. [Google Scholar] [CrossRef]
- Souto, P.H.S.; Santos, J.N.; Leite, H.R.; Hadders-Algra, M.; Guedes, S.C.; Nobre, J.N.P.; Santos, L.R.; Morais, R.L.d.S. Tablet Use in Young Children Is Associated with Advanced Fine Motor Skills. J. Mot. Behav. 2020, 52, 196–203. [Google Scholar] [CrossRef]
- Moon, J.H.; Cho, S.Y.; Lim, S.M.; Roh, J.H.; Koh, M.S.; Kim, Y.J.; Nam, E. Smart Device Usage in Early Childhood Is Differentially Associated with Fine Motor and Language Development. Acta Paediatr. Int. J. Paediatr. 2019, 108, 903–910. [Google Scholar] [CrossRef]
- Martzog, P.; Suggate, S.P. Screen Media Are Associated with Fine Motor Skill Development in Preschool Children. Early Child. Res. Q. 2022, 60, 363–373. [Google Scholar] [CrossRef]
- Lin, L.Y. Differences between Preschool Children Using Tablets and Non-Tablets in Visual Perception and Fine Motor Skills. Hong Kong J. Occup. Ther. 2019, 32, 118–126. [Google Scholar] [CrossRef]
- Lin, L.Y.; Cherng, R.J.; Chen, Y.J. Effect of Touch Screen Tablet Use on Fine Motor Development of Young Children. Phys. Occup. Ther. Pediatr. 2017, 37, 457–467. [Google Scholar] [CrossRef]
- Sono, C.; Hasegawa, T. Interaction to Support the Learning of Typing for Beginners on Physical Keyboard by Projection Mapping. In Proceedings of the 2019 7th International Conference on Information Technology: IoT and Smart City, Shanghai, China, 20–23 December 2019; Association for Computing Machinery: New York, NY, USA, 2019; pp. 565–569. [Google Scholar]
- Fitts, P.M.; Posner, M.I. Human Performance; Brooks/Cole Publishing Company, Ed.; Prentice-Hall: Hoboken, NJ, USA, 1967; Volume 5, ISBN 9780134452470. [Google Scholar]
- Rumelhart, D.E.; Norman, D.A. Simulating a Skilled Typist: A Study of Skilled Cognitive-Motor Performance. Cogn. Sci. 1982, 6, 1–36. [Google Scholar] [CrossRef]
- Salthouse, T.A. Perceptual, Cognitive, and Motoric Aspects of Transcription Typing. Psychol. Bull. 1986, 99, 303–319. [Google Scholar] [CrossRef] [PubMed]
- Salthouse, T.A. Effects of Age and Skill in Typing. J. Exp. Psychol. Gen. 1984, 113, 345–371. [Google Scholar] [CrossRef] [PubMed]
- Shaffer, L.H.; Hardwick, J. Typing Performance as a Function of Text. Q. J. Exp. Psychol. 1968, 20, 360–369. [Google Scholar] [CrossRef]
- Shaffer, L.H. Control Processes in Typing. Q. J. Exp. Psychol. 1975, 27, 419–432. [Google Scholar] [CrossRef]
- Gentner, D.R. Expertise in Typewriting. In The Nature of Expertise; Chi, M.T.H., Glaser, R., Farr, M.J., Eds.; Lawrence Erlbaum Associates, Inc.: Mahwah, NJ, USA, 1988; pp. 1–21. [Google Scholar]
- Dhakal, V.; Feit, A.M.; Kristensson, P.O.; Oulasvirta, A. Observations on Typing from 136 Million Keystrokes. In Proceedings of the Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 21–26 April 2018; Association for Computing Machinery: New York, NY, USA, 2018; Volume 2018. [Google Scholar]
- Feit, A.M.; Weir, D.; Oulasvirta, A. How We Type: Movement Strategies and Performance in Everyday Typing. In Proceedings of the Conference on Human Factors in Computing Systems, San Jose, CA, USA, 7–12 May 2016; Association for Computing Machinery: New York, NY, USA, 2016; pp. 4262–4273. [Google Scholar]
- Cerni, T.; Longcamp, M.; Job, R. Two Thumbs and One Index: A Comparison of Manual Coordination in Touch-Typing and Mobile-Typing. Acta Psychol. 2016, 167, 16–23. [Google Scholar] [CrossRef]
- Jiang, X.; Jokinen, J.P.P.; Oulasvirta, A.; Ren, X. Learning to Type with Mobile Keyboards: Findings with a Randomized Keyboard. Comput. Hum. Behav. 2022, 126, 106992. [Google Scholar] [CrossRef]
- Jiang, X.; Li, Y.; Jokinen, J.P.P.; Hirvola, V.B.; Oulasvirta, A.; Ren, X. How We Type: Eye and Finger Movement Strategies in Mobile Typing. In Proceedings of the Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; Association for Computing Machinery: New York, NY, USA, 2020; pp. 1–14. [Google Scholar]
- Pinet, S.; Zielinski, C.; Alario, F.X.; Longcamp, M. Typing Expertise in a Large Student Population. Cogn. Res. Princ. Implic. 2022, 7, 77. [Google Scholar] [CrossRef]
- Noyes, J. QWERTY-the Immortal Keyboard. Comput. Control Eng. 1998, 9, 117–122. [Google Scholar] [CrossRef]
- Azenkot, S.; Zhai, S. Touch Behavior with Different Postures on Soft Smartphone Keyboards. In Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services, San Francisco, CA, USA, 21–24 September 2012; ACM: New York, NY, USA, 2012; pp. 251–260. [Google Scholar]
- Pritom, A.I.; Mahmud, H.; Ahmed, S.; Hasan, M.K.; Khan, M.M. TYPEHEX Keyboard: A Virtual Keyboard for Faster Typing in Smartphone. In Proceedings of the International Conference on Computer and Information Technology, ICCIT 2015, Dhaka, Bangladesh, 21–23 December 2015; Institute of Electrical and Electronics Engineers Inc.: Dhaka, Bangladesh, 2015; pp. 522–526. [Google Scholar]
- Chang, J.; Choi, B.; Tjolleng, A.; Jung, K. Effects of Button Position on a Soft Keyboard: Muscle Activity, Touch Time, and Discomfort in Two-Thumb Text Entry. Appl. Ergon. 2017, 60, 282–292. [Google Scholar] [CrossRef] [PubMed]
- Ryu, T.; Lim, J.; Song, J.; Yun, M.H.; Shin, M. Performance Comparison between the Preferred Right and Preferred Left Hands in Text Entry Using Qwerty Touch Keyboard Smartphones. Int. J. Ind. Ergon. 2013, 43, 400–405. [Google Scholar] [CrossRef]
- Park, Y.S.; Han, S.H. One-Handed Thumb Interaction of Mobile Devices from the Input Accuracy Perspective. Int. J. Ind. Ergon. 2010, 40, 746–756. [Google Scholar] [CrossRef]
- Chang, J.; Jung, K. Effects of Button Width, Height, and Location on a Soft Keyboard: Task Completion Time, Error Rate, and Satisfaction in Two-Thumb Text Entry on Smartphone. IEEE Access 2019, 7, 69848–69857. [Google Scholar] [CrossRef]
- Hwangbo, H.; Yoon, S.H.; Jin, B.S.; Han, Y.S.; Ji, Y.G. A Study of Pointing Performance of Elderly Users on Smartphones. Int. J. Hum. Comput. Interact. 2013, 29, 604–618. [Google Scholar] [CrossRef]
- Parhi, P.; Karlson, A.K.; Bederson, B.B. Target Size Study for One-Handed Thumb Use on Small Touchscreen Devices. In Proceedings of the 8th Conference on Human-Computer Interaction with Mobile Devices and Services, Helsinki, Finland, 12–15 September 2006; Volume 159, pp. 203–210. [Google Scholar]
- Trudeau, M.B.; Udtamadilok, T.; Karlson, A.K.; Dennerlein, J.T. Thumb Motor Performance Varies by Movement Orientation, Direction, and Device Size during Single-Handed Mobile Phone Use. Hum. Factors 2012, 54, 52–59. [Google Scholar] [CrossRef]
- Yu, H.; Nam, K.; Shin, S.; Choi, M.; Son, Y.; Chang, J. Repetitive Patterns in the Locations of Touch Errors for Two-Thumb Text Entry on a Smartphone. Appl. Ergon. 2021, 97, 103541. [Google Scholar] [CrossRef]
- Kang, E.; Tjolleng, A.; Yu, H.; Jung, K.; Chang, J. Investigation into Touch Performance on a QWERTY Soft Keyboard on a Smartphone: Touch Time, Accuracy, and Satisfaction in Two-Thumb Key Entry. Appl. Sci. 2023, 13, 6825. [Google Scholar] [CrossRef]
- Iarygina, O.; Mottelson, A. A Systematic Review and Meta-Analysis of Text Entry Studies. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 26 April–1 May 2025; ACM: New York, NY, USA, 2025; pp. 1–14. [Google Scholar]
- Zhai, S.; Kristensson, P.O. Shorthand Writing on Stylus Keyboard. In Proceedings of the Conference on Human Factors in Computing Systems, Ft. Lauderdale, FL, USA, 5–10 April 2003; pp. 97–104. [Google Scholar]
- Zhai, S.; Kristensson, P.O. The Word-Gesture Keyboard: Reimagining Keyboard Interaction. Commun. ACM 2012, 55, 91–101. [Google Scholar] [CrossRef]
- Bi, X.; Chelba, C.; Ouyang, T.; Partridge, K.; Zhai, S. Bimanual Gesture Keyboard. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, Cambridge, MA, USA, 7–10 October 2012; ACM: New York, NY, USA, 2012; pp. 137–146. [Google Scholar] [CrossRef]
- Reyal, S.; Zhai, S.; Kristensson, P.O. Performance and User Experience of Touchscreen and Gesture Keyboards in a Lab Setting and in the Wild. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, 18–23 April 2015; ACM: New York, NY, USA, 2015; pp. 679–688. [Google Scholar]
- Leiva, L.A.; Kim, S.; Cui, W.; Bi, X.; Oulasvirta, A. How We Swipe: A Large-Scale Shape-Writing Dataset and Empirical Findings. In Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction, Toulouse, France, 27 September–1 October 2021; ACM: New York, NY, USA, 2021; pp. 1–13. [Google Scholar]
- Hoggan, E.; Brewster, S.A.; Johnston, J. Investigating the Effectiveness of Tactile Feedback for Mobile Touchscreens. In Proceedings of the Conference on Human Factors in Computing Systems, Florence, Italy, 5–10 April 2008; pp. 1573–1582. [Google Scholar]
- Palin, K.; Feit, A.M.; Kim, S.; Kristensson, P.O.; Oulasvirta, A. How Do People Type on Mobile Devices? Observations from a Study with 37,000 Volunteers. In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019, Taipei, Taiwan, 1–4 October 2019; Association for Computing Machinery, Inc.: New York, NY, USA, 2019; pp. 1–12. [Google Scholar]
- Varcholik, P.D.; Laviola, J.J.; Hughes, C.E. Establishing a Baseline for Text Entry for a Multi-Touch Virtual Keyboard. Int. J. Hum. Comput. Stud. 2012, 70, 657–672. [Google Scholar] [CrossRef]
- Nicolau, H.; Jorge, J. Elderly Text-Entry Performance on Touchscreens. In Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility, Boulder, CO, USA, 22–24 October 2012; pp. 127–134. [Google Scholar]
- Banovic, N.; Rao, V.; Saravanan, A.; Dey, A.K.; Mankoff, J. Quantifying Aversion to Costly Typing Errors in Expert Mobile Text Entry. In Proceedings of the Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017; Association for Computing Machinery: New York, NY, USA, 2017; Volume 2017, pp. 4229–4241. [Google Scholar]
- Xiong, J.; Muraki, S. An Ergonomics Study of Thumb Movements on Smartphone Touch Screen. Ergonomics 2014, 57, 943–955. [Google Scholar] [CrossRef] [PubMed]
- Jung, K.; Jang, J. Development of a Two-Step Touch Method for Website Navigation on Smartphones. Appl. Ergon. 2015, 48, 148–153. [Google Scholar] [CrossRef] [PubMed]
- Li, Y.; You, F.; You, X.; Ji, M. Smartphone Text Input: Effects of Experience and Phrase Complexity on User Performance, Physiological Reaction, and Perceived Usability. Appl. Ergon. 2019, 80, 200–208. [Google Scholar] [CrossRef]
- Ramati, I. Thumbs up and down: The Cultural Technique of Thumb-Typing. Convergence 2023, 29, 152–168. [Google Scholar] [CrossRef]
- Kristensson, P.O.; Jameson, A. Five Challenges for Intelligent Text Entry Methods. AI Mag. 2009, 30, 85–94. [Google Scholar] [CrossRef]
- Goodman, J.; Venolia, G.; Steury, K.; Parker, C. Language Modeling for Soft Keyboards. In Proceedings of the International Conference on Intelligent User Interfaces, San Francisco, CA, USA, 13–16 January 2002; pp. 194–195. [Google Scholar]
- Fowler, A.; Partridge, K.; Chelba, C.; Bi, X.; Ouyang, T.; Zhai, S. Effects of Language Modeling and Its Personalization on Touchscreen Typing Performance. In Proceedings of the Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, 18–23 April 2015; Association for Computing Machinery: New York, NY, USA, 2015; Volume 2015, pp. 649–658. [Google Scholar]
- Quinn, P.; Zhai, S. A Cost-Benefit Study of Text Entry Suggestion Interaction. In Proceedings of the Conference on Human Factors in Computing Systems, San Jose, CA, USA, 7–12 May 2016; Association for Computing Machinery: New York, NY, USA, 2016; pp. 83–88. [Google Scholar]
- Roy, Q.; Berlioux, S. Typing Efficiency and Suggestion Accuracy Influence the Benefits and Adoption of word Suggestions. In Proceedings of the Conference on Human Factors in Computing Systems, Yokohama, Japan, 8–13 May 2021; Association for Computing Machinery: New York, NY, USA, 2021. [Google Scholar]
- Brandt, D. The Rise of Writing; Cambridge University Press: Cambridge, UK, 2014; ISBN 9781107090316. [Google Scholar]
- Aldridge, M.; Fontaine, L. Using Keystroke Logging to Capture the Impact of Cognitive Complexity and Typing Fluency on Written Language Production. In Observing Writing; BRILL: Leiden, The Netherlands, 2019; pp. 285–305. [Google Scholar]
- van Weerdenburg, M.; Tesselhof, M.; van der Meijden, H. Touch-Typing for Better Spelling and Narrative-Writing Skills on the Computer. J. Comput. Assist. Learn. 2019, 35, 143–152. [Google Scholar] [CrossRef]
- Alves, R.A.; Castro, S.L.; de Sousa, L.; Strömqvist, S. Chapter 4: Influence of Typing Skill on Pause–Execution Cycles in Written Composition. In Writing and Cognition; BRILL: Leiden, The Netherlands, 2007; pp. 55–65. [Google Scholar]
- Van Waes, L.; Leijten, M.; Mariën, P.; Engelborghs, S. Typing Competencies in Alzheimer’s Disease: An Exploration of Copy Tasks. Comput. Hum. Behav. 2017, 73, 311–319. [Google Scholar] [CrossRef]
- Roeser, J.; De Maeyer, S.; Leijten, M.; Van Waes, L. Modelling Typing Disfluencies as Finite Mixture Process. Read. Writ. 2021, 37, 359–384. [Google Scholar] [CrossRef]
- Wobbrock, J.O. Measures of Text Entry Performance. In Text Entry Systems; Elsevier: Amsterdam, The Netherlands, 2007; pp. 47–74. [Google Scholar]
- Soukoreff, R.W.; Mackenzie, I.S. Measuring Errors in Text Entry Tasks: An Application of the Levenshtein String Distance Statistic Levenshtein Minimum String Distance, Text Entry, Error Rates. In Proceedings of the Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHI’01), Seattle WA, USA, 31 March–5 April 2001; ACM Press: New York, NY, USA, 2001; pp. 319–320. [Google Scholar]
- Wobbrock, J.O.; Myers, B.A. Analyzing the Input Stream for Character- Level Errors in Unconstrained Text Entry Evaluations. ACM Trans. Comput.-Hum. Interact. 2006, 13, 458–489. [Google Scholar] [CrossRef]
- Soukoreff, R.W.; MacKenzie, I.S. Metrics for Text Entry Research: An Evaluation of MSD and KSPC, and a New Unified Error Metric. In Proceedings of the Conference on Human Factors in Computing Systems, CHI, Ft. Lauderdale, FL, USA, 5–10 April 2003; Association for Computing Machinery (ACM): New York, NY, USA, 2003; pp. 113–120. [Google Scholar]
- Yamada, H. A Historical Study of Typewriters and Typing Methods: From the Position of Planning Japanese Parallels. J. Inf. Process. 1980, 2, 175–202. [Google Scholar]
- Matias, E.; MacKenzie, I.S.; Buxton, W. One-Handed Touch Typing on a QWERTY Keyboard. Hum. Comput. Interact. 1996, 11, 1–27. [Google Scholar] [CrossRef]
- Card, S.K.; Moran, T.P.; Newell, A. The Psychology of Human-Computer Interaction; CRC Press: Boca Raton, FL, USA, 1983; ISBN 9780203736166. [Google Scholar]
- Wagner, R.A.; Fischer, M.J. The String-to-String Correction Problem. J. ACM (JACM) 1974, 21, 168–173. [Google Scholar] [CrossRef]
- Levenshtein, V.I. Binary Codes Capable of Correcting Deletions, Insertions, and Reversals. Dokl. Akad. Nauk. SSSR 1965, 163, 845–848. [Google Scholar]
- MacKenzie, I.S. KSPC (Keystrokes per Character) as a Characteristic of Text Entry Techniques. In Proceedings of the 4th International Symposium on Human–Computer Interaction with Mobile Devices and Services (Mobile HCI02), Pisa, Italy, 18–20 September 2002; Springer: Berlin/Heidelberg, Germany, 2002; pp. 195–210. [Google Scholar]
- MacKenzie, I.S.; Soukoreff, R.W. Text Entry for Mobile Computing: Models and Methods, Theory and Practice. Hum. Comput. Interact. 2002, 17, 147–198. [Google Scholar]
- Gong, J.; Tarasewich, P. A New Error Metric for Text Entry Method Evaluation. In Proceedings of the Conference on Human Factors in Computing Systems, Montréal, QC, Canada, 22–27 April 2006; Association for Computing Machinery: New York, NY, USA, 2006; Volume 1, pp. 471–474. [Google Scholar]
- Leijten, M.; Van Waes, L. Keystroke Logging in Writing Research: Using Inputlog to Analyze and Visualize Writing Processes. Writ. Commun. 2013, 30, 358–392. [Google Scholar] [CrossRef]
- Wengelin, Å. Examining Pauses in Writing: Theory, Methods and Empirical Data. In Computer Key-Stroke Logging and Writing: Methods and Applications (Studies in Writing); Sullivan, K., Lindgren, E., Eds.; Elsevier: Amsterdam, The Netherlands, 2006; Volume 18, pp. 107–130. [Google Scholar]
- Galbraith, D.; Baaijen, V.M. Aligning Keystrokes with Cognitive Processes in Writing. In Observing Writing; BRILL: Leiden, The Netherlands, 2019; pp. 306–325. [Google Scholar]
- MacKenzie, I.S.; Zhang, S.X. The Design and Evaluation of a High-Performance Soft Keyboard. In Proceedings of the Conference on Human Factors in Computing Systems, Pittsburgh, PA, USA, 15–20 May 1999; pp. 25–31. [Google Scholar]
- Klimt, B.; Yang, Y. The Enron Corpus: A New Dataset for Email Classification Research. In Proceedings of the Machine Learning: ECML 2004. ECML 2004. Lecture Notes in Computer Science, Pisa, Italy, 20–24 September 2004; Boulicaut, J.F., Esposito, F., Giannotti, F., Pedreschi, D., Eds.; Springer: Berlin/Heidelberg, Germany, 2004; Volume 3201, pp. 217–226. [Google Scholar]
- Vertanen, K.; Kristensson, P.O. A Versatile Dataset for Text Entry Evaluations Based on Genuine Mobile Emails. In Proceedings of the 13th International Conference on Human-Computer Interaction with Mobile Devices and Services, Stockholm, Sweden, 30 August–2 September 2011; pp. 295–298. [Google Scholar]
- Mackenzie, I.S.; Soukoreff, R.W. Phrase Sets for Evaluating Text Entry Techniques. In Proceedings of the CHI’03 Extended Abstracts on Human Factors in Computing Systems, Ft. Lauderdale, FL, USA, 5–10 April 2003; pp. 744–745. [Google Scholar]
- Van Waes, L.V.; Leijten, M.; Roeser, J.; Olive, T.; Grabowski, J. Measuring and Assessing Typing Skills in Writing Research. J. Writ. Res. 2021, 13, 107–153. [Google Scholar] [CrossRef]
- Vertanen, K.; Kristensson, P.O. Complementing Text Entry Evaluations with a Composition Task. ACM Trans. Comput.-Hum. Interact. 2014, 21, 1–33. [Google Scholar] [CrossRef]
- Drouin, M.; Driver, B. Texting, Textese and Literacy Abilities: A Naturalistic Study. J. Res. Read. 2014, 37, 250–267. [Google Scholar] [CrossRef]
- Gaines, D.; Kristensson, P.O.; Vertanen, K. Enhancing the Composition Task in Text Entry Studies: Eliciting Difficult Text and Improving Error Rate Calculation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 8–13 May 2021; ACM: New York, NY, USA, 2021; pp. 1–8. [Google Scholar]
- Nicol, E.; Komninos, A.; Dunlop, M.D. A Participatory Design and Formal Study Investigation into Mobile Text Entry for Older Adults. Int. J. Mob. Hum. Comput. Interact. 2016, 8, 20–46. [Google Scholar] [CrossRef]
- Reinecke, K.; Gajos, K.Z. Labin the Wild: Conducting Large-Scale Online Experiments with Uncompensated Samples. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, Vancouver, BC, Canada, 14–18 March 2015; Association for Computing Machinery, Inc.: New York, NY, USA, 2015; pp. 1364–1378. [Google Scholar]
- Henze, N.; Poppinga, B.; Boll, S. Experiments in the Wild: Public Evaluation of Off-Screen in the Android Market. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, Reykjavik, Iceland, 16–20 October 2010; ACM Digital Library: New York, NY, USA, 2013; pp. 675–678. [Google Scholar]
- Vertanen, K.; Kristensson, P.O.; Emge, J.; Memmi, H. Text Blaster: A Multi-Player Touchscreen Typing Game. In Proceedings of the Conference on Human Factors in Computing Systems, Toronto, ON, Canada, 26 April–1 May 2014; Association for Computing Machinery: New York, NY, USA, 2014; pp. 379–382. [Google Scholar]
- Wimmer, C.; Schlögl, R.; Kappel, K.; Grechenig, T. Measuring Mobile Text Entry Performance and Behaviour in the Wild with a Serious Game. In Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia, Pisa, Italy, 26–29 November 2019; Association for Computing Machinery: New York, NY, USA, 2019. [Google Scholar]
- Lew, L.; Nguyen, T.; Messing, S.; Westwood, S. Of Course I Wouldn’t Do That in Real Life: Advancing the Arguments for Increasing Realism in HCI Experiments. In Proceedings of the CHI ’11 Extended Abstracts on Human Factors in Computing Systems, Vancouver, BC, Canada, 7–12 May 2011; Association for Computing Machinery: New York, NY, USA, 2011; pp. 419–428. [Google Scholar]
- Pan, C.S.; Shell, R.L.; Schleifer, L.M. Performance Variability as an Indicator of Fatigue and Boredom Effects in a VDT Data-Entry Task. Int. J. Hum. Comput. Interact. 1994, 6, 37–45. [Google Scholar] [CrossRef]
- Komninos, A.; Feit, A.M.; Leiva, L.A.; Lehmann, F.; Simou, I.; Minas, D.; Fotopoulos, A.; Xenos, M. An LLM-Driven Transcription Task for Mobile Text Entry Studies. In Proceedings of the International Conference on Mobile and Ubiquitous Multimedia, Stockholm, Sweden, 1–4 December 2024; Association for Computing Machinery: New York, NY, USA, 2024; pp. 264–279. [Google Scholar]
- Wobbrock, J.O.; Kane, S.K.; Gajos, K.Z.; Harada, S.; Froehlich, J. Ability-Based Design: Concept, Principles and Examples. ACM Trans. Access Comput. 2011, 3, 9. [Google Scholar] [CrossRef]
- Akpinar, E.; Yeşilada, Y.; Temizer, S. The Effect of Context on Small Screen and Wearable Device Users’ Performance—A Systematic Review. ACM Comput. Surv. (CSUR) 2020, 53, 1–44. [Google Scholar] [CrossRef]
- Komninos, A.; Dunlop, M.; Katsaris, K.; Garofalakis, J. A Glimpse of Mobile Text Entry Errors and Corrective Behaviour in the Wild. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, Barcelona, Spain, 3–6 September 2018; Association for Computing Machinery, Inc.: New York, NY, USA, 2018; pp. 221–228. [Google Scholar]
- Nicolau, H.; Montague, K.; Guerreiro, T.; Rodrigues, A.; Hanson, V.L. Investigating Laboratory and Everyday Typing Performance of Blind Users. ACM Trans. Access Comput. 2017, 10, 4. [Google Scholar] [CrossRef]
- Rodrigues, A.; Nicolau, H.; Santos, A.; Branco, D.; Rainey, J.; Verweij, D.; Smeddinck, J.D.; Montague, K.; Guerreiro, T. Investigating the Tradeoffs of Everyday Text-Entry Collection Methods. In Proceedings of the Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April–5 May 2022; Association for Computing Machinery: New York, NY, USA, 2022. [Google Scholar]
- Zhang, M.R.; Mariakakis, A.; Burke, J.; Wobbrock, J.O. A Comparative Study of Lexical and Semantic Emoji Suggestion Systems. In Proceedings of the Diversity, Divergence, Dialogue, Beijing, China, 17–31 March 2021; Toeppe, K., Yan, H., Chu, S.K.W., Eds.; Springer International Publishing: Cham, Switzerland, 2021; Volume 12645, pp. 229–247. [Google Scholar]
- Buschek, D.; Bisinger, B.; Alt, F. ResearchIME: A Mobile Keyboard Application for Studying Free Typing Behaviour in the Wild. In Proceedings of the Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 21–26 April 2018; Association for Computing Machinery: New York, NY, USA, 2018; Volume 2018, pp. 1–14. [Google Scholar]
- Evans, A.C.; Wobbrock, J.O. Taming Wild Behavior: The Input Observer for Obtaining Text Entry and Mouse Pointing Measures from Everyday Computer Use. In Proceedings of the Conference on Human Factors in Computing Systems, Austin, TX, USA, 5–10 May 2012; pp. 1947–1956. [Google Scholar]
- Ceolini, E.; Brunner, I.; Bunschoten, J.; Majoie, M.H.J.M.; Thijs, R.D.; Ghosh, A. A Model of Healthy Aging Based on Smartphone Interactions Reveals Advanced Behavioral Age in Neurological Disease. iScience 2022, 25, 104792. [Google Scholar] [CrossRef]
- Balerna, M.; Ghosh, A. The Details of Past Actions on a Smartphone Touchscreen Are Reflected by Intrinsic Sensorimotor Dynamics. NPJ Digit. Med. 2018, 1, 4. [Google Scholar] [CrossRef]
- van Nieuw Amerongen, A.R.; Meppelink, A.M.; Ghosh, A.; Thijs, R.D. Real-world Smartphone Data Can Trace the Behavioural Impact of Epilepsy: A Case Study. Eur. J. Neurol. 2024, 31, e16433. [Google Scholar] [CrossRef]
- Ceolini, E.; Ridderinkhof, K.R.; Ghosh, A. Age-Related Behavioral Resilience in Smartphone Touchscreen Interaction Dynamics. Proc. Natl. Acad. Sci. USA 2024, 121, e2311865121. [Google Scholar] [CrossRef]
- Wampfler, R.; Klingler, S.; Solenthaler, B.; Schinazi, V.R.; Gross, M.; Holz, C. Affective State Prediction from Smartphone Touch and Sensor Data in the Wild. In Proceedings of the Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April–5 May 2022; Association for Computing Machinery: New Orleans, LA, USA, 2022. [Google Scholar]
- Trifan, A.; Oliveira, M.; Oliveira, J.L. Passive Sensing of Health Outcomes through Smartphones: Systematic Review of Current Solutions and Possible Limitations. JMIR Mhealth Uhealth 2019, 7, e12649. [Google Scholar] [CrossRef] [PubMed]
- Arroyo-Gallego, T.; Ledesma-Carbayo, M.J.; Sanchez-Ferro, A.; Butterworth, I.; Mendoza, C.S.; Matarazzo, M.; Montero, P.; Lopez-Blanco, R.; Puertas-Martin, V.; Trincado, R.; et al. Detection of Motor Impairment in Parkinson’s Disease Via Mobile Touchscreen Typing. IEEE Trans. Biomed. Eng. 2017, 64, 1994–2002. [Google Scholar] [CrossRef] [PubMed]
- Dhir, N.; Edman, M.; Ferro, A.S.; Stafford, T.; Bannard, C. Identifying Robust Markers of Parkinson’s Disease in Typing Behaviour Using a CNN-LSTM Network. In Proceedings of the 24th Conference on Computational Natural Language Learning, Online, 19–20 November 2020; pp. 578–595. [Google Scholar]
- Iakovakis, D.; Hadjidimitriou, S.; Charisis, V.; Bostantjopoulou, S.; Katsarou, Z.; Klingelhoefer, L.; Reichmann, H.; Dias, S.B.; Diniz, J.A.; Trivedi, D.; et al. Motor Impairment Estimates via Touchscreen Typing Dynamics toward Parkinson’s Disease Detection from Data Harvested in-the-Wild. Front. ICT 2018, 5, 28. [Google Scholar] [CrossRef]
- Chen, M.H.; Leow, A.; Ross, M.K.; DeLuca, J.; Chiaravalloti, N.; Costa, S.L.; Genova, H.M.; Weber, E.; Hussain, F.; Demos, A.P. Associations between Smartphone Keystroke Dynamics and Cognition in MS. Digit. Health 2022, 8, 20552076221143234. [Google Scholar] [CrossRef] [PubMed]
- Hoeijmakers, A.; Licitra, G.; Meijer, K.; Lam, K.H.; Molenaar, P.; Strijbis, E.; Killestein, J. Disease Severity Classification Using Passively Collected Smartphone-Based Keystroke Dynamics within Multiple Sclerosis. Sci. Rep. 2023, 13, 1871. [Google Scholar] [CrossRef]
- Lam, K.H.; Meijer, K.A.; Loonstra, F.C.; Coerver, E.M.E.; Twose, J.; Redeman, E.; Moraal, B.; Barkhof, F.; de Groot, V.; Uitdehaag, B.M.J.; et al. Real-World Keystroke Dynamics Are a Potentially Valid Biomarker for Clinical Disability in Multiple Sclerosis. Mult. Scler. J. 2021, 27, 1421–1431. [Google Scholar] [CrossRef]
- Fadul, R.; AlShehhi, A.; Hadjileontiadis, L. Robust Remote Detection of Depressive Tendency Based on Keystroke Dynamics and Behavioural Characteristics. Sci. Rep. 2024, 14, 28025. [Google Scholar] [CrossRef]
- Mastoras, R.E.; Iakovakis, D.; Hadjidimitriou, S.; Charisis, V.; Kassie, S.; Alsaadi, T.; Khandoker, A.; Hadjileontiadis, L.J. Touchscreen Typing Pattern Analysis for Remote Detection of the Depressive Tendency. Sci. Rep. 2019, 9, 13414. [Google Scholar] [CrossRef]
- Knol, L.; Nagpal, A.; Leaning, I.E.; Idda, E.; Hussain, F.; Ning, E.; Eisenlohr-Moul, T.A.; Beckmann, C.F.; Marquand, A.F.; Leow, A. Smartphone Keyboard Dynamics Predict Affect in Suicidal Ideation. NPJ Digit. Med. 2024, 7, 54. [Google Scholar] [CrossRef]
- Bennett, C.C.; Ross, M.K.; Baek, E.G.; Kim, D.; Leow, A.D. Predicting Clinically Relevant Changes in Bipolar Disorder Outside the Clinic Walls Based on Pervasive Technology Interactions via Smartphone Typing Dynamics. Pervasive Mob. Comput. 2022, 83, 101598. [Google Scholar] [CrossRef]
- Zulueta, J.; Piscitello, A.; Rasic, M.; Easter, R.; Babu, P.; Langenecker, S.A.; McInnis, M.; Ajilore, O.; Nelson, P.C.; Ryan, K.; et al. Predicting Mood Disturbance Severity with Mobile Phone Keystroke Metadata: A Biaffect Digital Phenotyping Study. J. Med. Internet Res. 2018, 20, e241. [Google Scholar] [CrossRef]
- Al-Libawy, H.; Al-Ataby, A.; Al-Nuaimy, W.; Al-Taee, M.A.; Al-Jubouri, Q. Fatigue Detection Method Based on Smartphone Text Entry Performance Metrics. In Proceedings of the Proceedings—2016 9th International Conference on Developments in eSystems Engineering, DeSE 2016, Liverpool, UK, 31 August–2 September 2016; Institute of Electrical and Electronics Engineers Inc.: New York, NY, USA, 2017; pp. 40–44. [Google Scholar]
- Ross, M.K.; Demos, A.P.; Zulueta, J.; Piscitello, A.; Langenecker, S.A.; McInnis, M.; Ajilore, O.; Nelson, P.C.; Ryan, K.A.; Leow, A. Naturalistic Smartphone Keyboard Typing Reflects Processing Speed and Executive Function. Brain Behav. 2021, 11, e2363. [Google Scholar] [CrossRef]
- Akeret, K.; Vasella, F.; Geisseler, O.; Dannecker, N.; Ghosh, A.; Brugger, P.; Regli, L.; Stienen, M.N. Time to Be “Smart”—Opportunities Arising from Smartphone-Based Behavioral Analysis in Daily Patient Care. Front. Behav. Neurosci. 2018, 12, 303. [Google Scholar] [CrossRef]
- Kock, R.; Ceolini, E.; Groenewegen, L.; Ghosh, A. Neural Processing of Goal and Non-Goal-Directed Movements on the Smartphone. Neuroimage Rep. 2023, 3, 100. [Google Scholar] [CrossRef]
- Rizzolatti, G.; Craighero, L. The Mirror-Neuron System. Annu. Rev. Neurosci. 2004, 27, 169–192. [Google Scholar] [CrossRef]
Category of Approach | Methodologies and Key Characteristics |
---|---|
Controlled Laboratory Tasks | Description: Research conducted in a controlled environment, often using researcher-provided devices. Focus: Baseline performance, skill acquisition, controlled variable manipulation. Examples: Copy-typing tasks (transcribing standard phrases like Enron Mobile Email Database [96,97], MacKenzie Database [98]), composition tasks (generating freeform text or responses to prompts). Characteristics: High experimental control, precision, reproducibility. Limitations: Artificial context, limited ecological validity, potential for mental fatigue |
Real-World Data Collection | Description: Data collected from users in their natural daily environments. Focus: Typical performance, contextual factors influencing behavior, long-term monitoring. Examples: Experience Sampling Method (ESM) (users prompted for tasks on their own smartphones), Passive Sensing (background apps/custom keyboards unobtrusively collect typing data from everyday use). Characteristics: High ecological validity, allows for large-scale studies, captures naturalistic context (e.g., environment, mobility). Limitations: Lower experimental control, technical/ethical challenges (e.g., privacy, data consistency, noise from real-world contexts) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Viviani, L.; Liso, A.; Craighero, L. Mobile Typing as a Window into Sensorimotor and Cognitive Function. Brain Sci. 2025, 15, 1084. https://doi.org/10.3390/brainsci15101084
Viviani L, Liso A, Craighero L. Mobile Typing as a Window into Sensorimotor and Cognitive Function. Brain Sciences. 2025; 15(10):1084. https://doi.org/10.3390/brainsci15101084
Chicago/Turabian StyleViviani, Lorenzo, Alba Liso, and Laila Craighero. 2025. "Mobile Typing as a Window into Sensorimotor and Cognitive Function" Brain Sciences 15, no. 10: 1084. https://doi.org/10.3390/brainsci15101084
APA StyleViviani, L., Liso, A., & Craighero, L. (2025). Mobile Typing as a Window into Sensorimotor and Cognitive Function. Brain Sciences, 15(10), 1084. https://doi.org/10.3390/brainsci15101084