Next Article in Journal
Designing Slow Cities for More Than Human Enrichment: Dog Tales—Using Narrative Methods to Understand Co-Performative Place-Making
Next Article in Special Issue
Exploring Data-Driven Components of Socially Intelligent AI through Cooperative Game Paradigms
Previous Article in Journal
Choice, Control and Computers: Empowering Wildlife in Human Care
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

A Review of Automated Speech-Based Interaction for Cognitive Screening

SINTEF Digital, Forskningsveien 1, 0373 Oslo, Norway
Multimodal Technol. Interact. 2020, 4(4), 93;
Submission received: 24 October 2020 / Revised: 27 November 2020 / Accepted: 14 December 2020 / Published: 17 December 2020
(This article belongs to the Special Issue Speech-Based Interaction)


Language, speech and conversational behaviours reflect cognitive changes that may precede physiological changes and offer a much more cost-effective option for detecting preclinical cognitive decline. Artificial intelligence and machine learning have been established as a means to facilitate automated speech-based cognitive screening through automated recording and analysis of linguistic, speech and conversational behaviours. In this work, a scoping literature review was performed to document and analyse current automated speech-based implementations for cognitive screening from the perspective of human–computer interaction. At this stage, the goal was to identify and analyse the characteristics that define the interaction between the automated speech-based screening systems and the users, potentially revealing interaction-related patterns and gaps. In total, 65 articles were identified as appropriate for inclusion, from which 15 articles satisfied the inclusion criteria. The literature review led to the documentation and further analysis of five interaction-related themes: (i) user interface, (ii) modalities, (iii) speech-based communication, (iv) screening content and (v) screener. Cognitive screening through speech-based interaction might benefit from two practices: (1) implementing more multimodal user interfaces that facilitate—amongst others—speech-based screening and (2) introducing the element of motivation in the speech-based screening process.

1. Introduction

Language impairment is common in prodromal dementia states such as mild cognitive impairment (MCI). Well-documented literature has identified the early disruption of normative patterns and processing of speech and language to be characteristic in patients with Alzheimer’s disease and in MCI stages [1,2]. Early foundational clinical studies on language have highlighted changes in verbal fluency and naming [1,3].
Research has shown that MCI individuals with impairments in multiple domains, including language, are more likely to develop dementia compared to those with pure memory impairment [2]. Simple linguistic markers, such as word choice, verbal fluency, phrasing (i.e., ‘utterance’) and short speech patterns possess predictive power in assessing early cognitive decline status [1,4]. Language, speech and conversational behaviours reflect cognitive changes that may precede physiological changes and offer a much more cost-effective option for detecting preclinical cognitive decline [4,5].
Artificial intelligence (AI) and machine learning (ML) have been established as a means to facilitate automated speech-based cognitive screening through automated recording and analysis of linguistic, speech and conversational behaviours. Automatic speech recognition (ASR) and automatic speech analysis (ASA) can be used to (i) explore and discover latent speech patterns that are challenging to identify and analyse in a ‘manual’ way, (ii) provide a cost-effective method for cognitive screening by targeting the early onset of dementia and (iii) eventually place cognitive screening in non-clinical settings, away from ‘white coat’ effects and extra stress that the formal process may place on the screenee, potentially skewing the results.
In this work, a scoping literature review is performed to document and analyse current automated speech-based implementations for cognitive screening from the perspective of human–computer interaction (HCI). At this stage, the goal is to identify and analyse the characteristics that define the interaction between the automated speech-based screening systems and the users, potentially revealing interaction-related patterns and gaps. Therefore, this work can be of use to HCI designers, researchers and practitioners of the field aspiring to learn about existing practices related to automated speech-based cognitive screening and inform their designs accordingly. At an internal level, the scoping literature review is utilised to approach a specific topic and answer broad research questions within that context. It also serves as a preliminary stage for a systematic literature review that will cover the interaction themes discovered herein, as well as cognitive health-related aspects of the documented studies.
This paper is organised as follows. First, the literature review process is described (Section 2) and the review’s concept matrix is presented (Table 1). Next, findings from the review process are presented (Section 3). A discussion of the key findings and study limitations is presented in Section 4. The paper concludes in Section 5.

2. Literature Review

2.1. Research Questions

To assess the current state of the research in the field of cognitive screening using automated speech-based systems, the literature review will address two research questions (RQs):
  • RQ1: Which automated speech-based systems for cognitive screening have been studied?
  • RQ2: What are the interaction-related characteristics of the studied automated, speech-based systems for cognitive screening?
RQ1 focuses on identifying the automated speech-based systems that have been studied. The interaction aspects of the retrieved systems are important to further explore and analyse these systems. RQ2 examines these aspects.

2.2. Search Strategy

A literature search was performed in the Scopus academic search engine. The Scopus engine searches through the databases of publishers, such as ACM, IEEE, Elsevier, Springer, Taylor and Francis, Sage, Emerald, Oxford University Press, Cambridge University Press and many more. Scopus was chosen from amongst other academic search engines (e.g., Google Scholar, Web of Science) for the main search process owing to its wider coverage of scientific publishers, flexible result-filtering capabilities and the consistent accuracy of results [6,7]. The keywords used were: (‘dementia’ or ‘alzheimer’ or ‘impairment’) and (‘cognitive screening’ or ‘cognitive assessment’ or ‘preclinical screening’ or ‘preclinical assessment’) and (‘artificial intelligence’ or ‘AI’ or ‘speech recognition’ or ‘natural language processing’ or ‘speech analysis’). The publication’s title, abstract, and keywords were utilised for the retrieval of eligible articles. Eligible articles were also identified through backward reference searching, that is, by screening the reference lists of the retrieved publications [8].

2.3. Inclusion and Exclusion Criteria

Articles with the following characteristics were included in the literature review:
  • Written in English and accepted and presented in peer-reviewed publications,
  • Describing a speech-based system for cognitive screening that implements ASR and, possibly, ASA;
  • Describing a speech-based system that screens for cognitive impairment related to dementia; and
  • Evaluating the speech-based system for cognitive screening to some degree, either technically or empirically.
Based on inclusion criterion #4, conceptual work (e.g., conceptual description of systems, theoretical frameworks ) was excluded. Based on inclusion criterion #3, studies regarding cognitive screening outside the dementia domain were also excluded. Criterion #2 ensured that only automated approaches of speech-based interaction were included. Works in languages other than English or presented in pre-print form were also excluded (criterion #1).
The criteria’s formulation is based on the facts that: (i) the peer-review process adds to the credibility and reliability of the publications and the respective presented systems, and (ii) the evaluation of the speech-based system ensures that the systems are existing, usable, and beyond the conceptual level.

2.4. Screening Process and Results

In total, 65 articles (49 from Scopus and 16 from backward reference searching) were identified as appropriate for inclusion, from which 15 articles satisfied the inclusion criteria. The author, along with an independent expert of the field, reviewed all 15 articles independently. The categories and themes of the review were shaped by two reviewers, i.e., the author and an independent expert, based on the data extraction process. A high level of agreement (>80%) for the reviewers was achieved, regarding the inclusion/exclusion decisions and the formulated categories and themes. Any disagreements were discussed and settled.

2.5. Data Collection and Analysis

The data extracted from each article were as follows:
  • The source and full reference;
  • The description of the automated speech-based system(s) for cognitive screening, focusing on the technical and interaction aspects;
  • The cognitive screening tests that were utilised from the automated speech-based system(s) and their administration details.
The two reviewers jointly performed the data extraction process. Axial coding of the main themes took place so that each theme contains comparable and consistent categories. The main themes identified in the review and tabulated were
  • Studies presenting automated speech-based systems for cognitive screening (addressing RQ1);
  • The interaction aspects of these systems (addressing RQ2).
The themes were classified into a concept matrix to facilitate comparisons, provide structure and help clarify the concepts of the review for the reader [9,10]. Table 1 shows the concept matrix of the literature review.

3. Results

The literature review led to the documentation of five interaction-related themes, (i) user interface (UI), (ii) modalities, (iii) speech-based communication, (iv) screening content and (v) screener, all of which were further analysed in terms of their sub-categories and characteristics.

3.1. User Interface

The review showed that some automated speech-based systems for cognitive screening can screen exclusively for linguistic markers, thus featuring a voice user interface (VUI). Other systems can also feature multimodal UIs, which facilitate interaction in more ways than just speech. The target of these interfaces is to screen for additional markers to the linguistic ones, such as motor skills and auditory skills. Systems like those described by Tang et al. [4] and Mirheidari et al. [15] utilise VUI through several modalities, such as intelligent virtual agents (IVAs), robots, virtual assistants (VAs), and regular voice recorders. Naturally, this decision affects the nature of the screening test that accompanies these interfaces because its content should only support voice input and its results should be based on linguistic markers. Other systems, like the ones presented by Di Nuovo et al. [11] and Luperto et al. [14], utilise some of the modalities mentioned before to combine voice interfaces with other types of interfaces and offer multimodal interaction. Interaction then comes in the form of additional cognitive screening tasks, such as drawing a clock or a path, listening to an audio track or copying a cube, mainly originating from structured and validated cognitive screening tests, such as the Mini-Mental State Examination (MMSE) [24] and the Montreal Cognitive Assessment (MoCA) [25].

3.2. Modalities

Users interact with the automated speech-based systems for cognitive screening through several technologies. The literature review resulted in five modalities that are currently in speech-based systems: IVAs, VAs, socially assistive robots (SAR), audio recorders and telephone audio recorders. IVAs can ‘learn’ and be trained, thus enabling valuable real-time conversations between the system and the user [4] while VAs and SARs can provide screening tasks, recognise voice commands and facilitate multimodal interaction, as required from structured cognitive screening tests [14,19,23]. Voice recorders and telephones are less technologically advanced devices, which, however, can facilitate a less automated type of interaction that usually requires the assistance from a human screener to administer the cognitive test [12,18]. All the modalities implement ASR and, in cases, ASA to support the speech-based communication between the system and the user.

3.3. Speech-Based Communication

The way users/screenees communicate with the screeners can take three forms:
  • Structured speech-based communication: This type of communication is based on the screener evaluating the screenee on a particular set of predetermined, standardised, and speech-based screening tasks, where specific responses are expected for the screening score to be calculated. Speech-based, structured, and standardised cognitive screening tests, such as the speech-based versions of the MMSE and the MoCA, which ask for specific responses (e.g., what is the year or the season) from the screenee in order to score better, facilitate this type of communication [11,14,19,23].
  • Semi-structured speech-based communication: This type of communication is based on the screener evaluating the screenee on a particular set of predetermined, standardised screening tasks where the screenee’s responses are open-ended. Semi-structured and standardised cognitive screening tests, such as verbal fluency tests that, for example, ask the screenee to name as many animals as they can in one minute, facilitate this type of communication [15,18].
  • Unstructured speech-based communication: In this case, the screener and the screenee make conversation about various topics, and the evaluation of the screenee is based on linguistic markers and the quality of the dialogue. The user/screenee can converse with a human screener [16,17] or an AI agent [4].

3.4. Screening Content

The screening content affects interaction because its nature suggests the type of communication between the screener and the screenee, as mentioned above. Screening content for speech-based systems can come from validated tests (e.g., MMSE, MoCA and verbal fluency tests) and be used on an as-is basis [11,18,19] or adjusted to fit the purposes of the researcher and/or the screener [12,15]. Custom screening content is also utilised, especially in cases where the type of communication is unstructured (i.e., in conversations) [4,13,16,17].

3.5. Screener

The screening test can be administered by a human screener or an AI agent. In the former case, the human screener communicates the contents of the screening test to the user/screenee, and the ASR and ASA may take place later in an asynchronous manner. Audio recorders can document the communication and facilitate the asynchronous processing [5,12,18]. With an AI agent being the screener, the user communicates with IVAs, VAs, and SARs throughout the whole screening process. The ASR and ASA, in this case, may take place in a real-time manner, facilitated by AI and ML [4,11,15,19].

4. Discussion

A main observation that comes from the ‘speech-based communication’ category is the shortage of conversational agents for cognitive screening. The work of Tang et al. [4] presents an AI dialogue agent. However, the other two studies that support unstructured speech-based communication (i.e., [16,17]) are based on a human screener conducting the screening and the processing of asynchronous speech. A finely tuned speech-based, AI conversational agent would allow for more opportunistic use, thus screening for linguistic markers on a more frequent basis and increasing the chances of documenting early signs of cognitive decline. At the same time, its application for cognitive screening could be inexpensive, less stressful and presenting assessor neutrality, compared to having a formal healthcare process with a human screener [4,23]. However, more research is needed to construct reliable AI conversational agents since, currently, they present several limitations, such as their inability to fully capture the difference between human–AI and human–human conversations in real-world conversation (i.e., the human-in-the-loop problem [4,26]). Therefore, the accuracy of the dialogue simulation must be improved continuously.
When it comes to cognitive screening tests, Table 1 clearly shows that most studies utilise validated tests, either as-is (i.e., [19,23]) or adjusted to fit their research purposes (i.e., [12,15]). Pen-and-paper cognitive screening tests (e.g., MoCA and MMSE) are screening for various cognitive functions, such as orientation, attention, memory, language and abstraction [24,25]. Their standardised form supports structured speech-based communication between the screener and screenee. In this case, speech analysis is limited to identifying the screenee’s answers and mark them as right or wrong. The same speech analysis process is followed for standardised—and potentially validated—cognitive screening tests specifically tailored for speech-based screening (e.g., semantic verbal fluency tests); only in this case, the screenee’s responses are open-ended, and thus semi-structured speech-based communication with the screener is established. A number of studies utilise custom cognitive screening tests, mainly for conversations. These tests rely heavily on ASA for identifying linguistic markers in an unstructured communication context. Naturally, the unstructured nature of these screening tests makes it more challenging to get validated; thus, further research on their validity, sensitivity and specificity is necessary.
The results of this review suggest that cognitive screening through speech-based interaction might benefit from two practices. Firstly, multimodal UIs supporting speech-based interaction are scarce. Multimodal UIs can facilitate the screening of multiple cognitive functions. Apart from linguistic markers that can be covered by speech-based interaction and AI conversational agents, other cognitive and cognitive-related functions, such as motor, auditory, and visuospatial functions, can be screened by other modalities, including writing, listening to sounds and so on. Multimodal interventions have been shown to be significant in the fight against cognitive decline as they cover a wide range of cognitive decline markers [27]. Respective UIs that include speech-based interaction—and, potentially, screening conversations—can mark the next generation of multimodal cognitive screening systems. Secondly, the element that was missing from the reviewed studies and their screening processes was motivation. Motivation is an important element in cognitive screening for increasing the screening frequency (and thus the chances for detecting cognitive decline) and making the process less stressful and more enjoyable [28,29,30]. Gamified screening and serious games for screening have been suggested as effective ways of adding motivation in the cognitive screening process [28,29,30,31,32]. Therefore, a cognitive screening solution that (i) features multimodal functionality facilitating speech-based screening and (ii) introduces the element of motivation (e.g., by gamifying the screening process) might be an interesting subject for future research with true potential.

Study Limitations

As stated earlier, this review does not cover the cognitive health-related aspects of the documented studies. At this early stage, this study is focused on identifying the interaction aspects of speech-based screening because it is an important aspect that can affect screening results. Cognitive health-related aspects will be the subject of future work as described in Section 5. Moreover, in the analysis of results, no distinction was made between synchronous (real-time) and asynchronous ASR and ASA for cognitive screening. The reason for this was as follows: (i) in its essence, speech-based cognitive screening can facilitate and be of great value for detecting early signs of cognitive decline whether the processing of speech data takes place in real time or after a few minutes/hours, and (ii) in several reviewed articles, the descriptions around data processing were not clear or were missing. Naturally, in future work, this information will be retrieved by contacting respective authors as the timing of data processing is a feature that can affect the design and development of future speech-based cognitive screening systems.

5. Conclusions

Overall, this work—despite its descriptive nature—can lay the groundwork for researchers, designers and practitioners of the field to build on and further examine and produce prescriptive approaches around the use of automated speech analysis for cognitive screening. The herein documented studies reveal considerable interest in the topic over the last three years (with 9 out of 15 studies taking place from 2017 to 2020) and this work may act as a guide, assisting interesting parties in the field to get an overview on speech-based interaction for cognitive screening and the interaction-related practices and gaps in the field. Interaction is an important practice for speech-based cognitive screening systems that can operate as a lower implementation layer, on which the cognitive health-related content can be based and that—if not treated in an informed way—it may affect/skew the screening results. In this work, the conducted literature review managed to produce the interaction-related themes that are necessary for describing and classifying speech-based cognitive screening and the concept matrix of the review managed to highlight the underlying relationships among the interaction attributes of the systems.
Future work will address the cognitive health-related aspects of speech-based cognitive screening systems focusing, amongst others, on their screening evaluation methodologies, their screening validity and performance and the screenees’ cognitive status or impairment that they target. To that end, a systematic literature review will be utilised, being an extension of the herein presented scoping review, and the interaction elements of the studied systems will, naturally, be revisited.


This research was supported by the SINTEF project ‘Cognitive Screening with AI Conversational Agents’, funded by the Basic Funding through the Research Council of Norway.


I would like to thank Md Zia Uddin (PhD, IEEE Senior Member, and Research Scientist at SINTEF Digital) for his valuable assistance.

Conflicts of Interest

The author declares no conflict of interest.


  1. Asgari, M.; Kaye, J.; Dodge, H. Predicting mild cognitive impairment from spontaneous spoken utterances. Alzheimer’s Dement. Transl. Res. Clin. Interv. 2017, 3, 219–228. [Google Scholar] [CrossRef]
  2. Taler, V.; Phillips, N.A. Language performance in Alzheimer’s disease and mild cognitive impairment: A comparative review. J. Clin. Exp. Neuropsychol. 2008, 30, 501–556. [Google Scholar] [CrossRef] [PubMed]
  3. Murphy, K.J.; Rich, J.B.; Troyer, A.K. Verbal fluency patterns in amnestic mild cognitive impairment are characteristic of Alzheimer’s type dementia. J. Int. Neuropsychol. Soc. 2006, 12, 570–574. [Google Scholar] [CrossRef] [PubMed]
  4. Tang, F.; Uchendu, I.; Wang, F.; Dodge, H.H.; Zhou, J. Scalable diagnostic screening of mild cognitive impairment using AI dialogue agent. Sci. Rep. 2020, 10, 1–11. [Google Scholar] [CrossRef] [PubMed]
  5. König, A.; Satt, A.; Sorin, A.; Hoory, R.; Toledo-Ronen, O.; Derreumaux, A.; Manera, V.; Verhey, F.; Aalten, P.; Robert, P.H.; et al. Automatic speech analysis for the assessment of patients with predementia and Alzheimer’s disease. Alzheimer Dement. Diagn. Assess. Dis. Monit. 2015, 1, 112–124. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Falagas, M.E.; Pitsouni, E.I.; Malietzis, G.A.; Pappas, G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: Strengths and weaknesses. FASEB J. 2008, 22, 338–342. [Google Scholar] [CrossRef] [PubMed]
  7. Jacso, P. As we may search—Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Curr. Sci. 2005, 89, 1537–1547. [Google Scholar]
  8. Vom Brocke, J.; Simons, A.; Niehaves, B.; Riemer, K.; Plattfaut, R.; Cleven, A. Reconstructing the giant: On the importance of rigour in documenting the literature search process. In Proceedings of the European Conference on Information Systems, Verona, Italy, 8–10 June 2009; pp. 2206–2217. [Google Scholar]
  9. Webster, J.; Watson, R.T. Analyzing the past to prepare for the future: Writing a literature review. MIS Q. 2002, 26, 13–23. [Google Scholar]
  10. Boletsis, C. The new era of virtual reality locomotion: A systematic literature review of techniques and a proposed typology. Multimodal Technol. Interact. 2017, 1, 24. [Google Scholar] [CrossRef] [Green Version]
  11. Di Nuovo, A.; Varrasi, S.; Lucas, A.; Conti, D.; McNamara, J.; Soranzo, A. Assessment of cognitive skills via human-robot interaction and cloud computing. J. Bionic Eng. 2019, 16, 526–539. [Google Scholar] [CrossRef]
  12. Hakkani-Tür, D.; Vergyri, D.; Tur, G. Speech-based automated cognitive status assessment. In Proceedings of the Eleventh Annual Conference of the International Speech Communication Association, Chiba, Japan, 26–30 September 2010; pp. 258–261. [Google Scholar]
  13. López-de Ipiña, K.; Alonso, J.B.; Barroso, N.; Faundez-Zanuy, M.; Ecay, M.; Solé-Casals, J.; Travieso, C.M.; Estanga, A.; Ezeiza, A. New approaches for Alzheimer’s disease diagnosis based on automatic spontaneous speech analysis and emotional temperature. In International Workshop on Ambient Assisted Living; Springer-Verlag: Berlin/Heidelberg, Germany, 2012; pp. 407–414. [Google Scholar]
  14. Luperto, M.; Romeo, M.; Lunardini, F.; Basilico, N.; Abbate, C.; Jones, R.; Cangelosi, A.; Ferrante, S.; Borghese, N.A. Evaluating the Acceptability of Assistive Robots for Early Detection of Mild Cognitive Impairment. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, Macau, China, 3–8 November 2019; pp. 1–8. [Google Scholar]
  15. Mirheidari, B.; Blackburn, D.; O’Malley, R.; Walker, T.; Venneri, A.; Reuber, M.; Christensen, H. Computational cognitive assessment: Investigating the use of an intelligent virtual agent for the detection of early signs of dementia. In Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 12–17 May 2019; pp. 2732–2736. [Google Scholar]
  16. Mirheidari, B.; Blackburn, D.; Reuber, M.; Walker, T.; Christensen, H. Diagnosing people with dementia using automatic conversation analysis. In Proceedings of the Interspeech—ISCA, San Francisco, CA, USA, 8–12 September 2016; pp. 1220–1224. [Google Scholar]
  17. Mirheidari, B.; Blackburn, D.; Harkness, K.; Walker, T.; Venneri, A.; Reuber, M.; Christensen, H. Toward the automation of diagnostic conversation analysis in patients with memory complaints. J. Alzheimer’s Dis. 2017, 58, 373–387. [Google Scholar] [CrossRef] [Green Version]
  18. Pakhomov, S.V.; Marino, S.E.; Banks, S.; Bernick, C. Using automatic speech recognition to assess spoken responses to cognitive tests of semantic verbal fluency. Speech Commun. 2015, 75, 14–26. [Google Scholar] [CrossRef] [Green Version]
  19. Prange, A.; Niemann, M.; Latendorf, A.; Steinert, A.; Sonntag, D. Multimodal speech-based dialogue for the Mini-Mental State Examination. In Proceedings of the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK, 4–9 May 2019; pp. 1–8. [Google Scholar]
  20. Roark, B.; Hosom, J.P.; Mitchell, M.; Kaye, J.A. Automatically derived spoken language markers for detecting mild cognitive impairment. In Proceedings of the Second International Conference on Technology and Aging—ICTA, Toronto, ON, USA, 9–11 May 2007. [Google Scholar]
  21. Tóth, L.; Hoffmann, I.; Gosztolya, G.; Vincze, V.; Szatlóczki, G.; Bánréti, Z.; Pákáski, M.; Kálmán, J. A speech recognition-based solution for the automatic detection of mild cognitive impairment from spontaneous speech. Curr. Alzheimer Res. 2018, 15, 130–138. [Google Scholar] [CrossRef] [PubMed]
  22. Tröger, J.; Linz, N.; König, A.; Robert, P.; Alexandersson, J. Telephone-Based Dementia Screening I: Automated Semantic Verbal Fluency Assessment. In Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare, New York, NY, USA, 21–24 May 2018; pp. 59–66. [Google Scholar]
  23. Varrasi, S.; Di Nuovo, S.; Conti, D.; Di Nuovo, A. A social robot for cognitive assessment. In Proceedings of the Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 5–8 March 2018; pp. 269–270. [Google Scholar]
  24. Folstein, M.F.; Folstein, S.E.; McHugh, P.R. “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 1975, 12, 189–198. [Google Scholar] [CrossRef]
  25. Nasreddine, Z.S.; Phillips, N.A.; Bédirian, V.; Charbonneau, S.; Whitehead, V.; Collin, I.; Cummings, J.L.; Chertkow, H. The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 2005, 53, 695–699. [Google Scholar] [CrossRef] [PubMed]
  26. Waibel, A.; Steusloff, H.; Stiefelhagen, R.; Watson, K. Computers in the human interaction loop. In Computers in the Human Interaction Loop; Springer-Verlag: London, UK, 2009; pp. 3–6. [Google Scholar]
  27. Burgener, S.C.; Yang, Y.; Gilbert, R.; Marsh-Yant, S. The effects of a multimodal intervention on outcomes of persons with early-stage dementia. Am. J. Alzheimer’s Dis. Other Dementias® 2008, 23, 382–394. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Boletsis, C.; McCallum, S. Smartkuber: A serious game for cognitive health screening of elderly players. Games Health J. 2016, 5, 241–251. [Google Scholar] [CrossRef] [PubMed]
  29. Tong, T.; Guana, V.; Jovanovic, A.; Tran, F.; Mozafari, G.; Chignell, M.; Stroulia, E. Rapid deployment and evaluation of mobile serious games: A cognitive assessment case study. Procedia Comput. Sci. 2015, 69, 96–103. [Google Scholar] [CrossRef] [Green Version]
  30. Verhaegh, J.; Fontijn, W.F.; Aarts, E.H.; Resing, W.C. In-game assessment and training of nonverbal cognitive skills using TagTiles. Pers. Ubiquitous Comput. 2013, 17, 1637–1646. [Google Scholar] [CrossRef]
  31. Tong, T.; Chignell, M.; Tierney, M.C.; Lee, J. A serious game for clinical assessment of cognitive status: Validation study. JMIR Serious Games 2016, 4, e7. [Google Scholar] [CrossRef]
  32. Tong, T.; Chignell, M.; Lam, P.; Tierney, M.C.; Lee, J. Designing serious games for cognitive assessment of the elderly. In Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, Chicago, IL, USA, 16–19 March 2014; SAGE Publications Sage CA: Los Angeles, CA, USA, 2014; Volume 3, pp. 28–35. [Google Scholar]
Table 1. The concept matrix of the literature review.
Table 1. The concept matrix of the literature review.
ReferencesUser InterfaceModalitiesSpeech-BasedCommunicationScreeningContentScreener
VoiceMultimodalIntelligent Virtual AgentVirtual AssistantSocially Assistive RobotAudio RecorderAudio Recorder via TelephoneStructuredSemi-StructuredUnstructuredAs-Is Validated TestAdjusted Validated TestCustom TestAI AgentHuman
Di Nuovo et al. (2019) [11]X X XX X
Hakkani-Tür et al. (2010) [12]X X X X X
König et al. (2015) [5]X X XX X
López-de-Ipiña et al. (2012) [13]X X X XX
Luperto t al. (2019) [14]X X XX X
Mirheidari et al. (2019) [15]XX X X X
Mirheidari et al. (2016) [16]X X X XX
Mirheidari et al. (2017) [17]X X X XX
Pakhomov et al. (2015) [18]X X XX X
Prange et al. (2019) [19]X X XX X
Roark et al. (2007) [20]X X XX X
Tang et al. (2020) [4]XX X XX
Tóth et al. (2018) [21]X X XX X
Tröger et al. (2018) [22]X XXX X
Varrasi et al. (2018) [23]X X XX X
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Boletsis, C. A Review of Automated Speech-Based Interaction for Cognitive Screening. Multimodal Technol. Interact. 2020, 4, 93.

AMA Style

Boletsis C. A Review of Automated Speech-Based Interaction for Cognitive Screening. Multimodal Technologies and Interaction. 2020; 4(4):93.

Chicago/Turabian Style

Boletsis, Costas. 2020. "A Review of Automated Speech-Based Interaction for Cognitive Screening" Multimodal Technologies and Interaction 4, no. 4: 93.

Article Metrics

Back to TopTop