Next Article in Journal
Development of Air Quality Boxes Based on Low-Cost Sensor Technology for Ambient Air Quality Monitoring
Next Article in Special Issue
Development of a Non-Contacting Muscular Activity Measurement System for Evaluating Knee Extensors Training in Real-Time
Previous Article in Journal
Numerical Study of the Thermal Performance of a Mems Pressure Sensor with Self-Calibration Capabilities
Previous Article in Special Issue
Diagnostics of Articular Cartilage Damage Based on Generated Acoustic Signals Using ANN—Part II: Patellofemoral Joint
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 22
Issue 10
10.3390/s22103829
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assisting Personalized Healthcare of Elderly People: Developing a Rule-Based Virtual Caregiver System Using Mobile Chatbot †

by
Chisaki Miura
1,
Sinan Chen
1,*,
Sachio Saiki
2,
Masahide Nakamura
1,3 and
Kiyoshi Yasuda
4
1
Graduate School of System Informatics, Kobe University, 1-1 Rokkodai-cho, Nada, Kobe 657-8501, Japan
2
Department of Data & Innovation, Kochi University of Technology, 185 Miyanigutu, Tosayamada-cho, Kami-shi, Kochi 782-8502, Japan
3
RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan
4
Osaka Institute of Technology, 5-16-1 Omiya, Asahi-ku, Osaka 535-8585, Japan
*
Author to whom correspondence should be addressed.
†
This paper is an extended version of the conference papers: Chisaki, M.; Sachio, S.; Masahide, N.; Kiyoshi, Y. Implementing and Evaluating feedback feature of Mind Monitoring Service for Elderly People at Home. In Proceedings of the iiWAS 2020, Online, 30 November–2 December 2020.
Sensors 2022, 22(10), 3829; https://doi.org/10.3390/s22103829
Submission received: 6 April 2022 / Revised: 14 May 2022 / Accepted: 17 May 2022 / Published: 18 May 2022
(This article belongs to the Special Issue Recent Advancements in Sensor Technologies for Healthcare and Biomedical Applications)
Browse Figures
Review Reports Versions Notes

Abstract

:
To assist personalized healthcare of elderly people, our interest is to develop a virtual caregiver system that retrieves the expression of mental and physical health states through human–computer interaction in the form of dialogue. The purpose of this paper is to implement and evaluate a virtual caregiver system using mobile chatbot. Unlike the conventional health monitoring approach, our key idea is to integrate a rule-based virtual caregiver system (called “Mind Monitoring” service) with the physical, mental, and social questionnaires into the mobile chat application. The elderly person receives one question from the mobile chatbot per day, and answers it by pushing the optional button or using a speech recognition technique. Furthermore, a novel method is implemented to quantify the answers, generate visual graphs, and send the corresponding summaries or advice to the specific elder. In the experimental evaluation, we applied it to eight elderly subjects and 19 younger subjects within 14 months. As main results, its effects were significantly improved by the proposed method, including the above 80% in the response rate, the accurate reflection of their real lives from the responses, and high usefulness of the feedback messages with software quality requirements and evaluation. We also conducted interviews with subjects for health analysis and improvement.

1. Introduction

As an aging population worldwide, the shortage of human caregivers and nursing facilities is becoming a serious problem [1,2,3]. Although the Japanese government is focusing on the shift to in-home care from facility care [4], it takes big burdens on family caregivers, such as 24 h nursing care, one elderly person caring for another, and the elderly living alone. With the big wave of Information and Communication Technology (ICT) and Internet of Things (IoT), study and development for assisting personalized healthcare of elderly people using a machine instead of family caregivers have been gaining popularity in order to reduce families’ burdens [5,6,7,8,9]. In the personalized healthcare of the elderly using machines (i.e., smart healthcare), it often needs two parts of work for elderly people, including situational recognition and care nursing intervention. Studies of situation recognition include in-home fine-grained scenes (i.e., called context) recognition [10,11,12]), nonintrusive quality characterization of postural changes [13], and user-defined indoor location sensing [14]. As for care nursing intervention, it is common to use physical care robots [15,16,17,18], chatbots [19,20,21], and virtual agents [22,23,24,25,26]. The core point of these studies is to encourage dialogue-based interaction between machines and elderly people. However, assisting elderly people by merely relying on the progress of study and development is not enough. Specifically, for the case using the technology of nursing intervention for a long-term period, it is still not definite what effect was achieved from the standpoint of the elderly person. That contains not only technical performance, but also its usability and acceptability.
Our interest is to develop a novel virtual caregiver system (called “mind monitoring” service) for elderly people. We use the term “mind" to represent an expression from the heart. It includes the expression of mental and physical health states obtained by individual elderly people. Unlike the conventional approach, which detects external situations (e.g., temperature, motion, etc.) using various sensors [27,28], we principally focus on the inner expressions (i.e., thinking and feeling) of elderly people. In previous studies, to realize a smooth care nursing intervention, a speech-based interaction between virtual agents (i.e., MMDAgent [29]) and elderly people on a general computer has been proposed [30,31,32], such as confirming the daily state [33], answering the counseling [34,35,36], and extending to micro service such as watching favorite videos [37,38]. Our key concept is to regard a virtual agent as a caregiver, called a virtual caregiver [39]. We also present a method that allows users to define the appearance of virtual agents [40,41]. The ideal vision of a virtual caregiver is to perform various person-centric [42,43] care to improve the Quality of Life (QoL) [44,45,46] of not only elderly people but also family caregivers.
The main contribution of this paper is to implement a rule-based mind monitoring service using a mobile chatbot and apply it to eight elderly subjects (in their 50s to 80s) and 19 younger subjects (in their 20s to 40s) within 14 months to evaluate the effects. In existing approaches with the mind monitoring service, due to maintenance difficulties for the general computers (e.g., OS version update, Internet instability, etc.) at some time, most existing studies have to stop at technical development and short-term experiments. Although some problems relying on the developer using remote desktop tools can be improved, it is still a big challenge to the different network environments. Hence, a method to evaluate the effects of mind monitoring for personalized healthcare of elderly people is desperately needed, especially for evaluating whether elderly people can be accustomed to the form of virtual caregivers. In addition, more useful feedback messages from users need to be collected, and an extensive experiment for many elderly people to improve the monitoring service is also urgent.
The previous version of this paper was published as a conference paper [47]. The most significant change made to this version is the addition of implementation with the mind monitoring service, and experimental evaluation of the collected responses and feedback messages. Unlike the existing approach with the general computer, we advocate implementing the mind monitoring service using the mobile chatbot (i.e., LINE application [48,49,50]). Referring to the definition of health from the World Health Organization (WHO), the light spot of this paper is to integrate a rule-based virtual caregiver system with the physical, mental, and social questionnaires [51,52,53] into the mobile chat application. In the proposed method, the elderly person receives one question from the mobile chatbot per day, and first needs to push the optional button (e.g., have you slept well for the past week? Yes or no.). Then, the mobile chatbot further asks a question to collect detailed information from the last answer. To encourage the usage of the mind monitoring service based on the mobile chatbot, the most workout points in the implementation include developing a method to quantify the answers, generate visual graphs, and send the corresponding summaries or advice to the specific elder.
Based on the implementation of the mind monitoring service, we conducted an actual experimental evaluation of the collected answers, feedback messages, and the mind monitoring service itself with Software Quality Requirements and Evaluation (SQuaRE) [54,55]. As the main result, the response rate for six in eight elderly subjects was above 90%, and the average for 19 younger subjects was around 80%. Meanwhile, the collected response messages reflected the important events in their real lives, such as a lot of stress from COVID-19 due to social activities limited by the curfew. Through the questionnaires with the SQuaRE, most of the subjects were able to reflect on their own state using the implemented service, and interacting with the chatbot was not laborious, nor difficult to answer questions in terms of frequency. Thus, the effects including usability and acceptability of mind monitoring based on the mobile chatbot were significantly improved by the implemented method. The remainder of this paper is organized as follows. In Section 2, we discuss related works on technologies for assisting elderly people, including chatbot technology. We provide a detailed description of the proposed method in Section 3 and the experimental evaluation in Section 4, followed by the conclusions in Section 5.

2. Related Work

With the focus on technologies for assisting personalized healthcare of elderly people, exploiting smart care robots [16,56,57,58,59] is a promising way to support the independence of elderly people, as well as to relieve the burden on caregivers and families. However, the use of care robots is not widely spread yet, since deploying and operating a care robot at home is expensive. As one way to solve these challenges, the virtual agent technology [39,40,60,61] has been gathering great attention recently. The virtual agent (VA) is an animated, human-like graphical robot displayed on the screen, and serves as the intuitive user interface of the system through voice interactions in natural language. Since VA only requires a general computer for deployment and operation, the cost is much lower than that of physical robots. Moreover, another advantage is that developers can customize the VA according to the application. MMDAgent is one of the well-known powerful toolkits to implement VAs [29]. Using a three-dimensional movie model of MMD together with built-in speech recognition and synthesis technologies, MMDAgent allows a user to easily develop a custom VA with a spoken dialogue system. In [62], by utilizing the MMDAgent, we have developed a system called virtual care giver (VCG), where the virtual agent named “Mei” provides personalized cares for each elderly person [63]. The VCG supports a wide range of care, such as reminders to check for forgotten items, warnings for forgetting to take a medicine, and playing music videos on YouTube as a value-added service for entertainment [37,38]. Delegating the communication care with the agent, we believe human caregivers can concentrate on human-centric tasks that cannot be achieved with the machine. In order to provide appropriate care for each elderly person, it is essential to know what the person is thinking and what the person wants to do. However, internal states such as thoughts, feelings, or emotions cannot be observed from the outside, and cannot be detected by general sensors or the Internet of Things (IoT). Hence, our research group has been developing a new type of sensing technique, called mind sensing (“kokoro” sensing in Japanese) [60,64]. It aims to record the internal mind of a target person that cannot be observed externally by general sensors or IoT. In the mind sensing, firstly, a virtual agent (VA) or a chatbot asks the target elderly person various questions. Then, the person externalizes his/her mind as words by answering the question. Finally, the externalized data are stored as internal states, and utilized for appropriate supports or carers [31,32,35].
The history of chatbot technology dates back to 1950. A mathematician, Alan Turing, proposed the question “Can machines think?” in his paper “Computing Machinery and Intelligence”, which conceptualized chatbots [65]. Then, in 1966, the world’s first chatbot, ELIZA [66], was developed by Joseph, and since then, research on chatbots has developed. Today, the chatbot is a familiar technology for human beings, and it is used in many situations [67,68,69,70,71]. In particular, the use of chatbots in the field of mental health care has attracted a great deal of attention recently, because the chatbot has more potential to make people talk honestly than a real person [72,73,74]. According to the study of Lucas et al. in 2014 [75], it became clear that virtual humans increase the willingness to disclose. More specifically, researchers reported a study in which all participants spoke to a virtual human on a computer screen, but half of the participants were told the virtual human was a computer, and the other half were told it was controlled by an actual human. The results showed participants who believed they were interacting with a computer reported lower fear of self-disclosure, and more willingness to disclose their emotions. These results suggest that automated virtual humans can help overcome a significant barrier to obtaining truthful patient information. From this study, the fact that chatbots also have the potential to make users talk honestly about their feelings and thoughts was found, which led to an increasing usage of chatbots in the mental health field.
One example of developing a chatbot for supporting mental health care was carried out by Kowatsch et al. in 2017 [76]. Since the resources of health professionals are limited and they cannot monitor or support patients in their everyday life, Kowatsch et al. developed a text-based healthcare chatbot (THCB) which can effectively support both patients and health professionals. In the THCB, the main targets are young patients, and they can talk with a chatbot by answering questions regarding their conditions. During the interaction, if the user’s quality of life scores or other critical health states show a negative trend, the chatbot automatically sends notifications to health professionals, so that the professionals can conduct immediate and appropriate interventions. Since the professionals do not have to constantly monitor the status of their patients, it will help to reduce their burden. Other studies include the development of a mental health care chatbot on Discord for people who frequently use their PCs. In [77], the chatbot has a feature that assigns scores to the user’s emotions obtained during interaction with a chatbot, to realize continuous moods tracking of the user. Although the ease of interaction with a chatbot and usefulness of the moods tracking feature was confirmed through a subject experiment, the experiment was small, and further study is needed. Another example is an AI-based virtual health assistant named Vitalk [78]. This is an application for smart phones, and in this application, the chatbot gives advice and health information for users. The chatbot also adapts personalized user’s interests by applying machine learning. The engagement and effectiveness of Vitalk have been evaluated preliminarily in 2020 [79]. Moreover, using a mobile application for ecological momentary assessment (EMA) to collect data on stress and mood in a daily life setting is also presented [80]. The EMA is also called the experience sampling method that asks participants to respond to their thoughts, feelings, and actions and those of their environment in various situations and time streams. Although our approach in this paper is similar to the EMA method, the difference for us is to take older adults who require long-term care at home as the target population. In order to further reduce the burden of family caregivers, a health questionnaire and a chatbot is used to encourage homebound elderly to achieve self-care. The novelty contribution of this paper is that the virtual caregiver regularly asks the elderly appropriate questions from three perspectives: physical, mental, and social. The responses are automatically rated and visualized. This way gives the elderly a clearer picture of their responses over time and encourages better self-care.

3. Methodology

3.1. Previous Study

In order to provide appropriate care for each elderly person, it is essential to know what the person is thinking and what the person wants to do. However, internal states such as thoughts, feelings, or emotions cannot be observed from the outside, and cannot be detected by general sensors or IoT. An example of comparing the sensor-based and chatbot-based approaches for the elderly healthcare is shown in Figure 1. Hence, our research group has been developing a new type of sensing technique, called mind sensing (“kokoro” sensing in Japanese) [81]. It aims to record the internal mind of a target person that cannot be observed externally by general sensors or IoT. In the mind sensing, firstly, a virtual agent (VA) or a chatbot asks the target elderly person various questions. Then, the person externalizes his/her mind as words by answering the question. Finally, the externalized data are stored as internal states, and utilized for appropriate supports or cares.

3.2. Proposed Method

We propose a mind monitoring service in this section. Exploiting the mind sensing service with LINE chatbot, the service tries to externalize the internal state (i.e., mental state) of the target elderly person. The externalized state is recorded for monitoring. Then, feedbacks are sent to the elderly person to make the person reflect and improve his/her living. Figure 2 shows the overall architecture of the proposed mind monitoring service. The service consists of the following three elements: (A1) Interaction with a chatbot using mind sensing service. (A2) Inquiry method specialized for the acquisition of mental state. (A3) Self-care assistance and feedback by monitoring mental state.

3.2.1. A1: Interaction with Chatbot Using Mind Sensing Service

To establish continuous interaction between an elderly person and the proposed service, we exploit the mind sensing service with a LINE chatbot. LINE [82] is a widespread messaging smart phone application in Japan. The reason why we chose the LINE chatbot for the interaction tool is that the chatbot is easier and more reliable to exchange questions and answers, compared to the VA on the PC or email. As explained in the next section, inquiries for acquiring the mental states are somehow technical. Thus, the text message over LINE is clearer than the voice dialogue with the VA. In addition, LINE has an intuitive user interface, as well as text input via voice. In addition, it is already accepted by many elderly people. Hence, we consider that LINE is more suitable than email.
Using the time-based rule of the mind sensing service (see Section 3.1), the LINE chatbot delivers one question per day at the designated time to the elderly person. The time of the delivery can be configured based on the preference. Figure 3 (left) shows a screen shot of the proposed service, where the LINE chatbot asks a question about sleep. As will be explained in the next section, every question simply expects either yes (positive) or no (negative). The elderly person can easily answer the question by tapping one of the choices on the screen. As the elderly person answers the question, an event consisting of timestamps, user ID, question ID, and answer label is notified to our web server. For this, by utilizing the event-based rule of mind sensing service, we command the chatbot to a reply message based on the answer. Within the reply message, the chatbot asks an additional question to investigate more detailed context. Figure 3 (right) shows the case where the user answers “Yes, I’ve slept well.” to the question about the sleep. After receiving the event with a positive answer, the chatbot sends a reply with an additional inquiry to ask about any concerns regarding sleep. The elderly person then inputs detailed information by text. The chatbot finally replies with a stamp (a graphic) as the gratitude for externalizing the details.

3.2.2. A2: Inquiry Method Specialized for Acquisition of Mental State

It is essential for the proposed service to determine what to ask the elderly people and how. Since the primary goal of the service is to support healthy living, we first referred to WHO’s “health” definition [83], capturing the health as a complete state of physical, mental, and social well-being. Hence, we here introduce a framework evaluating the elderlies’ health from the three perspectives physicality, mentality, and sociality. More specifically, the physicality evaluates the physical symptoms that can be seen objectively, such as fatigue, pain, and sleep disorders. Mentality evaluates the subjective feelings such as emotions, moods, and stress. Sociality evaluates the self-evaluations or behaviors of a target person, such as happiness, self-esteem, and social behavior. We then develop questions that acquire the internal states for each of the three perspectives. In our previous study [84], we took questions from GDS-15 [85], PHQ-9 [86], GAD-7 [87], and GHQ60 [88], and implemented 42 questions over the three aspects in the proposed service. However, this approach failed. In the preliminary experiment, there were many complaints from elderly subjects, for instance, “the questions were quite technical and difficult to answer”, “I didn’t understand the meaning of the scale”,“I was tired and gave up for answering too many questions”, etc. Thus, we found that these clinical questionnaires were not suitable to the mind monitoring in their daily living. Hence, we consult a clinical expert to develop more casual questions, so as not to cause too much workload to elderly people. As a result, we have determined seven questions shown in Table 1. To promote easy and quick answers, each question simply expects either yes or no. In the table, “Feature” indicates what to survey by the question. “Category” represents a health category of the framework. In creating questions, we have focused on understanding the person’s conditions approximately. Namely, we narrowed down the seven fundamental features to be monitored within daily living: sleep, health, emotion, memory, psychology, motivation, and socialization. To reduce the burden, we configure the chatbot interaction (A1) to send one question once a day at the designated time. Thus, the seven questions are covered in a week, and the first question is sent again in the next week. Note that each question is asking the “condition of this past week” with respect to the feature.

3.2.3. A3: Self-Care Assistance and Feedback by Monitoring Mental State

Upon receiving every answer from the elderly person, the proposed service stores the answer in our time-series database. Since the seven questions are covered in a week, the proposed service evaluates the collected answers on a weekly basis, and sends feedback based on the result. This section first presents a method of scoring the answers for evaluating the mental states. We then describe a method of generating weekly feedback for promoting spontaneous self-care.
The challenge here is how to quantify (i.e., score) the collected answers. We consulted a clinical expert to consider how a third party (e.g., doctor, caregiver, family) tries to assess the target person’s condition from his/her answer. As a result, we found that there are typically three steps. The first step is to check if the answer is positive or negative, which is the direct objective of the question. The second step is to check how the answer has changed compared to the past. For example, if the answer is negative, we should consider if the answer had been negative for a long time or it has suddenly turned negative. The third step is to understand what the target person said additionally. Considering the above three steps, we propose the following three scoring methods: score answer, score observation, and score sentiment. The three scoring methods evaluate a given answer from different perspectives. Hence, we define the total score S t o t a l of the answer to be the weighted sum of them: S t o t a l = w 1 · S a n s w e r + w 2 · S o b s e r v a t i o n + w 3 · S s e n t i m e n t . Note that, here, w 1 + w 2 + w 3 = 1. Strictly speaking, the weight w i should be adjusted through clinical trials. In this study, however, we simply set w 1 = w 2 = w 3 = 1 / 3 , since this is the first experiment.
Based on the evaluation result, the proposed service generates and sends feedback containing greeting, summary of the answers, and advice/hints for better living. The purpose of the feedback is to promote the user’s self-reflection and spontaneous mental health care. As explained earlier, the evaluation of the collected answers is conducted on a weekly basis. Thus, the feedback is also generated once a week on a designated day of week. In the proposed service, the weekly feedback is executed as follows: (1) The service selects one question whose answer’s score ( S t o t a l ) is the worst in a week. (2) The service generates a feedback message based on the selected answer. To make the message as natural as possible, we structured the feedback message in four paragraphs: (a) greeting, (b) reflection, (c) advice, and (d) conclusion. In (a), the chatbot starts with greeting according to the current season or climate. In (b), the chatbot shows how the user answered the selected question, to get the person to look back him/herself. In (c), the chatbot shows advice or hints for better living with respect to the corresponding question. For this, we refer to the information of “Kenko-Choju Net” [89], which provides knowledge base about health and longevity for Japanese elderly people. In (d), the chatbot concludes the feedback with an encouraging remark. Figure 4 shows an example of the weekly feedback message with the structured four paragraphs. In this feedback, the question about “psychology” was picked up. The chatbot indicates that the user said she had been feeling anxiety before. The chatbot then suggests that she should have her family or friends listen to her anxiety. By receiving such feedback on a weekly basis, users can periodically reflect on their mental states. The service can also promote the user’s self-care consciousness.

3.3. Implementation

We have implemented the proposed mind monitoring service using the following technologies:
  • Chatbot interaction: mind sensing service [81] with Java, Eclipse Jersey [90], LINE Mmessaging API [48].
  • Question and message editing, nanagement: Google spreadsheet [91].
  • Automatic rule update: Python, cron [92].
  • Database: MySQL 5.7.27 [93].
To achieve efficient mind monitoring, we have also implemented a web application, called Data Visualizer, which supports users to see the collected time-series data. Data Visualizer is not only for the elderly person him/herself, but also for surrounding people (e.g., family, caregivers, and doctors) who want to monitor the target person. The technologies used for Data Visualizer are as follows.
  • Development language: Java, JavaScript, HTML, CSS [94].
  • Web service framework: Spring Boot 2.3.0 [95].
  • Template engine: Thymeleaf [96].
  • JavaScript library: jQuery 3.5.1 [97], Chart.js 2.7.2 [98].
  • Web server: Apache Tomcat 9.0.29 [99].
Data Visualizer is mainly designed for smart phones and has the four features.
  • Reviewing answer logs: The user can reflect the interaction with a chatbot.
  • Weekly score check: The user can check the score of mental states for each week.
  • Monthly score check: The user can check the score of mental states for each month.
  • Mind Monitoring: The user can check the score of mental states for each year.
Figure 5a,b show the top screen of Data Visualizer and Mind Monitoring feature page, respectively. In the graph of Figure 5b, the vertical axis represents the score value and the horizontal axis represents months. The horizontal axis can also be changed to weeks. The scores are averaged over the month for each of the categories: physicality, mentality, and sociality.

4. Evaluation

4.1. Experimental Setup

To evaluate the proposed mind monitoring service, we conducted an experiment in a real setting. In the experiment, subjects operate the service with their own smart phones for a long term. The operation period was from 1 November 2019 to 31 January 2021, one year and two months (14 months) in total. The demographic characteristics of the participants in the experimental evaluation are shown in Table 2. Eight elderly subjects (in their 50s to 80s) participated in the experiment. In addition to them, we also recruited 19 men and women in their 20s to 40s. The proposed service was originally intended for the elderly generation. Meanwhile, we also wanted to evaluate the effectiveness of the proposed service itself. This experiment was conducted by asking acquaintances of the research group members and others willing to cooperate. During the experiment, if the subject feels any burden, the person can quit answering questions from the chatbot at any time. In addition, we explained to the subjects that the logs of their interactions with the chatbot would be stored in our private database, and would not be disclosed to any third party except the experiment administrators. The experiment was approved by the research ethics committee of Graduate School of System Informatics, Kobe University (No. R01-02). Written informed consent was obtained from subjects for publication of this paper and accompanying images. Based on the data collected in the experiment, we evaluate the proposed mind sensing service from the following viewpoints: (1) responsiveness and continuity; (2) findings from mental state data. (3) Investigation of quality in use.

4.2. Results

4.2.1. Responsiveness and Continuity

Our first interest here is to see if elderly people can keep using the proposed service in their daily life. In the experiment, two elderly subjects quit using the service within a few months. As for one elderly person (male in the 70s), it was difficult for him to operate the service, because he was not using the smart phone frequently in his daily life. The other elderly person (male in the 70s) had been using the service for the first three months, but he eventually stopped using it. This was because his asthma worsened and the disease made it difficult for him to answer questions from the chatbot every day. For the remaining 25 subjects, we were able to get them to use the proposed service continuously. We especially show the response rates of elderly subjects for the entire 14 months in Table 3. The response rate is calculated by the ratio of the number of questions answered by the subject to the number of questions sent by the chatbot. From Table 3, we can see that four out of six elderly subjects responded to more than 90% of the questions from a chatbot. For subjects C and E, the overall response rate was low, not because they had stopped using the service, but simply because they responded less frequently. In other words, we could not obtain high frequency of responses from subjects C and E, but we were able to get them to answer the questions periodically. Regarding the other 19 subjects (in their 20s to 40s), the average of the response rates was around 80%. Considering these results, it was confirmed that the responsiveness and continuity of the service were relatively good.

4.2.2. Findings from Mental State Data

Our next interest was to see if the collected mental state data really reflect the minds of elderly people. Through 14 months of the experiment, the service obtained a large amount of time-series data. Using Data Visualizer, we first investigated the time-series data of each subject. Then, we conducted interviews with some elderly subjects to ask what happened by showing the visualized data. Figure 6 and Figure 7 show graphs of the mental state scores of two elderly subjects, subject A (male in 70s) and subject D (female in 70s), in 2020. In the graph, the vertical axis represents the average score value and the horizontal axis represents months. The blue, yellow, and green lines represent the scores of physicality, mentality, and sociality, respectively.
In Figure 6, we can see that subject A’s scores of each perspective are generally positive. This means his condition is relatively stable throughout the year, and his mental health in terms of physical, mental, and social is maintained to some extent. However, we can see that his physicality score dropped sharply in the middle of May. When we asked subject A for the reason, he told us that he had hurt his leg at that time because of too much walking. Afterwards, thanks to treatment and rehabilitation, his leg finally started to get better around July. In addition, we find that his sociality score dropped in early March. This was because the spread of coronavirus (COVID-19) reduced his opportunities to venture out. For a while after that, he stopped exercising at the gym, and his sociality score continued to stagnate. However, he started to go to the gym again around October, and his sociality score started to increase. In Figure 7, the mentality score and the sociality score of subject D are much lower than physicality score. When we asked subject D about her situation in 2020, she told us that her sister had passed away in January and she had been experiencing a terrible sense of loss. This sense of loss continued until around October, and her mentality went through a series of manic depressive cycles. She also had a lifestyle in which her days and nights were reversed. In contrast, her physicality score tended to be relatively positive, but we can also find a sudden decrease in her physicality score around July. In fact, at that time, she was suffering from dizziness caused by otolith detachment. Later, as she learned how to live well with her illness, her physicality score gradually recovered.
Thus, for the data in the months, we confirmed that the time-series mental state data were well reflected by important events in their real living. On the other hand, for the detail every day, further questioning based on the answer collected the exact reasons for physicality, mentality, and sociality positively or negatively. For this, we especially focus on the representative further explanations after the negative sociality answered by subjects in the experiment shown in Table 4. From a lot of text messages, although the detailed reasons were different by individual subjects, we can also see that either the older or younger subjects felt much stress from COVID-19, as the social activities were significantly limited by the curfew.

4.2.3. Investigation of Quality in Use

Our final interest is to see if the proposed mind monitoring service is satisfactory for users including not only the elderly but also young people. We believe that there is a need to analyze the experimented service using objective measures. In particular, the evaluation from the user’s viewpoints (e.g., the usability of the subjects) is indispensable. For this, we conducted a “quality evaluation during use” to evaluate the operation and usability of the system during use. More specifically, we evaluated the proposed service using the international software quality standard, SQuaRE (Systems and software engineering Systems and software Quality Requirements and Evaluation, ISO/IEC 25000 Series) [55].
It defines quality requirements, models, and attributes by which stakeholders evaluate the quality of the system and products from various viewpoints. The quality in use is evaluated through actual usage by the users. For this, we created a questionnaire to cover the above characteristics except for environmental risk mitigation. This is because our service does not have any influence on environmental risks. Table 5 shows the questions and results of the questionnaire regarding the above characteristics. In each question, our proposed service is named Mei-chan, which is the display name of the LINE chatbot, so that users can easily understand it. The questionnaire was answered by 18 subjects. Each question was answered by four level scales: strongly agree, agree, disagree, and strongly disagree. Based on the result, we evaluate the proposed service from the viewpoints of effectiveness, efficiency, usefulness, comfort, and flexibility.
Effectiveness:
Effectiveness means the degree of accuracy and completeness with which the user achieves the stated goals. Thus, the effectiveness of the proposed service can be assessed by whether the users were able to reflect on their own state and achieve spontaneous mental health care. From Q1, we can see that most of the subjects were able to reflect on their own states. However, Q2 shows that the subjects could not make much efforts to improve their mental states. Thereby, although we were able to confirm the effectiveness of the service to some extent, it was not sufficient.
Efficiency:
Efficiency means the degree of the resources used by a user for achieving a particular goal, so the efficiency of the proposed service can be evaluated by the ease of interacting with a chatbot. From Q3 and Q4, it can be seen that interacting with the chatbot was not laborious, nor difficult to answer questions in terms of frequency. Hence, the efficiency of the service was confirmed.
Usefulness:
Usefulness means the degree of user satisfaction. Hence, the usefulness can be measured by how much the user finds the service useful. From Q5, more than half of the subjects answered that interacting with the chatbot was useful for reflecting on their own states, while nearly 30% did not. In addition, since it was revealed that some subjects did not want to continue to use the service in Q6, we could not confirm the usefulness of the service very well.
Comfort:
Comfort means the degree of user satisfaction with comfort when using the system or software. Thus, the comfort can be evaluated by the ease of using the service including the operability. Since Q7 and Q8 show that the subjects were able to interact with the chatbot and reflect on their own conditions easily, we can confirm the comfort of the proposed service.
Flexibility:
Flexibility means the degree to which a product or system can maintain the quality even in a unexpected usage situation. In the proposed service, users are expected to answer one question from a chatbot every day. However, from Q9, we found that there were some cases where the subjects entered their answers at a later time in batches. Although these cases were not originally intended, the fact that there was no problem in using the service suggests that the service maintained a certain degree of flexibility. In addition, in Q10, the subjects said that this kind of flexibility was important.

4.3. Discussion

By looking at the changes in the scores for physicality, mentality, and sociality perspective, we were able to analyze the subjects’ mental states in more detail. Since these data are indispensable for understanding the user context, the ability to continuously obtain such information would be the most beneficial aspect of the proposed service. From further observing mental state scores in this experiment, we found two kinds of characteristics in the results. The first characteristic is acute change. In Figure 6 and Figure 7, two subjects had a sudden decrease of their physicality scores due to injury or illness (reasons collected by the subject’s further explanations). In both cases, there was a tendency that the score of physicality suddenly and drastically decreased, and the mentality and sociality scores also decreased along with the decrease of physicality. The second characteristic is chronic stagnation. Subject D felt disorders due to the loss experience from January to October. During this period, the physicality score was basically positive, while the mentality and sociality scores were negative. These results mean that subject D did not have objective symptoms of physical discomfort, but she felt mental discomfort such as lack of motivation. We found that when the scores of mentality and sociality are always stagnant at negative values, even though the physicality score is positive, there may be a possibility that the target person’s invisible disorder is hidden.
Features of the experimented service can be roughly contained into the following: visualization and monitoring of mental states and promotion of spontaneous self-care. Firstly, as for visualization and monitoring of mental state in the experimented service, since we were able to acquire mental state data for more than one year, and to formulate the monitoring process by analyzing the characteristics of scores, we believe this purpose has been achieved. As for the advantages, it is noteworthy that we were able to continuously acquire mental state data during such a long period of operation. We consider that this is because the efficiency and flexibility of the service had a positive impact on the subjects’ continuity. In other words, the fact that it was easy for the subjects to interact with a chatbot and that they were able to use the service whenever they wanted may have promoted the continuity of service use. As for the limitations, promotion of spontaneous mental health care has not been achieved, considering the results of the questionnaire. We thought that reflecting on one’s own condition would lead to self-care, but in fact, although the subjects were able to reflect on their own states, they could not achieve improvements or self-care. In addition, as revealed in the questionnaire, the usefulness of the service was also insufficient. Moreover, there were comments such as “I felt that chatbot’s replies were like fixed form sentences.” or “I think it would be easier to continue if the chatbot could provide more interesting information or unusual topics.” For this, we will devise methods to not make users feel boring, including to change questions or to provide interesting information for users. Meanwhile, we will also plan to focus on two aspects: (1) Analyzing the results of the mind monitoring automatically. (2) Introducing gamification into the mind monitoring in the future.

5. Conclusions

In this paper, to assist personalized healthcare for elderly people, we specifically focused on the implementation and evaluation of a virtual caregiver system using a mobile chatbot. The main outcomes include the following: (1) Integrating a rule-based mind monitoring service with the physical, mental, and social questionnaires into the mobile chat application. (2) Developing a method to quantify the answers, generate visual graphs, and send the corresponding summaries or advice to the specific elder. (3) Conducting an experiment to evaluate the effects, including usability and acceptability of mind monitoring based on the mobile chatbot. Through the experiments, the important events in subjects’ real lives could be reflected well by the collected response messages, the usability and acceptability of the proposed mind monitoring are significant, and confirming the advantage that the effect is improved by the implemented method.
Improving personalized healthcare using chatbot technology is a worthy research topic, especially for elderly people, even though this paper has some points that need to be improved. Although the experimental results show that the subjects were able to interact with the chatbot and reflect on their own conditions easily, some subjects could not make much efforts to improve their mental states. We consider that the individuality of its effects still needs to be deeply studied as a case study, including personal culture, habits, and life rhythms. Furthermore, integrating chatbot and sensor-based technologies is also an interesting task for us. Unlike the current time-based questionnaires for a static one-week window, it can be developed to event-based using the Internet of Things (IoT) technologies and sensor devices, such as physical sensors (e.g., motion and press) and environmental sensors (e.g., temperatures and humidity). Meanwhile, the retrieved feature values can apply to the Event Condition Action (ECA) rules. Users can define these conditions or generate them automatically by machine learning or deep learning. In this way, it can achieve the dynamic window size of the questionnaire depending on the individual events and users. Furthermore, linking a chatbot and wearable smartwatch to achieve real-time health tracking and timely personalized healthcare questionnaires is also a promising direction in the future.

Author Contributions

Writing—original draft preparation, C.M.; writing—review and editing, S.C. and S.S.; supervision, M.N. and K.Y.; validation, S.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was approved by the Research Ethics Committee of Kobe University Graduate School of System Informatics (No. R01-02).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Acknowledgments

This research was partially supported by JSPS KAKENHI Grant Numbers JP19H01138, JP18H03242, JP18H03342, JP19H04154, JP19K02973, JP20K11059, JP20H04014, JP20H05706 and Tateishi Science and Technology Foundation (C) (No.2207004).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Wright, J. Robots vs migrants? Reconfiguring the future of Japanese institutional eldercare. Crit. Asian Stud. 2019, 51, 331–354. [Google Scholar] [CrossRef]
  2. Wright, J. Tactile care, mechanical Hugs: Japanese caregivers and robotic lifting devices. Asian Anthropol. 2018, 17, 24–39. [Google Scholar] [CrossRef]
  3. Elsy, P. Elderly care in the society 5.0 and kaigo rishoku in Japanese hyper-ageing society. J. Studi Komun. 2020, 4, 435–452. [Google Scholar] [CrossRef]
  4. Song, P.; Tang, W. The community-based integrated care system in Japan: Health care and nursing care challenges posed by super-aged society. Biosci. Trends 2019, 13, 279–281. [Google Scholar] [CrossRef] [Green Version]
  5. Ho, A. Are we ready for artificial intelligence health monitoring in elder care? BMC Geriatr. 2020, 20, 1–7. [Google Scholar] [CrossRef]
  6. Parker, M.W.; Davis, C.; White, K.; Johnson, D.; Golden, M.; Zola, S. Reduced care burden and improved quality of life in African American family caregivers: Positive impact of personalized assistive technology. Technol. Health Care 2021, 30, 379–387. [Google Scholar] [CrossRef]
  7. Holthe, T.; Jentoft, R.; Arntzen, C.; Thorsen, K. Benefits and burdens: Family caregivers’ experiences of assistive technology (AT) in everyday life with persons with young-onset dementia (YOD). Disabil. Rehabil. Assist. Technol. 2018, 13, 754–762. [Google Scholar] [CrossRef]
  8. Sriram, V.; Jenkinson, C.; Peters, M. Informal carers’ experience of assistive technology use in dementia care at home: A systematic review. BMC Geriatr. 2019, 19, 1–25. [Google Scholar] [CrossRef] [Green Version]
  9. Aung, M.N.; Moolphate, S.; Yuasa, M.; Aung, T.N.N.; Koyanagi, Y.; Supakankunti, S.; Ahmad, I.; Kayano, R.; Ong, P. Community-Integrated Intermediary Care (CIIC) Service Model to Enhance Family-Based, Long-Term Care for Older People: Protocol for a Cluster Randomized Controlled Trial in Thailand. JMIR Res. Protoc. 2021, 10, e20196. [Google Scholar] [CrossRef]
  10. Chen, S.; Saiki, S.; Nakamura, M. Toward Affordable and Practical Home Context Recognition: Framework and Implementation with Image-based Cognitive API–. Int. J. Networked Distrib. Comput. 2019, 8, 16–24. [Google Scholar] [CrossRef] [Green Version]
  11. Chen, S.; Saiki, S.; Nakamura, M. Integrating Multiple Models Using Image-as-Documents Approach for Recognizing Fine-Grained Home Contexts. Sensors 2020, 20, 666. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Chen, S.; Saiki, S.; Nakamura, M. Toward flexible and efficient home context sensing: Capability evaluation and verification of image-based cognitive APIs. Sensors 2020, 20, 1442. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Chen, S.; Saiki, S.; Nakamura, M. Nonintrusive fine-grained home care monitoring: Characterizing quality of in-home postural changes using bone-based human sensing. Sensors 2020, 20, 5894. [Google Scholar] [CrossRef] [PubMed]
  14. Chen, S.; Saiki, S.; Nakamura, M. Using Human Pose Estimation for User-Defined Indoor Location Sensing. In Proceedings of the 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Kassel, Germany, 22–26 March 2021; pp. 495–501. [Google Scholar]
  15. Noda, K.; Wada, Y.; Saiki, S.; Nakamura, M.; Yasuda, K. Implementing Personalized Web News Delivery Service Using Tales of Familiar Framework. In Proceedings of the 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Athens, Greece, 19–23 March 2018; pp. 759–764. [Google Scholar]
  16. Abou Allaban, A.; Wang, M.; Padır, T. A systematic review of robotics research in support of in-home care for older adults. Information 2020, 11, 75. [Google Scholar] [CrossRef] [Green Version]
  17. Niemelä, M.; Melkas, H. Robots as social and physical assistants in elderly care. In Human-Centered Digitalization and Services; Springer: Berlin/Heidelberg, Germany, 2019; pp. 177–197. [Google Scholar]
  18. Yang, G.; Pang, Z.; Deen, M.J.; Dong, M.; Zhang, Y.T.; Lovell, N.; Rahmani, A.M. Homecare robotic systems for healthcare 4.0: Visions and enabling technologies. IEEE J. Biomed. Health Inform. 2020, 24, 2535–2549. [Google Scholar] [CrossRef]
  19. Ryu, H.; Kim, S.; Kim, D.; Han, S.; Lee, K.; Kang, Y. Simple and Steady Interactions Win the Healthy Mentality: Designing a Chatbot Service for the Elderly. Proc. ACM Hum.-Comput. Interact. 2020, 4, 1–25. [Google Scholar] [CrossRef]
  20. Zhang, H.; Zheng, J. The Application Analysis of Medical Chatbots and Virtual Assistant. Front. Soc. Sci. Technol. 2021, 3, 11–16. [Google Scholar]
  21. Da Paixao Pinto, N.; Garcia, A.C.B. Uso de Chatbots Personalizados para Monitoração do Desaprendizado e Esquecimento de Idosos. In Proceedings of the Anais Estendidos do XVII Simpósio Brasileiro de Sistemas Colaborativos, Porto Alegre, Brazil, 12 April 2021; pp. 42–47. [Google Scholar]
  22. Stara, V.; Vera, B.; Bolliger, D.; Rossi, L.; Felici, E.; Di Rosa, M.; de Jong, M.; Paolini, S. Usability and Acceptance of the Embodied Conversational Agent Anne by People with Dementia and Their Caregivers: Exploratory Study in Home Environment Settings. JMIR mHealth uHealth 2021, 9, e25891. [Google Scholar] [CrossRef]
  23. De Jong, M.; Hettinga, M.; Stara, V.; Evers, V.; Li, J. Eldertainment or functional necessity? how virtual agents affect the home lives of people with dementia using the quality of life (QOL-AD) scale. In Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, London, UK, 9–13 September 2019; pp. 41–44. [Google Scholar]
  24. Sidner, C.L.; Bickmore, T.; Nooraie, B.; Rich, C.; Ring, L.; Shayganfar, M.; Vardoulakis, L. Creating new technologies for companionable agents to support isolated older adults. ACM Trans. Interact. Intell. Syst. (TiiS) 2018, 8, 1–27. [Google Scholar] [CrossRef]
  25. Bravo, S.L.; Herrera, C.J.; Valdez, E.C.; Poliquit, K.J.; Ureta, J.C.; Cu, J.; Azcarraga, J.J.; Rivera, J.P. CATE: An Embodied Conversational Agent for the Elderly. In Proceedings of the ICAART (2), Valletta, Malta, 22–24 February 2020; pp. 941–948. [Google Scholar]
  26. Da Paixão Pinto, N.; dos Santos França, J.B.; de Sá Sousa, H.P.; Vivacqua, A.S.; Garcia, A.C.B. Conversational Agents for Elderly Interaction. In Proceedings of the 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Dalian, China, 28 May 2021; pp. 1–6. [Google Scholar]
  27. Sakakibara, S.; Saiki, S.; Nakamura, M.; Matsumoto, S. Implementing Autonomous Environmental Sensing in Smart City with IoT-Based Sensor Box and Cloud Services. Inf. Eng. Express 2018, 4, 1–10. [Google Scholar] [CrossRef]
  28. Niu, L.; Saiki, S.; Nakamura, M. Using non-intrusive environmental sensing for adls recognition in one-person household. Int. J. Softw. Innov. (IJSI) 2018, 6, 16–29. [Google Scholar] [CrossRef]
  29. Lee, A.; Oura, K.; Tokuda, K. MMDAgent-A fully open-source toolkit for voice interaction systems. In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 26–31 May 2013; pp. 8382–8385. [Google Scholar]
  30. Takatsuki, D.; Saiki, S.; Nakamura, M. Using Virtual Agent for Facilitating Online Questionnaire Surveys. In Proceedings of the 2018 International Conference on Signal Processing and Information Security (ICSPIS), Dubai, United Arab Emirates, 7–8 November 2018; pp. 1–4. [Google Scholar]
  31. Ozono, H.; Chen, S.; Nakamura, M. Study of microservice execution framework using spoken dialogue agents. In Proceedings of the 22th IEEE-A CIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel Distributed Computing (SNPD2021), Taichung, Taiwan, 24–26 November 2021. [Google Scholar]
  32. Chen, S.; Nakamura, M. Designing an Elderly Virtual Caregiver Using Dialogue Agents and WebRTC. In Proceedings of the 2021 4th International Conference on Signal Processing and Information Security (ICSPIS), Dubai, United Arab Emirates, 24–25 November 2021; pp. 53–56. [Google Scholar]
  33. Tokunaga, S.; Horiuchi, H.; Tamamizu, K.; Saiki, S.; Nakamura, M.; Yasuda, K. Deploying Service Integration Agent for Personalized Smart Elderly Care. In Proceedings of the 15th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2016), Okayama, Japan, 26–29 June 2016; pp. 897–902. [Google Scholar]
  34. Nakatani, S.; Saiki, S.; Nakamura, M.; Yasuda, K. Implementation and evaluation of personal ontology building system with virtual agent. In Proceedings of the International Conference on Human-Computer Interaction, Orlando, FL, USA, 26–31 July 2019; pp. 391–403. [Google Scholar]
  35. Chen, S.; Nakamura, M. Generating personalized dialogues based on conversation log summarization and sentiment analysis. In Proceedings of the 23rd International Conference on Information Integration and Web Intelligence, Linz, Austria, 29 November–1 December 2021; pp. 217–222. [Google Scholar]
  36. Chen, S.; Nakamura, M.; Saiki, S. Developing a platform of personalized conversation scenarios for in-home care assistance. In Proceedings of the 2021 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT), Bandung, Indonesia, 27–28 July 2021; pp. 148–153. [Google Scholar]
  37. Hirayama, K.; Saiki, S.; Nakamura, M. Evaluating Video Playing Application for Elderly People at Home by Facial Expression Sensing Service. In Proceedings of the 22nd International Conference on Information Integration and Web-based Applications & Services, Chiang Mai, Thailand, 30 November– 2 December 2020; pp. 21–27. [Google Scholar]
  38. Hirayama, K.; Chen, S.; Saiki, S.; Nakamura, M. Toward Capturing Scientific Evidence in Elderly Care: Efficient Extraction of Changing Facial Feature Points. Sensors 2021, 21, 6726. [Google Scholar] [CrossRef] [PubMed]
  39. Nakamura, M. Virtual Care Giver: Virtual Agent for Personalized Home Elderly Care. Impact 2018, 2018, 31–33. [Google Scholar] [CrossRef]
  40. Nakatani, S.; Saiki, S.; Nakamura, M.; Yasuda, K. Generating personalized virtual agent in speech dialogue system for people with dementia. In Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management; Springer: Berlin/Heidelberg, Germany, 2018; pp. 326–337. [Google Scholar]
  41. Nakatani, S.; Saiki, S.; Nakamura, M. Integrating 3d facial model with person-centered care support system for people with dementia. In Intelligent Human Systems Integration; Springer: Berlin/Heidelberg, Germany, 2018; pp. 216–222. [Google Scholar]
  42. Tamamizu, K.; Tokunaga, S.; Saiki, S.; Matsumoto, S.; Nakamura, M.; Yasuda, K. Towards Person-Centered Anomaly Detection and Support System for Home Dementia Care. In Proceedings of the 18th International Conference on Human-Computer Interaction (HCII2016), Toronto, ON, Canada, 17–22 July 2016; Springer International Publishing: Cham, Switzerland, 2016; Volume 9745, pp. 274–285. [Google Scholar]
  43. Maeda, H.; Saiki, S.; Nakamura, M.; Yasuda, K. Log4Care: Unified Event Logging Service for Personalized Care. In Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management; Springer: Berlin/Heidelberg, Germany, 2018; pp. 466–477. [Google Scholar]
  44. Suhr, R.; Lahmann, N.A. Urinary incontinence in home care: A representative multicenter study on prevalence, severity, impact on quality of life, and risk factors. Aging Clin. Exp. Res. 2018, 30, 589–594. [Google Scholar] [CrossRef]
  45. Pequeno, N.P.F.; de Araújo Cabral, N.L.; Marchioni, D.M.; Lima, S.C.V.C.; de Oliveira Lyra, C. Quality of life assessment instruments for adults: A systematic review of population-based studies. Health Qual. Life Outcomes 2020, 18, 1–13. [Google Scholar] [CrossRef]
  46. Burks, H.B.; des Bordes, J.K.; Chadha, R.; Holmes, H.M.; Rianon, N.J. Quality of Life Assessment in Older Adults with Dementia: A Systematic Review. Dement. Geriatr. Cogn. Disord. 2021, 50, 103–110. [Google Scholar] [CrossRef]
  47. Miura, C.; Saiki, S.; Nakamura, M.; Yasuda, K. Implementing and Evaluating feedback feature of Mind Monitoring Service for Elderly People at Home. In Proceedings of the 22nd International Conference on Information Integration and Web-Based Applications & Services, Chiang Mai, Thailand, 30 November–2 December 2020; pp. 390–395. [Google Scholar]
  48. Darmaastawan, K.; Sukarsa, I.; Buana, P.W. LINE Messenger as a Transport Layer to Distribute Messages to Partner Instant Messaging. Int. J. Mod. Educ. Comput. Sci. 2019, 11, 1–9. [Google Scholar] [CrossRef]
  49. Suana, W.; Distrik, I.W.; Herlina, K.; Maharta, N.; Putri, N.M.A.A. Supporting blended learning using mobile instant messaging application: Its effectiveness and limitations. Int. J. Instr. 2019, 12, 1011–1024. [Google Scholar] [CrossRef]
  50. Haristiani, N.; Danuwijaya, A.; Rifai, M.; Sarila, H. Gengobot: A chatbot-based grammar application on mobile instant messaging as language learning medium. J. Eng. Sci. Technol. 2019, 14, 3158–3173. [Google Scholar]
  51. Loos, E.; Kaufman, D. Positive impact of exergaming on older adults’ mental and social well-being: In search of evidence. In Human Aspects of IT for the Aged Population; Springer: Berlin/Heidelberg, Germany, 2018; pp. 101–112. [Google Scholar]
  52. Zhang, L.; Zhou, S.; Kwan, M.P. A comparative analysis of the impacts of objective versus subjective neighborhood environment on physical, mental, and social health. Health Place 2019, 59, 102170. [Google Scholar] [CrossRef]
  53. Gothe, N.P.; Ehlers, D.K.; Salerno, E.A.; Fanning, J.; Kramer, A.F.; McAuley, E. Physical activity, sleep and quality of life in older adults: Influence of physical, mental and social well-being. Behav. Sleep Med. 2020, 18, 797–808. [Google Scholar] [CrossRef] [PubMed]
  54. Komiyama, T.; Azuma, M. Establishing International Standards for Systems and Software Quality Requirements and Evaluation. In Proceedings of the IWESQ@ APSEC, Singapore, 1 December 2020; pp. 7–10. [Google Scholar]
  55. International Organization for Standardization. Systems and Software Quality Requirements and Evaluation (SQuaRE). Available online: https://www.iso.org/standard/35733.html/ (accessed on 1 January 2022).
  56. Do, H.M.; Pham, M.; Sheng, W.; Yang, D.; Liu, M. RiSH: A robot-integrated smart home for elderly care. Robot. Auton. Syst. 2018, 101, 74–92. [Google Scholar] [CrossRef]
  57. Bui, H.D.; Chong, N.Y. An integrated approach to human-robot-smart environment interaction interface for ambient assisted living. In Proceedings of the 2018 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), Genova, Italy, 27–29 September 2018; pp. 32–37. [Google Scholar]
  58. Pereyda, C.; Raghunath, N.; Minor, B.; Wilson, G.; Schmitter-Edgecombe, M.; Cook, D.J. Cyber-physical support of daily activities: A robot/smart home partnership. ACM Trans. Cyber-Phys. Syst. 2019, 4, 1–24. [Google Scholar] [CrossRef]
  59. Wilson, G.; Pereyda, C.; Raghunath, N.; de la Cruz, G.; Goel, S.; Nesaei, S.; Minor, B.; Schmitter-Edgecombe, M.; Taylor, M.E.; Cook, D.J. Robot-enabled support of daily activities in smart home environments. Cogn. Syst. Res. 2019, 54, 258–272. [Google Scholar] [CrossRef]
  60. Maeda, H.; Saiki, S.; Nakamura, M.; Yasuda, K. Memory Aid Service Using Mind Sensing and Daily Retrospective by Virtual Agent. In Human-Computer Interaction; Springer: Berlin/Heidelberg, Germany, 2019; pp. 353–364. [Google Scholar]
  61. Esposito, A.; Amorese, T.; Cuciniello, M.; Riviello, M.T.; Esposito, A.M.; Troncone, A.; Torres, M.I.; Schlögl, S.; Cordasco, G. Elder user’s attitude toward assistive virtual agents: The role of voice and gender. J. Ambient. Intell. Humaniz. Comput. 2021, 12, 4429–4436. [Google Scholar] [CrossRef]
  62. Tokunaga, S.; Tamamizu, K.; Saiki, S.; Nakamura, M.; Yasuda, K. VirtualCareGiver: Personalized smart elderly care. Int. J. Softw. Innov. (IJSI) 2017, 5, 30–43. [Google Scholar] [CrossRef]
  63. Nakamura Lab., Kobe Univ. Virtual Care Giver: Speech Dialogue between an Elderly and a Virtual Agent. Available online: https://www.youtube.com/watch?v=UTTigjg_fYs&t=2s (accessed on 1 January 2022).
  64. Nakamura, M.; Hatano, K.; Miyazaki, J.; Yasuda, K.; Kuwahara, N.; Kazui, H.; Saiki, S.; Tokunaga, S.; Otake, M.; Kodama, N.; et al. Developing a system for self-care and mutual-aids of elderly people at home based on externalization of internal states. In JSPS KAKENHI, Grant-in-Aid for Scientific Research (A); (19H01138); Japan Society for the Promotion of Science: Tokyo, Japan, 2019–2023. [Google Scholar]
  65. Turing, A.M. Computing machinery and intelligence. In Parsing the Turing Test; Springer: Berlin/Heidelberg, Germany, 2009; pp. 23–65. [Google Scholar]
  66. Weizenbaum, J. ELIZA—A computer program for the study of natural language communication between man and machine. Commun. ACM 1966, 9, 36–45. [Google Scholar] [CrossRef]
  67. Yin, J.; Goh, T.T.; Yang, B.; Xiaobin, Y. Conversation technology with micro-learning: The impact of chatbot-based learning on students’ learning motivation and performance. J. Educ. Comput. Res. 2021, 59, 154–177. [Google Scholar] [CrossRef]
  68. García-Méndez, S.; De Arriba-Pérez, F.; González-Castaño, F.J.; Regueiro-Janeiro, J.A.; Gil-Castiñeira, F. Entertainment Chatbot for the Digital Inclusion of Elderly People Without Abstraction Capabilities. IEEE Access 2021, 9, 75878–75891. [Google Scholar] [CrossRef]
  69. Kuo, P.H.; Lin, S.T.; Hu, J.; Huang, C.J. Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot. Sensors 2021, 21, 5132. [Google Scholar] [CrossRef]
  70. Kushwaha, A.K.; Kumar, P.; Kar, A.K. What impacts customer experience for B2B enterprises on using AI-enabled chatbots? Insights from Big data analytics. Ind. Mark. Manag. 2021, 98, 207–221. [Google Scholar] [CrossRef]
  71. Mundhra, R.; Lim, T.J.; Duong, H.N.; Yeo, K.H.; Niculescu, A.I. Towards a Humorous Chat-Bot Companion for Senior Citizens. In Conversational Dialogue Systems for the Next Decade; Springer: Berlin/Heidelberg, Germany, 2021; pp. 31–39. [Google Scholar]
  72. Sagar, R.H.; Ashraf, T.; Sharma, A.; Goud, K.S.R.; Sahana, S.; Sagar, A.K. Revolution of AI-Enabled Health Care Chat-Bot System for Patient Assistance. In Applications of Artificial Intelligence and Machine Learning; Springer: Berlin/Heidelberg, Germany, 2021; pp. 229–249. [Google Scholar]
  73. Schario, M.E.; Bahner, C.A.; Widenhofer, T.V.; Rajaballey, J.I.; Thatcher, E.J. Chatbot-Assisted Care Management. Prof. Case Manag. 2022, 27, 19–25. [Google Scholar] [CrossRef] [PubMed]
  74. Chen, J.; Agbodike, O.; Kuo, W.L.; Wang, L.; Huang, C.H.; Shen, Y.S.; Chen, B.H. Online Textual Symptomatic Assessment Chatbot Based on Q&A Weighted Scoring for Female Breast Cancer Prescreening. Appl. Sci. 2021, 11, 5079. [Google Scholar]
  75. Lucas, G.M.; Gratch, J.; King, A.; Morency, L.P. It’s only a computer: Virtual humans increase willingness to disclose. Comput. Hum. Behav. 2014, 37, 94–100. [Google Scholar] [CrossRef]
  76. Kowatsch, T.; Nißen, M.; Shih, C.H.I.; Rüegger, D.; Volland, D.; Filler, A.; Künzler, F.; Barata, F.; Büchter, D.; Brogle, B.; et al. Text-based healthcare chatbots supporting patient and health professional teams: Preliminary results of a randomized controlled trial on childhood obesity. In Proceedings of the Persuasive Embodied Agents for Behavior Change (PEACH2017) Workshop, co-located with the 17th International Conference on Intelligent Virtual Agents (IVA 2017), Stockholm, Sweden, 14 December 2017. [Google Scholar]
  77. Beiley, M.R. Mental Health And Wellness Chatbot. 2019. Available online: https://repository.arizona.edu/handle/10150/632811 (accessed on 18 May 2022).
  78. Health, T. Vitalk. Available online: https://vitalk.health/ (accessed on 1 January 2022).
  79. Daley, K.; Hungerbuehler, I.; Cavanagh, K.; Claro, H.G.; Swinton, P.A.; Kapps, M. Preliminary evaluation of the engagement and effectiveness of a mental health chatbot. Front. Digit. Health 2020, 2, 41. [Google Scholar] [CrossRef]
  80. Yang, Y.S.; Ryu, G.W.; Han, I.; Oh, S.; Choi, M. Ecological momentary assessment using smartphone-based mobile application for affect and stress assessment. Healthc. Inform. Res. 2018, 24, 381–386. [Google Scholar] [CrossRef] [PubMed]
  81. Maeda, H.; Saiki, S.; Nakamura, M.; Yasuda, K. Rule-Based Inquiry Service to Elderly at Home for Efficient Mind Sensing. In Proceedings of the 21st International Conference on Information Integration and Web-based Applications & Services (iiWAS2019), Munich, Germany, 2–4 December 2019; pp. 666–670. [Google Scholar]
  82. Corporation, L. LINE. Available online: https://line.me/en/ (accessed on 1 January 2022).
  83. World Health Organization. Constitution of the World Health Organization. Available online: https://www.who.int/ (accessed on 1 January 2022).
  84. Miura, C.; Maeda, H.; Saiki, S.; Nakamura, M.; Yasuda, K. Prototyping and preliminary evaluation of mind monitoring service for elderly people at home. In Proceedings of the 21st International Conference on Information Integration and Web-Based Applications & Services, Munich, Germany, 2–4 December 2019; pp. 437–443. [Google Scholar]
  85. Shin, C.; Park, M.H.; Lee, S.H.; Ko, Y.H.; Kim, Y.K.; Han, K.M.; Jeong, H.G.; Han, C. Usefulness of the 15-item geriatric depression scale (GDS-15) for classifying minor and major depressive disorders among community-dwelling elders. J. Affect. Disord. 2019, 259, 370–375. [Google Scholar] [CrossRef]
  86. Levis, B.; Sun, Y.; He, C.; Wu, Y.; Krishnan, A.; Bhandari, P.M.; Neupane, D.; Imran, M.; Brehaut, E.; Negeri, Z.; et al. Accuracy of the PHQ-2 alone and in combination with the PHQ-9 for screening to detect major depression: Systematic review and meta-analysis. JAMA 2020, 323, 2290–2300. [Google Scholar] [CrossRef]
  87. Johnson, S.U.; Ulvenes, P.G.; ktedalen, T.; Hoffart, A. Psychometric properties of the general anxiety disorder 7-item (GAD-7) scale in a heterogeneous psychiatric sample. Front. Psychol. 2019, 10, 1713. [Google Scholar] [CrossRef] [Green Version]
  88. Tabatabaei, S.; Rasouli, Z. The reliability and validity of Persian version of the General Health Questionnaire (GHQ-60) in industry staff. Iran Occup. Health 2018, 15, 45–54. [Google Scholar]
  89. The Japan Foundation for Aging and Health. Kenko-Choju Net. Available online: https://www.tyojyu.or.jp/net/ (accessed on 1 January 2022).
  90. Eclipse Foundation. Eclipse Jersey. Available online: https://projects.eclipse.org/projects/ee4j.jersey (accessed on 1 January 2022).
  91. Grossman, T.A. Source code protection for applications written in Microsoft Excel and Google Spreadsheet. arXiv 2008, arXiv:0801.4774. [Google Scholar]
  92. Choi, B. Python Network Automation Labs: Cron and SNMPv3. In Introduction to Python Network Automation; Springer: Berlin/Heidelberg, Germany, 2021; pp. 629–673. [Google Scholar]
  93. Ongo, G.; Kusuma, G.P. Hybrid database system of MySQL and MongoDB in web application development. In Proceedings of the 2018 International Conference on Information Management and Technology (ICIMTech), Jakarta, Indonesia, 3–5 September 2018; pp. 256–260. [Google Scholar]
  94. Niederst Robbins, J. Learning Web Design: A Beginner_s Guide to HTML, CSS, Javascript, and Web Graphics; O_Reilly Media: Sebastopol, CA, USA, 2020. [Google Scholar]
  95. García, M.M. A Basic Spring Boot Application. In Learn Microservices with Spring Boot; Springer: Berlin/Heidelberg, Germany, 2020; pp. 25–74. [Google Scholar]
  96. Carvalho, F.M.; Duarte, L.; Gouesse, J. Text Web Templates Considered Harmful. In Proceedings of the International Conference on Web Information Systems and Technologies, Vienna, Austria, 18–20 September 2019; pp. 69–95. [Google Scholar]
  97. Ceklosky, S. Client side web programming more or less: AJAX or JQUERY. J. Comput. Sci. Coll. 2018, 33, 63–64. [Google Scholar]
  98. Da Rocha, H. Learn Chart. js: Create Interactive Visualizations for the Web with Chart. js 2; Packt Publishing Ltd.: Birmingham, UK, 2019. [Google Scholar]
  99. Yari, I.A.; Abdullahi, B.; Adeshina, S.A. Towards a framework of configuring and evaluating modsecurity WAF on tomcat and apache web servers. In Proceedings of the 2019 15th International Conference on Electronics, Computer and Computation (ICECCO), Abuja, Nigeria, 10–12 December 2019; pp. 1–7. [Google Scholar]
Sensors 22 03829 g001 550
Figure 1. The comparison of the sensor-based and chatbot-based approach for elderly people. Copyright 2009–2018 Nagoya Institute of Technology (MMDAgent Motion “Info of Mei”).
Figure 1. The comparison of the sensor-based and chatbot-based approach for elderly people. Copyright 2009–2018 Nagoya Institute of Technology (MMDAgent Motion “Info of Mei”).
Sensors 22 03829 g001
Sensors 22 03829 g002 550
Figure 2. Overall architecture of the proposed rule-based virtual caregiver system using mobile chatbot.
Figure 2. Overall architecture of the proposed rule-based virtual caregiver system using mobile chatbot.
Sensors 22 03829 g002
Sensors 22 03829 g003 550
Figure 3. Screenshots of user’s LINE application operated by the proposed service (added notes).
Figure 3. Screenshots of user’s LINE application operated by the proposed service (added notes).
Sensors 22 03829 g003
Sensors 22 03829 g004 550
Figure 4. Example of the weekly feedback message with the structured four paragraphs.
Figure 4. Example of the weekly feedback message with the structured four paragraphs.
Sensors 22 03829 g004
Sensors 22 03829 g005 550
Figure 5. Screenshots of the implemented web application: (a) Data Visualizer. (b) Mind Monitoring.
Figure 5. Screenshots of the implemented web application: (a) Data Visualizer. (b) Mind Monitoring.
Sensors 22 03829 g005
Sensors 22 03829 g006 550
Figure 6. Self−evaluation results of subject A in the period of this experiment.
Figure 6. Self−evaluation results of subject A in the period of this experiment.
Sensors 22 03829 g006
Sensors 22 03829 g007 550
Figure 7. Self−evaluation results of subject D in the period of this experiment.
Figure 7. Self−evaluation results of subject D in the period of this experiment.
Sensors 22 03829 g007
Table
Table 1. Seven questions for capturing the health state from physical, mental, and social well-being.
Table 1. Seven questions for capturing the health state from physical, mental, and social well-being.
QuestionSurvey ItemCategory
Have you slept well in the past week?SleepPhysicality
Have you felt sick, pain, or tired during the past week?HealthPhysicality
Have you had something fun in the past week?EmotionMentality
Have you felt you could not remember something, or forgotten something in the past week?PsychologyMentality
Have you felt anxiety or unwell during the past week?PsychologyMentality
Have you felt not motivated or appetite in the past week?MotivationSociality
Have you had many opportunities to go out, to talk and to have hobbies in the past week?SocializationSociality
Table
Table 2. Demographic characteristics of the participants in the experimental evaluations.
Table 2. Demographic characteristics of the participants in the experimental evaluations.
Demographic Characteristics Frequency%
GenderMale1659.3
Female1140.7
Total 27100
Age20∼291244.4
30∼39414.8
40∼4927.4
50∼5927.4
60∼6913.7
70∼79518.5
≥8013.7
Total 27100
Marital statusMarried1348.1
Single1451.9
Total 27100
Table
Table 3. Total response rate of elderly subjects.
Table 3. Total response rate of elderly subjects.
SubjectAgeGenderRate
A70∼79M91%
B60∼69M92%
C80∼89F30%
D70∼79F90%
E70∼79F54%
F50∼59F95%
Table
Table 4. Representative further explanations after the negative sociality answered by subjects.
Table 4. Representative further explanations after the negative sociality answered by subjects.
Answers in Text Messages to “What Is the Reason for Sociality Negatively?”Age and Sex of the Subject
I don’t want to go out because of coronavirus.Female in 80s
I’m bored as I can’t go out much because of the coronavirus.Female in 50s
I don’t get to meet many people, and my life consists mainly of being at home, which is not that much fun.Male in 70s
The curfew has made me even less inclined to go out, my body has become even more stiff, and I may not be able to walk when the curfew is ended.Female in 70s
I’ve been a little stressed out by my self-restrained lifestyle. I miss my normal life.Female in 40s
I miss seeing my friends. I wonder how long this kind of life will continue...Female in 20s
I can’t help but get stressed out when I stay at home and work in my room.Male in 20s
Since I don’t leave the house anymore, I am less aware of dates and days of the week, so I don’t remember the timeline of episodes. I especially can’t remember what happened on the weekend.Male in 20s
Table
Table 5. Results of questionnaire.
Table 5. Results of questionnaire.
QuestionStrongly AgreeAgreeDisagreeStrongly Disagree
Q1Do you think you were able to reflect on your mental state through the interaction with Mei-chan?27.8%50.0%16.7%5.6%
Q2Do you think the interaction with Mei-chan helped you to learn about your condition and to improve it?5.6%55.6%33.3%5.6%
Q3Do you think the daily interaction with Mei-chan was laborious and difficult?0%27.8%27.8%44.4%
Q4Do you think it was difficult to answer Mei-chan’s questions because they were so frequent?0%11.1%27.8%61.1%
Q5Do you think the interaction with Mei-chan was useful to reflect on your mental state?16.7%50.0%33.3%0%
Q6Do you want to continue to use this service?33.3%38.9%16.7%11.1%
Q7Do you think it was easy for you to interact with Mei-chan?72.2%22.2%5.6%0%
Q8Do you think you were able to check your mental state easily by interacting with Mei-chan?33.3%61.1%5.6%0%
Q9Mei-chan can record the data even if you enter the answers at a later time in batches. Did you do this way of answering?33.3%11.1%11.1%44.4%
Q10Do you think the flexibility, for example, the service does not require to respond immediately as in Q9, is important?61.1%33.3%5.6%0%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Miura, C.; Chen, S.; Saiki, S.; Nakamura, M.; Yasuda, K. Assisting Personalized Healthcare of Elderly People: Developing a Rule-Based Virtual Caregiver System Using Mobile Chatbot. Sensors 2022, 22, 3829. https://doi.org/10.3390/s22103829

AMA Style

Miura C, Chen S, Saiki S, Nakamura M, Yasuda K. Assisting Personalized Healthcare of Elderly People: Developing a Rule-Based Virtual Caregiver System Using Mobile Chatbot. Sensors. 2022; 22(10):3829. https://doi.org/10.3390/s22103829

Chicago/Turabian Style

Miura, Chisaki, Sinan Chen, Sachio Saiki, Masahide Nakamura, and Kiyoshi Yasuda. 2022. "Assisting Personalized Healthcare of Elderly People: Developing a Rule-Based Virtual Caregiver System Using Mobile Chatbot" Sensors 22, no. 10: 3829. https://doi.org/10.3390/s22103829

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop