i-PreventDiabetes was evaluated using a multi-method approach involving 50 participants with prediabetes aged between 19 and 75, between November 2017 and January 2018. The participants were recruited through advertisements shared on social media and posted on notice boards of selected healthcare institutions. They were selected randomly from the public, as they volunteered to participate in this study. Table 4
illustrates a summary of the demographic profile of the study participants. In total, eight out of 50 participants reported using a self-care tool but did not specifically state what type of tool.
Participants were encouraged to use i-PreventDiabetes (which was made available over the Web) at their own pace for six weeks, depending on their requirements and availability. The primary tasks in the system (blood glucose, food, weight, stress, and physical activities) were logged in the system for analysis purpose. Participants were asked to complete questionnaires at the beginning of the study and at the end of the sixth week of the study. Table 5
summarizes the scales used to evaluate i-PreventDiabetes, their purpose, and when they were administered in the study. The statistical package for the social sciences (SPSS) tool (version 25) was used for statistical analysis of the results obtained. At the end of the quantitative study, a short interview was conducted with a purposive sample of eight participants to obtain further insights into their experience with and views of i-PreventDiabetes. According to Six and Macefield, five to 10 participants are sufficient to determine most of the usability problems of a system [30
]. The eight interviewees comprised of four participants who used the system extensively and four others who made little use of the system.
Both the TTM and MHLC were administered twice, that is, at the beginning and at the end of the study. The TTM consisted of 24 statements with five response choices, ranging from strongly disagree (1) to strongly agree (5). Participants were expected to respond to the TTM statements to determine if there were any differences in their stage of behavior change toward T2D prevention before and after using the system for six weeks. At the end of the study, the TTM was administered again to see if there was any difference in their stage of behavior change toward diabetes prevention. The McNemar–Bowker test and the Wilcoxon signed-rank test were used to compare the participants’ initial stage and final stage. The statistical significance was set at p < 0.05. The MHLC consists of three subscales: internal health locus of control (IHLC), powerful others health locus of control (PHLC), and chance health locus of control (CHLC). There were 18 statements with six answer options ranging from strongly disagree (1) to strongly agree (6). The MHLC was administered pre- and post-study to note any difference in participants’ perception whether health is controlled by internal or external factors. The change scores for each MHLC subscale were then computed by subtracting the baseline and follow-up values.
The final questionnaire of the evaluation study comprised of three scales (IMI, TAM, and UEQ), and this was used only once after the participants had used i-PreventDiabetes for six weeks. The IMI was adopted from the self-determination theory [34
]. This scale comprised of 20 items rated on a seven-point Likert scale, from completely untrue (1) to completely true (7). It was employed to identify the motivation level of users in using the system based on five (out of seven) relevant factors: interest/enjoyment, perceived competence, effort/importance, pressure/tension, and value/usefulness. Using the TAM, the perceived usefulness (comprising of 10 statements) and perceived ease of use (comprising of nine statements) of i-PreventDiabetes were assessed. Each TAM question was accompanied by a seven-point Likert scale, from one (strongly disagree) to seven (strongly agree). The UEQ scale comprised of 26 items to measure user experience of the system from six different perspectives, including attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty of the system. Each item of the UEQ was assessed using a five-point Likert scale, ranging from very difficult (1) to very easy (5).
The majority of the study participants were female (70%). Throughout the six weeks, 1500 logins were recorded with a mean (µ) = 37.5. The total number of tasks performed by all the participants is 3639. Figure 5
shows the details of the recorded activities. Food and physical activities were the most frequently recorded tasks, indicating that users primarily use the system to track their food intake and physical activities. At the end of the study, 40 participants completed the six-week evaluation, which had a success rate of 80% participation (40 out of 50 participants). The remaining 10 participants did not participate until the end of the six weeks due to their other responsibilities and work engagements.
The six-week study shows that there were changes in behavior among the participants. Table 6
shows the descriptive values of the initial (IniStage) and final (FStage) behavior change stages. As per our findings, the mean (µ = 3.93) of FStage is higher than the mean (µ = 3.200) of IniStage. This indicates that the participants’ behavior changed at the end of the study, as the responses shifted toward the maximum value on the Likert scale, with higher 50th and 75th percentile values. The p
-values of the Wilcoxon signed-rank test (<0.001) also indicate that the change in the behavioral change stages of the participants is statistically significant.
Moreover, this study also reveals that the changes in the behavior stages were more positive than negative. Table 7
depicts the results of the ranks of the stages. In total, 17 participants were found to have advanced in the behavior change stages, whereas four participants regressed. The other 19 participants neither progressed nor regressed, they maintained their behavior stages. This contributes to the conclusion that i-PreventDiabetes has the potential to change people’s behavior toward better health management.
Based on the results, some changes were noted in the participants’ attitudes toward the end of the study. Table 8
shows the difference between participants’ responses before and at the end of week six for that subscale. The participants became more aware that their health control comes from within themselves (IHLC), and not from others (PHLC), or their health conditions do not happen by chance (CHLC). The final IHLC mean (µ = 5.021) is higher than the initial IHLC mean (µ = 5.004), whereby the mean values of initial PHLC (µ = 3.346) and CHLC (µ = 2.725) were higher than their final values (PHLC µ = 3.262 and CHLC µ = 2.604). This shows that there were changes in the attitude of the participants toward their healthcare. This could indicate that participants have instilled the self-care habit in them to a certain level.
The participants’ level of motivation to use i-PreventDiabetes indicates that they were pleased with the system. Table 9
shows the mean values of the selected five IMI subscales. The participants felt that the system has value, it is useful for their healthcare management (µ = 5.338), and their level of interest and enjoyment in the system is also quite high. In addition, the results show that the participants imposed less pressure and tension during their involvement with the system. Based on the results, it can be interpreted that the participants were motivated to use the system.
Based on the results, it was revealed that the participants found the system to be useful and user-friendly. Table 10
depicts the results of the TAM scale. Both the subscales of TAM have reasonable mean values to indicate the perceived usefulness (µ = 5.258) and ease of use (µ = 5.345) of the system. The participants agree that using i-PreventDiabetes improves the quality of their lives. It also improves their health performance and effectiveness in caring for their health. Besides making their health easier to manage, it also gives them more control over their lives. Moreover, the participants felt that it was easy to get the system to do what they wanted to do and to remember how to operate the system.
As per the UEQ results, it was found that the participants experienced the system in a positive manner. Figure 6
illustrates the UEQ results and benchmark, which compares i-PreventDiabetes with the other 246 products [38
]. Benchmarking analysis of UEQ compares the calculated mean score of i-PreventDiabetes with the result of other items such as websites, online retailers, social media networks, and business applications on the market that were evaluated using this scale. By comparing the evaluated product’s results to the benchmark’s data, conclusions can be drawn about the evaluated product’s relative quality. Based on the benchmark, dependability, stimulation, and novelty are rated as “excellent,” while the other three scales hold “good” ratings.
All the six subscales of UEQ were above 0.8 and have positive values. According to UEQ, values above 0.8 are considered positive, −0.8 to 0.8 neutral, and below −0.8 negative values [37
]. The lowest value is 1.431 (novelty), while the highest value is 1.750 (attractiveness and stimulation). The highest value of 1.750 shows that the participants liked the system. It also seems that it is easier to get familiarized with the system. Nevertheless, a few participants felt that the system could be much more creative and innovative.
Based on the results, it can be concluded that i-PreventDiabetes is well accepted and supported by the target users. The analysis shows that the system has great potential to improve the participants’ behavior toward T2D prevention. The majority of the study participants (40 of the initial 50) had continuously used the system for six weeks and were keen on the idea of having one system that comprised all the necessary features to prevent or delay diabetes. They also preferred setting their own health goals and meeting them by using selected features of the system at their own pace. Moreover, 32 out of 40 active participants (80%) mentioned that they would like to continue using i-PreventDiabetes after the study. Although the majority of research participants were female, women are less likely to adhere to a healthy lifestyle, despite their increased awareness of the benefits of healthy living [39
]. Women’s lack of adherence to healthy lifestyle is mostly associated with socioeconomic factors, as healthy lifestyles are expensive and time-consuming, and women mostly spend their time fulfilling their social roles in the family [40
]. This system also positively affected the attitude of people with prediabetes in terms of managing their own health. Their attitude has shifted to the point where, after six weeks, they were more convinced that their health control comes from within rather than from others or by chance. This indicates that systems, such as i-PreventDiabetes, can instill self-care habits among users.
The study participants were also motivated to use the system. They enjoyed using the system and also acknowledged the value and importance of the system. Generally, the participants felt less tense when using the system. In addition, i-PreventDiabetes has been found to be useful in managing the participants’ health. Most of the participants found the system easy to use, although a few of them thought that the system could be further simplified. Goal setting and activity planning have been determined to play a vital role in self-care applications. Evidence indicates that such a feature would foster users to take responsibility for their health [41
]. The participants prefer to set their own “achievable” goals and meet the targets by executing the tailored activities using the application. Achievable goals can also motivate the participants; this would make them commit to their goals as they are not being pressured due to strict goals.
Moreover, the user experience of the system is noted to be positive. A few participants shared that the system acts as a constant reminder that triggers them to monitor their blood glucose levels actively. The participants found the system to be convenient. Overall, the study shows that i-PreventDiabetes fosters positive changes in users to prevent diabetes. It is a well-designed system that incorporates the perspectives of both people with prediabetes and healthcare professionals; further, it can enable users to prevent or delay diabetes, provided that the user takes responsibility for making effective use of the features offered in the system. The inclusion of healthcare professionals is suggested to foster people with prediabetes to advance rapidly through behavioral stages in the process of lifestyle modification [42
]. i-PreventDiabetes, being a self-care system, did not include healthcare professionals directly in the process of using the system to prevent the progression of prediabetes to diabetes. Results of the evaluation study reveal that it is possible to develop a system that is acceptable by target users provided the design incorporates perspectives of both the target users and healthcare professionals. However, some of the participants expressed that the system should be more autonomous and attractive. According to them, the system should be configured in order to accept and store automatic inputs of their physical activity details from external devices. They also wanted more automatic alerts and reminders from the system.
Furthermore, it was found that participants were a bit skeptical about using the social support features in the system as they did not feel good about sharing their information with others. The users’ attitudes toward receiving social support to help them achieve their health goals were mixed. Social support is often seen as an essential component of health management systems, although it may raise privacy issues. It has been observed that users were not comfortable with communicating with other users or strangers on the system. A cultural difference may be a factor. Users in the United States, for example, have welcomed platforms, such as PatientsLikeMe, where users are eager to freely disclose their symptoms and treatments as well as participate in conversations with other patients dealing with similar health problems [43
Diabetes prevention systems, such as i-PreventDiabetes, have immense potential and a role in preventing the progression of prediabetes to diabetes. Potential users of such systems are more likely to leverage these systems for their healthcare if they are endorsed and promoted by diabetes institutes, healthcare institutions, hospitals, and wellness centers [44
]. Such interventions can complement or be integrated with existing T2D prevention programs.