From Symptom Tracking to Contact Tracing: A Framework to Explore and Assess COVID-19 Apps

Abstract

Smartphone applications related to coronavirus disease 2019 (COVID-19) continue to emerge and evolve, but despite a wide variety of different app functions, there has yet to be a comprehensive study of what the most prevalent publicly available apps provide, and there exists no standardized evaluation system for end users to determine the safety and efficacy of an app before they download it. Furthermore, limited oversight means that the rapidly growing space creates challenges for end users trying to find a relevant app. We adapted the M-Health Index and Navigation Database (MIND) from apps.digitalpsych.org that previously has been used to evaluate mental health applications to guide the assessment of COVID apps. Using this framework, we conducted a thorough analysis of the top-100 returned coronavirus apps on two separate dates a month apart to understand the clinical utility and features of COVID-19 apps and how these change in a short period of time. We ultimately identified a significant turnover rate, as well as privacy concerns around lack of privacy policies and disclosure of personal information. Our research offers insight into the current status of COVID-19 apps and provides a comprehensive and adaptable framework to help individuals assess the growing number of such digital tools in the wake of the pandemic.

1. Introduction

Five months into the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2, coronavirus disease 2019, COVID-19) pandemic, infection and deaths continue to increase globally, surpassing over two million confirmed cases in the United States and eight million worldwide as of June 20 [1]. Although there have been numerous statewide and nationwide efforts to contain and mitigate the spread of the virus, such as stay-at-home orders, social distancing, and lockdowns, there has been growing consensus for the need to combat the pandemic through a combined strategy of both medical and technological tools [2]. As a result, mobile health apps focused on education, symptom tracking, contact tracing, and access to telehealth services have risen to the forefront of the COVID-19-related apps to support users remotely and monitor the spread of the virus. While these tools hold enormous potential, little is known about the availability and offerings, and there remains a critical need for a thorough system of evaluation that can help interested users find a safe and effective COVID-19-related app to meet their needs.

Numerous states and countries have attempted to create their own COVID-19 apps—some for contact tracing and others for self-help and to share information regarding local and national protocols, resources, and the coronavirus. Some apps, such as TraceCOVID, are designed primarily for contact tracing to limit the spread of the virus, while other apps are created for general constituent use, such as JamCOVID19, the national COVID-19 app for Jamaica, which not only includes a contact-tracing feature but also provides educational information and local news and updates regarding the governmental protocol. While some apps have become popular and well known within the COVID-19 apps space, such as the Apple COVID-19 apps, we currently do not have a broader understanding of what apps exist in the COVID-19 space and how many of these apps are created for contact tracing, educational purposes, symptom tracking, telehealth, and other resources to support users through COVID-19 pandemic. Most analyses of COVID-19 apps have focused primarily on contact-tracing tools. Given the lack of public willingness to engage with contact-tracing apps [3] and limited evidence of their efficacy [4], we sought to analyze the full swath of available apps for an individual searching for digital COVID-19 resources on the app store. We investigated the developers, features, functions, and privacy settings of the top-100 returned apps.

In addition, it is unknown how these apps are changing and how turnover within the app stores affects the availability of emerging apps. High rates of turnover have already been characterized and emphasized in other health app domains, such as the mental health app space, but there has yet to be an estimation of how fast the availability of apps is changing in the COVID app space [5]. As new COVID-19 apps are continuously created and added to the app store, we sought to determine the rate at which the space is changing and identify which apps are stable. We conducted two analyses of the top-returned COVID apps a month apart, allowing us to both analyze app content and characterize the field’s dynamic nature.

In the absence of stringent FDA oversight of COVID apps, there is no comprehensive or standardized vetting system to evaluate apps and address the privacy concerns that have been raised [6]. Prior research has focused on the privacy issues around contact-tracing apps, many of which may collect and share users’ personal data [7]. To date, however, no research has been conducted to understand the privacy and data collection considerations for all COVID-19 apps, beyond the contact-tracing ones. Given the input of potentially sensitive information, such as symptoms and health data, into COVID-related digital tools, we sought to explore the potential privacy concerns across the app space using a framework that can be replicated by users seeking to understand data use and privacy policies.

We adapted a thorough, reproducible framework that has been previously used to evaluate mental health apps called M-Health Index and Navigation Database (MIND) [8,9] (see Figure 1 and apps.digitalpsych.org for more details) to answer a series of key questions around COVID apps: Where are COVID-19 apps being developed? What are they doing? Are they adequately protecting user data? How are they changing over time? The ultimate aim of these analyses is to inform guidance for the future development and dissemination of apps related to COVID-19, using a framework that can be adapted and utilized in the coming months as technology continues to emerge to keep pace with the COVID-19 pandemic.

2. Materials and Methods

On 3 May 2020, we identified the first 100 coronavirus-related iOS apps by searching “COVID-19” into the Apple store. Because some of the apps required local phone numbers, national identification codes, and research study codes, we were only able to access a total of 58 apps. On 8 June 2020, we again gathered data on the first 100 coronavirus-related iOS apps, using the same search terms and, due to the same constraints, were only able to access 69 apps. In addition, we recollected data on apps that were originally on the top 100 list in May (and were not in the top 100 in June), if they were still available in the iOS Store as on 8 June 2020.

We collected data related to app origin, app features and tools, app functionality, privacy policies, and other qualities derived from the M-Health Index and Navigation Database (MIND) at apps.digitalpsych.org [9]. Questions for app origin, country, app functionality, and privacy policies were based on prior work developing a framework for the evaluation of mental health apps [8]. We selected questions relevant to an analysis of COVID apps, including those pertaining to app developer, origin, functionality, accessibility, and privacy and security settings. These questions can all be answered with binary or numeric responses so as to minimize subjectivity.

Questions in regards to app features were based on an understanding of what features would be most important for users regarding the COVID-19 pandemic. For example, features such as updated statistics, a map, educational materials, and national/local news provide users updated information about confirmed cases, death rates, where potential hotspots are within their county, state, or country, and updated national and/or local protocols. Other features specific to COVID-19 such as contact tracing, telehealth services, community-engagement efforts, and symptom tracking were chosen to be included in the questions as these tools are specific to the COVID-19 pandemic and could possibly help with monitoring the spread of the virus or support the community and patients virtually [10]. We also collected data regarding the presence of mental health resources, through direct support, links to points of care, or other educational information. Ultimately, our questions were chosen to answer four primary questions: Where are COVID-19 apps being developed? What are they doing? Are they adequately protecting user data? How are they changing over time?

3. Results

3.1. Accessibility

The framework’s questions around app accessibility prompt users to consider what apps and features are available and readily accessible to them. Our analysis revealed that for COVID-19 apps, many were not applicable or accessible to the general public. Of the first 100 apps generated for COVID-19 in both May and June, only 58 and 69, respectively, were able to be downloaded and used, whereas the remaining 42 and 31 apps, respectively, required national identification codes, local country phone numbers, or research study codes to access. Moreover, when specifically looking at the top 10 results as of June 8 2020, we found that half of the apps were created for or by countries other than the US, with apps from Vietnam, Pakistan, Czech Republic, Russia, and the UAE being generated in some of the top ranked spots. Thus, we were restricted in our ability to apply the framework across many apps due to the lack of app accessibility in the COVID-19 app space.

3.2. App Origin

While particular scrutiny has been paid to the development of COVID-19 apps by governments, the utilization of our framework was able to capture the wider variety of distinct developers in this space: governments, for-profit companies, non-profit organizations, trusted healthcare companies, and academic institutions. Overall, app origins changed notably between May and June as shown in

Table 1. Origin of COVID Apps in May and June ¹.

	May	June
Government	49	53
For-Profit	20	28
Non-Profit	1	3
Trusted healthcare company	13	7
Academic Institution	10	5

¹ We removed eight apps for the analysis in May and four apps for the analysis in June, as either the apps became unavailable within the one week that we analyzed them or the app’s name was in a language that required a different script to notate.

. There was a large increase in the number of apps developed by for-profit companies, increasing by half from 20 to 28, while the number of apps created by the government increased by 4 between our initial and final data collection points. Likewise, we saw the number of apps from non-profits tripling, while the apps from trusted healthcare companies and academic institutions reduced by almost half for both.

3.3. Features and App Uses

Contact-tracing apps have received the bulk of media attention. However, the top-100 returned apps on an app store search for “COVID-19” offered a range of diverse features and functions (see Figure 2 and Figure 3). In both May and June, the most common features in coronavirus-related apps were educational materials related to the virus (74.1%, 79.7%), updated statistics (56.9%, 53.6%), updated local news (39.7%, 44.9%), and a screening tool to determine whether the user should get further testing (37.9%, 40.6%). Less common in both May and June were apps for telehealth (3.4%, 7.2%), research studies (5.2%, 10.1%), community service efforts (6.9%, 5.8%). There was a notable increase in apps that provided content or support for mental health between May and June, rising from 13.8% to 23.2% of the apps having resources or support available to users. In terms of use of the apps, 23 were patient-facing, 42 were reference, and 6 were for first responder/essential worker use. We did not identify strong correlations between app origins and app features, but the apps from non-profit developers appeared most strongly correlated with symptom tracking (R = 0.504, p = 0.00025) and research study (R = 0.504, p = 0.00025) offerings.

3.4. Privacy Policies and Harm

Our framework provides a comprehensive method to analyze the complexities of a privacy policy for apps. Utilizing this framework, we were able to highlight many of the dangers behind COVID-19 apps, specifically for contact-tracing and telehealth-focused apps. Of the 77 apps we analyzed that were available as of 8 June 2020, only 63 provided a working privacy policy, with the other 14 apps linked to a faulty page with no privacy policy. As shown in

Table 2. Privacy Policies for Non-Contact-Tracing Apps (N = 65). HIPAA, Health Insurance Portability and Accountability Act.

Privacy Policy	51
Declare data use/purpose	45
Security measures	35
Personal health information shared	13
De-identified data shared	8
Anonymized/aggregated data shared	18
Opt out of data collection	8
Delete your data	21
User data stored only on device	35
User data stored on server	11
Crisis management feature	0
Claim to meet HIPAA	4
Average reading level (avg)	11.7
App uses third party vendors	36

, a large number of apps were found to share personal health information or anonymized or aggregated data, with few apps allowing users to opt out of data collection. Of the 12 apps that had a contact-tracing feature, every app provided a working privacy policy, but only half the apps allowed users to delete their data, and even less allowed users to opt out of data collection, as shown in

Table 3. Privacy Policies for Contact-Tracing Apps (N = 12).

Privacy Policy	12
Declare data use/purpose	11
Security measures	9
Personal health information shared	7
De-identified data shared	2
Anonymized/aggregated data shared	5
Opt out of data collection	2
Delete your data	6
User data stored only on device	4
User data stored on server	8
Crisis management feature	0

. Four of the telehealth-focused apps were created for use in the United States, yet only one app claimed to meet the Health Insurance Portability and Accountability Act (HIPAA) guidelines. Moreover, despite providing remote healthcare to their patients, none of the telehealth apps and, more broadly, none of the COVID-19 apps, specified providing a crisis management feature in their privacy policies. We further analyzed the correlation between app origins and privacy policies and found no significant correlations.

3.5. Turbulence and Turnover

The repetition of our analysis on two separate occasions enabled us to characterize the turnover rate of the emerging COVID-19 app space. Eighteen of the top 100 apps collected in May had become unavailable or were not listed in the top-100 returned apps in June, indicating a significant turnover rate of 18%. We also found that there was an average change of 7.5 ranks in terms of placement either below or above the app’s initial rank as of May, with 41 apps having changed less than five spots in the span of a month. Some notable apps that drastically decreased in rank in the iOS store were: StopCovid, Central Coast COVID-19 Info, Speetar COVID19, Asistencia COVID-19, and Nuahealth Video Consultations, which decreased by 81, 18, 34, 40, and 21 ranks, respectively. As apps related to COVID-19 inevitably progress over the coming months, this initial analysis highlights the space’s dynamic nature, further underscoring the importance of an evaluation system beyond app store metrics.

4. Discussion

The framework utilized in this paper, derived from the M-Health Index and Navigation Database (MIND) at apps.digitalpsych.org, prompts users to consider various dimensions of the COVID-19 app space: app accessibility, app origin, app functionality, app features and app use, and privacy and security. Our results revealed high heterogeneity of these apps, as well as propose considerations for the next generation of these tools.

While smartphone apps continue to assume new roles in the international response to COVID-19, our results suggest their purpose is still evolving. Our findings of 20% turnover of these apps within one month as well as changes in functions offered by newer apps highlight the dynamic nature of digital health marketplaces, as apps try to evolve to the changing needs. In contrast to this change, we noticed few differences concerning privacy flaws in these apps, with equal concerns for both contact tracing and non-contact-tracing apps. As these apps continue to evolve to meet the global needs, our results highlighted trends towards more comprehensive apps with features ranging from educational resources and updated statistics to local news and governmental updates and a need for both stronger privacy policies that protect users’ data and more educational material and resources to support users’ mental health.

In terms of app accessibility, we found a significant portion of the top 100 COVID-19 apps to be inaccessible to the general public, which may indicate great confusion for a user that stems from the oversaturation of the app space, with so many COVID-19 apps returned in a search. It also raises questions about how research in the space is being disseminated and translated to the broader public, since so many of the top-returned apps from the U.S. store were not actually available in the United States, making it difficult for an individual searching in the app store to come across a research-supported and available app.

Utilizing the framework, we found that the majority of apps in the COVID-19 space are being created by the government, with an upsurge of apps created by for-profit or individual developers in the past month. However, with the exception of non-profit apps being more likely to offer symptom tracking or participation in a research study, we found no strong correlation between the origin of the apps and the presence of certain features or privacy policies, which may indicate a more uniform response among the different types of developers. This may indicate that more developers are understanding the usage of and the needs for a COVID-19 app during this time and are adapting and creating apps in response to these needs, while governments continue to roll out apps for their constituents both at a national and statewide level, as seen for apps like StopCovid France, JamCOVID-19, COVID-19 UAE, Coronavirus Australia, COVID-19 Wisconsin Connect, and myColorado.

For app features, we only saw a slight decrease in the number of apps that provided updated statistics and in those that were utilizing the users’ data for research purposes, while all remaining categories saw an increase in the number of apps offering these features. The largest increase was seen in the number of apps that offered educational materials, followed by apps that offered updated news or a map or had a mental health component within. As many COVID apps did not have extensive mental health information, we adapted our prior framework to the current model to better understand the overarching trends in mental health support among COVID apps, often in the form of links to sources of care or psychoeducation and mindfulness techniques. This may be a result of more apps providing educational content and of using the apps to disseminate important information about the spread of disease, local protocols, and sources of care to their users, while also pushing away from providing information that may be more readily acquired online through multiple different platforms like updated statistics. In terms of app use, we saw an increase in all categories, indicating the responsiveness of the COVID-19 app space to better cater to their users’ needs.

The privacy concerns for COVID-19 apps extend far beyond the contact tracing realm. While many previous papers have expressed privacy concerns with contact-tracing apps, no prior paper has comprehensively looked at the privacy policies and concerns of all COVID-19 apps that include apps with symptom tracking, screening tools, and telehealth, which often require users to share personal health information to gain further information regarding whether they should receive further testing or to virtually access healthcare through a physician. While every single app linked to a privacy policy, only 81% linked to a valid, up-to-date privacy policy, whereas the other apps linked to faulty pages or provided error messages. However, as problematic as the contact-tracing apps were the telehealth and screening tool apps that also failed to protect users’ data or allow users to opt out of data collection. Although contact-tracing apps have recently come under significant scrutiny, other apps under the COVID-19 app space deserve equal attention for privacy and security concerns and need to be downloaded and used with caution.

We applied this framework towards the evaluation of COVID-specific apps, some of which are now in the M-Health Index and Navigation Database (MIND) at apps.digitalpsych.org. For example, we used our framework to find and evaluate COVID-related apps with both psychoeducational, mental health-focused content, and links to up-to-date information about the pandemic. The COVID Coach app, which is developed by the United States Department of Veterans Affairs, emerged as a frontrunner due its comprehensive mental health resources, secure privacy protections, and links to resources for users. Going forward, we intend to use this framework in more case studies and with researchers and clinicians who may be seeking secure COVID-related apps to recommend.

Limitations

We were unable to analyze the apps that appeared in the top-100 returned apps but were not English or required local phone numbers or national identification codes. Regarding privacy policies, we were only able to analyze what apps claimed and were not able to verify if apps actually fulfilled the claims made. Moreover, in terms of research study apps, we did not analyze apps that required a research study code to access the app, and for apps related to data collection, we did not analyze whether data were publicly available for users to view. Additionally, given the dynamic, rapidly changing nature of this space, we recognize that our findings no longer describe exactly the current state of COVID-19 apps. Our monthly turnover rate of 20% was calculated during the initial emergence of COVID-19 apps, and since then, the monthly turnover rate may have changed. We did not focus on the popularity of apps or take into account the number of downloads of an app when collecting data and instead evaluated the top-returned applications that were generated in the iOS Store search, as these would be the apps most readily accessible and populated for users upon a search for COVID-19 or Coronavirus in the App Store. Finally, we concentrated our analysis on the iOS apple store and did not conduct any analysis with the Android App Store or Google Play stores, as we have seen a recent convergence and collaboration in the space of COVID-19 apps between the two app stores [11,12]. Many contact-tracing apps cannot go on the apple store, and we were thus limited in our analysis of contact-tracing apps.

5. Conclusions

In the era of the COVID-19 pandemic and its aftermath, digital health holds enormous potential. Smartphone apps can augment the “test, trace, isolate, quarantine” strategy espoused by the WHO to directly mitigate the spread of the virus through symptom tracking and contact tracing. However, our analysis reveals that resources that will serve the pandemic’s aftermath, especially around mental health concerns, are far less common and represent an area in desperate need of continued innovation [13]. We echo and extend the privacy concerns identified in previous analyses of contact-tracing and symptom-tracking apps. The lack of stated correspondence with HIPAA guidelines among telehealth apps is especially concerning, given the exchange of sensitive personal health information that these apps facilitate. If digital health is to meet its full potential in the fight against COVID-19, there must be a renewed focus on protecting user data and on ensuring that research-supported apps are accessible and available to the general public. The framework provided comprises objective questions across various categories that can be answered by a user, serving as a widely accessible tool to help any interested user find a safe and effective COVID app.