Lessons Learned from Development of a Mobile App for Cardiovascular Health Awareness

: Cardiovascular disease (CVD) is a major public health concern in the United States. In response to the federally sponsored Million Hearts Risk Check Challenge, a team of programmers, software developers, health-information technologists, and clinicians in an integrated healthcare system in Wisconsin collaborated to develop Heart Health Mobile TM (HHM), designed to improve awareness of cardiovascular disease risk and promote risk factor control among users. This paper outlines the development processes and highlights key lessons learned for mobile health applications. An agile project management methodology was used to dedicate adequate resources and employ adaptive planning and iterative development processes with a self-organized, cross-functional team. The initial HHM iOS app was developed and tested, and after additional modiﬁcations, gamiﬁed and HTML 5 versions of the app were released. The development of an iOS app is low in cost and sustainable by a healthcare system. Future app modiﬁcations to enhance data security and link self-reported cardiovascular risk assessment data to patient medical records may improve performance, patient relevance, and clinician acceptance of HHM in the primary-care setting. Legal and institutional barriers regarding the capture and analyses of protected health information must be mitigated to fully capture, analyze, and report patient health outcomes for future studies.


Introduction
Mobile health (mHealth) technologies and applications (apps) have dramatically increased in number and adoption across the globe in recent years. Articles in the peerreviewed and gray literature provide extensive examples of adoption by both providers and their patients. Furthermore, many studies have shown promise in various technologies' positive improvement in health and health-behavior outcomes, while many others' impact is inconclusive [1][2][3][4][5][6]. Applications and implementation is widespread across many aspects of healthcare, including public and private health organizations' foci towards some of the most significant of public health concerns.
Cardiovascular disease (CVD) is a major public health concern worldwide and is growing. The number of individuals with CVD has nearly doubled over the past three decades, affecting an estimated 523 million individuals in 2019 [7]. Accordingly, global trends in CVD mortality during this timeframe have also increased steadily, accounting for approximately 19 million deaths and 34 million years of life lost due to CVD death or disability. In the United States, an estimated 86 million American adults have CVD [8,9], and the CVD mortality rate is equivalent to one death every 40 s [10]. Heart disease remains

Materials and Methods
The goals of the HHS Million Hearts Risk Check Challenge were to achieve the following: (1) reach individuals across the country, taking special aim at those who may be at risk for CVD and are unaware of it; (2) deploy an engaging user interface that provides consumers with a quick health risk assessment and motivate them to obtain a more accurate risk assessment by entering their blood pressure and cholesterol values; and (3) direct users to nearby community pharmacy locations offering affordable and convenient blood pressure and cholesterol screenings.
To create an app that fulfills these primary goals, a mix of specialties and expertise was needed. A multidisciplinary team of 24 members was assembled that included a broad cross-section of clinical professionals from medicine, epidemiology, health information technology, graphic design, business analytics, and marketing. The core team consisted of an iOS developer, a graphic designer, and a project manager. An informal agile project management methodology was employed to promote dedicated team resources, adaptive planning, and iterative development (usually 2-week sprints of predefined tasks) with a self-organized, cross-functional team. Development of the initial app prototype was completed over a 30-day timeframe, followed by roughly 30-45 additional days spent on the gamified and HTML5 versions. A second software developer joined the team to work on the HTML5 version. The entirety of HHM's development took place over a 2-month period with an estimated internal cost of $50,000, which included project management, graphic design, programming, server and domain setup, language translation, promotional video and script production, and business plan development.

App Development
Since the purpose of this app was to rapidly identify and highlight risk factors for CVD, partnership with an existing risk score engine was permitted by the Office of the National Coordinator (ONC) for Health Information Technology at the onset of the project. The application programing interface (API) created and maintained by Archimedes (acquired by Evidera) was made available to all contestants [14,15]. The engine was based on current risk factor guidelines published by the American Heart Association. Major risk factors for CVD in adults include body mass index (BMI) ≥30 kg/m 2 , current smoker, sedentary lifestyle, poor diet, total cholesterol levels ≥240 mg/dL, blood pressure at or above 140 mm Hg/90 mm Hg, and fasting plasma glucose ≥126 mg/dL 2 . Leveraging the Archimedes API technology, we began with the development of an iOS version of the CVD risk assessment tool. Physician representatives on the team reviewed and refined the app assessment items. In keeping with the rules of the contest, this app was developed in compliance with ONC, along with recommendations from internal advisors from legal/corporate risk management and information systems' security.
Global positioning system (GPS) was also employed to suggest nearby screening locations within the user interface (UI). If a user reported having not yet completed a blood pressure or cholesterol screening, the locations of local options were presented. Part of this screening location dataset was available via an API maintained by a third party. Other location data were loaded through a manually updated and disseminated spreadsheet, supplied by contest coordinators at the Million Hearts Risk Check Challenge, who collected these data from partner organizations interested in being listed as available locations within the contestant apps' UIs.
The HHM iOS app (for Apple devices) prototype was developed, tested, and made available for judging within 30 days, and was initially selected as one of five finalists to receive an award (see Figure 1). Additional modifications to the app were developed over the following 2 months, taking into account the judges' feedback. The app then underwent preliminary testing with users, both internal and external to the team's healthcare system. During those 2 months, a gamified version of the app was also added, as well as an HTML 5 web application (see Figure 2), which allowed users to access the application through their browser on virtually any device. The final product was available at www. hearthealthmobile.com until 2017. The mobile app and its associated web-based app were launched in a production-ready state and made available for download in the Apple Store in January 2013 and taken down in 2017.
pliance with ONC, along with recommendations from internal ad rate risk management and information systems' security.
Global positioning system (GPS) was also employed to sug cations within the user interface (UI). If a user reported having n pressure or cholesterol screening, the locations of local options this screening location dataset was available via an API maintaine location data were loaded through a manually updated and di supplied by contest coordinators at the Million Hearts Risk Ch lected these data from partner organizations interested in being tions within the contestant apps' UIs.
The HHM iOS app (for Apple devices) prototype was deve available for judging within 30 days, and was initially selected receive an award (see Figure 1). Additional modifications to the a the following 2 months, taking into account the judges' feedback. preliminary testing with users, both internal and external to the t During those 2 months, a gamified version of the app was also ad 5 web application (see Figure 2), which allowed users to access their browser on virtually any device. The final product was a ealthmobile.com until 2017. The mobile app and its associate launched in a production-ready state and made available for dow in January 2013 and taken down in 2017.

Dissemination
A mix of communications comprised the dissemination pla releases came from HSS and MCHS [16,17]. Social media pos MCHS Corporate Communications team and by personal accou team and their colleagues, friends, and family. No formal social m ducted. Additional releases from local promotions within the f cities may have occurred, but those were not tracked by this pro The Million Hearts Risk Check Challenge partnered with county: Baltimore, Tulsa, Chicago, Philadelphia, and San Diego C locations had different campaigns surrounding the Million Hea cluded public walks, public service announcements, and media acted autonomously during the campaign and after the release o ample, Chicago's event mentioned HHM at a public park even health official who was interviewed on a morning television stat demonstration of the HHM app. Events and efforts at other site were reported back to the HHM team.

Data Collection and Analysis
The primary methods of data collection were Adobe Omnit form, publicly available user reviews through the Apple Store, an reviews through AppAdvice.com. Usage data, not including pat

Dissemination
A mix of communications comprised the dissemination plan. The only known press releases came from HSS and MCHS [16,17]. Social media posts were followed by the MCHS Corporate Communications team and by personal accounts of the development team and their colleagues, friends, and family. No formal social media campaign was conducted. Additional releases from local promotions within the five participating launch cities may have occurred, but those were not tracked by this project team.
The Million Hearts Risk Check Challenge partnered with four US cities and one county: Baltimore, Tulsa, Chicago, Philadelphia, and San Diego County. Each of these five locations had different campaigns surrounding the Million Hearts Campaign, which included public walks, public service announcements, and media coverage. These entities acted autonomously during the campaign and after the release of the HHM app. For example, Chicago's event mentioned HHM at a public park event, and also had a public health official who was interviewed on a morning television station while showing a live demonstration of the HHM app. Events and efforts at other sites varied, none of which were reported back to the HHM team.

Data Collection and Analysis
The primary methods of data collection were Adobe Omniture's web analytics platform, publicly available user reviews through the Apple Store, and publicly available user reviews through AppAdvice.com. Usage data, not including patient-protected health information (e.g., weight, blood pressure, and cholesterol), were collected and stored through the Adobe Omniture cloud platform within MCHS's corporate account. Research staff collected de-identified user data from the app to tabulate the number of unique users, identify geographic distribution of users, and determine the overall profile and in-app behaviors of HHM users.

Results
The ONC of Health Information Technology announced in February 2013 that the HHM development team was the winner of the competition. From 1 January 2013 through 1 April 2017, the mobile app and the web app have been accessed by 7613 and 2573 unique visitors, respectively. Descriptive statistics of user behavior while on the app are depicted in Table 1. As expected given the nature of apps, most users spent less than 5 min on HHM.

Devices
The analytics data also show that users on larger devices will spend more time engaged in the application. Table 2 shows device-specific data for the use of the mobile app. Among mobile app HHM users, the average time spent was nearly a full minute longer on iPads than it was on the iPhone or iPad touch. Similarly, the number of device types that accessed the web app (n = 145) over the same time period was much larger than the number that accessed the iOS-based mobile app, shown in Table 3. The vast majority of visits was from devices categorized as "nonmobile" (n = 2748; 87.5%). Of the devices that logged at least 10 or more visits, the Google Nexus 7 had the longest "Average Time Spent on Site" at nearly 5 min. While the devices indicate some correlation with time spent on the mobile and web applications, we cannot confirm whether users with tablets, for example, were engaged longer than those with phones. Some users may have found the app on their phone, but decided to use a tablet to engage for a longer period of time and/or preferred a larger screen size.

Sharing
During the assessed time period, users shared the application 78 times through the web app and 1195 through the mobile app. Users were most likely to share with others through the Email button (n = 1024), rather than social media channels such as Facebook (n = 103), Twitter (n = 130), or LinkedIn (n = 15). During the same time period, 7613 unique users accessed the mobile app and 2573 accessed the web app (see Tables 4 and 5).

User Retention
As shown in Figure 3, the usage of the HHM app faded quickly after initial dissemination efforts. Initial promotions appeared to make an impact, depicted in the usage spikes and highlighted by the green arrows in Figure 4. These spikes correlated with the ONC's preplanned efforts within the collaborating US cities. Based on these data, it is anticipated that similar development efforts will have similar results. If not injected with a dedicated dissemination plan, a new app will not likely have extended reach. Simply loading an app onto an app store does not mean users will find, download, install, or use it. nation efforts. Initial promotions appeared to make an impact, depicted in the usage spikes and highlighted by the green arrows in Figure 4. These spikes correlated with the ONC's preplanned efforts within the collaborating US cities. Based on these data, it is anticipated that similar development efforts will have similar results. If not injected with a dedicated dissemination plan, a new app will not likely have extended reach. Simply loading an app onto an app store does not mean users will find, download, install, or use it.   nation efforts. Initial promotions appeared to make an impact, depicted in the usage spikes and highlighted by the green arrows in Figure 4. These spikes correlated with the ONC's preplanned efforts within the collaborating US cities. Based on these data, it is anticipated that similar development efforts will have similar results. If not injected with a dedicated dissemination plan, a new app will not likely have extended reach. Simply loading an app onto an app store does not mean users will find, download, install, or use it.

Gamifying Health
Gamification is a technique used to drive, improve, and sustain user engagement by incorporating game-like elements, such as digital awards, trophies, badges, or points [18]. The HHM gamified version runs seamlessly with the standard version, adding points and incentive to unlock a mini-game. Users were able to make the version selection after entry into the app and were also able to manually switch back and forth between versions. Unfortunately, analytics were not configured to track version variation in detail. However, an analysis of visits by page showed a significant interest in the act of switching versions (see Figure 5). While we cannot confirm the direction change (gamified to standard, or standard to gamified), we do know that users were indeed interested in trying the alternative version. Users made the choice to switch versions more than 16,000 times. Further research is needed to test various aspects of the different versions to confirm specific user appeals, intentions, and any offline actions taken as a result of extended or sustained app engagement. (see Figure 5). While we cannot confirm the direction change (gamified to standard, or standard to gamified), we do know that users were indeed interested in trying the alternative version. Users made the choice to switch versions more than 16,000 times. Further research is needed to test various aspects of the different versions to confirm specific user appeals, intentions, and any offline actions taken as a result of extended or sustained app engagement.

International Reach
The HHM app was originally designed for US users only. Even in the absence of follow-on promotions outside of initial efforts (and zero promotion internationally), HHM was downloaded and used by countries around the world (see Figures 6-7), suggesting that the mere existence within the Apple Store and on the web allows for some level of global reach. Content of the HHM app had been translated into six different languages, but was only published in English due to limited resource allocation and short timeline. Users' epidemiological data actually suggested a broader reach to international users

International Reach
The HHM app was originally designed for US users only. Even in the absence of follow-on promotions outside of initial efforts (and zero promotion internationally), HHM was downloaded and used by countries around the world (see Figures 6 and 7), suggesting that the mere existence within the Apple Store and on the web allows for some level of global reach. Content of the HHM app had been translated into six different languages, but was only published in English due to limited resource allocation and short timeline. Users' epidemiological data actually suggested a broader reach to international users incentive to unlock a mini-game. Users were able to make the version selection after entry into the app and were also able to manually switch back and forth between versions. Unfortunately, analytics were not configured to track version variation in detail. However, an analysis of visits by page showed a significant interest in the act of switching versions (see Figure 5). While we cannot confirm the direction change (gamified to standard, or standard to gamified), we do know that users were indeed interested in trying the alternative version. Users made the choice to switch versions more than 16,000 times. Further research is needed to test various aspects of the different versions to confirm specific user appeals, intentions, and any offline actions taken as a result of extended or sustained app engagement.

International Reach
The HHM app was originally designed for US users only. Even in the absence of follow-on promotions outside of initial efforts (and zero promotion internationally), HHM was downloaded and used by countries around the world (see Figures 6-7), suggesting that the mere existence within the Apple Store and on the web allows for some level of global reach. Content of the HHM app had been translated into six different languages, but was only published in English due to limited resource allocation and short timeline. Users' epidemiological data actually suggested a broader reach to international users  The addition of local adaptations might be appropriate to accommodate international users. First, native language translation may promote usage. Secondly, we observed Canadian users seldom use Facebook or Twitter for sharing; perhaps HHM's social media sharing feature may need to be redesigned, presenting users different sharing options based on country of residence. Interestingly, but not surprisingly, our data suggest that apps can deliver public health interventions globally, and at a low cost. Literature has reported other international strategies across medical specialties and public health efforts [19][20][21][22][23][24] including the development of a framework for studying complex mHealth interventions in diverse cultural settings [25]. The addition of local adaptations might be appropriate to accommodate international users. First, native language translation may promote usage. Secondly, we observed Canadian users seldom use Facebook or Twitter for sharing; perhaps HHM's social media sharing feature may need to be redesigned, presenting users different sharing options based on country of residence. Interestingly, but not surprisingly, our data suggest that apps can deliver public health interventions globally, and at a low cost. Literature has reported other international strategies across medical specialties and public health efforts [19][20][21][22][23][24] including the development of a framework for studying complex mHealth interventions in diverse cultural settings [25].

Discussion
Reducing CVD risk is essential for improving public health. The HHM app was designed to rapidly assess CVD risk and connect individuals to nearby locations for followup risk factor screenings (e.g., high blood pressure and dyslipidemia), which could eventually lead to needed clinical care and healthier lifestyle choices. HHM was modestly successful following its launch, reaching over 10,000 unique users, mainly via mobile phones, from many countries around the globe. Users typically spent 2-3 min on the app, and some also shared it within their peer networks. In contrast to the developmental effort of clinical software as part of enterprise electronic health records (EHR), consumer health app development can be accomplished at much lower cost and within a shorter timeframe, assuming a proper app development team is integrated with a clinical care team, and assisted by a library of APIs.
Although the app met the ONC criteria for data security and was placed on the Million Hearts Campaign website, legal and institutional barriers precluded additional features that may have had a stronger public health impact. Specifically, HHM did not store PHI, upload patient data to an EHR, or include direct referrals to nearby clinical centers. Such personalized features would permit a more robust assessment of health impact, such as changes in users' body weight, blood pressure, or cholesterol numbers. In general, more relevant and connected mobile-and web-based mHealth apps will need to safely link EHR and app-based PHI data, which will require careful reconsideration of existing privacy policies at most healthcare institutions. Sustainability should also be a target of future research. HHM was part of the citywide CVD risk reduction campaigns for a limited time, and HHM was also dependent on limited-time access to two API's for data-the Surescripts' list of screening locations and the Archimedes risk engine. Agreements with such entities would need to be continued, or local risk and referral algorithms would need to be created, to maintain HHM.

Discussion
Reducing CVD risk is essential for improving public health. The HHM app was designed to rapidly assess CVD risk and connect individuals to nearby locations for follow-up risk factor screenings (e.g., high blood pressure and dyslipidemia), which could eventually lead to needed clinical care and healthier lifestyle choices. HHM was modestly successful following its launch, reaching over 10,000 unique users, mainly via mobile phones, from many countries around the globe. Users typically spent 2-3 min on the app, and some also shared it within their peer networks. In contrast to the developmental effort of clinical software as part of enterprise electronic health records (EHR), consumer health app development can be accomplished at much lower cost and within a shorter timeframe, assuming a proper app development team is integrated with a clinical care team, and assisted by a library of APIs.
Although the app met the ONC criteria for data security and was placed on the Million Hearts Campaign website, legal and institutional barriers precluded additional features that may have had a stronger public health impact. Specifically, HHM did not store PHI, upload patient data to an EHR, or include direct referrals to nearby clinical centers. Such personalized features would permit a more robust assessment of health impact, such as changes in users' body weight, blood pressure, or cholesterol numbers. In general, more relevant and connected mobile-and web-based mHealth apps will need to safely link EHR and app-based PHI data, which will require careful reconsideration of existing privacy policies at most healthcare institutions. Sustainability should also be a target of future research. HHM was part of the citywide CVD risk reduction campaigns for a limited time, and HHM was also dependent on limited-time access to two API's for data-the Surescripts' list of screening locations and the Archimedes risk engine. Agreements with such entities would need to be continued, or local risk and referral algorithms would need to be created, to maintain HHM.

FDA Regulations
There are already tens of thousands of health-related consumer apps available for public use in app stores (e.g., Google Play and iTunes), with unabated growth likely. The FDA released guidance on the regulation of mHealth apps, but this was primarily focused on medical devices, rather than the whole spectrum of mobile apps [26]. Some elements of these FDA app guidelines remain ambiguous. For example, the FDA states it regulates products that display medical data. This might be considered data conversion or analytics, but unlike advanced analytical programs, the agency implies regulation of even rudimentary displays (e.g., graphing medical-data time trends). The FDA also indicated they will not regulate "software that allows a doctor to enter or store a patient's health history in a computer file", presumably referring generally to EHRs [20].
Regardless of the app's platform, clinical and patient tools vary in their intentions and outcomes, from risk calculators to drug-genomic interaction recommendations and alerts. Technological innovations in mHealth apps will likely continue at a very rapid pace, quickly "dating" some FDA regulations in this regard and highlighting the importance for researchers and health professionals to stay current on new FDA guidance as it is released [27].

Data Security and Privacy
There are vulnerable layers within most applications, including the operating system, the device, the network, and also the servers used to transmit and store the data [28]. As patient use of mobile devices increases, privacy concerns also increase. These concerns span across electronic data transmission to unintended capture of PHI.
The emergence of mobile phones and tablets amplifies these risks. New technologies have brought substantial benefits to consumers and healthcare, but they also bring new risks (e.g., data breeches). HHM did not have audio or video communication within its feature set; however, it did transmit data to and from an API, to and from Adobe Omniture (analytics), and to social media if the user chose. There was no specific identifying information collected through the application's interface; however, there is always the potential to match a unique identity to a combination of identifiers, such as weight, age, smoking status, etc. At the advice of the authors' Legal team, the unique PHI entered into the application is not tracked or stored in any way by our systems. This bolsters the confidence of security and limits PHI leakage, but it also hinders the ability to track and report on change in health outcomes over time (further discussed below) or link the app with EHR data.

Technology Platforms
A persistent question in app development is regarding the optimal platform (e.g., iOS, Android, and web-based) for user experience, engagement, and resource investment. Newly developed technologies now offer developers the tools to parse code into two separate platforms (e.g., C# to iOS and Android), but this was not available at the time HMM was developed. Thus, we chose to move forward with iOS, based mainly on the experience of our development team. As the project progressed, it was strongly suggested by ONC leadership that a parallel app was needed to accommodate more users-a web app, usable across more devices than the iOS version. The development team needed to utilize another programmer, skilled in HTML5, to develop the second version. We recommend that each organization and team assess their current market, intended target audience and the availability of programming and support skills prior to committing resources to any app development project.

Third-Party Software and Services
As noted earlier, the risk engine was provided via API for contestants entering this national competition. An API or other third-party software can provide a much needed shortcut for a healthcare organization seeking to develop this type of application, as was the case with the HHM project. Without the opportunity to connect with this existing risk engine, we would not have been in the position to develop such an application-certainly not for a competition, and also not within the same timeline or budget. These APIs also made it possible for the development team to stay focused on design, testing, and implementation. Serving as a connection to an externally managed engine or dataset, APIs simplify the development and shorten the development cycle. Furthermore, app updates were streamlined, as core functionality from the risk engine and screening location listings were managed externally, through third parties. Specifically, within the iOS version (and also applicable to an Android version), this strategy insulates segments of code so that API upgrades can be performed inside the service layer, without altering the core codebase of the app-a process that requires a release to and review by Apple (or Google Play Store for Android apps). Thus, updates to those components were seamless, requiring limited testing and deployment effort by the internal HHM team. Equipped with intimate knowledge of healthcare processes and patient preferences, providers may be well-positioned and at a competitive advantage when developing mHealth applications for patient care.
In healthcare and other disciplines, there is often extreme variability in costs and deliverables of software projects. Outside of budget, scope, and timelines, development teams themselves can vary drastically in leadership, experience, focus, and motivation. Longer timelines will also often see staff turnover that can lead to delays from hiring, training, and knowledge transfer before new members are acclimated and proficient in the tasks assigned. Although some provider organizations have information technology development teams on board, they usually do not have a very large development team, much less a large team capable of shifting focus quickly to a new effort. Once again, this is where third-party services and APIs can play significant roles, enabling healthcare providers to take on this type of specialized software development.

Limitations
The major limitation of the project is that the team was not able to capture patient health outcomes due to PHI concerns by the governing institution. As such, the foci of this paper were lessons from development and dissemination, rather than outcomes of testing the app as mHealth intervention. Furthermore, the rollout and dissemination of the app, a part of the larger national campaign, was inconsistent across the different US cities, thus complicating any evaluation of health change impact from the app itself. Each of the aforementioned limitations present critical considerations for future and similar privately or federally funded programs.

Conclusions
mHealth apps, both those related to CVD and other chronic diseases, are still in relative infancy, with inconclusive evidence to date in terms of the quality, effectiveness, and sustainability of mHealth interventions [4,6]. How such apps may ultimately improve the public's health remains unclear, but HHM provides a useful model by which healthcare delivery systems, even those in rural areas or outside of major academic health settings or technological hubs, can play a lead role in their development. Furthermore, if mHealth apps are found to be effective, health systems and insurers could shift resource foci towards the development and expansion. It seems likely that an effective app would be far less costly than additional in-person healthcare for chronic disease prevention. As mobile phone technologies are approaching universal accessibility, mHealth apps are broadly scalable across geographic and economic strata.
Future research should focus on rigorous designs to test the value-added contribution of CVD mobile apps as a component of (broader) CVD risk reduction campaigns such as those communities involved in the Million Hearts Campaign. This hypothesis-generating project also raised a number of new research questions on topics including the following: (1) organizational readiness to adopt and/or promote mHealth technologies for sustainable physician or patient use [29]; (2) patients' variability in willingness to share health apps via social media, particularly those with CVD risk factors or other chronic conditions; (3) effectiveness of CVD mHealth apps as part of an intervention program; (4) effectiveness of federally funded competitions/challenges that encourage the development of new technologies; (5) international reach, variability, and impact; (6) impact and effectiveness of gamification strategies in mHealth design; and (7) challenges and opportunities in leveraging augmented [30] and virtual reality in mHealth application designs. We recommend that agencies hosting similar challenge/competitions consider the collection of PHI as a requirement, enabling future analyses of health outcomes and impact.