Data-Driven Activities Involving Electronic Health Records: An Activity and Task Analysis Framework for Interactive Visualization Tools

.


Introduction
An electronic health record (EHR) contains patient data, such as demographics, prescriptions, medical history, diagnosis, surgical notes, and discharge summaries. Healthcare providers use EHRs to make critical decisions, study the effects of treatments, determine the effectiveness of treatments, and monitor patient improvement after a particular treatment. In addition to these benefits, EHRs can potentially aid clinical researchers in detecting hidden trends and missing events, revealing unexpected sequences, reducing the incidence of medical errors, and establishing quality control [1,2]. Recently, several healthcare organizations have used systems that incorporate EHR data to improve the quality of care; these systems are intended to replace traditional paper-based medical records [3]. However, a few studies reveal that these EHR-based systems hardly improve the quality of care. One of the reasons for this is that they do not allow for human-data interaction in a manner that fits and supports the needs of healthcare providers [4,5]. A set of technologies and techniques that can improve the efficacy and utility of these EHR-based systems can be found in information visualization [5], or broadly speaking interactive visualization tools (IVTs).
IVTs can be defined as computational technologies that use visual representations (i.e., visualizations) to amplify human cognition when working with data [6,7]. IVTs can help people who use them gain better insight by providing the means to explore the data at various levels of granularity and abstraction. An important feature of IVTs that makes them suitable for the exploration of EHRs is the ability to show relevant data quickly by mapping it to visualizations [5]. Another feature is interaction. Making the visualization interactive allows healthcare providers to perform various data-driven tasks and activities. Interaction helps users accomplish their overall goals by dynamically changing the mapping, view, and scope of EHR data. In recent years, a number of EHR-based IVTs have been developed and deployed to support healthcare providers in performing data-driven activities.
To provide a clear and systematic approach in examining EHR-based IVTs for clinical decision support, this paper provides a framework for analyzing tasks and activities supported by these tools. To do so, we will first provide a brief survey of some of the existing IVTs that support the exploration and querying of EHR data and examine overall patterns in these tools. This survey does not include EHR-based IVTs that are designed for clinical documentation, administration, and billing processes.
There are a few studies that review EHR-based IVTs and their applications. Rind et al. [5] reviewed and compared state-of-the-art information visualization tools that involve EHR data using four criteria: (1) data types that they cover, (2) support for multiple variables, (3) support for one versus multiple patient records, and (4) support for user intents. Lesselroth and Pieczkiewicz [8] surveyed different visualization techniques for EHRs. They cover a large number of visualization tools (e.g., Lifelines, MIVA, WBIVS, and VISITORS). Their survey is organized into five sections: (1) multimedia, (2) smart dashboards to improve situational awareness, (3) longitudinal and problem-oriented views to tell clinical narratives, (4) iconography and context links to support just-in-time information, and (5) probability analysis and decision heuristics to support decision analysis and bias identification. Combi et al. [9] reviewed a few visualization tools (e.g., IPBC, KHOSPAD, KNAVE II, Paint Strips, and VISITORS) and described them based on the following features: subject cardinality (single/multiple patients), concept cardinality (single/multiple variables), abstraction level (raw data, abstract concepts, knowledge), and temporal granularity (single, single but variable, multiple). Finally, in a book chapter, Aigner et al. [10] described strategies to visualize (1) clinical guidelines seen as plans (e.g., GEM Cutter, DELT/A), (2) patients' data seen as multidimensional information space (e.g., Midgaard, VIE-VISU, Gravi++), and (3) patients' data related to clinical guidelines (e.g., Tallis Tester, CareVis).
A careful examination of the above surveys shows that a systematic analysis of IVTs with a focus on how they support EHR-data-driven tasks and activities is lacking. The purpose of the current paper is to fill this gap. Here, we present a framework for analyzing how IVTs can support different EHR-based tasks and activities. The framework can help designers and researchers to conceptualize the functionalities of EHR-based IVTs in an organized manner. In addition, this paper is suggestive of how this framework can be used to evaluate existing EHR-based IVTs and design new ones systematically. This paper also leads to the development of best practices for designing similar frameworks in similar areas.
The rest of this paper is organized as follows. Section 2 discusses how the proposed framework is formed and examines the relationships among the three concepts of activities, tasks, and low-level interactions in the context of the framework. Section 3 presents our strategy for searching relevant literature and explains our selection criteria. Section 4 provides a brief survey of a set of IVTs and outlines their main goal(s). In this section, using the proposed analytical framework, we identify the tasks and activities that IVTs support. Finally, Section 5 discusses how the framework can be used to evaluate the surveyed EHR-based IVTs.

A Proposed Activity and Task Analysis Framework
In the context of IVTs, user-tool interaction can be conceptualized as actions that are performed by users and consequent reactions that occur via the tool's interface. This bi-directional relationship between the user and the tool supports the flow of information between the two. Interaction allows for human-information discourse [11]. Furthermore, it allows users to adjust different features of the IVT to suit their analytical needs. Interaction can be characterized at different levels of granularity [7,12]. As displayed in Figure 1, an activity can be conceptualized at the highest level, where it is composed of multiple lower-level tasks (e.g., ranking, categorizing, and identifying) that work together to accomplish the activity's overall goal. An activity and a task can consist of multiple sub-activities and sub-tasks, respectively. At the lower level, tasks can be considered to have visual and interactive aspects; tasks that are supported by visual processing are called visual tasks. For instance, consider a scenario in which a user is working with a stacked bar chart that aggregates laboratory test results. The user needs to understand the distribution of a specific test of a collection of patients after surgery over time. Some of the visual tasks that the user may need to perform can include detecting the time when the test is at its peak and observing the average test result at different times. Interactive tasks require users to act upon visualizations. For instance, in the example above, the user may want to cluster the test results based on different time granularities (e.g., over an hour, over a day, or over a month). Each interactive task is made up of a number of lower-level actions (i.e., interactions) that are carried out to complete the task.
Multimodal Technologies and Interact. 2020, 4, 7 4 of 28 hypotheses by analyzing patient history and status [50]. Lastly, monitoring refers to the activity of repetitive testing with the aim of adjusting and guiding the management of recurrent or chronic diseases [51].

Hierarchical Structure of Activities, Sub-Activities, Tasks, and Sub-Tasks
In this section, we identify sub-activities, tasks, and sub-tasks that blend and combine together to give rise to the three activities of interpreting, predicting, and monitoring. Interpreting, as a higher-level activity, can be comprised of four sub-activities: (i) understanding (e.g., gaining insight into patient medical records), (ii) discovering (e.g., finding patients with interesting medical event patterns), (iii) exploring (e.g., observing patient data in different temporal granularities), and (iv) overviewing (e.g., providing compact visual summaries of all event sequences found in the data). Likewise, predicting can be comprised of two sub-activities: (i) learning (e.g., generating new In most complex situations, activities, sub-activities, tasks, and sub-tasks are combined to support users in accomplishing their overall goal. It is important to note two perspectives from which we can view human-data discourse. From a top-down perspective, users' goals flow from higher-level activities that need to be accomplished. From here, we go down to a number of tasks and sub-tasks (visual and interactive), and then to a set of low-level interactions. From a bottom-up perspective, the performance of a series of low-level interactions that users perform with visual representations gives emergence to tasks. Similarly, the performance of a sequence of tasks gives emergence to activities all the way up until an overall goal is accomplished.
In this paper, we present an activity and task analysis framework for examining EHR-based IVTs (i.e., ones that involve EHRs as their main source of data with which users perform data-driven tasks and activities). To identify what activities, sub-activities, tasks, and sub-tasks are supported in EHR-based IVTs, we have examined a number of such tools that have been developed by different researchers and have been reported in the literature (see Wang et al. [13]; Wongsuphasawat et al. [14]; Wongsuphasawat and Gotz [15]; Malik et al. [16]; Fails [17]; Klimov et al. [18]; Wongsuphasawat [19]; Monroe et al. [20]; Brodbeck et al. [21]; Chittaro et al. [22]; Rind et al. [23]; Plaisant et al. [24]; Faiola and Newlon [25]; Pieczkiewicz et al. [26]; Bade et al. [27]; Hinum et al. [28]; Rind et al. [29]; and Ordonez et al. [30]; Gresh et al. [31]; Horn et al. [32]). To conceptualize and develop the elements of the framework, our focus is the identification of activities and tasks that are independent of any specific technology or platform. To be consistent, we re-interpret how activities and tasks are named by the authors of the afore-listed sources in light of the unified language of our proposed framework. The activity and task terms we use might differ from the language of the existing literature since the authors have described their tools using their own vocabulary. Unfortunately, the language that different authors use is not consistent. Such inconsistency makes it difficult to analyze how well and comprehensively such tools support EHR-based tasks and how they can be improved. In the next section, we define and categorize the higher-level activities that result from interaction and combination of different sub-activities, tasks, and sub-tasks.

Hierarchical Structure of Activities, Sub-Activities, Tasks, and Sub-Tasks
In this section, we identify sub-activities, tasks, and sub-tasks that blend and combine together to give rise to the three activities of interpreting, predicting, and monitoring. Interpreting, as a higher-level activity, can be comprised of four sub-activities: (i) understanding (e.g., gaining insight into patient medical records), (ii) discovering (e.g., finding patients with interesting medical event patterns), (iii) exploring (e.g., observing patient data in different temporal granularities), and (iv) overviewing (e.g., providing compact visual summaries of all event sequences found in the data). Likewise, predicting can be comprised of two sub-activities: (i) learning (e.g., generating new hypotheses from the data), and (ii) discovering (e.g., recognizing the deterioration of the disease). Finally, monitoring is composed of (i) investigating (e.g., examining the development of a patient after treatment), (ii) analyzing (e.g., studying the aggregated event sequences for quality assurance), and (iii) evaluating (e.g., assessing the quality of care based on clinical parameters). At the next level of the hierarchy, as shown in Figure 2, each sub-activity can be composed of a number of visual (e.g., specifying, recognizing, and detecting) as well as interactive tasks (e.g., locating, ordering, querying, and clustering). Moreover, as shown in Table 1, each task consists of different sub-tasks; for instance, ordering can be carried out by a combination of sub-tasks such as ranking, aggregating, identifying, and classifying.

Search Strategy
We conducted an electronic literature search in order to collect the research papers that describe the design, implementation, or evaluation of EHR-based IVTs. In order to assure a comprehensive document search, we included all the keywords that are relevant to the goal of the research and also covered all the synonyms and related terms, both for EHRs and visualization tools. We further broadened our search by adding an * to the end of a term to make sure the search engines picked out different variations of the term. We also added quotation marks around phrases to ensure that the exact sequence of words is found. To ensure that relevant papers were not missed in our search, we used a relatively large set of keywords. We used two categories of keywords. The first category concerned visualization tools and included the following terms: "visualization*", "visualization tool*", "information visualization*", "interactive visualization*", "interactive visualization tool*", "visualization system*", and "information visualization system*". For the second category, EHR, we used the following terms: "Health Record*", "Electronic Health Record*", "EHR*", "Electronic Patient Record*", "Electronic Medical Record*", "Patients Record*", and "Patient Record*". As we were looking for papers about EHR-based visualization tools, we used the keywords shown in Table 2.
We used the following search engines based on their relevance to the field: PubMed, the ACM Digital Library, the IEEE Library, and Google Scholar. We also looked for relevant papers in two medical informatics journals (International Journal of Medical Informatics and Journal of the American Medical Informatics Association). Furthermore, additional papers were collected in conference proceedings (e.g., IEEE Conference on Visual Analytics Science and Technology (VAST), HCIL Workshop 2015, and IEEE VisWeek Workshop on Visual Analytics in Health Care) that were published in 2007 and later. We then manually reviewed the reference lists of the papers that met the selection criteria to find other relevant studies that had not been identified in the database search. All the studies included in this survey were published from 1998 until 2015. We reviewed all of the abstracts, removed the duplicates, and shortlisted abstracts for a more detailed assessment.

Selection Criteria
Out of all the studies that survived the initial filtering, we only included those that described an interactive visualization tool and provided a detailed description of the tool's visualization and its interaction design in order to analyze how the tool can support different EHR-data-driven tasks and activities. All the papers related to the visualization of any administrative tasks with patient data, medical guidelines, genetics data, and syndromic surveillance were excluded from our survey as we only focused on clinical EHR data. We also excluded the studies that were solely focused on the visualization of free text (e.g., the patient's progress notes) and medical images (e.g., magnetic resonance imaging, and X-ray images).

Results
A total of 912 articles were identified from our initial search of electronic databases. A search of the gray literature and manually searching references from articles resulted in an additional 34 papers. We removed a total number of 205 duplicates that were included in the 946 articles, both within and between search engines. We then reviewed all the abstracts and excluded 685 further articles. Next, we read the full text of 56 remaining articles and excluded the ones that did not meet the selection criteria. Finally, 24 studies remained for the analysis. The results of the selection procedure are displayed in the flow diagram in Figure 3. Table 2. Overview of the search terms used.

Survey of the Interactive Visualization Tools
In this section, we provide a survey of 19 IVTs that are described in the chosen articles and use our proposed activity and task framework to analyze them. The survey includes an overview of the goal of the IVT, a brief description of its visualization, and an analysis of how sub-activities, tasks, and sub-tasks blend and combine to accomplish the tool's main higher-level activities of interpreting, predicting and, monitoring. A very important criterion to differentiate IVTs is whether they support activities that involve multiple patient records or exploration of an individual patient. We divide our survey into two different types of IVTs based on this criterion: population-based tools and single-patient tools. Initially, studies were focused on single-patient tools, but since 2010, most of the IVTs are developed to support large numbers of patient records. Our survey includes more population-based tools, as it seems that these are more prevalent than single-patient tools. For the first type, we survey 14 tools, and, for the second type, we survey five tools.

Population-Based Tools
Population-based IVTs support data-driven activities that involve multiplicity of patient records in aggregate form and simultaneously. Although these types of tools display fewer details about a particular patient, they provide users with the ability to recognize patterns, detect anomalies, find desired records, and cluster and aggregate records into different groups. In this section, we survey fourteen population-based IVTs.

Survey of the Interactive Visualization Tools
In this section, we provide a survey of 19 IVTs that are described in the chosen articles and use our proposed activity and task framework to analyze them. The survey includes an overview of the goal of the IVT, a brief description of its visualization, and an analysis of how sub-activities, tasks, and sub-tasks blend and combine to accomplish the tool's main higher-level activities of interpreting, predicting and, monitoring. A very important criterion to differentiate IVTs is whether they support activities that involve multiple patient records or exploration of an individual patient. We divide our survey into two different types of IVTs based on this criterion: population-based tools and single-patient tools. Initially, studies were focused on single-patient tools, but since 2010, most of the IVTs are developed to support large numbers of patient records. Our survey includes more population-based tools, as it seems that these are more prevalent than single-patient tools. For the first type, we survey 14 tools, and, for the second type, we survey five tools.

Population-Based Tools
Population-based IVTs support data-driven activities that involve multiplicity of patient records in aggregate form and simultaneously. Although these types of tools display fewer details about a particular patient, they provide users with the ability to recognize patterns, detect anomalies, find desired records, and cluster and aggregate records into different groups. In this section, we survey fourteen population-based IVTs.

Lifelines2
Lifelines2 [13,52] enables users to explore and analyze a set of temporal categorical patient records interactively. As shown in Figure 4, each record is represented by a horizontal strip containing patient ID and multiple events in patient history that occur at various times. Each event shows up as a color-coded triangle icon on a horizontal timeline. Lifelines2 allows the detection of temporal patterns and trends across EHRs to facilitate hypothesis generation and identify cause-and-effect relationships between patient records.
Multimodal Technologies and Interact. 2020, 4, 7 10 of 28 Lifelines2 [13,52] enables users to explore and analyze a set of temporal categorical patient records interactively. As shown in Figure 4, each record is represented by a horizontal strip containing patient ID and multiple events in patient history that occur at various times. Each event shows up as a color-coded triangle icon on a horizontal timeline. Lifelines2 allows the detection of temporal patterns and trends across EHRs to facilitate hypothesis generation and identify cause-and-effect relationships between patient records.
This tool supports the activity of interpreting by allowing users to get a better understanding of clinical problems and discovering patients with interesting medical event patterns. It also supports monitoring by investigating the impact of hospital protocol changes in patient care. It allows for temporal ordering of event sequences, observing the distribution of temporal events, and locating records with particular event sequences. These tasks (ordering, observing, locating) are supported by sub-tasks such as ranking, aggregating, and identifying. Lifeflow [14,53] provides a visual summary of the exploration and analysis of event sequences in EHR data. While in Lifelines2, due to limited screen space, it is not possible to see all records simultaneously; Lifeflow gives users the ability to answer questions that require an overview of all the records. To convert from Lifelines2 view to Lifeflow, a data structure called "tree of sequences" is created by aggregating all the records. This structure is then converted into a Lifeflow view with each node representing an event bar. Figure 5 shows Lifeflow visualization where all the records are vertically stacked on the horizontal timeline and all the events are represented using color-coded triangles. This tool supports the activity of interpreting by allowing users to get a better understanding of clinical problems and discovering patients with interesting medical event patterns. It also supports monitoring by investigating the impact of hospital protocol changes in patient care. It allows for temporal ordering of event sequences, observing the distribution of temporal events, and locating records with particular event sequences. These tasks (ordering, observing, locating) are supported by sub-tasks such as ranking, aggregating, and identifying.

Lifeflow
Lifeflow [14,53] provides a visual summary of the exploration and analysis of event sequences in EHR data. While in Lifelines2, due to limited screen space, it is not possible to see all records simultaneously; Lifeflow gives users the ability to answer questions that require an overview of all the records. To convert from Lifelines2 view to Lifeflow, a data structure called "tree of sequences" is created by aggregating all the records. This structure is then converted into a Lifeflow view with each node representing an event bar. Figure 5 shows Lifeflow visualization where all the records are vertically stacked on the horizontal timeline and all the events are represented using color-coded triangles.
In this IVT, the sub-activities of exploring and overviewing medical events support the activity of interpreting, while analyzing aggregated event sequences for quality assurance supports the activity of monitoring. Recognizing patterns and temporal ordering of aggregated event sequences are two tasks that enable Lifeflow to support exploring, overviewing, and analyzing sub-activities. Finally, sub-tasks such as aggregating, identifying, and classifying work together to accomplish higher-level tasks.

Eventflow
Eventflow [20] provides users with the ability to query, explore, and visualize interval data interactively. It allows pattern recognition by visualizing events in both a timeline that displays all individual records and an aggregated overview that shows common and rare patterns. As displayed in Figure 6, all the records are shown on a scrollable timeline browser. On the horizontal timeline, point-based events are displayed as triangles, while interval events are represented by the connected rectangles. In the center, an aggregated display gives users an overview of all event sequences in EHR data. The aggregation method works exactly like the one in Lifeflow, but it has been extended to work for interval events in the Eventflow. All the records with the same event sequence are aggregated into a single bar and the average time between two events among the records in the group is represented by the horizontal gap between two bars. This tool supports interpreting by providing an overview of all event sequences found in the data and exploring medical events (point-based events as well as interval events). The overviewing and exploring sub-activities can be accomplished by recognizing temporal patterns and simplifying temporal event sequences. Monitoring can be accomplished by investigating aggregated event sequences. The investigating sub-activity is supported by detecting anomalies in the data. Eventflow In this IVT, the sub-activities of exploring and overviewing medical events support the activity of interpreting, while analyzing aggregated event sequences for quality assurance supports the activity of monitoring. Recognizing patterns and temporal ordering of aggregated event sequences are two tasks that enable Lifeflow to support exploring, overviewing, and analyzing sub-activities. Finally, sub-tasks such as aggregating, identifying, and classifying work together to accomplish higher-level tasks.

Eventflow
Eventflow [20] provides users with the ability to query, explore, and visualize interval data interactively. It allows pattern recognition by visualizing events in both a timeline that displays all individual records and an aggregated overview that shows common and rare patterns. As displayed in Figure 6, all the records are shown on a scrollable timeline browser. On the horizontal timeline, point-based events are displayed as triangles, while interval events are represented by the connected rectangles. In the center, an aggregated display gives users an overview of all event sequences in EHR data. The aggregation method works exactly like the one in Lifeflow, but it has been extended to work for interval events in the Eventflow. All the records with the same event sequence are aggregated into a single bar and the average time between two events among the records in the group is represented by the horizontal gap between two bars. This tool supports interpreting by providing an overview of all event sequences found in the data and exploring medical events (point-based events as well as interval events). The overviewing and exploring sub-activities can be accomplished by recognizing temporal patterns and simplifying temporal event sequences. Monitoring can be accomplished by investigating aggregated event sequences. The investigating sub-activity is supported by detecting anomalies in the data. Eventflow supports predicting by learning new hypotheses where this sub-activity can be carried out by tasks such as specifying temporal patterns and simplifying temporal event sequences. Aggregating, identifying, classifying are the lowest-level sub-tasks for Eventflow.

Caregiver
Caregiver [21] is an IVT that supports therapeutic decision making, intervention, and monitoring. As displayed in Figure 7, the tool has three different views where the upper view displays the duration and size of the patient groups that are chosen by physicians to receive interventions. A common timeline for each patient is shown in the lower view of the chosen attributes. Caregiver allows users to create new cohorts from the search results based on a combination of values of any number of variables.
In this tool, the activity of interpreting can be accomplished by discovering trends, critical incidents, and cause-effect relationships. Caregiver also supports predicting by allowing users to learn about the deterioration in the status of a disease. It supports these sub-activities (discovering and learning) by specifying temporal relationships and clustering. Specifying and clustering can be carried out by sub-tasks such as identifying, classifying, and ranking.

CoCo
CoCo [16,54] is an IVT for comparing cohorts of sequences of events recorded in EHRs. It provides users with overview and event-level statistics of the chosen dataset along with a list of available metrics to generate new hypotheses. It consists of a file manager pane, a dataset statistics pane, an event legend, a list of available metrics, the main window, and options for filtering and sorting the results (as shown in Figure 8). The summary panel includes high-level statistics containing the total number of records and events in each record.
CoCo supports the activity of interpreting by allowing users to explore and investigate two groups of temporal event sequences simultaneously. The activity of predicting can be accomplished by learning new hypotheses from the statistical analysis while comparing the event sequences (i.e., detecting differences among groups of patients). Ranking, classifying, and identifying are the lowest-level sub-tasks in CoCo.

Caregiver
Caregiver [21] is an IVT that supports therapeutic decision making, intervention, and monitoring. As displayed in Figure 7, the tool has three different views where the upper view displays the duration and size of the patient groups that are chosen by physicians to receive interventions. A common timeline for each patient is shown in the lower view of the chosen attributes. Caregiver allows users to create new cohorts from the search results based on a combination of values of any number of variables.
In this tool, the activity of interpreting can be accomplished by discovering trends, critical incidents, and cause-effect relationships. Caregiver also supports predicting by allowing users to learn about the deterioration in the status of a disease. It supports these sub-activities (discovering and learning) by specifying temporal relationships and clustering. Specifying and clustering can be carried out by sub-tasks such as identifying, classifying, and ranking.

CoCo
CoCo [16,54] is an IVT for comparing cohorts of sequences of events recorded in EHRs. It provides users with overview and event-level statistics of the chosen dataset along with a list of available metrics to generate new hypotheses. It consists of a file manager pane, a dataset statistics pane, an event legend, a list of available metrics, the main window, and options for filtering and sorting the results (as shown in Figure 8). The summary panel includes high-level statistics containing the total number of records and events in each record.
CoCo supports the activity of interpreting by allowing users to explore and investigate two groups of temporal event sequences simultaneously. The activity of predicting can be accomplished by learning new hypotheses from the statistical analysis while comparing the event sequences (i.e., detecting differences among groups of patients). Ranking, classifying, and identifying are the lowest-level sub-tasks in CoCo.

Similan
Similan [19] is a tool that provides users with the ability to discover and explore similar records in the temporal categorical dataset. Records are ranked by their similarity to a target record that can be either a reference record or a user's specified sequence of events. The similarity measure considers the transposition of events, addition, removal, and temporal differences of matching to estimate the

Similan
Similan [19] is a tool that provides users with the ability to discover and explore similar records in the temporal categorical dataset. Records are ranked by their similarity to a target record that can be either a reference record or a user's specified sequence of events. The similarity measure considers the transposition of events, addition, removal, and temporal differences of matching to estimate the

Similan
Similan [19] is a tool that provides users with the ability to discover and explore similar records in the temporal categorical dataset. Records are ranked by their similarity to a target record that can be either a reference record or a user's specified sequence of events. The similarity measure considers the transposition of events, addition, removal, and temporal differences of matching to estimate the similarity of temporal sequences. Simian lets users to visually compare the selected target with a set of records and rank those records based on the matching score, as shown in the left side middle panel in Figure 9.
similarity of temporal sequences. Simian lets users to visually compare the selected target with a set of records and rank those records based on the matching score, as shown in the left side middle panel in Figure 9.
In this IVT, interpreting can be carried out by exploring and discovering similar records in temporal categorical data where these sub-activities themselves are supported by detecting (calculating similarity measure among records) and recognizing similarity among records. Predicting is accomplished by discovering patients with similar symptoms to a certain target patient. The sub-activity discovering can be carried out by tasks such as temporal ordering and dynamic query. Finally, sub-tasks such as ranking, identifying, and classifying work together to accomplish higher-level tasks.

Outflow
Outflow [15,55] is a graph-based visualization that shows the eventual outcome across the event sequences in patient records. It aggregates and displays event progression pathways and their corresponding properties, such as cardinality, outcomes, and timing. The tool allows users to interactively analyze the event sequences and detect their correlation with external factors (e.g., beyond the collection of event types that specify an event sequence). The tool is a state transition diagram, which is represented by a directed acyclic graph. The states (nodes) are unique combinations of patient symptoms that are mapped to rectangles, where the height of each rectangle is proportional to the number of patients. The graph is divided into different layers vertically, where layer i consists of all states in the graph with i symptoms. These layers are arranged from left to right, displaying patient history from past to future. Edges display transitions among symptoms where each edge encodes the number of patents that are involved in the transition and the average time In this IVT, interpreting can be carried out by exploring and discovering similar records in temporal categorical data where these sub-activities themselves are supported by detecting (calculating similarity measure among records) and recognizing similarity among records. Predicting is accomplished by discovering patients with similar symptoms to a certain target patient. The sub-activity discovering can be carried out by tasks such as temporal ordering and dynamic query. Finally, sub-tasks such as ranking, identifying, and classifying work together to accomplish higher-level tasks.

Outflow
Outflow [15,55] is a graph-based visualization that shows the eventual outcome across the event sequences in patient records. It aggregates and displays event progression pathways and their corresponding properties, such as cardinality, outcomes, and timing. The tool allows users to interactively analyze the event sequences and detect their correlation with external factors (e.g., beyond the collection of event types that specify an event sequence). The tool is a state transition diagram, which is represented by a directed acyclic graph. The states (nodes) are unique combinations of patient symptoms that are mapped to rectangles, where the height of each rectangle is proportional to the number of patients. The graph is divided into different layers vertically, where layer i consists of all states in the graph with i symptoms. These layers are arranged from left to right, displaying patient history from past to future. Edges display transitions among symptoms where each edge encodes the number of patents that are involved in the transition and the average time interval between different states. The end state that is represented by a trapezoid followed by a circle is used to mark points where the patient paths have ended. Finally, the color of the edges and end states represents the average outcome for the corresponding group of patients.
In this tool, sub-activities of exploring and overviewing event sequences work together to accomplish the activity of interpreting. Outflow also supports predicting by allowing users to discover the progression of temporal event sequences. The sub-activities of exploring, overviewing, and discovering can be accomplished by summarizing temporal event sequences, specifying temporal relationships, and detecting patterns from statistical summaries. Finally, aggregating, identifying, and classifying are the lowest-level sub-tasks.

IPBC
IPBC [22] (interactive parallel bar charts) is an interactive 3D visualization of temporal data. IPBC applies visual data mining to a real medical problem such as the management of multiple hemodialysis sessions. It provides users with the ability to make various decisions regarding such things as therapy, management, and medical research. Each time series is displayed as a 3D bar chart where one of the horizontal axes shows time and the vertical axis represents the value, as displayed in Figure 10. Lined up bar charts on the second horizontal axis enable users to view all the series simultaneously. interval between different states. The end state that is represented by a trapezoid followed by a circle is used to mark points where the patient paths have ended. Finally, the color of the edges and end states represents the average outcome for the corresponding group of patients. In this tool, sub-activities of exploring and overviewing event sequences work together to accomplish the activity of interpreting. Outflow also supports predicting by allowing users to discover the progression of temporal event sequences. The sub-activities of exploring, overviewing, and discovering can be accomplished by summarizing temporal event sequences, specifying temporal relationships, and detecting patterns from statistical summaries. Finally, aggregating, identifying, and classifying are the lowest-level sub-tasks.

IPBC
IPBC [22] (interactive parallel bar charts) is an interactive 3D visualization of temporal data. IPBC applies visual data mining to a real medical problem such as the management of multiple hemodialysis sessions. It provides users with the ability to make various decisions regarding such things as therapy, management, and medical research. Each time series is displayed as a 3D bar chart where one of the horizontal axes shows time and the vertical axis represents the value, as displayed in Figure 10. Lined up bar charts on the second horizontal axis enable users to view all the series simultaneously.
IPBC supports interpreting by allowing users to explore patient data interactively. Monitoring can be carried out by evaluating the quality of care based on certain clinical parameters. The sub-activities of exploring and evaluating are supported by specifying temporal relationships and recognizing similar patterns where these tasks themselves can be accomplished by sub-tasks such as identifying, classifying, and ranking.  IPBC supports interpreting by allowing users to explore patient data interactively. Monitoring can be carried out by evaluating the quality of care based on certain clinical parameters. The sub-activities of exploring and evaluating are supported by specifying temporal relationships and recognizing similar patterns where these tasks themselves can be accomplished by sub-tasks such as identifying, classifying, and ranking.

Gravi++
Gravi++ [28] allows users to explore and analyze multiple categorical variables using interactive visual clustering. This tool uses a spring-based layout to place both patient and variable icons across the visualization, where the value of a variable for a patient identifies the distance between that patient's icon and the variable's icon. Gravi++ provides users with the ability to detect clusters since patients with similar values are placed together on screen. In order to visualize the exact values of each variable for each patient, the tool shows each patient's value as a circle around variables. The patient icons are represented by spheres while the variable icons are encoded by squares. Moreover, the tool can encode different patient attributes using patient icons; for instance, the size of the sphere can be mapped to the body mass index of the patient and its color can encode the patient's gender or therapeutic outcome.
This tool supports the activity of interpreting by allowing users to explore patient data and discover clusters of similar patients. Monitoring can be accomplished by investigating the development of a patient after a certain treatment. The sub-activities of exploring, discovering, and investigating are supported by tasks such as recognizing patterns and specifying temporal relationships. Finally, identifying and classifying are the lowest-level sub-tasks that are supported by the tool.

PatternFinder
PatternFinder [17] is a query-based tool for data visualization and visual query that can help users search and discover temporal patterns within multivariate categorical data. PatternFinder allows users to specify queries for temporal events with time span and value constraints and enables them to look for temporally ordered events/values/trends as well as the existence of events. Also, users can set a range of possible time spans among the events to specify how far apart the events are from each other. The tool has two main panels: the pattern design and query specification panel and the result visualization panel. The leftmost part of the pattern design panel is the Person/People panel that enables users to limit the types of patients by name, by choosing from a list of patients, or by typing a text string. Any modifications that are done in this panel are dynamic queries that lead to an immediate update of the results in the result visualization panel. The temporal panel that is placed to the right of the Person/People panel enables users to form temporal pattern queries by chaining the events together. Users are able to search for the presence of events, the temporal sequence of events (e.g., an emergency doctor's visit followed by a hospitalization), the temporal sequence of values (e.g., 200 or below cholesterol followed by 240 or higher), and the temporal value patterns (e.g., monotonically decreasing). The result visualization panel displays a graphical table of all the matches where each row shows a single pattern match for one patient. Pattern matches are represented as a timeline in a "ball-and-chain" visualization fashion where the event points are shown as circles and time spans are displayed by blue bars between the events. The color of the event point in the result visualization panel matches the color of the associated event in the query specification panel. All the events that match the query pattern specified by users are linked together by horizontal lines.
In this tool, the activity of interpreting is supported by discovering patterns and exploring patient data dynamically, where these sub-activities themselves can be carried out by tasks such as specifying temporal relationships and issuing dynamic queries. Identifying and ranking are the two low-level sub-tasks that work together to support the aforementioned tasks.

TimeRider
TimeRider [23] offers an animated scatter plot to help users discover patterns in irregularly sampled patient data covering several time spans. As shown in Figure 11, time is represented by either traces or animation in TimeRider. Color, shape, and size of marks are used to encode up to three additional variables. Users can compare patient records of different time spans by synchronizing patients' age, calendar date, and the start and end of the treatment. three additional variables. Users can compare patient records of different time spans by synchronizing patients' age, calendar date, and the start and end of the treatment. This tool supports interpreting by allowing users to detect trends, clusters, and correlations and providing them with an overview to visually compare patient data in parallel. The sub-activities of detecting and overviewing can be carried out by tasks such as specifying temporal relationships, clustering, and recognizing patterns. Identifying and aligning are the sub-tasks that work together to support the aforementioned tasks.

VISITORS
VISITORS [18,56] is an IVT that allows for exploration, analysis, and retrieval of raw temporal data. The tool uses raw numerical data (e.g., white blood cell counts) across time to derive temporal abstractions (e.g., durations of low, normal, or high blood-cell-count levels for patients). It then uses lower-level temporal abstractions in conjunction with raw data to generate higher-level abstractions. Finally, patient groups' values are aggregated and displayed. Figure 12 shows this tool's visualization environment, where raw numerical data is represented by line charts, whereas categorical data is displayed as tick marks or bars on a horizontal zoomable timeline.
In this tool, the activity of interpreting is supported by exploring patient data in different temporal granularities. The sub-activity of exploring can be carried out by tasks such as specifying relationships, observing the distribution of aggregated values of a group of patients, and locating records based on specific time and value constraints. VISITORS supports the activity of monitoring by sub-activities, such as investigating treatment effects, clinical trial results, and quality of clinical management processes. The latter sub-activity, investigating, can be carried out by the task of This tool supports interpreting by allowing users to detect trends, clusters, and correlations and providing them with an overview to visually compare patient data in parallel. The sub-activities of detecting and overviewing can be carried out by tasks such as specifying temporal relationships, clustering, and recognizing patterns. Identifying and aligning are the sub-tasks that work together to support the aforementioned tasks.

VISITORS
VISITORS [18,56] is an IVT that allows for exploration, analysis, and retrieval of raw temporal data. The tool uses raw numerical data (e.g., white blood cell counts) across time to derive temporal abstractions (e.g., durations of low, normal, or high blood-cell-count levels for patients). It then uses lower-level temporal abstractions in conjunction with raw data to generate higher-level abstractions. Finally, patient groups' values are aggregated and displayed. Figure 12 shows this tool's visualization environment, where raw numerical data is represented by line charts, whereas categorical data is displayed as tick marks or bars on a horizontal zoomable timeline.
In this tool, the activity of interpreting is supported by exploring patient data in different temporal granularities. The sub-activity of exploring can be carried out by tasks such as specifying relationships, observing the distribution of aggregated values of a group of patients, and locating records based on specific time and value constraints. VISITORS supports the activity of monitoring by sub-activities, such as investigating treatment effects, clinical trial results, and quality of clinical management processes. The latter sub-activity, investigating, can be carried out by the task of recognizing patterns as well as all the other tasks needed to support the activity of interpreting. Finally, aggregating, classifying, aligning, and identifying are the lowest-level sub-tasks that are supported by this tool. recognizing patterns as well as all the other tasks needed to support the activity of interpreting. Finally, aggregating, classifying, aligning, and identifying are the lowest-level sub-tasks that are supported by this tool.

Prima
Prima [31] is a population-based IVT that allows users to explore the categorical and numerical data by constructing different linked views. This helps users to not only understand the large set of patient records but also discover patterns and trends in the dataset. The aggregated window provides an overview of the categorical variables by showing the proportions of patients in each category for those variables using stacked bar charts. This window enables users to filter patients by applying a color "brush". It also displays correlations among different categorical variables through interactive coloring. Another view displays a histogram of numerical variables. The data can also be explored with a 2D scatter plot. Another view of the data is called multiple category tables. It shows the values of either a single variable or multiple categories. Finally, the tool incorporates the Kaplan-Meier curve to estimate the survival function from the patient data.
Prima supports the activity of interpreting by allowing users to explore patient data interactively, where this sub-activity itself can be accomplished by recognizing patterns and specifying temporal relationships. Finally, aggregating and ranking are the lowest-level sub-tasks that are supported by the tool.

Prima
Prima [31] is a population-based IVT that allows users to explore the categorical and numerical data by constructing different linked views. This helps users to not only understand the large set of patient records but also discover patterns and trends in the dataset. The aggregated window provides an overview of the categorical variables by showing the proportions of patients in each category for those variables using stacked bar charts. This window enables users to filter patients by applying a color "brush". It also displays correlations among different categorical variables through interactive coloring. Another view displays a histogram of numerical variables. The data can also be explored with a 2D scatter plot. Another view of the data is called multiple category tables. It shows the values of either a single variable or multiple categories. Finally, the tool incorporates the Kaplan-Meier curve to estimate the survival function from the patient data.
Prima supports the activity of interpreting by allowing users to explore patient data interactively, where this sub-activity itself can be accomplished by recognizing patterns and specifying temporal relationships. Finally, aggregating and ranking are the lowest-level sub-tasks that are supported by the tool.

WBIVS
WBIVS [26] is a web-based interactive tool that visualizes numerical and categorical variables for lung transplant home monitoring data. Numerical variables are displayed in line plots, while categorical variables are visualized in matrix plots. The tool visualizes ten variables in total. When a data point gets selected, all the other data points that belong to the same time period will get highlighted in the other charts. Moreover, users can find details about the last two chosen data points on the right part of the graph.
This tool supports the interpreting activity by allowing users to explore patient data interactively and discover patterns. Monitoring is supported by investigating treatment effects. The exploring and discovering sub-activities can be accomplished by tasks such as specifying temporal relationships among data points and organizing data for pattern recognition. These tasks can be composed of lowest-level sub-tasks, such as identifying, classifying, and highlighting.

Single-Patient Tools
Single-patient IVTs provide visualizations of one single-patient record at a time. These tools enable users to overview a given patient's historical data, detect important events in the patient's history, and recognize trends. In this section, we survey five single-patient IVTs.

Midgaard
Midgaard [27] allows for exploration of the intensive care units' data at different levels of abstraction from overview to details. It uses visualizations to display numerical variables of treatment plans. It incorporates a complex semantic zoom method for numerical variables by calculating their categorical abstractions based on the available screen area and zoom level. Midgaard provides users with the ability to switch between different views such as a colored background, colored bars, area charts, or augmented line charts based on the level of details. The tool can progressively switches to a more detailed view to display all the individual data points when users zoom in or switch back to more compact graphical elements when they zoom out.
Midgaard can also visualize medical treatment plans using colored bars where each bar can contain further bars displaying sub-plans. It allows users to navigate and zoom by interacting with two time axes that are placed below the visualization area. The bottom axis displays a temporal overview of the patient record while the middle axis allows users to see specific time intervals in more detail.
The activity of interpreting is supported by exploring patient data at different levels of abstraction, where this sub-activity itself can be accomplished by tasks such as recognizing fluctuations in data. Identifying and classifying are the two sub-tasks that are supported by this tool.

MIVA
MIVA [25] (Medical information visualization assistant) is a tool that transforms and organizes biometric data into temporal resolutions to provide healthcare providers with contextual knowledge. It allows users to prioritize and customize visualizations based on specific clinical problems. It visualizes the data using point plots to display temporal changes in numerical values, where each variable is represented by a separate plot, as shown in Figure 13. MIVA enables users to detect changes in multiple physiological data points over time for faster and more accurate diagnosis. Users can control the data source, time resolutions, and time periods to narrow down the assessment of a patient's condition.
This tool supports the activity of interpreting by enabling users to carry out sub-activities such as exploring longitudinal relationships in patient data where this sub-activity can be accomplished by tasks such as specifying temporal relationships and recognizing patterns. At the level of sub-tasks, this tool supports identifying as well as classifying.

VIE-VISU
VIE-VISU [32] uses a set of glyphs to display changes in a patient's status over time in intensive care. Each glyph's geometrical shape and color encodes categorical variables, while the numerical variables are represented by size of the glyph's elements. Every glyph can encode 15 variables that are classified by physiological systems. For instance, the respiratory parameters are mapped to a rectangle in the middle of the glyph; circulatory parameters are mapped to a triangle on top of the glyph, and the fluid balance parameters are shown by two smaller rectangles at the bottom of the glyph. By default, the tool displays 24 glyphs, one per hour.
The activity of interpreting can be accomplished by overviewing a patient's status, where this sub-activity is supported by tasks such as recognizing patterns. This tool supports monitoring by evaluating changes in patient's status over time. The task of identifying temporal relationships supports the sub-activity of evaluating. Finally, aggregating and classifying are two sub-tasks that can be carried out by the tool.

Lifelines
Lifelines [24] offers a visualization environment to show patient history on a zoomable timeline, where a patient's medical record is displayed by a set of events and lines. Episodes and events in a patient record are represented by a set of multiple line segments as shown in Figure 14. Color can be used to encode the states of categorical variables. This IVT provides an overview of a patient history to recognize trends, specify important events, and detect omissions in data.
The activity of interpreting is supported by understanding patient's status where this sub-activity itself can be carried out by tasks such as recognizing patterns and specifying temporal relationships. The tool supports monitoring by allowing users to carry out sub-activities such as investigating trends and anomalies in patient data. The investigating sub-activity is supported by outlining and summarizing the patient data. Finally, aggregating, classifying, and identifying are the sub-tasks that are supported by the tool. The activity of interpreting can be accomplished by overviewing a patient's status, where this sub-activity is supported by tasks such as recognizing patterns. This tool supports monitoring by evaluating changes in patient's status over time. The task of identifying temporal relationships supports the sub-activity of evaluating. Finally, aggregating and classifying are two sub-tasks that can be carried out by the tool.

Lifelines
Lifelines [24] offers a visualization environment to show patient history on a zoomable timeline, where a patient's medical record is displayed by a set of events and lines. Episodes and events in a patient record are represented by a set of multiple line segments as shown in Figure 14. Color can be used to encode the states of categorical variables. This IVT provides an overview of a patient history to recognize trends, specify important events, and detect omissions in data.
The activity of interpreting is supported by understanding patient's status where this sub-activity itself can be carried out by tasks such as recognizing patterns and specifying temporal relationships. The tool supports monitoring by allowing users to carry out sub-activities such as investigating trends and anomalies in patient data. The investigating sub-activity is supported by outlining and summarizing the patient data. Finally, aggregating, classifying, and identifying are the sub-tasks that are supported by the tool.

VisuExplore
VisuExplore [29,57] displays patient data in different views aligned with a horizontal timeline, where each view shows multiple variables. This IVT uses common visualization techniques that make it easy to use and learn. In this tool, numerical data are displayed using bar charts and line plots, whereas categorical data are represented using event charts and timeline charts, as shown in Figure 15.

VisuExplore
VisuExplore [29,57] displays patient data in different views aligned with a horizontal timeline, where each view shows multiple variables. This IVT uses common visualization techniques that make it easy to use and learn. In this tool, numerical data are displayed using bar charts and line plots, whereas categorical data are represented using event charts and timeline charts, as shown in Figure 15.
In this tool, the activity of interpreting is supported by exploring temporal data of patients with chronic diseases, where this sub-activity can be carried out by tasks such as specifying temporal relationships. Finally, aligning and identifying are two sub-tasks that can be carried out by the tool.

Discussion and Limitations
In this paper, we have presented and proposed a framework to identify and analyze EHR-data-driven tasks and activities in the context of IVTs-that is, all the activities, sub-activities, tasks, and sub-tasks that are supported by EHR-based IVTs. Using a survey of 19 EHR-based IVTs, we demonstrate how these IVTs support activities by identifying the combination of sub-activities, tasks, and sub-tasks that work together to help users carry out the three higher-level activities as displayed in Table 3. Interpreting is supported by all IVTs surveyed in this paper. Eventflow, Similan, CoCo, Outflow, and Caregiver are the only IVTs that support predicting, whereas Lifelines2, Lifeflow, Eventflow, Gravi++, IPBC, TimeRider, VISITORS, WBIVS, VIE-VISU, Lifelines, CoCo, and Visu-Explore are the tools that facilitate monitoring. Going down from high-level activities, recognizing patterns and specifying temporal relationships are the most common sub-activities that help users with the activity of interpreting in most of the IVTs. The existing EHR-based IVTs support predicting by giving users the ability to perform sub-activities such as  Our proposed framework can offer a number of benefits for designers, researchers, and evaluators of EHR-based IVTs. Firstly, the framework can help the designer to conceptualize activities, tasks, and sub-tasks of EHR-based IVTs systematically. Secondly, it can assist researchers in making sense of IVTs by providing them with all the activities that can be accomplished by carrying out different sets of sub-activities, tasks, and sub-tasks. Thirdly, this framework can be used by evaluators to identify the gaps in support of higher-level activities supported by existing IVTs. It appears that almost all existing IVTs focus on the activity of interpreting, while only a few of them support predicting despite the importance of this activity in supporting users to find the patients In this tool, the activity of interpreting is supported by exploring temporal data of patients with chronic diseases, where this sub-activity can be carried out by tasks such as specifying temporal relationships. Finally, aligning and identifying are two sub-tasks that can be carried out by the tool.

Discussion and Limitations
In this paper, we have presented and proposed a framework to identify and analyze EHR-data-driven tasks and activities in the context of IVTs-that is, all the activities, sub-activities, tasks, and sub-tasks that are supported by EHR-based IVTs. Using a survey of 19 EHR-based IVTs, we demonstrate how these IVTs support activities by identifying the combination of sub-activities, tasks, and sub-tasks that work together to help users carry out the three higher-level activities as displayed in Table 3. Interpreting is supported by all IVTs surveyed in this paper. Eventflow, Similan, CoCo, Outflow, and Caregiver are the only IVTs that support predicting, whereas Lifelines2, Lifeflow, Eventflow, Gravi++, IPBC, TimeRider, VISITORS, WBIVS, VIE-VISU, Lifelines, CoCo, and Visu-Explore are the tools that facilitate monitoring. Going down from high-level activities, recognizing patterns and specifying temporal relationships are the most common sub-activities that help users with the activity of interpreting in most of the IVTs. The existing EHR-based IVTs support predicting by giving users the ability to perform sub-activities such as learning new hypotheses, discovering patients with similar symptoms to a target patient, and detecting early deterioration of a disease. Finally, the most common sub-activities that facilitate monitoring are evaluating the quality of care and investigating the development of a patient's status after treatment.
Our proposed framework can offer a number of benefits for designers, researchers, and evaluators of EHR-based IVTs. Firstly, the framework can help the designer to conceptualize activities, tasks, and sub-tasks of EHR-based IVTs systematically. Secondly, it can assist researchers in making sense of IVTs by providing them with all the activities that can be accomplished by carrying out different sets of sub-activities, tasks, and sub-tasks. Thirdly, this framework can be used by evaluators to identify the gaps in support of higher-level activities supported by existing IVTs. It appears that almost all existing IVTs focus on the activity of interpreting, while only a few of them support predicting despite the importance of this activity in supporting users to find the patients that are at high risk and identify the risk factors of various diseases. Also, some of the EHR-based IVTs do not pay enough attention to monitoring, even though this activity is beneficial in investigating the quality of clinical management processes. All these higher-level activities should be an integral part of a properly designed EHR-based IVT since healthcare providers use such tools to (1) better understand patients' condition, (2) anticipate the course of a specific disease, and (3) track patients' condition after treatment. Most of the tools surveyed in this paper can only satisfy a certain aspect of users' needs. According to a recent survey in the US, 40% of the clinicians are not satisfied with the existing EHR-based system [58]. Therefore, a framework is needed to guide the designer of an IVT in choosing which activities, tasks, and sub-tasks the tool should support. Using questions such as, "What activities can users accomplish by executing a set of tasks?" or "What tasks should be supported to provide users with the ability to perform their activities?", we demonstrate how the proposed framework can be used by designers of EHR-based IVTs to systematically conceptualize and design the tasks and activities of such tools. Given the framework, all designers need to know is, which low-level sub-tasks, tasks, and sub-activities to select and how to blend and combine them to support higher-level activities and allow users to accomplish their overall goal. For instance, if a designer wants to design an IVT to monitor an infant's condition in the neonatal intensive care unit, they can choose different sets of sub-activities, such as investigating the effect of a specific treatment or evaluating changes in infant's status over time. Then, the designer selects a combination of tasks such as the temporal ordering of event sequences or displaying the distribution of temporal events to support the chosen sub-activities. Finally, a set of sub-tasks, such as ranking, aggregating, and identifying, are chosen to support the selected tasks.
We believe a successful EHR-based tool should be capable of doing more than just storing, retrieving, and exchanging patient data. It should support more complex activities, tasks, and sub-tasks to allow healthcare providers to accomplish their goals. Our proposed framework promises a new means for designers of EHR-based IVTs to understand the effectiveness of incorporating such activities, tasks, and sub-tasks in their tool. The use of our framework in EHR-based IVTs will also help physicians to make better treatment decisions and track changes in a patient's condition over time.  This paper has three key limitations. First, we do not investigate the completeness and accuracy of the data sources that IVTs are using as our survey relies on the descriptions of the IVTs found in publications and video tutorials. Second, as the main goal of this paper is the analysis of EHR-based IVTs, we exclude tools that are mainly dependent on statistical and machine learning methods. Finally, we do not consider commercial tools in this paper. This is because online descriptions of such tools do not systematically and thoroughly cover the features of these tools, i.e., their visualizations, interactions, and results.
The findings of this paper will lead to the development of best practices for creating similar frameworks in other domains. A possible area of future research involves developing frameworks for visual analytics tools that incorporate automated analysis techniques along with interactive visualizations to support the increasingly large and complex datasets in EHRs.