Supporting Sensemaking of Complex Objects with Visualizations: Visibility and Complementarity of Interactions

Abstract

Making sense of complex objects is difficult, and typically requires the use of external representations to support cognitive demands while reasoning about the objects. Visualizations are one type of external representation that can be used to support sensemaking activities. In this paper, we investigate the role of two design strategies in making the interactive features of visualizations more supportive of users’ exploratory needs when trying to make sense of complex objects. These two strategies are visibility and complementarity of interactions. We employ a theoretical framework concerned with human–information interaction and complex cognitive activities to inform, contextualize, and interpret the effects of the design strategies. The two strategies are incorporated in the design of Polyvise, a visualization tool that supports making sense of complex four-dimensional geometric objects. A mixed-methods study was conducted to evaluate the design strategies and the overall usability of Polyvise. We report the findings of the study, discuss some implications for the design of visualization tools that support sensemaking of complex objects, and propose five design guidelines. We anticipate that our results are transferrable to other contexts, and that these two design strategies can be used broadly in visualization tools intended to support activities with complex objects and information spaces.

1. Introduction

Making sense of complex objects—both physical and abstract—is difficult. Objects are complex when they are composed of many constituent components that are arranged in complicated ways and have intricate interrelationships. One study suggests that variation, symmetry, part count, simpler part decomposability, intricate details, and topology are six factors that contribute towards visual shape complexity [1]. Examples of complex objects are multi-dimensional geometric shapes, elaborate chemical compounds, large architectural structures, large graphs, and the human brain. To make sense of such objects, people need to reason about and mentally manipulate representations of the objects as they carry out different cognitive tasks [2,3]. In doing so, mental models of the objects are developed, elaborated, and refined—all important components of sensemaking activities [4,5]. As the complexity of objects increases, sensemaking and reasoning become increasingly difficult, especially when the objects exist in three or more dimensions [6,7].

Due to the difficulty of mentally manipulating complex information, external representations are typically used during sensemaking activities to support cognitive demands [8,9]. External representations facilitate sensemaking and reasoning by enabling cognitive processing to be offloaded onto and distributed throughout the external environment [10,11]. This enables a coordination among internal and external representations, through which they function as one distributed representational system [12]. Furthermore, by using external representations, cognitive processing can be offloaded onto the perceptual system, which leads to a lessened cognitive load [11]. For example, by visually representing objects diagrammatically (i.e., as external representations), cognitive load is lessened by enabling perceptual recognition of topological and geometric properties and relations of and within the objects, which would otherwise have to be done cognitively “in the head” [13].

Although visual representations (also referred to as visualizations) can decrease cognitive burden and facilitate reasoning, it is still difficult to make sense of visualizations of complex objects due to their intricacy, density, and multidimensionality. One strategy for alleviating such difficulties is to make visualizations interactive, which can provide users with more flexibility and control over what they see and how they see it. It is well-accepted that interaction can extend the expressive power of visualizations and can enhance users’ exploration abilities [14]. This is especially true for complex objects, as their semantic-richness and density can be extremely difficult to explore through non-interactive media. Although previous research has led to some recommendations for interaction and interface design for complex objects [1,15,16,17,18], there is no conclusive set of principles, guidelines, or models that captures relevant design strategies and considerations. There is a need for more research to identify and validate design strategies for supporting sensemaking of complex objects with interactive visualizations.

In this paper, we investigate the role of two design strategies in making the interactive features of visualizations more supportive of users’ exploratory needs when trying to make sense of complex objects. These two strategies are: (1) visibility; and (2) complementarity of interactions. We extend the notion of visibility commonly found in the literature to make it more suitable to complex contexts. We employ a theoretical framework concerned with human–information interaction and complex cognitive activities to inform, contextualize, and interpret the effects of the design strategies. The two strategies are incorporated in Polyvise, a visualization tool for supporting exploration of complex four-dimensional geometric objects. A mixed-methods study was conducted to evaluate the design strategies and the overall usability of Polyvise. We report the findings of the study, discuss some implications for the design of visualization tools that support sensemaking of complex objects, and propose five design guidelines. We anticipate that our results are transferrable to other contexts, and that these two design strategies can be used broadly in visualization tools intended to support activities with complex objects and information spaces.

The paper is organized into the following sections: Section 2: background information, including an overview of the two main concepts behind the design strategies—namely, visibility and complementarity of interactions; Section 3: a description of the theoretical framework concerned with human–information interaction and complex cognitive activities; Section 4: an introduction to the problem domain for our study: complex four-dimensional geometric shapes; Section 5: a description of Polyvise and its features; Section 6: method for the usability evaluation; Section 7: presentation of the results of the evaluation; Section 8: discussion of the results and implications; and Section 9: conclusions and recommendations.

2. Background

2.1. Complex Objects

In this paper, we are concerned with complex objects. Although a rigorous characterization is outside the scope of this paper, a number of characteristics that contribute to objects’ complexity can be identified: number and types of constituent components, number and types of relationships among components, intricacy of relationships among components, symmetry, topology, and number of dimensions in which objects exist [1]. Objects can be viewed as fitting somewhere along a continuum from low complexity to high complexity, rather than as fitting into binary classifications of simple or complex. In this paper, however, the specific degree of complexity is not of interest; yet, as will be discussed in Section 4, the objects in our study were four-dimensional shapes, with hundreds or thousands of components of multiple types, and multiple hierarchical levels. We do not try to limit the scope of our work to specific degrees of complexity, but suggest that our findings are applicable in situations where information is too complex to be entirely visualized, and sensemaking of the visualized information is not straightforward.

2.2. Visibility

There is widespread agreement that visibility is an important concept in interaction and interface design [19,20,21,22]. Although visibility takes on different meanings in the literature, three main definitions seem to be used: (1) Making action possibilities visible to users—e.g., when buttons are used in the interface of a word processing application to convey action possibilities for copying, pasting, saving, and so on. This type of visibility helps users to understand what they can do with a tool and reduces the need to remember the tool’s functions. This emphasis on recognition over recall serves both pragmatic and epistemic purposes, as visible actions become part of users’ cognitive system by which they think and reason about phenomena [23]. (2) Making the current state of a system visible to users—e.g., when a user launches an application and a progress bar is displayed to convey the process and progress of loading the application. This type of visibility decreases frustration and confusion by keeping the user informed of the system’s state. (3) Making concepts, data, information, and other “content” visible to users—e.g., when a user is performing a task with a visualized protein, and all of the content relevant for the task is made visible in the interface. In regard to this third meaning, to an extent, the temporality of dynamic and interactive visualizations can be viewed as an extended form of static visibility. For example, animations can increase the explicitness of visualized information and, when carefully designed and judiciously used, can lead to enhanced comprehension, learning, memory, and inference [24]. However, despite explicitly making information visible, the visibility of the information is transient. This transience may negatively impact reasoning, as once an animation ends, the information becomes invisible again, which places a cognitive burden on users to recall the information and manipulate it in working memory. One possibility in such contexts is to make the information being conveyed in the animation more permanently visible in the interface. In one study [25], it was demonstrated that capturing intermediate states of animated transformations of 3D geometric shapes, and then making the states permanently visible, added a degree of advantage to users’ reasoning and sensemaking of the relationships among shapes.

2.3. Complementary Interactions

Interactions are complementary when they together enhance the performance of users’ tasks and activities. For example, in a sensemaking activity, a user may need to perform a task of categorizing pieces of information. If an interface is designed such that the user can filter, arrange, and annotate the information in ways that facilitate the categorization task, the actions can be viewed as complementary. Empirical studies (e.g., [25,26,27,28,29,30]) suggest that complementary interactions can contribute positively towards performing diverse sensemaking activities with visualizations. For example, Groth and Streefkerk [26] found that allowing users to interact with 3D molecular visualizations, by rotating and annotating their elements, seemed to support users’ performance of knowledge discovery tasks. Among other benefits, complementary interactions can support flexibility of action, which in turn encourages more autonomous and self-regulated exploratory processes [31]. In addition, complementary interactions, when designed to work in concert, allow users to switch from one interaction to another in order to engage in different forms of exploratory activities and styles [28]. Conjunctively, different interactions enable users to perform more integrated and coordinated activities [17].

It is not easy, however, to design complementary interactions that can work in concert with one another. In some cases, users are provided with too few action possibilities, while, in other cases, users may be given too many. In the former case, lack of possibilities can make exploratory activities ineffective and inefficient, even frustrating. In the latter case, some costs might be related to time consumption and increased cognitive demand [28,31,32,33,34,35], as users might need to spend time testing all available interactions, figuring out their functions and benefits, and remembering how and when to use them. The challenge of designing complementary interactions is to balance the need to provide interactions that are conjunctively supportive, yet do not impose additional unintended load on users. This is particularly the case when dealing with complex cognitive activities, as users need to be provided with a set of interactions whose number is appropriate, that are complementary to each other, and that also allow users carry out tasks effectively.

3. Theoretical Framework

To inform, contextualize, and interpret the effects of our design strategies, we employed a broad theoretical framework that is concerned with technology-mediated human–information interaction and complex cognitive activities. As an interdisciplinary field of study, human–information interaction is concerned with how and why humans use, find, consume, work with, and interact with information (e.g., data, concepts, models, etc.) to solve problems, make decisions, learn, plan, make sense, discover, and carry out other such tasks and activities [18,36,37,38]. Although technologies mediate interaction with information, the focus of human–information interaction research and design is not on technological issues per se; rather, the focus is on the interaction between humans and information. Scholars have written about human–information interaction in various domains and contexts, such as information retrieval in digital libraries, or information categorization in personal information management. The framework we have employed here has been developed in recent years by Sedig and colleagues (see [14,18,39]), and focuses on human–information interaction, in the context of complex cognitive activities, that is mediated by interactive computational technologies. Furthermore, the framework is focused on visualizations rather than other components or interface modalities. As it is a general, high-level framework that encompasses all complex cognitive activities that are performed with interactive technologies, it can function as a theoretical lens to help guide design, implementation, and evaluation. In the following two subsections, we elaborate on components of the framework that are relevant here: complex cognitive activities and conceptual tools for interactive visualization design. In the third subsection, we synthesize these ideas and discuss how they are relevant to the design of the tool and the subsequent study.

3.1. Complex Cognitive Activities

Cognitive activities range from simple to complex. Cognitive activities are complex when they possess two essential attributes: (1) complex mental processes—e.g., those that combine and integrate elementary perceptual and attentive processes with higher-level reasoning and analytical processes; and (2) complex external conditions—e.g., dynamic, uncertain, large, heterogeneous, interdependent pieces of information [40,41]. Examples of complex cognitive activities are analyzing phone records during intelligence analysis, solving problems related to climate change, making sense of very large ontological datasets, understanding how machine learning algorithms are constructed and work in the context of visual analytics tools, and making sense of complex four-dimensional geometric objects. All of these activities involve complex mental processes (e.g., reasoning, apprehension, memory encoding, mental modeling) and complex external conditions (e.g., millions of phone records of undetermined authenticity, dozens of interdependent variables that are continuously changing, and many layers of intricately connected shapes and sub-shapes).

Complex cognitive activities can be characterized at different levels of abstraction, in order to conceptualize and discuss them in a consistent manner. In the theoretical framework we are using, they are characterized at four levels (from high to low): activity, task, individual interaction, and event. Most activities comprise a number of sub-activities, each comprising a number of tasks, each comprising a number of sub-tasks, and so on. For example, consider an activity such as making sense of a large body of medical documents. To do so, an analytical reasoning sub-activity may be required, in order to systematically decompose the documents and reason about their similarities. To perform this sub-activity, a series of tasks may need to be executed, such as browsing through the documents from a particular decade, categorizing them according to a major theme, ranking them according to their importance, and so on. These tasks may be repeated throughout the activity, sometimes being sub-tasks of other tasks. Each of these tasks may then require sub-tasks, and so on. Each task would likely involve the performance of a number of interactions. For instance, to categorize the documents, the user may need to filter them according to a particular attribute or drill into a particular document to access more information. To complete any one of these actions, a number of micro-level events such as mouse clicks, finger swipes, or keystrokes may be required. Figure 1, which is adapted from [39], depicts the performance of a generic complex cognitive activity over time. Three main levels are shown: activity (and sub-activities), tasks (and sub-tasks), and individual interactions. Events are not depicted, as they are physical and implementation-dependent. The diagram can be viewed from a bottom-up and/or top-down perspective. From a bottom-up view, the performance of a task gradually emerges over time through a sequence of interactions that users perform with visualizations. Likewise, an activity emerges over time through a sequence of tasks that users carry out. From a top-down view, activities can be broken down into a sequence of tasks, and tasks can be broken down into a sequence of interactions that are performed at the interface.

Cognitive scientists have demonstrated that complex activities do not follow pre-defined trajectories, as goals are often not fully formed in the mind of a user until there is an ongoing discourse between the user and the information [42]. In other words, as users interact with the information, goals are continually formed, revised, and abandoned based on what they are seeing and thinking. Thus, interaction and interface design decisions should be considered carefully, as they influence goal formation, task performance, and ultimately affect the performance of the overall cognitive activity.

3.2. Conceptual Tools for Interactive Visualization Design

When designing visualizations, designers have a set of information items (facts, concepts, data, objects, processes) that need to be visually represented for users to see and interact with. These information items can be conceptualized as consisting within an information space—where information space is a metaphoric space comprising all information items that are relevant to the design situation. The information space is a conceptual lens that can help the designer in thinking about the design situation. For example, if a tool is being developed to support sensemaking of protein structures, the information space might be conceptualized as comprising certain concepts (e.g., amino acid, polypeptide), processes (e.g., folding, translating), and datasets (e.g., protein folding datasets). The design challenge, then, is how to visually encode such items in ways that meet the needs of users. This process can be described as mapping items from an information space—the conceptual, metaphoric space—to a representation space—the physical space in which the items are encoded and made perceptually accessible to users.

The framework employs the conceptual lens of a system—that is, an organized whole, composed of parts that generate emergent properties through their interrelationships—to aid in thinking about the process of mapping from information space to representation space. Complex information spaces can typically be viewed as multi-level systems—i.e., systems with multiple levels of sub-systems and super-systems. For instance, a protein can be viewed as an information space, which can be viewed as a system composed of sub-systems (e.g., amino acids), each of which is composed of sub-systems (e.g., chemical elements), each of which is composed of sub-systems, and so on. Most complex information spaces can be viewed through such a lens.

When designing visualizations, designers can think about which systems and sub-systems should be visually encoded. In complex contexts, only a portion of the information space can ever be visible in the representation space at any given time. Thus, designers have to make decisions about which information to encode and how to design interactions so that users can access and work with all of the information effectively. Using a systems lens can help designers think about complex information spaces in a systematic, hierarchical fashion, and about designing interactions that are geared towards specific systems and sub-systems within an information space.

3.3. Application of the Theoretical Framework

In this paper, we are concerned specifically with sensemaking, which can be viewed as a complex cognitive activity that relies on a human–information interactive discourse [14,43]. Making sense of complex objects involves complex mental processes, including, among others, those related to mental model formation, hypothesis generation and testing, and spatial reasoning. It also involves complex external conditions, as the objects themselves are large, multidimensional, and intricately related. Thus, we can use the model of complex cognitive activities described above (see Figure 1) to abstractly conceive how the sensemaking activity will be carried out. For instance, we know that users need to perform many different interactions to achieve the goal of any given task, and that users need to perform many different tasks to make sense of complex objects. We know that it is not possible to pre-determine how this will happen, but we can identify tasks that are likely to be carried out during a sensemaking activity with complex objects, and can also identify different interactions that would likely be beneficial in supporting such tasks. In designing our study, we chose a set of tasks that are common in sensemaking activities—e.g., locating different 3D cells in the visualized object, identifying the connecting 2D elements, and ranking the complexity of different 3D cells.

We also know that in complex contexts it is not possible to visually encode all information at a single time. Thus we need to carefully consider which information is encoded, and how users can and should be able to access relevant latent information. To do so, we can conceptualize the information space (i.e., the complex object) as a multi-level system. As we encode one level of the system, the sub-systems remain latent, and can be made visible through user interaction. Rather than conceptualizing this interactive discourse in an ad hoc fashion, we can conceptualize it systematically using a systems lens, and design interactions that are intended for exploring different hierarchical levels of an object (this is also where our extended notion of visibility becomes relevant). Through this process of exploring systems and latent sub-systems (e.g., of a four-dimensional geometric object), tasks can be carried out, and there is eventually an emergence of the overall sensemaking activity, as depicted in Figure 1. Different design strategies, such as the two being examined in our research, can be used to facilitate this process of viewing and interacting with different levels of objects or information spaces, which can alleviate some of the cognitive demands that exist during sensemaking, particularly in complex contexts.

4. Problem Domain and Justification

The problem domain chosen to conduct this research is four-dimensional (4D) geometry, an area of higher discrete mathematics. In mathematics, 4D space is a Euclidian space that generalizes the rules of three-dimensional Euclidian space to four dimensions. It is important to note that four-dimensional Euclidian space is not the same as space–time, a popular construct in which time is viewed as the fourth dimension in addition to the three spatial dimensions. Space–time is non-Euclidian, and is fundamentally different from the four-dimensional space that is of interest here.

Four-dimensional objects are composed of lower-dimensional components: vertices, edges (1D), faces (2D), and cells (3D)—components that define the surface of the 4D objects. The complex and abstract nature of geometric objects within 4D space makes their exploration challenging. For example, a tesseract, which is the four-dimensional analog of a cube (see Figure 2a), is a simple example of a 4D object, and is composed of 16 vertices, 32 edges, 24 faces, and 8 cells. As the number of their constituent components increases, 4D objects can be extremely difficult to manipulate mentally without external representations. For instance, a dodecacontachoron (see Figure 2b), which is only moderate in complexity, contains 600 vertices, 1200 edges, 720 faces, and 120 cells. Other objects of higher complexity contain many times the number of elements. Besides the large number of elements, 4D objects have different types of faces and cells, adding to the difficulty of their exploration (see Figure 2c). To make sense of such objects, numerous tasks must be performed, including locating and identifying cells, comparing their attributes, and assessing relationships among components within an object. Appendix A provides a full list of tasks that we had subjects perform in our study.

One way to make sensemaking of these 4D objects more attainable is to render them as interactive visualizations. Visualizations of these objects, however, can still be highly complex, making their exploration non-trivial. As mentioned, the first challenge comes from the huge number of elements, and with it, the manifold sub-patterns and structures embedded within these objects. The second challenge comes from encoding and displaying these 4D objects as 2D representations, which by necessity may cause much information not to be readily shown. This hidden information may never be explored, but remain latent, if the necessary and sufficient interactive support mechanisms are not provided to users. For these two main reasons, visualizations of 4D objects provide a viable testbed for investigating how to design interactive mechanisms for enabling effective exploration and sense-making of these complex objects and information spaces.

Although 4D objects do not represent all types of complex information spaces, we anticipate that the results from this research can transfer to other contexts in which complex information spaces are being explored. Such spaces have similar general characteristics, such as their multi-level organization, large number of components, and intricate relationships (e.g., large-scale, complex ontological datasets). As a result, at a general level, designing visualizations for complex information spaces leads to a number of common design issues, such as those related to visibility and complementarity of interactions. The results of our usability evaluation should be able to inform designers in other similar contexts.

5. Polyvise

Polyvise is a visualization tool that allows users to explore and make sense of complex 4D geometric shapes. The tool has gone through several design and modification iterations [44]. A pilot study previously examined the usability of a prototype of Polyvise [45]. The two interaction concepts of visibility and complementarity of interactions were not well developed and connected to underlying theory. The concepts have since been crystallized and developed in much more detail, and have been connected to relevant theory. Subsequently, these concepts are analyzed and presented as potentially important interaction design strategies for exploring complex information spaces. The study data have been analyzed with a focus on complementarity and visibility, and the results of this work are reported herein.

A screen capture of Polyvise’s interface is shown in Figure 3. The interface has five main components: Display (1); Context (2); Scope (3); View (4); and Detail (5). The Display Panel shows the visualized 4D object, while the Detail, Scope, and View panels contain the interactive mechanisms, or interaction techniques, through which users are able to control and adjust the degree of internal structural display and visual complexity of the main object. These three techniques are discussed briefly next; afterwards, we describe how Polyvise implements the two design strategies of visibility and complementary interactions to make these techniques more supportive of users’ sensemaking activities.

5.1. Interaction Techniques

In this section, we describe the three main interaction techniques that have been implemented in Polyvise—namely, filtering, focus+scoping, and stacking–unstacking. A series of videos that illustrate the interaction techniques are included as Supplementary Materials on the publisher’s website.

5.1.1. Filtering

In Polyvise, discrete filters are intended to facilitate the exploration of 4D objects by allowing users to be selective about what elements and substructures they wish to observe. This selective control enables users to adjust the level of detail, so as to emphasize certain elements while deemphasizing or hiding others. Being able to adjust the level of detail can contribute to the process of abstraction when dealing with information spaces which are noisy and have layered structures. Among other benefits, filters help users to formulate hypotheses and test them [14].

Polyvise provides several different discrete filters. The filters are designed to manage the dimensional characteristics of the visualized polytopes: one filter for vertices, four filters for 1D edges, six filters for 2D faces, four filters for 3D cells, and one filter for the 4D frame (see Figure 3, Panel 5). Figure 4 shows an example of using the filters to adjust the degree of visual detail. Figure 4a is a rendering of the object in full detail. Figure 4b results from applying two types of face filters. Figure 4c results from applying an edge filter and another face filter. Further filtering operations can still be applied to the visualized object. These filtering options give users a high degree of control over what elements to view in a dynamic and systematic manner, thereby enhancing users’ abilities to explore them. In sum, filters are especially useful for exploring densely-populated visualization spaces, which can be very complex, noisy, and may be rendered unexplorable if users do not have interactive controls to adjust how much of the object is represented.

5.1.2. Focus+Scoping

Polyvise uses another interaction technique called focus+scoping. This technique gives users the ability to experiment with the structural growth of the spaces’ internal elements around specific focal points. Polyvise gives users this ability by integrating two techniques: focus+context and scoping. When exploring a space, users need to focus on an aspect of this space, while keeping in mind the overall context of exploration. Focus+context techniques aim to support continuous visual transition between focus and context [46,47]. Scoping, on the other hand, refers to dynamically working forwards and backwards to view compositional development and growth, either spatially or temporally. Our scoping technique here is essentially a user-controlled fluid form of faded scaffolding. For example, a user can adjust the temporal scope of a social network to view its growth and development over time. Similarly, a user can adjust the spatial scope of a biological cell to view its growth and development in space. Spatial and temporal scoping can be combined if the intent is to maintain the accuracy of changes in both space and time; alternatively, the intent can be to maintain the accuracy of only one of the two. For instance, although a cell develops over time, scoping may be implemented such that a user adjusts its development in space without regard for the accuracy of changes through time. In Polyvise, continual increase or decrease in spatial development and growth (i.e., scope) not only allows users to locate an object’s starting point of growth, but also to discover the process and sequence of its growth and construction—e.g., how internal patterns are formed, or how elements connect to one another. Focus+scoping brings together the ideas of focus and context, continuous transition between them, and scoping. In addition, rather than just one starting point, in Polyvise, focus+scoping enables users to select different starting focal points, with different growth trajectories. Figure 5 shows three examples of a user selecting different focal points of an object. The shape parameter map in the Context Panel diagrammatically represents contextual neighborhoods of polytopes, with a number of landmark points encoding different positions of what is called the kaleidoscopic vertex. The three visualized objects in Figure 5 are the same object viewed from different focal points, which have been selected by the user in the diagrammatic map.

Once a focal point has been selected, users can increase the growth and development of the object from that particular point in a controlled, dynamic, and fluid manner—a process that can help users link the focal point with the overall context. Similarly, when users incrementally add elements around a focal point, they can explore different aspects of localized sub-structures and patterns around this focal point. Figure 6 shows a sequence of images produced by focus+scoping from the focal point shown in the rightmost example in Figure 5. In this example, the user is gradually working forwards from the selected focal point of the object, in order to view its compositional growth and to reason about how the constituent components are interrelated.

5.1.3. Stacking–Unstacking

Polyvise uses a third interaction technique called stacking–unstacking, which is intended to allow users to adjust the perspective from which the objects are viewed. These objects can exhibit varying degrees of visual complexity, according to their viewing position and orientation. Stacking–unstacking enables users to move back and forth between views of diverse complexity. Stacking refers to aligning subsets of internal elements, aimed at reducing the number of elements being displayed on the visualization. Stacking of elements provides simplified views, helping users in abstracting key constituent elements of the object. Conversely, unstacking refers to separating the aligned elements and revealing the fuller complexity of the explored object. Both stacking and unstacking are variants of the transforming action pattern, which is known to have various cognitive benefits [14]. By blending two complementary techniques, stacking–unstacking enables users to compare simplified (stacked) views with richer, more complex (unstacked) views. This interaction technique can support users’ comparative reasoning and analysis, as it facilitates the comparison of contrasting views of the visualized object.

Polyvise implements stacking by providing a number of simplified views which can be accessed through the View Panel (Figure 3, Panel 4). For any visualized object, Polyvise dynamically generates six stacked views. Selecting one of these stacked views would match the orientation of the visualized object to that of the selected view. Similarly, Polyvise implements unstacking by enabling users to directly manipulate the visualized object and to adjust its orientation continuously so as to reveal the object’s elements hidden by stacking. The continuous interactive adjustment gives users precise control over the visual complexity of the visualized object. Figure 7 shows three examples (L to R) of objects that have been stacked (top) and unstacked (bottom). After performing a stacking action (top row), the objects can be viewed in a simplified manner where much of the complexity is hidden. From the stacked views, the objects can be unstacked through manipulation and re-orientation (bottom row). Users can quickly return to the stacked views at any time. The stacking–unstacking technique allows users to dynamically control the degree of visual complexity of these visualizations, thus enhancing their exploration.

5.2. Interaction Design Strategies

In this section, we describe how the two design strategies related to visibility and complementarity inform the above techniques.

5.2.1. Visibility

In the context of Polyvise, we are mainly concerned with the first and third meanings of visibility described in Section 2.2—making action possibilities visible and making content or data visible in the interface. The novelty of our extended notion of visibility actually lies in the combination of these two meanings of visibility. A common assumption in interface design is that simply making the existence of action possibilities visible is sufficient—e.g., by providing a menu list of possible actions. This may be true in contexts where action possibilities do not require significant mental effort to understand—e.g., simple actions in a word processor or web browser. In more complex contexts, however, even when possibilities are made visible it can be difficult to know and remember the intended targets and effects of the actions. For example, 4D objects are composed of many shapes in different planes. Although the possibility of filtering a 3D octagonal prism may be clearly visible in the interface via a menu label, users may not be able to mentally visualize the shape to perceive what the targets of the action will be within the main 4D object, and thus may have a difficult time mentally projecting what the effects of the filtering action will be. This lack of knowing and/or remembering the targets and effects of actions may lead to users performing actions randomly, in a trial-and-error fashion, thus likely not developing robust mental models of the shapes and their constituent components and interrelationships. Figure 8a shows one way of making the action possibilities visible using a typical menu list. Figure 8b shows how the same action possibilities are made visible in Polyvise. In both cases, action possibilities are made visible in the interface. However, in Polyvise, the action possibilities are made visible by also making the content visible—i.e., the constituent components of the shapes and the common viewing perspectives are visually represented. We propose that this combination of the two meanings of visibility leads to enhanced benefit for sensemaking of complex objects and spaces.

In addition to the visualizations of the object’s components (i.e., 3D cells and 2D faces), the quantity of each component is displayed. Together, these two features may assist users in identifying the types and number of the constituent elements of the objects. For example, in Figure 8, it is relatively easy to determine what 2D faces and 3D cells are forming the 4D object by examining the information provided in the Detail Panel. In this case, the object is made up of squares, hexagons, and octagons, with the object having more squares than hexagons and octagons (see “face” column of the Detail Panel). Similarly, the object is composed of four different types of 3D cells, having more hexagonal prisms than other cells (see “cell” column of the Detail Panel).

The visibility strategy we employed in Polyvise can help users perform fairly simple tasks without necessarily having to perform any actions. For instance, users may be able to perceive that all 3D cells have squares in them (see Figure 8). In addition, although there are two types of hexagons (green and yellow), it is clear that the green hexagons are part of the hexagonal prisms and truncated octahedra, while the yellow hexagons belong to the truncated cubocathedra and the truncated octahedra. When users come to this realization, they may then be able to infer that, in forming the 4D object, a hexagonal prism joins a truncated octahedron through green hexagons, as both 3D cells have the same colored 2D shape.

An additional aspect of visibility in Polyvise is that each of the 3D cell filters is interactive, allowing users to rotate and view the cells from different angles directly in the Detail Panel. This features makes more of the content visible in the interface, which is especially important for 3D objects, since they may have occluded elements when rendered in 2D space. Users can interact with the cells without affecting other elements in the interface, including the main object in the Display Panel. The design strategy for the filter controls is intended to facilitate users’ understanding of how the components are connected. At any time (i.e., with or without applying any filters), users can interact with the 3D cells to reason about them and make sense of their place within the main object. Figure 9 shows four snapshots of a user interacting with the octagonal prism in the Detail Panel from Figure 8.

The View Panel also makes certain content visible in an attempt to help decrease users’ reliance on trial-and-error interaction strategies. Each representation in the View Panel is a snapshot of the 4D object if a stacked view were to be applied to it. With complex objects there are usually some important standard viewing perspectives—e.g., top, bottom, left, right, and so on. Providing interactive mechanisms for speedy access to these views may be important in supporting users’ exploration, as these views can serve as reference points. This snapshot gives users some anticipatory information, enabling users to offload some memory load onto the tool since they can rely on perceptual recognition over the need for recall.

It is worth noting that two popular concepts from the HCI and visualization literature share some similarity with the strategies employed in Polyvise: feedforward and multiple coordinated views. Feedforward refers to a design strategy in which the purpose of an action is communicated to users before the action takes place [48]. Feedforward is used to reveal functional affordances through well-defined sensory affordances (e.g., a label or the shape of an object) [49]. Although the strategy in Polyvise can be viewed as similar to feedforward, it is more particular in that both the targets and the effects of actions are actually visualized (i.e., represented analogically) rather than only described (i.e., represented symbolically). The design intention goes beyond simply revealing functional affordances. Multiple view interfaces use two or more distinct views to support the investigation of a single conceptual entity [50]. These views may be coordinated in various ways [51]. The stacked views in Polyvise are similar to this idea, although they provide dynamic snapshots of common views rather than providing distinct, alternate views of the 4D object under investigation. In this way, they are more similar to buttons than they are to separate views.

5.2.2. Complementary Interactions

When deciding whether to include an interaction technique during the design of Polyvise, two criteria were used: the technique must add value to users’ exploration, and it must complement and work in concert with the other chosen techniques. When designing the interaction techniques that would meet these two criteria, a minimalist approach was followed. This approach keeps the number of interaction techniques low, but still makes them simple to use and sufficiently supportive of users’ sense-making activities. The three main interaction techniques of filtering, focus+scoping, and stacking–unstacking, as described earlier, attempt to make exploration more feasible by providing different methods to reduce the visual complexity of the explored space. These three techniques are also intended to work together. It is by bringing these techniques together that perhaps their benefits are even greater as they allow users to act on the visualizations in concerted ways, helping to overcome the limitations that may emerge from these techniques individually. It is important to note that effective interaction design relies on incorporating many more criteria than only the two above. For instance, there may be multiple different combinations of effective interactions, and the order in which interactions are made available to users can influence their task performance. We do not suggest that these two criteria are the only ones of importance; rather, we suggest that they are valuable and should be used along with other relevant criteria within a broader interaction design strategy.

To think about the complementary nature of the interactions, consider the following scenario, in which a user is trying to determine how the first two types of 3D cells (i.e., rhombicubotahedra and triangular prisms) are connected together in the visualized object shown in Figure 10a. In this case, the user starts by selecting a standard view to obtain a simplified, stacked display of the object. From Figure 10b, it appears that the boundary and center shapes are rhombicubotahedra, as indicated by the light-blue colored triangle elements. Next, the user decides to filter out all other 3D cells, except the ones of interest. Figure 10c shows the visualized object with highlighted rhombicubotahedra. Figure 10d, on the other hand, displays the object with only highlighted triangular prisms. Up to this point, it seems that rhombicubotahedra are connected to triangular prisms through shared triangle faces. This conjecture can be confirmed after applying unstacking coupled with focus+scoping interactions to further reduce the visual complexity, until the user is certain of his/her observations (see Figure 10e).

Besides allowing for concerted tasks on the visualized 4D objects, the interface components in Polyvise interact with each other—that is, they are dynamically linked. These components update continuously in a coordinated manner as actions are carried out on the objects. For instance, following the previous example, the effects of reducing the scope of an object are also reflected in the representations of the controls for accessing the standard views (see Figure 11). Similarly, the filtering controls are synchronized to assist users better. When 3D cells are filtered out, the filter controls for 2D polygons belonging to the filtered-out cells are also deselected (see Figure 11). As Figure 11a shows (see highlighted components), only the controls for the 2D polygons belonging to the first 3D cell type (i.e., rhombicuoctahedron), which is the only type of 3D cell selected, are on. In addition, although the controls for the other unselected 3D cells are off, the faces, which the unselected cells share with the selected cells, are also displayed on the controls with colored faces. This feature may allow users to get a better idea of how the cells are associated with each other within the visualized 4D object, as only those shared faces are displayed. Figure 11b shows another instance of controls being orchestrated in response to users’ actions.

6. Usability Evaluation

This section describes the empirical usability evaluation that was conducted to evaluate the interface of Polyvise. The evaluation was intended to investigate whether and how the design strategies of visibility and complementarity of interactions were useful in making Polyvise’s interaction techniques supportive of users’ exploration of complex 4D objects.

6.1. Design and Participants

Usability testing is often conducted in visualization contexts, and involves observing users carrying out pre-defined tasks that are concerned with important features of the tool [52]. This study used a mix-method (quantitative and qualitative) research design, which can help triangulate and cross-validate the different types of findings [53]. Four types of data-collection instruments were used: (1) answers to exploratory activities; (2) video and interview transcripts; (3) answers to the questionnaire; and (4) direct observations. Twenty university students from STEM (Science, Technology, Engineering, and Mathematics) disciplines participated in the study. They were recruited through an e-mail invitation. The first ones who responded were selected to participate in the study. None of the participants had used the tool before.

6.2. Procedure

Participants were randomly paired, forming eleven (11) pairs. Co-discovery learning was employed as the primary usability method (see [54]). In this method, two participants learn how to use an interface while conversing with each other. This conversation helps evaluators to gain a better understanding of the usability of the interface.

Each pair was given a tour of Polyvise, where the researcher described its visuo-interactive features. This tour lasted about 12 min. Then, the pair was allowed 15 min to familiarize themselves with the application. Afterwards, each pair was requested to complete a set of nine exploratory tasks using Polyvise (see Appendix A for the list of tasks). While participants interacted with the tool, they were videotaped and the researchers wrote notes about their overall usage patterns and verbal comments. Additionally, a screen-capturing program was used to record participants’ interactions with the application and their voices. Finally, at the end of the session, participants completed a questionnaire that collected feedback about their impressions of the tool. Some participants were also interviewed to clarify some of their comments made during interaction with the tool and their answers given in the questionnaire.

6.3. Tasks

As described in Section 3.1, tasks are goal-oriented behaviors in which users engage during the performance of an activity. Users typically carry out a number of tasks during the performance of a complex cognitive activity, each of which may comprise a number of individual actions. We chose a set of tasks that are commonly performed during sensemaking activities [55]. There were two main reasons for providing goal-oriented tasks to participants, rather than simply letting them explore Polyvise freely without an aim. First, because Polyvise is concerned with non-trivial visualizations, it seems appropriate to give some direction to participants’ exploration so that it can be focused. Visualization tools like Polyvise are created for specific purposes that are known a priori to designers. By observing if and how participants were able to complete anticipated tasks, we could then assess in a more direct way if Polyvise could meet its intended goals. Second, we did not make the tasks prescriptive, but rather made them open to participants’ perception of how they could be accomplished. That is, although participants would have focused aims, they would still be able to engage in discovery-based interactions.

As can be seen in Appendix A, these tasks include: describe, identify, locate, reveal, cluster, rank, distinguish, compare, and correlate. Each task was composed of several smaller sub-tasks. Some tasks in a given activity would require one type of action, whereas others a combination of several actions. Examples of the type of tasks included locating different 3D cells in the visualized object, identifying the connecting 2D elements, describing how 3D cells were connected within 4D objects, and ranking the complexity of different 3D cells. All the tasks could be accomplished in multiple ways. The instructions did not specify which components of Polyvise participants should use.

7. Results

Analysis of the data was based on a “thematic” approach (see [56,57]) and followed these five adapted steps: (1) data familiarization; (2) code generation; (3) theme search; (4) theme revision; and (5) theme refinement. In terms of code generation (in Step 2), we followed a two-coding phase adapted from Saldaña’s two-coding cycle framework [58]. In the first phase, we focused on Magnitude and Structural coding, while during the second phase we focused on Pattern coding. Magnitude coding was applied to the quantitative data. This process resulted in

Table 1. Participants’ assessment of the overall usability of Polyvise.

Question	Mean Score
All in all, Polyvise is useful in helping me develop an understanding of 4D geometric objects. (5) strongly agree, (4) agree, (3) undecided, (2) disagree, (1) strongly disagree.	4.5
Compared to what you knew about 3D and 4D mathematical structures before using Polyvise, how much have you learned about these 3D and 4D structures now that you have used Polyvise? (5) I have learned all that there is to know, (4) I have learned quite a bit, (3) I have learned some, (2) I have learned very little, (1) I have not learned anything at all.	3.5
Polyvise made the 3D and 4D concepts less challenging (or easier) to explore and learn; (5) strongly agree, (4) agree, (3) undecided, (2) disagree, (1) strongly disagree.	4.4
Mean usability index	4.13

,

Table 2. Participants’ assessment of the overall usefulness of each interaction technique in Polyvise.

Interaction Technique	Rating
Stacking–Unstacking. Rate the usefulness of this feature in helping you explore and understand 4D structures using a scale from 1 to 10 (1 = not useful at all; 10 = extremely useful)	9.25
Focus+Scoping. Rate the usefulness of this feature in helping you explore and understand 4D structures using a scale from 1 to 10 (1 = not useful at all; 10 = extremely useful)	8.75
Filtering. Rate the usefulness of this feature in helping you explore and understand 4D structures using a scale from 1 to 10 (1 = not useful at all; 10 = extremely useful)	8.75
Overall. How would you rate (1–10; 1 = not effective at all; 10 = extremely effective) the overall effectiveness of Polyvise in supporting you in exploring and understand 4D structures	8.58
Mean usefulness index	8.83

and

Table 3. Participants’ selection of interactions for completing some tasks.

Tasks	Single Interaction	S–U a F+S b	S–U Filtering	F+S Filtering	S–U F+S Filtering
Identify	8 *	15	15	0	62
Locate	0	38	0	23	38
Distinguish	15	0	23	0	62
Categorize	8	8	31	0	54
Compare	0	0	31	0	69
Rank	15	15	8	8	54
Generalize	0	8	15	0	77
Emphasize	8	0	8	15	69
Reveal	8	8	23	15	46

* in percentage of participants, rounded to the nearest integer; ^a stacking–unstacking; ^b focus+scoping.

. We used Structural coding due to its suitability for studies dealing with multiple participants and semi-structured data-gathering protocols and exploratory investigations. The process helped us identify clusters of comments that were specific to the themes of our research—that is, the effect of complementarity of interactions and visibility. It also helped us to group comments that were specific to participants’ patterns of interaction within Polyvise (e.g., Filtering, Focus+Scoping, or Stacking–Unstacking). The comments below were the result of doing Structural coding. The end of our analysis involved Pattern coding with the goal of distilling patterns of behavior when relating tasks to interactions used in Polyvise.

7.1. Overall Effectiveness and Usability of Polyvise

In general, all participants were able to successfully carry out the tasks that were assigned to them. They used a combination of the three interaction techniques provided in Polyvise because of their perceived functional complementarity of these techniques. Pattern coding analysis of participants’ engagement with Polyvise, using video recordings and researchers’ observations, indicated a general behavioral pattern among the groups. Initially, each pair would read a task and contemplate it. While thinking about the task, participants would interact with a displayed 4D object through the Display Panel to get an overview and general sense of it. Afterwards, all pairs would interact with the software to reduce the visual complexity of the displayed object to a more manageable visualization. Often, after rotating the object a number of times, participants would also play with the different standard views to obtain simplified stacked views. Then, they would unstack the 4D object by rotating it again in different directions. At other times, participants would rely on the filtering options to reduce unwanted details or to emphasize desirable elements. Similarly, with high frequency, participants would use scoping to explore structural connections and growth patterns.

Participants’ perception of the usability and usefulness of the application seemed quite positive. Table 1 summarizes participants’ responses regarding the overall usability of Polyvise. These responses were distilled from the questionnaire given at the end of the study (see questions in the lower section of the table). They were rated on a 5-point Likert scale, where strongly positive responses were assigned a value of 5, and strongly negative responses were assigned a value of 1. The mean usability index appears to be somewhat high (4.13).

Comments made by participants in post-hoc interviews and in the questionnaire support the data provided in Table 1. Our Structural coding process helped us to collate participants’ comments related to the usefulness of the tool. For example, in commenting about the effectiveness and usefulness of the tool, some participants stated that “[The tool] gave me a lot of understanding”, “In a short time, it increased my understanding a lot”, “It was very useful and I have enjoyed the way to understand 4D geometry”, and “I would say that I have a much better understanding now of not only necessarily the structures, but how they are built and their basic shapes”. Similarly, others commented that “It is a non-trivial task to present 4D geometry and I think Polyvise helped me to visualize better”, “It gives me a good understanding of construction and joining of shapes”, and “Great tool for visualizing, understanding of building cells, complexity, differences between structures”. Finally, one participant said that “These sorts of things [structures] for me without this software are very hard to even imagine!”

Through interaction with tool before and during the tasks, all participants seemed to have developed a good understanding of what each interaction technique did (and did not do). It helped that we used a co-discovery approach, because we found from our observations and video recordings that one participant would often explain to the other how a technique would work if the other participant appeared to show signs of uncertainty. To inquire further about the usefulness of each technique on its own, we asked participants to rate and quantify the degree of usefulness the main interaction techniques had in helping them to explore and understand the 4D structures (see Table 2 below). The perceived usefulness was quite high (almost 9 out of 10).

The above two tables to a great extent cross-validated each other at the levels of the overall tool and its individual components. Overall, Stacking–Unstacking was rated as the most useful. Participants were observing using it often as a “point-of-reference” as one participant put it. It was especially useful when, after rotating the 4D object in the Display Panel, participants could go back to some known position (that is, Stacking) and (re)start the exploration all over again (that is, Unstacking). In a way, this suggests that for complex visualizations, the ability to access specific points of references could be helpful to ameliorate the cognitive load caused by the transient nature of interactive visualizations. It was also observed in the screen capturing videos that participants used Stacking–Unstacking often in combination with the other two techniques; however, the combination of Focus+Scoping and Filtering was used less frequently in comparison—this will be further elaborated in the next section reporting our findings on the effect of the complementarity of interactions.

7.2. Effect of Visibility

Our extended visibility strategy had a positive effect on participants’ exploration of the 4D objects. A substantial amount of participants’ interaction with Polyvise while performing the tasks was aimed at observing and comparing the visible elements of these two panels, especially after they performed an action. Participants would often rely on the visible information to formulate hypotheses and conjectures, and later test them. That is, to an extent, our extended visibility strategy provided multifaceted scaffolds: (1) they allowed for easier initial visual assimilation of the simpler elements forming the more complex 4D objects; (2) they rendered visual comparison of these simpler elements more plausible; and (3) they acted as their cognitive and transitional aids when going from one interaction to the next. These scaffolds in turn facilitated a greater sense of confidence in participants’ approach to making sense of the intricate 4D objects.

Participants were often observed making comparisons using the View Panel’s six snapshots of the standard displays. Comparisons occurred across the six displays and also between the displays and the displayed 4D object. Without the six displays, which make visible certain features of the objects, comparisons, often leading to interesting discoveries, would have been extremely difficult given the complexity of the visualized 4D objects. For example, in one occasion, a pair of participants were solving a task, and while applying focus+scoping to a 4D object, they kept looking at the six standard displays and how the smaller components grew. By comparing the different views, they were quick to identify the 2D objects and 3D patterns that were added to the 4D object, as each view provided a different perspective to observe and make comparative analyses. The ability to see one displayed object from different perspectives at once seemed to have been beneficial, as one participant put it: “the [View] Panel is very useful to synthesize the whole complex structure.” Similarly, other participants referring to the benefits of the panel commented: “It helps to understand different projections on a plane”; “It helps obtain the desired view immediately”; and “It gives me a decent view of all sides of the shapes”. Participants were asked whether the items in the View Panel would be of use if they did not have the snapshots present but contained only the labels—i.e., if the design had followed the common strategy of simply making action possibilities visible as text labels. In response, participants suggested that without the snapshot images, it would have been difficult to know what the items could do, thereby minimizing their value. More importantly, some participants commented that it was the continuous update, or the harmonious interplay between these images and other interaction techniques (e.g., scoping), that really increased their benefits. Participants also emphasized that it was the possibility of seeing and visualizing the images as one of their most important aspects.

Participants found the visibility of elemental parts of the shapes in the Detail Panel to be supportive while performing many tasks. Many participants, after having acquired a basic understanding of the entire 4D object through the Display Panel, often relied on the Detail Panel when attempting to probe deeper and develop their mental model further. The visual provision of the elements along with their number helped participants in their exploration, as attested by these comments: “[The Detail Panel] makes it easy to visualize the specified substructures; the use of numbers, color and elemental components helped with the visualization”, “[The panel] was very useful, as you can rotate, observe and count basic shapes”, and “[The panel] helped by showing different 3D shape elements and their numbers in the 4D figures”. Two characteristics of the components in this panel were salient for participants: (1) the use of color scheme; and (2) the 3D cells being interactive—i.e., employing our extended notion of visibility.

The consistent color scheme was used to link related elements across interface components, but, more specifically, to indicate how these elements would fit into the displayed 4D structure. Participants found the use of color to be important, as indicated by these comments: “Color coding is one of the most important features”; “Color can definitely help to establish the understanding of connectivity”; “The coloring shows the matching. It eliminates any difficulty to locate and identify basic elements”; and “It showed direct interactions between 2 geometric objects. I don’t need to see the entire object, but only need to look for the colors”. Complementary to this color scheme, the provision of interactive 3D cells was appreciated by participants, as they made comments such as: “The most useful part was the cell part [in the Detail Panel]”; and “3D cells makes it much easier to understand the underlying structures of the overall shape. The coloring small portions, creating unit-structures help me to imagine and build more complex ones!”

A special characteristic of the visibility of the 3D cells was that they were interactive, allowing participants to rotate and explore them from different perspectives. This was a result of our extended notion of visibility—i.e., making both action possibilities and content visible (combining the first and third meanings of visibility in Section 2.2) and, additionally, making the visible content interactive. These strategies seem to have increased the utility of the different interface components. Participants valued this characteristic of the cells with statements such as:

“By rotation, we are able to explore the object and find out its features from different angles. The 3D cells themselves can be complicated. This way, I can first understand the 3D cells, then understand the big picture. Some angles are better than others for understanding the cells.”

“The way we can rotate and see different faces with color makes it easier to understand.”

“For me it was very important aspect of understanding, to be able to rotate.”

“[The cells] were very useful as you can rotate, observe and COUNT basic shapes.”

7.3. Effect of Complementary Interactions

Complementary interactions had a clear effect on the participants’ ability to complete the tasks successfully. In most cases, participants had to use two or more interactions to tackle the tasks. From our Structural coding analysis of screen-captured videos, we noted that participants found these interactions to be complementary. Aside from the videos, we wanted participants to tell us in an explicit and direct way about the interplay among these interactions. To this end, we gave them the list of exploratory tasks used in the experiment and asked them what single interactions—or their combinations—they would use to accomplish them again. The results are displayed in Table 3. For their responses, participants may have believed that the interplay of these interactions was essential for the completion of the tasks, as in all cases they would choose all three interactions (see last column of Table 3).

The data in Table 3 are consistent with the observations made during the completion of the tasks. Observations by researchers and video recordings indicated that participants often used at least two interactions. It can also be noted that in terms of combinations of two interactions, Stacking–Unstacking was seen as a good complement to the other interactions that could be used in many of the tasks (see 3rd and 4th columns of the table). The combination of Focus+Scoping and Filtering was chosen for only four tasks (Locate, Rank, Emphasize, and Reveal; 5th column).

Further support to the data could be derived from participants’ comments collated from our Structural coding analysis. One participant, for instance, stated that “With the aid of view and detail and scope [interaction techniques], complex displays can be divided into several simple problems”. Another participant said that “The connection is possible through the display+coloring [stacking–unstacking] and the detail+cell [filtering]” to assist in exploring the connection of different substructures; while another participant suggested that for him “[Scoping] combined with display [stacking–unstacking], [he] can finally determine the geometry of the 4D structures”. Similarly, one participant said that although stacking–unstacking was useful, “It is still too detailed to explore, so I must to do so with scope”. Participants in reference to the interplay between the Detail and Scoping panels suggested that “[The Detail Panel] shows internal structure from viewing angles. When combined with scope, it shows how things are built from simpler pieces”, and “Because [Detail Panel] can be used in conjunction with the scope panel and show the sequence of construction, they help explore the 4D structures”.

8. Discussion

The results of the evaluation indicate that there are benefits for providing complementary interactions and extending the notion of visibility in interaction and interface design when dealing with complex objects. The importance of complementary interactions rests upon their ability to support different, coordinated operations, offering users the possibility to switch between exploration styles. The appropriateness of complementary interactions should perhaps not be measured in terms of their large number. Van Wijk [33] suggests that interactions should be provided “carefully and sparingly” due to costs associated with having a large number of available interactions. Our study demonstrates that although Polyvise offers users with a relatively small number of interactions, their combination and, more importantly, complementary nature, can support numerous strategies for the exploration of 4D objects. The strategies conjunctively empower users to perform their tasks during a sensemaking activity. One approach that can be adopted to design effective complementary interactions is minimalism. A minimalist approach attempts to keep the number of interactions low, while making certain that they integrate well and that this integration offers the sufficient means to support users’ tasks. The greater number of interactions provided in many applications may end up not being used much or at all. If interactions do not contribute in any significant manner to the exploration process, instead of being beneficial, they may actually lead to unwanted consequences—sometimes referred to as “interaction costs” [59]. Some of these consequences include: spending time finding an appropriate interaction, figuring out what combinations of interactions to use, and spending time to try them out [60]. This is also in line with research in the personalization of user interfaces for productivity applications, where task performance was found to be faster for a minimalist interface than for an interface with a fuller set of choices [61]. However, although a low number of interactions can make the interface simpler, it is important to ensure that enough interactions are made available for users to carry out their tasks effectively. It is perhaps impossible to provide prescriptive guidelines on this matter, except to say that designers should be aware of the trade-offs that come with too many or too few interactions. Thus, we are not proposing that a minimalist approach is the only good design strategy; rather, we are simply suggesting that an additive minimalist approach could be an efficient and effective process to design and develop visualization tools dealing with complex objects. Furthermore, we are not suggesting that additional interactions should not be added as a tool evolves and/or the context of use changes. If designers find that certain needs are not met, they can include other interactions to avoid the risk of the tool not being relevant or useful. Interactive features could and should be added when there is a need—this is part of a flexible design approach.

As the data from the study demonstrate, designing the interface such that deep, layered content is made visible and interactive—i.e., using our visibility related design strategy—can offer users additional cognitive support. This support can lessen users’ cognitive load during their exploration and, at the same time, can provide the means for users to conduct more planned and strategic explorations. As stated earlier, some participants mentioned that without the possibility of seeing the visualized content in the six standard views, it would have made them difficult to use. Some even suggested that they would likely not make use of them at all if the visuals were not there. In the same way, the findings suggest that it was the visuals (i.e., our visibility design strategy) on the interface controls in the Detail Panel that made these controls more useful. This was especially true for the cells, as they were interactive (in addition to the main Detail panel). Participants said that without them being interactive, it would have been difficult to make sense of the three-dimensionality of the cells and their structural features properly. As the interface controls in both the View and Detail panels prove, design strategies related to visibility may not be limited to static visualizations, but can actually be employed in interactive contexts to make visible the effects of users’ actions.

8.1. Design Guidelines

Based on the design of Polyvise, and on the results from our usability evaluation, we propose the following five design guidelines. Each design guideline (DG) is briefly described below:

• DG1: Make key sub-components of objects and information spaces visible in the interface. Visualize the sub-components themselves, rather than simply making their existence and/or function visible via textual labels. Additionally, make them interactive and dynamically linked if possible. This was the key to our visibility strategy, which we found to be effective in supporting users’ sensemaking activities.
• DG2: Provide frames of reference that users can access quickly to restart their exploration. The stacked views, along with the stacking–unstacking interaction techniques, supported rapid and easy access to the references for users to restart their exploration and were found to be supportive of users’ tasks.
• DG3: Use varied levels of detail to support continuous back-and-forth comparative visual reasoning. Polyvise allows for decomposing complex 4D objects into their smaller parts that users can manipulate and interact with (e.g., through scoping or filtering). Participants found these features helpful in understanding how elements can come together.
• DG4: Provide different reference points from which the complexity of objects can be adjusted—e.g., with focus+scoping techniques. We found this to be supportive of reasoning through the complexity of the 4D objects and making sense of their composition and structure.
• DG5: Integrate multiple, mutually-supportive interactions to enable fluid and complementary activities. Multiplicity of interactions is essential for exploring complex visualizations. However, they should be chosen carefully such that they are also complementary.

Similar to other interaction design guidelines in visualization contexts (e.g., [62]), this list is not exhaustive, and is intended to serve as a starting point for other researchers to build upon. Our long term goal is to have a set of guidelines that can form a framework to guide the design of visualizations of complex objects and information spaces.

9. Summary and Conclusions

It is well known that complex objects are very difficult to understand without the support of external representations (e.g., visualizations). Research has shown that making such visualizations interactive can help support users’ cognitive and perceptual processes while reasoning about the objects. There are still many challenges, however, when it comes to effectively designing such interactive visualizations. In this paper we investigated the role of two interaction design strategies in supporting sensemaking of complex objects. These two strategies are related to visibility and complementarity of interactions. We situated our investigation within a particular context—namely, sensemaking of four-dimensional mathematical shapes. A usability study was conducted in which participants completed a set of diverse tasks that are typical in sensemaking activities. These tasks were performed through interaction with visualizations of 4D mathematical structures. Results of the study suggest that two design strategies—(1) extending the common notion of visibility; and (2) providing complementary interactions—can provide enhanced support for sensemaking activities.

Although the study was confined to 4D geometric objects, we anticipate that the results can be useful in informing the design of visualizations and interfaces for other types of complex information spaces. From the results, it can be concluded that the notion of visibility can be extended in ways that facilitate exploration of complex objects. The strategies we employed may be beneficial in supporting sensemaking of other information spaces with similar characteristics—e.g., multidimensionality, intricateness, density, and embeddedness of the information. The results of the study also suggest that it might be necessary to provide complementary interactions which, individually and in concert, enable users to perform sophisticated yet coordinated exploratory activities. An important observation from this study is that effectiveness of complementary interactions may not necessarily be measured in terms of having a large number of interactions; rather, it is perhaps more beneficial to follow a minimalist approach, where users are provided with only a small collection of interactions, the combination of which allows performing diverse actions on and with the information. Even though the interactive visualizations in this research are algorithm-driven, the design lessons are applicable in the context of data-driven visualizations as well. One aspect of data modeling can involve mapping datasets onto and visualizing them as geometric structures. Consequently, studying how to design interactive visualizations that support making sense of complex four-dimensional geometry and lessons learned can transfer to data-driven visualizations.