Making Sense of Complexity : Using SenseMaker as a Research Tool

There is growing interest in studying processes of human sensemaking, as this strongly influences human and organizational behavior as well as complex system dynamics due to the diverse lenses people use to interpret and act in the world. The Cognitive Edge SenseMaker® tool is one method for capturing and making sense of people’s attitudes, perceptions, and experiences. It is used for monitoring and evaluation; mapping ideas, mind-sets, and attitudes; and detecting trends and weak signals. However, academic literature describing the tool-set and method is lacking. This introduction aims to guide researchers in choosing when to use SenseMaker and to facilitate understanding of its execution and limitations. SenseMaker can provide nuanced insight into system-level patterns of human sensemaking that can provide insight to nudge systems towards more desirable futures, and enable researchers to measure beyond what they know.


Introduction
Social systems continuously engage in sensemaking processes that shape behavior and emergent structure [1].The process starts when people sense cues from the environment, make sense of it, then enact their interpretation to express their identity and shape their world [2].Sensemaking is a cognitive process that allows us to structure the unknown, to understand and explain the world, and to inform action [1,[3][4][5][6][7].Through sensemaking processes, information is interpreted and meaning assigned so as to inform behavior on both the individual and collective scale [1,8,9].In addition, all sensemaking processes are informed by culture, prevailing narratives, knowledge systems, and experiences [1,3].How people act, in turn, shapes their social realities and influences future sensemaking in an ongoing cyclical process [4].
People make sense of the world and give meaning to life through the construction of narratives [10][11][12][13].How people make sense of the world is reflected in their everyday micro-narratives, the anecdotes or "small stories" people tell in social interactions.Narratives are, therefore, particularly useful for exploring social patterns of cognition [14].Social knowledge extracted from daily rhetoric can point to what informs decisions, actions, interests, and principles and, thus, may be useful for discovering what is considered public truth and preferable behavior [10].Micro-narratives contribute more to participative modes of sensemaking than "big stories" do [15].Furthermore, these fragments collectively disclose identities, motivations, and attitudes [12,16].Sensemaking, narrative, and culture are, therefore, interwoven and give feedback to one another in complex ways.
The patterns that emerge in the narratives, heuristics, and memes of individuals, groups, or organizations are avenues for systemic meaning-making that enable researchers and decision-makers to explore the complex dynamics of social systems [17][18][19].Reductionist approaches to analyze social complexity are limited in the explanations they can offer [20][21][22][23], as they ignore key dynamics and features of complex adaptive systems [24].
The purpose of this article is to introduce the SenseMaker tool, which is based on the Cynefin sensemaking framework.SenseMaker®is Software as a Service, available through Cognitive Edge, of whom David John Snowden is the founder and chief scientific officer [37].Originally developed as a narrative-based mixed method tool to inform response options in organizations [38], SenseMaker is now increasingly being used in trans-and interdisciplinary academic domains [39][40][41][42][43][44].While much has been written about different management applications, mainly in grey literature, the use of SenseMaker as a research approach is not well described.This article aims to help researchers understand SenseMaker and evaluate whether it is a suitable tool for their research.To this end, the article provides an overview of the four steps of designing and implementing a SenseMaker study and reflects on its limitations for research.

The Cynefin Framework and SenseMaker Tool
The Cynefin 1 sensemaking framework is recognized as one of the first practical applications of complexity thinking in the field of management science [45].The Cynefin framework emerged at the turn of the century as a guide to distinguish between different types of problem contexts (obvious, complicated, complex, chaotic) based on degrees of predictable order [14].Once decision-makers can situate their context in one of the domains, the framework guides suitable responses for each decision context by clarifying the kinds of challenges and types of practice associated with each [46].While the obvious and complicated domains are characterized by known and knowable causality, the complex and chaotic domains are characterized by uncertainty and non-linearity, and in the central domain of disorder, the appropriate response is still unknown [47].As situations and circumstances change, the problem context can dynamically move between domains.
Within the complex domain, Snowden and Boone suggest interacting with the system through what they call "probing" [46]-small safe-to-fail experiments.As a result of this probing, managers sense patterns that emerge from the system and respond in ways that either encourage or discourage undesirable system behavior-a response described as pattern management [48].Each response initiates the next probe, in an ongoing adaptive cycle.
SenseMaker originated in the field of knowledge management and was specifically developed to explore emergent narrative patterns in the complex domain of Cynefin, although its use is not confined to this domain [49].SenseMaker entails probing to solicit micro-narratives from the system context, then looking for emergent patterns from the narratives, and responding with pattern management to explore what Juarrero refers to as the "adjacent possible" [50].
SenseMaker is both a tool and a method for collective inquiry into people's attitudes, perceptions, experiences, and reflections.Conceptually SenseMaker is grounded in insights gleaned from complex adaptive systems and the cognitive sciences applied to the analysis of social systems [51][52][53].It links micro-narratives with human sensemaking to create advanced decision support, research, and monitoring capability [54].The narrative-based method and accompanying software support the process of sensemaking of patterns in complex social systems and allows one to assess and monitor their continuously evolving nature [55,56].
SenseMaker differs significantly from other narrative-based research methods [38] that typically involve immersive field-based observations and multiple in-depth interviews.The latter can result in an intermingling of the views, voices, and values of the researcher and the research participants [57,58].The SenseMaker method replaces immersive interviews by the researcher, as micro-narratives are sourced from people's lived experiences by facilitators from the community.However, narratives may also be obtained directly using collection tools available from a webpage or as an application (app) on a smart device.SenseMaker is a form of distributed ethnography, as it transfers the onus of interpretation of narratives from the researcher to participants.Through this self-signification, SenseMaker removes ethnographic coding and expert re-interpretation, as participants assign meaning to their own micro-narratives, which enables large scale explorations, reduces researcher bias, and allows for more objective analysis [59,60].
SenseMaker is a mixed method that combines first-hand narratives with the statistical authority of quantitative data [61,62].Research questions are built into the SenseMaker instrument as signifiers (concepts that anchor the plotting of micro-narratives in space).A signification framework consists of predetermined questions that guide the inquiry, solicit micro-narratives, and clarify interpretations which extract associated meanings of the narratives from participants [56,62].Signification takes place when participants answer signifier questions on the micro-narrative they shared, and in the background numerical coordinates are associated with the micro-narrative relative to the signification framework.This innovation provides linked qualitative and quantitative data that can be assessed in parallel.When the set of signifiers is plotted in SenseMaker, it displays mathematical patterns that reveal a map of the social landscape, to detect and distil variations in patterns of cognition [55,56,60].Thus, the signification framework becomes the basis for exploring the relevance of patterns, underlying relationships, norms, and dynamics in a social system [56].

Applications
SenseMaker supports participative processes of collective inquiry and shared sensemaking and has been used for: monitoring and evaluation; impact assessment; and the facilitation of complex development and social intervention planning across various disciplines and sectors [59,61,63].Although many applications in the commercial and intelligence areas are not documented, known SenseMaker cases can be grouped into three areas: (i) monitoring, evaluation, and measuring impact; (ii) mapping of ideation cultures, mind-sets, and attitudes; and (iii) detecting trends (See Table 1 below).References to these application objectives have been distinguished based on the formality of the literature available at the time.The second application objective, mapping of ideation cultures, mind-sets, and attitudes, has been most popular to date, both in terms of the overall number of references and formal publications.The approach is deemed particularly well suited to resilience research, as the ability to detect a change in mind-sets or pick up on emerging trends can feedback into adaptive management practices and monitoring and evaluation of programme effectiveness can inform resilience-building interventions within communities.Its contribution to resilience assessments is worth highlighting: airline flight operations are assessed against resilience engineering principles [71,78]; communities targeted for economic development and upliftment are assessed based on social pathways to cope, adapt, and transform towards prosperity [80,82]; the primary researcher performs resilience assessments in an essential service organization for the effectiveness of sensemaking in extreme events response and the overall utilization of types of social resilience resources.SenseMaker provides evidence and supports assessments, allows researchers to measure beyond what they know, and may reveal views not expected and "rehumanize" data by bringing in participants' voices [55,66].Anomalies and outliers can serve as weak signal detection that might represent emerging opportunities or obstacles [60].

Guidelines for Using SenseMaker as a Research Approach
The four high-level steps in the SenseMaker method consist of: (1) initiation of the tool, either through selecting an off-the-shelf scan, or designing a signification framework from first principles; (2) story collection and data capture; (3) explorative pattern analysis and collective sensemaking of the meaning behind the patterns; and (4) responses to act on the insight gained through safe-to-fail intervention strategies (Figure 1).The SenseMaker toolset that supports the method is available as an online Software-as-a-Service suite and consists of various software modules covering design, collection, view, and analysis (Table 2).The approach is deemed particularly well suited to resilience research, as the ability to detect a change in mind-sets or pick up on emerging trends can feedback into adaptive management practices and monitoring and evaluation of programme effectiveness can inform resilience-building interventions within communities.Its contribution to resilience assessments is worth highlighting: airline flight operations are assessed against resilience engineering principles [71,78]; communities targeted for economic development and upliftment are assessed based on social pathways to cope, adapt, and transform towards prosperity [80,82]; the primary researcher performs resilience assessments in an essential service organization for the effectiveness of sensemaking in extreme events response and the overall utilization of types of social resilience resources.SenseMaker provides evidence and supports assessments, allows researchers to measure beyond what they know, and may reveal views not expected and "rehumanize" data by bringing in participants' voices [55,66].Anomalies and outliers can serve as weak signal detection that might represent emerging opportunities or obstacles [60].

Guidelines for Using SenseMaker as a Research Approach
The four high-level steps in the SenseMaker method consist of: (1) initiation of the tool, either through selecting an off-the-shelf scan, or designing a signification framework from first principles; (2) story collection and data capture; (3) explorative pattern analysis and collective sensemaking of the meaning behind the patterns; and (4) responses to act on the insight gained through safe-to-fail intervention strategies (Figure 1).The SenseMaker toolset that supports the method is available as an online Software-as-a-Service suite and consists of various software modules covering design, collection, view, and analysis (Table 2).(2) probe a social context for narratives using distributed ethnography; (3) make sense of patterns across the narratives; and (4) respond based on the insights and adaptively nudge the system towards more desirable futures.

SenseMaker Analyst
A web-based analysis tool, currently less powerful than Explorer and Modeler, but with a more intuitive user interface and new functionality like x-y-plots.
Web interface to an online analysis tool.

SenseMaker Explorer and SenseMaker Modeler
An application that runs off a computer and access data locally that was extracted from the online-database.An analytical platform for finding statistical correlations, to slice and dice data; build 3D landscapes; compare patterns among sub groups and more.Has largely been replaced by the online SenseMaker Analyst.
Software installed on an Intel based platform, requires a Java plug-in.

Step 1: Initiation through to Instrument Deployment
The questions that drive the inquiry determine the approach to be followed during design, which in turn determines what can be derived from the analysis afterwards.This initial step is critical, as the effectiveness of the SenseMaker approach hinges on the combined effect of instrument design and capturing sufficient numbers of narratives.

Conceptual Design
The conceptual design includes a view of the problem context, potential participants, and the degree of participation that will be followed in the research [46,55].The purpose of the exploration guides identification of ideation constructs to use in the signification framework.Usefulness of results is strongly influenced by the social fit of the constructs, so attention must be paid to aligning concept clarity and structure, described by Podsakoff, MacKenzie & Podsakoff [91], to within-group language and meaning of those concepts, described by James & Jones [92].Off-the-shelf frameworks may be utilized to circumvent design effort [55].If there is a conflict to be resolved, one might focus on making visible the perspectives of the different groups involved.However, in a monitoring application, one would typically compare results with an earlier collection to see whether shifts have occurred.To address participant literacy concerns, one could consider recording oral narratives and using pictures in the signification framework.If joint sensemaking will be used, it is useful to consider following a participative design approach [66].

Detailed Design: Signification Framework
During detailed design, researchers establish the signification framework and prepare the online instrument for data capture.A signification framework is designed to elicit the concepts researchers want to explore and is used as the basis for analysis later.The framework solicits a micro-narrative from participants, followed by questions of clarification, through which participants self-signify their narrative.The clarification questions use widgets that create conceptual space relative to the concepts utilized in a framework.These widgets are novel relational filters like triads, dyads, and stones, which capture nuances in the experiences of participants that traditional surveys cannot convey.Deliberate ambiguity among options in the signification framework invites people to exercise their own judgement, which triggers slow thinking and retrospective sensemaking [1,93].Ambiguity also adds flexibility to the framework to support abductive research or pre-hypothesis explorations [94].
Eliciting micro-narratives: To encourage participants to contribute micro-narratives, the framework starts with an open-ended question that serves to invite the participant to relay an observation or an anecdote (Figure 2).The best stories to work with are told from a person's own perspective [35].The elicitation question should implicitly or explicitly invite positive and negative stories.The researcher should stay clear of prompts that can be answered with platitudes, as these do not stimulate personal reflection or useful signification patterns.Photos can be used to trigger a response; or, there might be two prompts that allow the participant to choose the prompt that resonates most.Although stories are typically collected anonymously to ensure confidentiality, participants can be given a choice whether their story can be shared or not.Once the participants have shared their story, they are asked to provide a title that triggers another layer of meaning-making and is used as a handle for the micro-narrative and associated metadata during analysis.relative to the concepts utilized in a framework.These widgets are novel relational filters like triads, dyads, and stones, which capture nuances in the experiences of participants that traditional surveys cannot convey.Deliberate ambiguity among options in the signification framework invites people to exercise their own judgement, which triggers slow thinking and retrospective sensemaking [1,93].Ambiguity also adds flexibility to the framework to support abductive research or pre-hypothesis explorations [94].
Eliciting micro-narratives: To encourage participants to contribute micro-narratives, the framework starts with an open-ended question that serves to invite the participant to relay an observation or an anecdote (Figure 2).The best stories to work with are told from a person's own perspective [35].The elicitation question should implicitly or explicitly invite positive and negative stories.The researcher should stay clear of prompts that can be answered with platitudes, as these do not stimulate personal reflection or useful signification patterns.Photos can be used to trigger a response; or, there might be two prompts that allow the participant to choose the prompt that resonates most.Although stories are typically collected anonymously to ensure confidentiality, participants can be given a choice whether their story can be shared or not.Once the participants have shared their story, they are asked to provide a title that triggers another layer of meaning-making and is used as a handle for the micro-narrative and associated metadata during analysis.Self-signification of narratives: The process of self-signification of the narratives is done by participants through interactive widgets that consist of dyads, triads, stones, and multiple choice Dyads: A dyad widget consists of a slider used to indicate the relative strength of a concept, quality, belief, or outcome along a continuum between two opposite extremes, where the extremes are described as either mutually negative, positive, or neutral [56] (Figure 3).A dyad is suitable to determine the value of a concept with a bipolar structure and based on Aristotle's "golden mean" carry exaggerated opposites at the ends, like ranging from extreme excess to deficiency [41,91,96].Respondents are invited to indicate a location along the slider that best fits their anecdote [65].SenseMaker assigns a percentage to the two opposite variables based on the given location as an indication of the relative strength for each variable.Dyads are used to explore underlying beliefs, test a disguised hypothesis, or to measure the strength of assumed modulators [60].Snowden describes modulators as the underlying factors that interact with each other and the system and may include attitudes, perceptions, and emotional intensity [97].Dyads are also useful to detect movement and direction of change in desirable system behavior to verify the effectiveness of systems-level interventions.Self-signification of narratives: The process of self-signification of the narratives is done by participants through interactive widgets that consist of dyads, triads, stones, and multiple choice.
Dyads: A dyad widget consists of a slider used to indicate the relative strength of a concept, quality, belief, or outcome along a continuum between two opposite extremes, where the extremes are described as either mutually negative, positive, or neutral [56] (Figure 3).A dyad is suitable to determine the value of a concept with a bipolar structure and based on Aristotle's "golden mean" carry exaggerated opposites at the ends, like ranging from extreme excess to deficiency [41,91,96].Respondents are invited to indicate a location along the slider that best fits their anecdote [65].SenseMaker assigns a percentage to the two opposite variables based on the given location as an indication of the relative strength for each variable.Dyads are used to explore underlying beliefs, test a disguised hypothesis, or to measure the strength of assumed modulators [60].Snowden describes modulators as the underlying factors that interact with each other and the system and may include attitudes, perceptions, and emotional intensity [97].Dyads are also useful to detect movement and direction of change in desirable system behavior to verify the effectiveness of systems-level interventions.Triads: A triad widget is a triangle with labelled corners and is used to convey the relative importance of three concepts [56].Triads are useful to probe trade-offs and reveal subtleties and undercurrents in the system.Participants are invited to place a dot in a triangle to indicate the relative strength or influence of the concepts in the corners relative to their anecdote [73,98].The indicated position in the triad will return three numerical results which always sums to 100, representing the relative weight of the named corners (refer to Figure 4 for an example of values for the three variables for a given location).Triads: A triad widget is a triangle with labelled corners and is used to convey the relative importance of three concepts [56].Triads are useful to probe trade-offs and reveal subtleties and undercurrents in the system.Participants are invited to place a dot in a triangle to indicate the relative strength or influence of the concepts in the corners relative to their anecdote [73,98].The indicated position in the triad will return three numerical results which always sums to 100, representing the relative weight of the named corners (refer to Figure 4 for an example of values for the three variables for a given location).Triads: A triad widget is a triangle with labelled corners and is used to convey the relative importance of three concepts [56].Triads are useful to probe trade-offs and reveal subtleties and undercurrents in the system.Participants are invited to place a dot in a triangle to indicate the relative strength or influence of the concepts in the corners relative to their anecdote [73,98].The indicated position in the triad will return three numerical results which always sums to 100, representing the relative weight of the named corners (refer to Figure 4 for an example of values for the three variables for a given location).Stones: A stone widget consists of a canvas, on which named markers can be placed, where meaning is assigned on the canvas either through named axes or pictures.A stone widget is used for a relative comparison of more than three elements along two axes that make visible how different parties perceive the same issue [56].When participants place named stones on the canvas, Cartesian coordinates are recorded as percentages for future analysis.Stones are considered more evaluative than triads but can be challenging for participants.It is a good idea to limit the number of stones to be placed on a canvas to avoid cluttering the user interface.Another version of this filter blends the slider and stone concept by asking participants to place various stones on a canvas with effectively a single dimension, which simplifies the user experience (Figure 5).signifier data, while the percentages reflect the number of responses that fall within the respective circles [85].
Stones: A stone widget consists of a canvas, on which named markers can be placed, where meaning is assigned on the canvas either through named axes or pictures.A stone widget is used for a relative comparison of more than three elements along two axes that make visible how different parties perceive the same issue [56].When participants place named stones on the canvas, Cartesian coordinates are recorded as percentages for future analysis.Stones are considered more evaluative than triads but can be challenging for participants.It is a good idea to limit the number of stones to be placed on a canvas to avoid cluttering the user interface.Another version of this filter blends the slider and stone concept by asking participants to place various stones on a canvas with effectively a single dimension, which simplifies the user experience (Figure 5).The one dimensional stone slider is a variation of the canvas.In this example, a filter is applied to different coping strategies (e.g., evacuated, got shelter, sold assets) to reveal the coping strategies employed by a particular respondent as well as evaluate their relative effectiveness [82].
Multiple choice questions can serve to clarify aspects of the anecdote or capture participant demographics [56].The multiple-choice options should be informed by consideration of the various categorizations that would be useful to filter results during analysis, e.g., age, gender, location, or feelings linked to the anecdote (Figure 6).(b) The one dimensional stone slider is a variation of the canvas.In this example, a filter is applied to different coping strategies (e.g., evacuated, got shelter, sold assets) to reveal the coping strategies employed by a particular respondent as well as evaluate their relative effectiveness [82].
Multiple choice questions can serve to clarify aspects of the anecdote or capture participant demographics [56].The multiple-choice options should be informed by consideration of the various categorizations that would be useful to filter results during analysis, e.g., age, gender, location, or feelings linked to the anecdote (Figure 6).signifier data, while the percentages reflect the number of responses that fall within the respective circles [85].
Stones: A stone widget consists of a canvas, on which named markers can be placed, where meaning is assigned on the canvas either through named axes or pictures.A stone widget is used for a relative comparison of more than three elements along two axes that make visible how different parties perceive the same issue [56].When participants place named stones on the canvas, Cartesian coordinates are recorded as percentages for future analysis.Stones are considered more evaluative than triads but can be challenging for participants.It is a good idea to limit the number of stones to be placed on a canvas to avoid cluttering the user interface.Another version of this filter blends the slider and stone concept by asking participants to place various stones on a canvas with effectively a single dimension, which simplifies the user experience (Figure 5).The one dimensional stone slider is a variation of the canvas.In this example, a filter is applied to different coping strategies (e.g., evacuated, got shelter, sold assets) to reveal the coping strategies employed by a particular respondent as well as evaluate their relative effectiveness [82].
Multiple choice questions can serve to clarify aspects of the anecdote or capture participant demographics [56].The multiple-choice options should be informed by consideration of the various categorizations that would be useful to filter results during analysis, e.g., age, gender, location, or feelings linked to the anecdote (Figure 6).

Instrument Usability Tests and Online Deployment
The SenseMaker method and instrument design are particularly sensitive to a cultural fit between the language used and the meaning of the concepts in the signification framework and among the target population.Anthropological considerations during field tests are essential to verify usability by the intended community [99].Social usability should consider contextual differences of ethnography, belief systems, knowledge systems, social formulations, language, and meaning [100].A good social fit will be subjectively pleasing to intended participants, and options in the signification framework will resonate with them [101].
Usability tests with the target community follow an iterative refinement process and are valuable towards meaningful data and useful patterns from the study.Paper versions can be tested initially; but, following deployment of the online collection instrument, thorough usability tests have to be performed with members of the target community before full-on capturing commences.

Step 2: Story Collection: Narrative Collection Strategy and Data Capture
Once the instrument is ready to go live, data capturing can be done on paper, online on the web, or through the SenseMaker Collector app.All paper-based input has to be transcribed into the system.For online capturing, the link to the website can be circulated electronically to request participation.When using the SenseMaker app, Internet access is not required during interviews, as the data is saved locally on the device and synchronized once connectivity is established.
Members of the target community may act as gatekeepers to solicit participation.Interviewers should be comfortable with the data collection tool and trained in the project objectives to improve the quality of the narrative collection.They should also know how to elicit a relevant anecdote as well as how the data is to be used afterwards.In addition, they should understand and be able to explain the signifiers [56,69].Interviewers can type the anecdote shared by participants, take a photo, or make an audio recording.If participants are comfortable with electronic devices and abstract concepts, they can be shown how to complete the first signifier, after which they can continue themselves.
Creative strategies are required to ensure capturing at scale, as it can be challenging to collect sufficient numbers of narratives.A nonparametric sampling framework is recommended to ensure validity and the statistical significance of findings [66].Broad sampling ensures depth of penetration through sufficient responses, especially if differentiated insight is required within and between sub-groups [60,63].SenseMaker is best suited to studies with large samples, and Mager et al. [56] recommend that researchers aim for a minimum of 200 stories for a basic level of confidence.Mass collection methods to operationalize the concept of distributed ethnography (allowing for qualitative data capture at quantitative scales) include the following: kiosk-based capture at social gathering places; participant observations from students sent out as apprentices; situational gathering at special events like games, parties, or festivals; samples of employees requested to collect micro-narratives from people matching specific criteria; and arrangements across an organization that target a focused time for participation [102,103].
Ongoing capture enables continuous monitoring of shifting data patterns to enable adaptive management as interventions are rolled out [104].To achieve ongoing data capture, researchers might consider replacing regular organizational reports with captured narratives or daily journaling.They could also build human sensor networks consisting of customers or employees, which can be activated as the need arises [105].

Step 3: Exploratory Data Analysis and Sensemaking
SenseMaker analysis utilizes abductive reasoning and serves as the primary sensemaking step in the method.Sensemaking commences once sufficient data volumes are reached, or once data collection has been completed.User input from the triads, dyads, and stone widgets provides numerical data that can be statistically analyzed and visualized, while results from multiple choice questions are used to filter and further explore the data.
Based on distinctions between quantitative and qualitative research paradigms by Goertz and Mahoney [106], SenseMaker analysis is suitable to explore associated meaning and relationships between concepts, and not for measurements or inference on causality.
Sensemaking consists of exploratory analysis, which involves pattern seeking by the researcher followed by collective sensemaking including interactive and participative sessions [55].It is done by iterating between patterns, associated signifiers, and narratives to find out what constitutes the pattern and search for the contexts which inform such dispositions [17].
Narratives should always be treated systematically.Consideration should be given to how the narratives are handled, and a choice should be made before the analysis begins, as dipping randomly into the stories can lead to interpreter bias.Where applicable, researchers should note the permissions assigned by participants to their narratives when analyzing data (Figure 2).

Exploratory Analysis
A data preparation step is often overlooked but is crucial to the quality of the analysis.The quality of the data in relation to the purpose of the study should be assessed, issues corrected where possible, and the limitations imposed on the conclusions by the unresolved issues noted.
It is recommended to start the analysis with a broad bottom-up exploration of patterns across the dataset.The signifier framework provides a broad theoretical foundation from which to investigate emergent patterns of how narratives cluster, utilize theories that underlie the design to support and develop a contextual understanding of patterns in the dataset.Researchers should also look for statistical correlations between signifiers, keeping in mind that variables derived from triads have negative correlations owing to a loss of the degrees of freedom [98].A geometric means is suggested to investigate the central tendency of triad data because of the "closure" constraint on variables that add to 100 [98].
The bottom-up exploration can be followed by a top-down process that starts by listing the guiding questions and identifying which combinations of signifiers are relevant to those questions.The exploratory analysis notes links or contradictions between patterns and interesting patterns that emerge should be followed up.
The interactive exploration of patterns and anomalies is supported by SenseMaker's visualization and statistical analysis tools; or, researchers can use R or Tableau if preferred [107].Clusters of stories may be identified from signifier plots and subjected to thematic analysis to assist with the interpretation of the patterns.Disintermediation is achieved by giving decision-makers direct access to the raw narratives for a rich description of what informs the patterns and by allowing them to make their own interpretation of the meaning behind the data, without being constrained by expert interpretation [94,108,109] (Figure 7).
The abductive nature of the exploratory analysis enables researchers to measure beyond what they knew.Furthermore, SenseMaker analysis can reveal surprising results as correlations may emerge among seemingly unrelated signifiers.The variety of patterns that emerge from the analysis is owing to the ability to correlate all the signifiers in the framework with one another, as well as against the themes from the narratives behind these patterns.This, in turn, is as the micro-narratives are associated with all the numerical data derived from the self-signification process.
The next step is then to confirm the conjectures that arise after exploration through further analysis, e.g., by checking whether the responses to other signifiers support the conjecture or by thematic analysis of the narratives associated with the patterns.

Collective Sensemaking
Collective sensemaking involves returning the information to a wider audience to solicit reflections on the emergent patterns and what underlies them.Chilisa [100] argues that culturally sensitive research ethics ensure that research findings are validated by the researched community.To facilitate collective sensemaking, a useful practice is to share the results from the exploratory analysis with participants in a workshop setting.The purpose of collective sensemaking is, through conversation, to seek to uncover and make sense of the patterns of thinking that led to the behavior made visible in the patterns.If participants in the study give permission for their voices to be heard, story packs can be compiled to clarify patterns by extracting illustrative stories to support meaning making.Participants and other stakeholders often enjoy having story packs returned to them and discussing the significance of the findings to deepen insights and inform action [111].The ultimate test of conjectures is to subject them to collective sensemaking or verify them through safe-to-fail experiments.
Collective sensemaking produces a shared map of what is going on and enables coordinated action [111].Some development practitioners consider this participative collective human sensemaking process, with actors in the system, as the core of the methodology [54].It provides change agents in the system with the ability to: detect signals from the results; gain insight into correlations not previously understood; verify the effectiveness of interventions; develop new ways to engage in the system; adaptively improve their strategies [62].These collaborative interpretive processes can lead to in-depth interpretations; grounded perspectives; shared understanding which ensures alignment and buy-in; accelerated innovation; and enlarge the diverse brainpower collectively engaged in change-making [55,112].

Step 4: Response to Affect Change in the System
For Snowden, the purpose of a sensemaking approach is "making sense of the world so that we can act in it" [37].Pattern management aims to "amplify the positive and dampen the negative" to achieve "more stories like these, fewer stories like those", which is the guiding heuristic for safe-to-fail intervention strategies [45,113].Pattern management works to influence feedback loops in complex adaptive systems that constrain and enable the evolution of a system [114].
Continuous monitoring provides the ability to track shifting patterns over time, monitor their rate and trajectory of change, and explore reasons for shifts from the narratives [54].This awareness brings valuable insight and learning opportunities and can inform adaptive management [103].Early detection can lead to quick feedback loops in response to changing system dynamics and enables fast exploitation or fast recovery by enacting small course correcting adjustments to nudge the system towards desirable outcomes [16].Nudging a system in this way to an "adjacent possible" is easier than bringing about large shifts [115,116].

SenseMaker Limitations and Suggestions for Future Development
Researchers should consider limitations to the SenseMaker approach to assess and manage risk implications to their research objectives.One of the main challenges is that the initial investment in becoming familiar with the approach sets a high barrier to entry; but, with experience, the tool becomes easier and more cost effective to use [54].

Research Method
SenseMaker is primarily an exploratory method suitable for abductive research and pre-hypothesis studies, as it explores patterns and relationships to reveal what is not known, and can produce novel surprises [40,42,116].On the other hand, SenseMaker data can validate or challenge a hypothesis, pre-defined assumptions or expectations [55].Furthermore, Mager et al. advise to only embark on a SenseMaker study if key stakeholders are prepared to actively engage in sensemaking, value hearing different voices, and would be receptive to the whole distribution of patterns, including outliers [55].
Contrasting responses have been reported from the application of the SenseMaker method.Respondents often welcomed the opportunity to share their concerns [60].Some participants offered to recruit other participants from outside their immediate circles whilst others observed that the process of completing the survey felt therapeutic and asked for further opportunities to participate.On the other hand, it has also been reported that participants abandoned a survey and decision-makers that commissioned the study abandoned the project [16].
Various ethical issues may arise in the use of SenseMaker.Technology that engages with social dynamics may be "beneficial or harmful, empowering or dangerous, depending on the context in which they are used" (p.118 [117]).The researcher needs to consider whether this form of research makes participants vulnerable by revealing information that might be used against them.Although stories are typically collected anonymously, it may be possible to deduce where some stories are from.Respondents should be given the option to restrict access to their own narrative.It is important to consider who may access the stories and how they plan to use them.It is also important to manage expectations, as participants may assume that telling their stories will lead to tangible responses.A wider ethical consideration, aligned with possible transformation agendas that may inform the research, is whether those researched will be treated with dignity and integrity, or whether insight into the stories opens participants to be manipulated [99,118].
Diverse schools of thought exist on the validity of quantifying social phenomena.While the strength of the SenseMaker method is to transpose personal responses into numerical values, the subjective self-signification process may paradoxically contribute to a perceived limitation in this research approach.Since the SenseMaker approach applies techniques from natural sciences to social sciences, it may be criticized as scientism after Hayek [119].Haack [120] describes scientism as an inflated admiration of natural science, while losing sight of the fallibility of its limitations.Due to criticism that social phenomena cannot be quantified or measured meaningfully, the numerical data derived from SenseMaker can be seen as weak.While numbers lend it to scientific-looking techniques, the application of statistical analysis to weak data may be criticized as scientistic.Similarly, SenseMaker analysis produces impressive apparently quantitative diagrams, but its strengths must not be overestimated.Hayek [119] warns that the application of natural science techniques to social phenomena may lead to a loss of the human factor, which according to Miles [121] may lead to an inhuman and depersonalized science.To counter this impersonal inclination towards a scientistic reduction in social sciences, Miles [121] advocates that researchers adopt a humanistic outlook that views participants and their communities as irreducibly complex, and Haack [120] call for human mindedness.For this reason, Goertz and Mahoney [105] should be heeded when they say quantitative results should not be overanalyzed, as they are indicative at best.Furthermore, any conjectures that arise from SenseMaker analysis should be held lightly, verified collectively, and tested adaptively through safe-to-fail experiments, and remain open to surprise.

Socio-Technical Considerations
Implications of the digital revolution easily crowd the space required for narrative research using electronic media.Miller and Horst [122] warn that an implication of digital approaches is that possibilities of abstraction and quantification have expanded at the expense of social connection and face-to-face interaction.When people are asked to construct a micro-narrative on an electronic device, the effects of the technological acceleration may lead to a loss of depth and meaning.After years of narrative-based sensemaking, Kurtz [123] advises that narratives shared by participants who engage with the narrative collection process and have time to reflect on their narrative increase the utility of patterns revealed: "by orders of magnitude".This awareness highlights the need to intentionally design the data collection process and train the interviewers to maximum effect.It is both tough and vital to create a liberating space where narrative work can take place [42].
The project team needs to consider information security controls and how to implement them.The software does not prevent anyone from reading the stories that participants asked to be confidential.

Figure 1 .
Figure 1.The SenseMaker involves four iterative steps to: (1) design and set up the instrument; (2) probe a social context for narratives using distributed ethnography; (3) make sense of patterns across the narratives; and (4) respond based on the insights and adaptively nudge the system towards more desirable futures.

Figure 1 .
Figure 1.The SenseMaker involves four iterative steps to: (1) design and set up the instrument;(2) probe a social context for narratives using distributed ethnography; (3) make sense of patterns across the narratives; and (4) respond based on the insights and adaptively nudge the system towards more desirable futures.

Figure 2 .
Figure 2.An example of a prompt for a micro-narrative.Prompts should elicit a personal anecdote of an experience that left an emotional impression on participants.This prompt is from the off-the-shelf Culture Scan by Cognitive Edge used to get a sense of organizational culture seen from employees' perspectives [95].

Figure 2 .
Figure 2.An example of a prompt for a micro-narrative.Prompts should elicit a personal anecdote of an experience that left an emotional impression on participants.This prompt is from the off-the-shelf Culture Scan by Cognitive Edge used to get a sense of organizational culture seen from employees' perspectives [95].

Figure 3 .
Figure 3. (a) A SenseMaker dyad is a continuum between two opposite extremes.Participants use the slider to indicate where their narrative is located.(b)The results shown reflect views on this question from 1000 stories collected in Southern Africa for a grassroots needs assessment among communities subjected to consecutive droughts and floods.These results indicate that few participants think answers can come from a combination of both traditional and modern ideas[67].The colors reflect age groups, with over 50s shown in purple.

Figure 4 .
Figure 4.A SenseMaker triad has labelled corners that are all either positive, negative, or neutral.Participants use the context of their micro-narrative to indicate a position in the triangle closer to the labelled corners with a stronger influence.(a) An example showing the numerical values that would be assigned relative to each corner for the indicated location of the marker, which always sum to 100.This data point indicates that quality was compromised in order to finish on time: the position indicated is closer to "Finish on time" with little "Stick to budget" and almost no "Ensure Quality".(b) Results from a Futures study conducted during 2013 among 600 participants in Singapore regarding their expectations of national governance in 10 years' time.This shows the government's clear preference for focusing on constituent well-being.Each dot refers to a narrative and associated

Figure 3 .
Figure 3. (a) A SenseMaker dyad is a continuum between two opposite extremes.Participants use the slider to indicate where their narrative is located.(b)The results shown reflect views on this question from 1000 stories collected in Southern Africa for a grassroots needs assessment among communities subjected to consecutive droughts and floods.These results indicate that few participants think answers can come from a combination of both traditional and modern ideas[67].The colors reflect age groups, with over 50s shown in purple.

Systems 2019, 7 ,Figure 3 .
Figure 3. (a) A SenseMaker dyad is a continuum between two opposite extremes.Participants use the slider to indicate where their narrative is located.(b)The results shown reflect views on this question from 1000 stories collected in Southern Africa for a grassroots needs assessment among communities subjected to consecutive droughts and floods.These results indicate that few participants think answers can come from a combination of both traditional and modern ideas[67].The colors reflect age groups, with over 50s shown in purple.

Figure 4 .
Figure 4.A SenseMaker triad has labelled corners that are all either positive, negative, or neutral.Participants use the context of their micro-narrative to indicate a position in the triangle closer to the labelled corners with a stronger influence.(a) An example showing the numerical values that would be assigned relative to each corner for the indicated location of the marker, which always sum to 100.This data point indicates that quality was compromised in order to finish on time: the position indicated is closer to "Finish on time" with little "Stick to budget" and almost no "Ensure Quality".(b) Results from a Futures study conducted during 2013 among 600 participants in Singapore regarding their expectations of national governance in 10 years' time.This shows the government's clear preference for focusing on constituent well-being.Each dot refers to a narrative and associated

Figure 4 .
Figure 4.A SenseMaker triad has labelled corners that are all either positive, negative, or neutral.Participants use the context of their micro-narrative to indicate a position in the triangle closer to the labelled corners with a stronger influence.(a) An example showing the numerical values that would be assigned relative to each corner for the indicated location of the marker, which always sum to 100.This data point indicates that quality was compromised in order to finish on time: the position indicated is closer to "Finish on time" with little "Stick to budget" and almost no "Ensure Quality".(b) Results from a Futures study conducted during 2013 among 600 participants in Singapore regarding their expectations of national governance in 10 years' time.This shows the government's clear preference for focusing on constituent well-being.Each dot refers to a narrative and associated signifier data, while the percentages reflect the number of responses that fall within the respective circles [85].

Figure 5 .
Figure 5. (a) Stones are placed on a canvas to illustrate relative location, as illustrated in this Inclusive Business Scan performed by Rikolto VECO to gain insight into smallholder farmers' inclusion in formal markets across the world[69].Respondents can place one or more of M, P or T that apply to their story.(b) The one dimensional stone slider is a variation of the canvas.In this example, a filter is applied to different coping strategies (e.g., evacuated, got shelter, sold assets) to reveal the coping strategies employed by a particular respondent as well as evaluate their relative effectiveness[82].

Figure 6 .Figure 5 .
Figure 6.(a) Multiple choice questions are useful for filtering results during analysis, as illustrated in the triad by results from a corporate investigation into the nature of decisions made in crisis.The

Figure 5 .
Figure 5. (a) Stones are placed on a canvas to illustrate relative location, as illustrated in this Inclusive Business Scan performed by Rikolto VECO to gain insight into smallholder farmers' inclusion in formal markets across the world[69].Respondents can place one or more of M, P or T that apply to their story.(b) The one dimensional stone slider is a variation of the canvas.In this example, a filter is applied to different coping strategies (e.g., evacuated, got shelter, sold assets) to reveal the coping strategies employed by a particular respondent as well as evaluate their relative effectiveness[82].

Figure 6 .Figure 6 .
Figure 6.(a) Multiple choice questions are useful for filtering results during analysis, as illustrated in the triad by results from a corporate investigation into the nature of decisions made in crisis.TheFigure 6.(a) Multiple choice questions are useful for filtering results during analysis, as illustrated in the triad by results from a corporate investigation into the nature of decisions made in crisis.The results are color coded according to the emotion people associate with their narrative to highlight the emotional effect different types of decisions have on the workforce (unpublished).(b) Multiple choice questions can also be summarized using bar graphs.This example shows the distribution of narratives relative to a multiple choice question about how common a particular type of occurrence is.This was especially useful for picking up weak signals among the 20 incidents that had never happened before[70].

Table 1 .
Documented SenseMaker applications grouped into three key application objectives.

Table 2 .
The SenseMaker software suite is available from Singapore-based Cognitive Edge to support the execution of the method and consists of six modules to enable design, data collection, exploration, and analysis.