Insights from an Umbrella Review of Digital Storytelling

Abstract

This study synthesised reviews to identify methodological indicators that are instrumental in informing claims regarding digital storytelling (DST) outcomes and effectiveness across fields. An umbrella review methodology was employed, following the PRISMA-ScR reporting framework, with the Population–Concept–Context (PCC) framework guiding research question formulation. Four databases were searched, yielding 1656 records. Following independent screening by two reviewers, 19 reviews met the inclusion criteria. Critical appraisal was conducted using the JBI checklist, and content and thematic synthesis were subsequently performed using MAXQDA 26. Findings related to publication indicators revealed a gradual increase in DST reviews between 2016 and 2025, with qualitative systematic reviews being the most prevalent type. In terms of geographic distribution, most reviews were conducted by researchers based in Türkiye, with notable gaps in South American and African contexts. Analysis on contextual indicators demonstrated that the majority of reviews adopted a configurative review orientation regarding micro-level practices across health-related, general education, and language education fields. Drawing on participants from both formal and non-formal settings, these reviews primarily reported outcomes across eight domains. Regarding methodological indicators, the analysis revealed a lack of uniformity and considerable variation in the use and reporting of quality assessment, theoretical and pedagogical frameworks, technological tools, and measurement instruments. Key recommendations include prioritising meta-analytic and mixed-method reviews, encouraging research in underrepresented regions, examining DST at meso- and macro-levels, and systematically integrating and reporting key methodological indicators in future reviews.

1. Introduction

Technology has fundamentally transformed the ways in which individuals communicate, learn, and exchange information across personal, professional, and educational contexts. Digital storytelling (DST) has emerged as a prominent multimedia narrative approach that integrates digitised images, text, sound, and other digital elements to create short visual narratives, typically three to five minutes in length, focused on specific themes or perspectives (Chan & Sage, 2019; Lambert, 2013; Robin, 2008, 2016).

Initially developed as an arts-based research method for capturing and sharing participant experiences (de Jager et al., 2017; Stenhouse et al., 2013), the use and benefits of DST practices have increasingly expanded across diverse fields. In education, DST fosters transferable skills through multidimensional processes, including topic selection, research, scriptwriting, and multimedia composition (Robin, 2006, 2016). Moreover, it positively influences cognitive and socio-emotional competencies, such as creativity, collaboration, higher-order thinking, and digital literacy (Hung et al., 2012; Niemi et al., 2014; Quah & Ng, 2022; Robin, 2008; Ware, 2006; Yang & Wu, 2012). However, the realisation of these benefits is not automatic; rather, it depends on the quality of instructional design, implementation fidelity, and the contextual conditions under which DST activities are enacted (Smeda et al., 2014; Robin, 2016). In second language (L2) learning, DST activities with varied instructional designs enhance learners’ linguistic repertoire, including vocabulary, grammar, and overall proficiency (Emert, 2014; S. Kim, 2014) as well as learner autonomy, identity development, critical thinking, and willingness to communicate (Hafner & Miller, 2011; Huang, 2022; D. Kim & Li, 2021; Yang & Wu, 2012).

Existing reviews on DST examine multiple disciplinary areas within education, mapping its use as a pedagogical tool and evaluating its effects on learning outcomes across educational levels and subject areas. Systematic and scoping reviews have explored how DST is implemented in formal educational settings. Sarıca and Usluel (2016) provide an early overview of DST in education, establishing a foundational understanding of its pedagogical applications. Wu and Chen (2020) identify five pedagogical orientations and eight types of learning outcomes, including affective, cognitive, linguistic, and social gains, across 57 studies spanning K–12 to higher education. DST facilitates emotional engagement, reflection, and identity construction in education (Gita et al., 2025; Sarıca, 2023). The process of creating digital stories also activates metacognitive, motivational, and emotional regulation strategies that are integral to self-regulated learning. A review mapping research trends identified academic achievement, attitudes, writing skills, and motivation as the most frequently examined variables, with language courses representing the dominant disciplinary context (Bilici & Yılmaz, 2021).

More recent reviews further examine the influence of DST on engagement, academic achievement, and twenty-first-century competencies (Nasir et al., 2024), as well as the design and technological features of DST authoring tools (Quah & Ng, 2022). Bibliometric evidence from Tian and Suki (2023) further indicates that, although research on DST in higher education has been expanding annually, significant regional and disciplinary gaps persist.

In language learning, for instance, positive outcomes are associated with gains in reading and writing skills, though the evidence for listening and speaking remains limited. Additionally, the reliability of measurement instruments is often unreported in the reviewed studies (Lim et al., 2022). Evidence of gains in oral and narrative competencies through storytelling practices in both bilingual and monolingual foreign language contexts suggests that DST affordances foster improvements beyond receptive literacy skills (Pérez Agustín & De la Peña Álvarez, 2025). In addition, Shen et al. (2024) demonstrate that DST can enhance second language willingness to communicate by creating opportunities for language practice, identity negotiation, and community building.

In health disciplines, DST reviews pursue two interrelated purposes: evaluating DST as a clinical or community health intervention for behaviour change and health promotion and examining it as an arts-based research method for capturing lived experiences of health and illness. Regarding the first purpose, Akinosun et al. (2021), by mapping the mechanisms, contexts, and outcomes of DST interventions, identify applications targeting cardiovascular risk factors such as hypertension, diabetes, obesity, and smoking. Along the same lines, Lohr et al. (2022) claim that although DST has been used to address health equity and foster community engagement, long-term outcome data and culturally sensitive evaluation tools remain underdeveloped. With respect to the second purpose, Moreau et al. (2018), by reviewing the use of DST in health professions education, demonstrate that co-creating and viewing patient digital stories enhances learning, empathy, and reflective practice among health professionals, particularly in undergraduate nursing programmes. However, viewing patient stories in isolation yields minimal educational impact, and high-quality research on behavioural outcomes remains limited.

DST has also been applied to specialised populations, including older adults, migrants, refugees, and discipline-specific contexts such as pharmacy education. Research involving older adults, including those with mild cognitive impairment or dementia, shows that DST supports memory, reminiscence, identity, and self-confidence, although the overall level of clinical evidence remains limited (Rios Rincon et al., 2022). Similarly, Botfield et al. (2017) explore DST for promoting sexual health and wellbeing among migrant and refugee youth, revealing benefits for social activism and community development; however, limited funding and a lack of long-term impact evidence constrains its broader adoption in the field. In pharmacy education, Mills et al. (2022) demonstrate that a DST-based exam review enhances student confidence and engagement compared to traditional lecture-based methods, although knowledge gains are not statistically significant across most assessment items.

More broadly, using DST as a research method in health contexts highlights its role as an empowering and transformative approach, particularly in valuing local and cultural knowledge and fostering meaningful change, while also identifying ongoing challenges related to theoretical inconsistencies, diverse analytical approaches, and complex ethical considerations (West et al., 2022). Despite the expanding body of reviews reporting diverse outcomes across disciplines, no meta-review has examined the methodological quality indicators that underpin review rigour and the strength of the DST evidence base. Individual reviews have identified several methodological concerns, including unreported instrument reliability (Lim et al., 2022), low levels of clinical evidence (Rios Rincon et al., 2022), theoretical inconsistency (West et al., 2022), and a lack of long-term effectiveness data (Botfield et al., 2017; Lohr et al., 2022). Taken together, these methodological issues are not isolated to any single discipline or review type; rather, they constitute a common pattern that individual systematic or scoping reviews cannot adequately address on their own. Therefore, a higher-level synthesis that examines review quality indicators across the entire eligible corpus, rather than adding yet another domain-specific review, is necessary to map the state of methodological practice, identify cross-disciplinary gaps, and provide actionable guidance for strengthening future DST evidence syntheses.

Given the methodological concerns and meta-review gap, an umbrella review (UR) of DST reviews is needed to inform practitioners, researchers, and policymakers about the quality and strength of the current evidence base. An umbrella review can examine methodological indicators across studies and provide insights into the robustness of DST evidence. This UR focuses on five key methodological indicators: quality assessment, theoretical frameworks, pedagogical design, technological tools, and measurement instruments. These indicators were selected through a combined deductive–inductive process. An initial set was derived from established methodological literature on evidence synthesis and was subsequently refined through a preliminary examination of the retrieved DST reviews. This examination confirmed that the five dimensions were both consistently identifiable across review types and directly relevant to the strength and credibility of DST outcome claims. Accordingly, a conceptual framework is employed to assess the extent to which reviews address these indicators (see Figure 1). This framework also points to the interrelationships among indicators within individual reviews and across the eligible review corpus (C. S. Collins & Stockton, 2018), enabling a more comprehensive understanding of DST practices (Bingham et al., 2024). Rather than functioning as discrete criteria, these indicators operate as interdependent facets, each informing and constraining the others and collectively underpinning the credibility of reviews and their claims regarding DST outcomes and effectiveness.

2. Conceptual Framework

This umbrella review (UR) adopts a conceptual framework comprising five interrelated methodological indicators to evaluate the quality and rigour of DST reviews. These indicators, including quality assessment, theoretical frameworks, pedagogical design, technological tools, and measurement instruments, do not operate as discrete criteria but function as interdependent dimensions that collectively shape and underpin the credibility of review conclusions. This section elaborates on each indicator and establishes its significance for evaluating DST review quality. It is worth noting that ethical considerations are not included as a sixth indicator. Although ethics is undeniably relevant to DST research, particularly in health and community contexts involving vulnerable populations (West et al., 2022), the present framework focuses specifically on review-level methodological indicators related to appraisal, theory, pedagogy, tools, and measurement. Ethical dimensions are typically addressed within individual primary studies rather than at the synthesis level, and their inclusion would require a distinct evaluative lens that falls outside the methodological scope of this umbrella review.

The first methodological indicator, quality assessment, relates to critical appraisal, which is a defining characteristic of systematic reviews (SRs) that directly influences the reliability and validity of their conclusions (De Cassai et al., 2025; Fromm et al., 2025). Without quality assessment, SRs may synthesise findings from methodologically weak or biased studies alongside rigorous ones, leading to distorted conclusions (De Cassai et al., 2025; Higgins et al., 2019). Along similar lines, Grant and Booth (2009) caution that, without formal appraisal, scoping reviews may rely on research quantity rather than methodological rigour. Assessing the quality of underlying evidence is therefore essential for providing meaningful insights into the evidence base (Daudt et al., 2013; Pham et al., 2014). This is particularly important given the growing prevalence of scoping reviews (M. D. Peters et al., 2021) and their role in informing clinical practice, policy, and guideline development (Daudt et al., 2013; Khalil et al., 2016), especially where they serve as precursors to systematic reviews. Accordingly, quality assessment is a fundamental step for ensuring that included studies meet minimum methodological standards (Cumpston et al., 2019; Shea et al., 2017). Therefore, this umbrella review also examines quality assessment practices in scoping reviews.

The second methodological indicator, theoretical frameworks, is essential in guiding research toward meaningful synthesis and developing generalisable knowledge base (Jones & Czerniewicz, 2011). A theory serves as a basis for describing, explaining, and predicting the phenomena under investigation, and this contributes to a better understanding of the research domain (Mueller & Urbach, 2017). In the context of educational technology, theory should be descriptive in explaining how specific instructional materials contribute to students’ learning experiences (Issroff & Scanlon, 2002). Further, A. Collins et al. (2016) argue that the design science of education must examine how different learning environments influence outcomes such as learning, collaboration, and motivation. Overall, theoretical frameworks serve an explanatory function by clarifying the relationships between instructional design and learning outcomes.

In this regard, theories can be categorised as either explanatory or design-oriented (Hew et al., 2019). Explanatory theories account for why particular phenomena occur; for example, cognitive load theory explains how information-processing constraints influence learning. In contrast, design-oriented theories prescribe how instruction should be structured to achieve specific goals. Newman and Gough (2020) emphasise that theoretical frameworks shape the entire review process, including the formulation of research questions, selection criteria, quality assessment, and synthesis strategies. As interpretive lenses, they support the organisation, explanation, and meaningful interpretation of findings, thereby advancing theoretical understanding in DST research.

In practical terms, theoretical frameworks help explain DST effectiveness across diverse contexts. For instance, constructivist learning theory illustrates how active story creation facilitates knowledge construction through engagement with authentic tasks.

The third methodological indicator, pedagogical design, refers to the concrete instructional activities, sequences, tools, and strategies through which DST is operationalised in practice (Goodyear, 2005). While theoretical frameworks explain why DST leads to particular learning outcomes, pedagogical design specifies how those outcomes are achieved. In DST research, pedagogical design encompasses decisions such as whether story creation is individual or collaborative, whether the process is scaffolded across defined phases such as pre-production, production, and post-production, the degree of learner autonomy afforded at each stage, the role of peer and instructor feedback, and the integration of multimodal composing tools into the instructional sequence (Smeda et al., 2014; Robin, 2016). These design choices shape the nature of learner engagement and directly influence the types of outcomes that DST interventions can support. Clear reporting of pedagogical design in reviews is therefore essential for determining whether observed outcomes are attributable to DST itself or to the specific instructional conditions under which it is implemented.

The fourth methodological indicator, technological tools, refers to the software, applications, and platforms used to create multimedia narratives. These tools vary considerably in terms of features, complexity, accessibility, and pedagogical affordances (Sarıca & Usluel, 2016). From a methodological perspective, the significance of technological tools in DST research extends beyond simple cataloguing. Tool selection shapes the nature of learner production, such as multimodal composition versus linear narration; interaction patterns, such as individual versus collaborative authoring; multimodal affordances, such as the integration of audio, video, images, and text; data sources available for outcome measurement, such as artefact analysis versus performance data; accessibility conditions, such as device requirements and internet dependency; and ultimately, the range of outcomes that can realistically be achieved and measured. Consequently, failing to report or classify tools systematically limits the interpretability of DST outcome evidence and hinders cross-study comparison. Analysis of DST environments further demonstrates that different tools can produce distinct learning outcomes (Psomos & Kordaki, 2012). For example, mobile devices may facilitate peer interaction in collaborative DST settings, while Ren’Py can support the development of communication skills through gamified experiences (Nasir et al., 2024). Furthermore, pedagogical designs that leverage different technological affordances may foster a range of outcomes across cognitive domains, from surface-level recall to deep comprehension and higher-order thinking, as well as self-regulatory competencies and affective dimensions (Reigeluth, 1999).

The fifth methodological indicator, measurement instruments, refers to standardised tests and researcher-developed tools used to assess DST outcomes and effectiveness. The quality of these instruments directly influences the validity and reliability of outcome evidence in reviews. Validity refers to the extent to which an instrument measures what it is intended to measure (Cohen et al., 2002). In this context, Prinsen et al. (2016) caution that including non-validated or uncommon measures in systematic reviews can complicate the interpretation and application of findings. Echoing these concerns, Sullivan (2011) emphasises that high reliability alone does not ensure credible research outcomes unless multiple forms of validity are also established.

These five methodological indicators provide the conceptual foundation for this umbrella review. To examine these indicators across the eligible corpus of DST reviews, three overarching research questions were developed, addressing publication, contextual, and methodological characteristics. By addressing multiple dimensions, each research question enables a comprehensive evaluation of DST reviews.

To advance understanding of how five interrelated methodological indicators inform and underpin claims regarding DST outcomes and effectiveness, this umbrella review was guided by the following research questions:

RQ1: What are the publication characteristics of DST research in terms of (a) publication years, (b) review types, (c) publication types, (d) authorship, and (e) countries?

RQ2: What are the contextual characteristics of DST research in terms of (a) review orientations, (b) disciplines, (c) number of included studies, (d) participants, and (e) targeted outcomes?

RQ3: What are the methodological characteristics of DST research in terms of (a) quality assessments, (b) theoretical frameworks, (c) pedagogical designs, (d) DST delivery tools, and (e) measurement instruments?

3. Methodology

An umbrella review (UR) methodology was deemed appropriate to provide a higher-order synthesis of the existing DST reviews (Becker & Oxman, 2008; Grant & Booth, 2009). To ensure comprehensive and transparent reporting (Tricco et al., 2018), the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) was adopted. The UR also employed the Population–Concept–Context (PCC) framework, recommended by the Joanna Briggs Institute (JBI), to guide the formulation of research questions and ensure clarity and specificity in defining the review scope (Aromataris et al., 2020; M. D. J. Peters et al., 2020).

Within this framework, the population covered systematic reviews, scoping reviews, meta-analyses, and mixed-method reviews examining DST outcomes and effectiveness across disciplines. The concept comprised five methodological quality indicators: quality assessment, theoretical frameworks, pedagogical design, technological tools, and measurement instruments. The context included reviews published across diverse settings, with no restrictions on publication date, and was limited to studies published in English and Turkish. English was included because it is the primary language of international academic publishing in educational technology and health sciences (Meneghini & Packer, 2007). Turkish was included because Türkiye represents an active research hub in digital storytelling, with a notable volume of peer-reviewed work published in Turkish-language journals that would otherwise be excluded by a strictly English-only strategy. It is acknowledged that this bilingual criterion may have influenced the geographical distribution of the retrieved corpus. The UR followed four phases: identification and screening, critical appraisal, data extraction, and synthesis.

In the first phase, relevant reviews were identified and screened through database searches and backward reference tracking. In the second phase, the methodological quality of the reviews was assessed using the JBI checklist (Appendix A). Based on this assessment, all 19 reviews demonstrated sufficient methodological rigour for their respective designs and were included in the synthesis. In the third phase, data on relevant indicators were extracted from each review using a standardised extraction form (see Appendix B for the brief data extraction form). The form captured the following fields: bibliographic details, including title, authors, year, journal or outlet, and country of the first author; review type and methodology; disciplinary domain; participant context; targeted outcome domains; quality appraisal tool used, if any; theoretical framework(s) referenced and their mode of application, such as background mention, guiding lens, or moderator variable; pedagogical design features reported; DST tools mentioned and the level of tool-related reporting, categorised as none, incidental, or systematic; and measurement instruments, including instrument names, constructs measured, and reported psychometric properties. For reviews that addressed multiple indicators simultaneously, each applicable field was coded independently. Ambiguous cases were discussed by the two reviewers until consensus was reached, with unresolved discrepancies referred to a third reviewer. In the fourth phase, content and thematic synthesis were conducted using both deductive and inductive coding approaches (Braun & Clarke, 2006; Kyngäs et al., 2020), while two reviewers independently analysed the data and resolved discrepancies through consensus.

3.1. Search Strategy

Four databases, including three multidisciplinary databases (Web of Science, Academic Search Ultimate, and Scopus) and one education-specific database (ERIC), were searched. These databases were selected based on their comprehensive coverage of educational technology and health sciences literature (AlRyalat et al., 2019). Specialised databases such as PsycINFO, CINAHL, and LLBA were not included in the primary search strategy, as the four selected databases collectively cover the majority of education, health, and language-learning journals relevant to DST. It is acknowledged, however, that reviews indexed exclusively in these specialised databases may have been missed, potentially affecting the comprehensiveness of the retrieved health-related, psychological, or linguistics-focused evidence.

The following search strategy was employed: (“digital storytelling” AND review) OR (“digital story” AND review) OR (“digital stories” AND review) OR (“digital stor” AND review). This strategy captured variations in terminology while maintaining a focus on review-type publications.

To maximise comprehensive retrieval, given variations in database indexing practices, all fields (title, abstract, keywords, and full text where available) were searched without date restrictions. Additionally, the titles and abstracts of all retrieved records were manually screened for explicit review identifiers (e.g., “systematic review,” “meta-analysis,” “scoping review,” “research synthesis,” and “literature review”) to address limitations in database indexing of review-type studies (Holly, 2017). Backward reference tracking of all included reviews was also conducted to capture additional studies not identified during database searches (Greenhalgh & Peacock, 2005).

The initial search yielded 1656 records across the four databases: Academic Search Ultimate (n = 219), ERIC (n = 480), Scopus (n = 464), and Web of Science (n = 493). Duplicate removal was performed using Citavi 7, after which the remaining unique records were screened at the title and abstract level to identify potentially eligible studies. Records that were clearly irrelevant at this stage were excluded. The remaining records then proceeded to full-text assessment. Following this process, 110 records were assessed for full-text eligibility (see Figure 2). The detailed record counts at each stage, including duplicates removed, records screened, and records excluded at each level are presented in the PRISMA flow diagram (Figure 2).

3.2. Review Selection Process

Following PRISMA-ScR guidelines, the review selection process proceeded in two stages. In the first stage, two reviewers independently screened the titles and abstracts of the records retained after duplicate removal to identify potentially eligible studies for full-text retrieval. In the second stage, the same two reviewers independently assessed the full texts of the 110 records that passed title and abstract screening against the inclusion and exclusion criteria. Eligible reviews were those that (a) adopted a review methodology addressing digital storytelling across any field or context, (b) explicitly reported DST outcomes or effectiveness, and (c) were published as peer-reviewed journal articles or book chapters in English or Turkish.

Reviews were excluded if they (a) focused on bibliometric or descriptive analyses of publication trends rather than DST outcomes and effectiveness (n = 28), (b) did not report outcomes or effectiveness (n = 19), (c) were overly broad in scope or entirely conceptual in nature (n = 15), (d) consisted of doctoral dissertations, master’s theses, or conference proceedings (n = 11), (e) did not maintain a primary focus on DST practice (n = 9), or (f) were published in languages other than English or Turkish due to translation fidelity concerns (n = 9).

After applying these inclusion and exclusion criteria, 19 reviews were selected for the final synthesis (see Figure 2).

3.3. Analysis Process

In accordance with the JBI umbrella review protocol (Aromataris et al., 2015) and scoping review guidelines (Tricco et al., 2018), two reviewers conducted content and thematic analysis using MAXQDA 26. Both inductive and deductive coding approaches were employed throughout the analysis (Miles & Huberman, 1994).

For publication and contextual characteristics, a deductive coding approach (Creswell & Clark, 2017) was applied. Relevant characteristics of each review were examined and organised within a theme–category–code structure. For methodological characteristics, which involved more complex and diverse issues, both deductive (theme–category–code) and inductive (code–category–theme) approaches were used.

The two reviewers independently analysed all 19 reviews. Throughout the iterative coding process, they discussed and refined codes. A third reviewer resolved discrepancies through discussion until consensus was reached. This process ensured the consistency and reliability of the results (Schreier, 2014). Intercoder reliability was calculated using Cohen’s kappa prior to the consensus-building phase, based on a randomly selected subset comprising 30% of the eligible corpus (n = 6 reviews). Reliability was computed at the level of individual codes within the theme–category–code structure. The resulting Cohen’s κ of 0.86 indicated a high level of agreement between the two independent reviewers (Landis & Koch, 1977).

4. Results

4.1. Results on Publication Characteristics of DST Reviews

4.1.1. Publication Years

The number of DST reviews published increased over time, with one review each in 2016, 2020, and 2021; four in 2022; three in 2023; and six in 2025 (see Figure 3).

4.1.2. Review Types

The 19 reviews employed diverse methodologies, including qualitative systematic reviews (n = 8), scoping reviews (n = 5), meta-analyses (n = 2), systematic literature reviews (n = 2), and mixed-method reviews (n = 2).

4.1.3. Publication Types

Seventeen reviews were published as journal articles and two as book chapters. Two reviews were published in Technology, Pedagogy and Education (C.-Y. Chang & Su, 2025; Isaacs et al., 2024). Two additional reviews by Y. Chi et al. (2025a, 2025b) were published in Theory and Practice in Language Studies and Educational Process: International Journal, respectively. A further seven reviews were published in education-focused journals, including Pamukkale University Journal of Education, Journal of Science Education and Technology, International Journal of Progressive Education, and Journal of Qualitative Research in Education.

Notably, five journals were from the fields of health, psychology, and public health, such as International Journal of Environmental Research and Public Health, BMC Public Health, Journal of Medical Internet Research, Journal of Psychotherapy Integration, and International Journal of Mental Health Nursing. Additionally, one review appeared in Behavioral Sciences (Xu et al., 2023), and two reviews were published as book chapters in Communications in Computer and Information Science (Khan et al., 2023) and Multimodal Learning Environments in Southern Africa (Olugbara et al., 2022).

Taken together, the results underscore the broad applicability of DST across diverse disciplinary and publication contexts.

4.1.4. Authorship

Author teams ranged from one to five members. This pattern likely reflects both the collaborative nature of the DST field and the methodological complexity of review-level research on digital storytelling.

4.1.5. Countries

Based on the institutional affiliations of the first authors, the reviews originated from 10 countries: Türkiye (n = 7), Malaysia (n = 3), Australia (n = 2), the United States (n = 2), and five additional countries with one review each (n = 5). No eligible reviews from South America or Africa were identified in the retrieved corpus. This finding should be interpreted cautiously, as reviews published in other languages, in regional journals not indexed by the selected databases, or in grey literature may have been missed. Nonetheless, the absence of eligible reviews from these regions points to a notable geographical gap in the currently accessible review-level DST evidence base.

4.2. Results on Contextual Characteristics of DST Reviews

4.2.1. Review Orientations

Using Gough et al.’s (2012) framework, the reviews were classified according to their synthesis orientation. Fourteen reviews adopted configurative approaches, employing qualitative thematic synthesis, content analysis, or narrative interpretation (e.g., Bezen & Çıralı Sarıca, 2025; C.-Y. Chang & Su, 2025). Three reviews used aggregative approaches, applying meta-analytic methods to calculate pooled effect sizes (e.g., Sahin, 2022). Two reviews integrated both orientations, combining quantitative aggregation with configurative interpretation (e.g., Ates, 2023).

4.2.2. Disciplines

Reviews were distributed across three disciplinary domains: health and medical sciences (n = 7), general education (n = 7), and language education (n = 5). Reviews in health and medical sciences addressed diverse subfields, including nursing education and health communication (C.-Y. Chang & Su, 2025), gerontological nursing and public health (H. Chang et al., 2023), health psychology (Stargatt et al., 2022), social work and gerontology (Xu et al., 2023), mental health nursing (De Vecchi et al., 2016), psychotherapy (Ogbeiwi et al., 2024), and immigrant and refugee public health (Kisa & Kisa, 2025). These reviews predominantly framed DST as a therapeutic or social intervention, emphasising patient outcomes, recovery-oriented practices, and psychosocial wellbeing (see Figure 4).

Reviews in general education (n = 7) addressed diverse fields, including educational technology and instructional methodology (Sahin, 2022; Talan, 2021), primary and classroom education (Demirbas & Sahin, 2020; Ispir & Yıldız, 2023), physics education (Bezen & Çıralı Sarıca, 2025), twenty-first-century competencies (Isaacs et al., 2024), and multimodal learning in Southern Africa (Olugbara et al., 2022). These reviews generally positioned DST as a pedagogical tool, primarily focusing on enhancing instructional processes, learner engagement, and competency development. A notable exception is Olugbara et al. (2022), which extends this pedagogical framing to include community of practice formation and identity construction within regionally specific sociotechnical contexts and incorporates a broader social dimension into the analysis.

Reviews in language education (n = 5) address affective dimensions of EFL/ESL pedagogy (Y. Chi et al., 2025a, 2025b), productive skills in English (Khan et al., 2023), cross-linguistic performance (Veyis et al., 2025), and both first and foreign language instruction (Ates, 2023). In this domain, DST is widely framed as a pedagogical tool. This framing reflects the field’s emphasis on language acquisition, skill development, and affective engagement in second and foreign language learning contexts.

4.2.3. Number of Included Studies

The number of primary studies across reviews varied considerably, ranging from 8 (Stargatt et al., 2022) to 96 (Ates, 2023). More than half of the reviews included between 11 and 40 studies. Smaller sample sizes tended to reflect either the nascent or highly specialised nature of the research domain, as seen in reviews focusing on immigrant health (n = 9; Kisa & Kisa, 2025) and physics education (n = 12; Bezen & Çıralı Sarıca, 2025). In contrast, larger samples were characteristic of reviews with broader disciplinary or geographic scopes, such as those examining nursing education (n = 45; C.-Y. Chang & Su, 2025) and Turkish educational research (n = 65; Demirbas & Sahin, 2020).

Across the reviews, primary studies included in systematic and scoping reviews within health-related contexts were predominantly qualitative or mixed-method in design, including case studies, ethnography, narrative inquiry, and other exploratory approaches (e.g., Ogbeiwi et al., 2024; Xu et al., 2023). In contrast, primary studies in educational contexts more commonly employed quasi-experimental and mixed-method designs (e.g., Khan et al., 2023; Olugbara et al., 2022).

4.2.4. Participants

Reviews addressed two main participant contexts: those in formal education (n = 12) and those in non-educational contexts, including health, community, and clinical environments (n = 7). Within formal education, participants were distributed across three categories. Five reviews focused on K–12 learners (e.g., Isaacs et al., 2024; Ispir & Yıldız, 2023), four on higher education participants, particularly in EFL/ESL and pre-service teacher education contexts (e.g., Y. Chi et al., 2025b; Olugbara et al., 2022), and three included participants spanning multiple educational levels from early childhood through higher education (e.g., Bezen & Çıralı Sarıca, 2025; Demirbas & Sahin, 2020).

The remaining seven reviews addressed non-educational contexts, with participants including older adults across community-dwelling, residential care, and clinical gerontological contexts (e.g., H. Chang et al., 2023). Two reviews examined participants in mental health and therapeutic settings, including populations such as individuals with dementia, veterans with PTSD, suicide-bereaved youth, and adolescents in foster care (e.g., Ogbeiwi et al., 2024). A further review focused on immigrant and refugee populations in community-based health promotion programmes (Kisa & Kisa, 2025).

Notably, some reviews integrated both educational and non-educational contexts. For example, C.-Y. Chang and Su (2025) examined three distinct participant groups, including patients in care settings, nursing students in formal education programmes, and healthcare professionals in continuing professional development settings (see Figure 4).

4.2.5. Targeted Outcome Domains

Using Wu and Chen’s (2020) eight-category typology, outcome domains were identified, with frequent overlap across categories. Affective outcomes were the most prevalent (n = 17), including motivation, engagement, and self-confidence (e.g., Talan, 2021), as well as emotional well-being and empowerment (e.g., C.-Y. Chang & Su, 2025). Cognitive outcomes (n = 11) included critical thinking, creative thinking, and problem-solving (e.g., Isaacs et al., 2024). Social outcomes (n = 10) encompassed collaboration, communication, and community building (e.g., H. Chang et al., 2023). Technological outcomes (n = 9) covered digital literacy and ICT competence (e.g., Demirbas & Sahin, 2020). Ontological outcomes (n = 8) included identity construction and narrative agency (e.g., Stargatt et al., 2022). Academic achievement and linguistic outcomes were each reported in six reviews (e.g., Khan et al., 2023), while conceptual outcomes were identified in three reviews (e.g., Bezen & Çıralı Sarıca, 2025).

Notably, most reviews reported on multiple outcome domains simultaneously. For example, Ispir and Yıldız (2023) identified seven distinct outcome categories across 36 primary studies in Turkish classroom education. In contrast, only one review adopted an explicitly regional comparative focus on the Southern African context, identifying outcomes such as student engagement, communities of practice, digital literacy, identity construction, and collaborative learning (Olugbara et al., 2022).

4.3. Results on Methodological Characteristics of DST Reviews

4.3.1. Quality Assessment

Only five reviews, including three systematic reviews (e.g., Bezen & Çıralı Sarıca, 2025) and two scoping reviews (e.g., H. Chang et al., 2023), employed standardised critical appraisal instruments. Two reviews used the Mixed Methods Appraisal Tool (MMAT), two applied Joanna Briggs Institute (JBI) checklists, and one used adapted forms based on Pluye et al. (2009) and the Critical Appraisal Skills Programme (CASP).

These reviews were conducted across health-related disciplines (n = 3), general education (n = 1), and physics education (n = 1). In contrast, fourteen reviews did not report using any critical appraisal tool to assess the quality of primary studies (see Figure 5).

4.3.2. Theoretical and Pedagogical Frameworks

Reviews varied in their use of theoretical frameworks and pedagogical designs. Three reviews employed such frameworks as operational analytical lenses guiding the entire synthesis process, including two that used theoretical frameworks and one that applied a pedagogical design framework. For example, Y. Chi et al. (2025a) adopted a dual framework combining Ohler’s (2013) five-phase DST process model with MacIntyre et al.’s (1998) willingness-to-communicate pyramid. The former was used to structure the categorisation of findings by DST phase, whereas the latter mapped outcomes onto hierarchical levels of communicative readiness. Similarly, Isaacs et al. (2024) operationalised the Voogt and Pareja Roblin (2012) twenty-first-century skills framework as a top-down classification tool, informing the search strategy, guiding data extraction, and structuring the coding of qualitative data across eight skill domains.

Three reviews relied on DST-inherent characteristics (e.g., multimodal features, active learning affordances, and student-centred processes) as explanatory mechanisms (e.g., Ates, 2023). These reviews attributed positive outcomes or the perceived effectiveness of DST practices to its inherent multimodal, audiovisual, and technology-mediated features, including active learning affordances, multisensory stimulation through the integration of text, images, audio, and video, and the student-centred nature of the story creation process (see Figure 5).

Nine reviews referenced theoretical or pedagogical frameworks in their discussion sections but did not use them to guide the review process. Among the most frequently cited were reminiscence therapy (n = 4), learner-centred pedagogical principles (n = 4), constructivist learning theory (n = 3), Vygotsky’s sociocultural theory and the Zone of Proximal Development (n = 2), and Mayer’s Cognitive Theory of Multimedia Learning (n = 2).

Across meta-analytic reviews (n = 3), neither pedagogical design nor theoretical frameworks were examined as moderator variables, despite the potential influence of theoretical and instructional variations on DST effectiveness. These reviews investigated a broad range of moderators, including learner characteristics (e.g., education level, grade level), instructional design features (e.g., course type, experiment duration, story source, instructor type), linguistic factors (e.g., language studied, language skills), and study-level attributes (e.g., publication type, country, publication year). However, only a limited number of these moderators reached statistical significance. For instance, education level emerged as the most consistent moderator, significantly favouring university students for attitudinal outcomes (ES = 1.73; Sahin, 2022) and high school students for academic achievement (ES = 1.302; Talan, 2021). Course type was significant only in Talan (2021), where participants in social sciences (ES = 0.977) and science (ES = 0.944) outperformed those in other disciplines (p < 0.001).

Among implementation-related variables, experiment duration and country were the only moderators identified as significant (Veyis et al., 2025). In this review, short-term interventions lasting one to four weeks produced stronger effects (g = 1.660) than longer interventions, and country explained approximately 42% of the between-study variance. All other moderators, including publication type, language studied, language skills, grade level, story source, instructor type, and publication year, were reported as non-significant across the three reviews. Notably, one review did not report any explicit theoretical or pedagogical framework (Talan, 2021).

4.3.3. DST Tool Typologies

Reviews varied considerably in their reporting of DST tools. Ten did not report specific tools (e.g., Isaacs et al., 2024; Xu et al., 2023), while six mentioned them only incidentally within study descriptions (e.g., Ates, 2023; Talan, 2021). Only three reviews systematically extracted tool-related data using frequency tables, typological categorizations, or functional classifications (Bezen & Çıralı Sarıca, 2025; Ispir & Yıldız, 2023; Ogbeiwi et al., 2024).

For instance, Ispir and Yıldız (2023) provided the most detailed analysis, presenting tools alongside outcome categories in frequency tables. In their review, PhotoStory was associated with cooperative, cognitive, and psychomotor outcomes, whereas PowToon and Toondoo were primarily linked to cognitive and social outcomes. This descriptive co-occurrence approach extends beyond simple tool listing; however, such associations do not establish causality. For example, the co-occurrence of PhotoStory and cooperative outcomes may reflect the collaborative instructional design rather than the inherent affordances of the tool itself. While tool–outcome mapping is useful for identifying patterns, these associations remain strictly descriptive within the current evidence base and should not be interpreted as establishing causal relationships. Whether the observed effects are attributable to the technological affordances of specific tools, to the pedagogical designs in which those tools are embedded, or to the interaction between tools and pedagogy remains an open empirical question and constitutes an important direction for future experimental and quasi-experimental DST research.

Notably, none of the reviews examined the relationship between the pedagogical affordances of tools and the reported outcomes or effectiveness of DST practices. The absence of such an analysis points to a significant gap in the field.

4.3.4. Measurement Instruments

Across the 19 included reviews, reporting of measurement instruments varied considerably by discipline. The findings presented here are based only on what the included reviews explicitly reported and should therefore be interpreted as evidence of review-level reporting practices, rather than as a comprehensive account of measurement practices across the broader DST primary literature. Because reviews may summarise, aggregate, or omit details from the primary studies they include the absence of detailed instrument-level reporting in a review does not necessarily indicate the absence of such instruments in the primary studies.

In general education, seven reviews reported instruments only at a categorical level, referring broadly to tools such as tests, scales, interviews, rubrics, or assessment forms, without consistently identifying specific standardised measures. Researcher-developed instruments, including achievement tests, rubrics, and assessment scales, were reported in seven reviews (e.g., Demirbas & Sahin, 2020; Isaacs et al., 2024). At the review level, this suggests that general education reviews tended to describe measurement approaches in broad methodological categories rather than providing detailed instrument-level information.

A different reporting pattern was observed in health-related reviews. Three reviews reported standardised instruments in greater detail (Ogbeiwi et al., 2024; Stargatt et al., 2022; Xu et al., 2023). For example, Stargatt et al. (2022), using a tabulated format, documented instruments such as the State–Trait Anxiety Inventory (STAI), Hospital Anxiety and Depression Scale (HADS), Caregiver Questionnaire (CQ), Geriatric Depression Scale (GDS), Autobiographical Memory Inventory (AMI), Quality of Life in Alzheimer’s Disease scale (QOL-AD), Quality of the Caregiving Relationship Questionnaire (QCPR), and Dementia Care Mapping. As reported in that review, these instruments were used primarily to assess mood, well-being, memory, caregiving relationships, and quality of life among older adults with dementia.

The remaining four health-related reviews reported the use of non-standardised qualitative methods, including interviews, focus groups, observational notes, consumer narratives, ethnographic approaches, reflective journals, open-ended questionnaires, and workshop observations (e.g., De Vecchi et al., 2016; Kisa & Kisa, 2025). For instance, Kisa and Kisa (2025) described the use of ethnographic methods, workshop-based narrative creation, and participatory designs to capture participants’ experiences through personal storytelling rather than the use of standardised measurement instruments.

In language education reviews (n = 5), only one study (Veyis et al., 2025) provided detailed instrument-level reporting, including measures such as the Preliminary English Test (PET) and the Cambridge Flyers Listening examination. In contrast, Y. Chi et al. (2025a) made only a single reference to the Foreign Language Classroom Anxiety Scale.

Overall, the included reviews suggest a disciplinary contrast in measurement reporting. Health-related reviews more frequently reported named standardised instruments, whereas general education and language education reviews more often described instruments at a broader categorical level or reported researcher-developed tools. Qualitative instruments (e.g., interviews, focus groups, observations, and reflective journals) were widely reported across all three domains. This pattern reflects the exploratory, process-oriented, and participatory nature of digital storytelling research as reported in the 19 included reviews, rather than providing definitive evidence about the quality, validity, or comprehensiveness of measurement practices in the wider DST evidence base.

5. Discussion

This umbrella review synthesised findings from 19 reviews to provide a comprehensive overview of digital storytelling (DST) practices across publication, contextual, and methodological indicators. The findings show that existing DST reviews are predominantly qualitative systematic reviews and scoping reviews published across diverse journals and disciplines. Most were conducted by multi-author teams based in Türkiye, Malaysia, Australia, and the United States between 2016 and 2025, indicating both disciplinary breadth and geographical concentration. In particular, the absence of reviews originating from South America and Africa points to a notable regional gap in the DST evidence base.

Although the retrieved corpus demonstrates diversity in review types, meta-analytic and mixed-method reviews remain scarce. This limits the field’s ability to generate pooled effect size estimates, conduct moderator analyses, and integrate quantitative and qualitative evidence for multidimensional interpretation. Meta-analytic reviews are important for quantifying DST effectiveness and identifying the conditions under which interventions are more or less successful (Hennessy et al., 2019). Mixed-method reviews, in turn, integrate quantitative and qualitative evidence, offering multidimensional insights for informed decision-making (Harden & Thomas, 2010; Pearson et al., 2015). The limited number of such reviews therefore restricts the availability of clear, evidence-based guidance for practitioners, policymakers, and researchers. In addition, the geographical concentration of reviews may limit the transferability of findings to underrepresented educational, clinical, and cultural contexts (Schloemer & Schröder-Bäck, 2018). Future reviews should therefore expand both their methodological range and geographical scope to better reflect the diverse contexts in which DST is implemented globally.

Findings related to contextual indicators show that most reviews adopt configurative orientations, using qualitative thematic synthesis, content analysis, or narrative interpretation to generate conceptual understanding. Only a small number employ aggregative approaches through meta-analytic methods or integrate both orientations. This pattern suggests that DST research has developed a rich conceptual understanding of practices and outcomes but still lacks sufficient pooled evidence regarding effectiveness. As Gough et al. (2012) note, robust evidence synthesis benefits from the integration of both aggregative and configurative approaches. The predominance of configurative reviews also means that moderator analyses, which could clarify how factors such as educational level, disciplinary context, learner characteristics, instructional design, or technology use influence DST outcomes, remain limited. Furthermore, the scarcity of mixed-orientation reviews limits the availability of multi-dimensional evidence needed for informed decision-making (Pearson et al., 2015; Snilstveit et al., 2012). Moreover, most reviews focus on micro-level practices, while meso-level organisational structures and macro-level policy frameworks receive little attention. These higher-level dimensions are essential for understanding institutional adoption, policy integration, and large-scale implementation (Ganakas et al., 2025). Future evidence syntheses should therefore address multiple levels of analysis to produce a more comprehensive understanding of DST implementation.

A central methodological finding concerns the limited use of formal quality appraisal across reviews. Regardless of review type, only a small number of studies apply established critical appraisal tools or clearly report assessment procedures. This issue is important because quality appraisal is a foundational component of rigorous evidence synthesis; it helps evaluate methodological rigour, identify potential sources of bias, and determine the credibility and applicability of review findings (De Cassai et al., 2025; Guo et al., 2025; Shaheen et al., 2023). Similar concerns have been reported in broader educational technology research. For example, Zawacki-Richter et al. (2025) found that only 25.9% of 576 systematic reviews in digital education conducted comprehensive quality appraisal, while Buntins et al. (2023) reported that only 4.4% of educational technology evidence syntheses met full replicability criteria. Uttley et al. (2023) also identified numerous issues in the conduct and reporting of systematic reviews that may threaten reliability and validity.

The absence of quality appraisal is particularly problematic in meta-analytic reviews, where pooling data from methodologically weak studies may produce statistically precise but potentially misleading effect estimates, thereby undermining the validity of conclusions (Cheungpasitporn et al., 2025; Turner et al., 2025). None of the three meta-analyses included in this corpus employed formal appraisal tools, raising concerns about the credibility of their reported effect sizes. Although some scholars argue that critical appraisal may be less necessary for scoping reviews because their primary purpose is descriptive mapping rather than evaluative synthesis (Kysh et al., 2025), others maintain that appraisal becomes increasingly important when scoping reviews inform policy, practice, or future intervention design (Daudt et al., 2013; Khalil et al., 2016; M. D. Peters et al., 2021). Accordingly, regardless of review type, quality assessment should adhere to fundamental standards of rigour and transparency (Kysh et al., 2025; Viksveen et al., 2021). Future DST reviews should therefore incorporate quality appraisal procedures appropriate to their review type and purpose, thereby strengthening transparency, methodological rigour, and confidence in review conclusions.

The second finding related to methodological indicators concerns the limited use of theoretical frameworks and pedagogical designs as analytical lenses across reviews. To interpret this finding precisely, it is important to distinguish between three levels of theoretical use identified in the retrieved corpus: (a) theory mentioned in the background or introduction without informing the review methodology; (b) theory used to guide the coding, synthesis, or classification of findings throughout the review process; and (c) theory examined as a moderator or explanatory variable in meta-analytic or mixed-method reviews. The majority of reviews in this corpus fall into the first category, referencing theoretical perspectives retrospectively rather than using them to shape analytical decisions. Only a small number of reviews employed theory at the second level, and none of the meta-analytic reviews examined theoretical grounding as a moderator variable. This distinction is important because only the second and third levels represent methodologically robust uses of theory in evidence synthesis (Newman & Gough, 2020; Schad et al., 2021).

This finding aligns with previous concerns in DST and related educational technology fields. Lim et al. (2022), for example, report that many DST studies fail to identify an underlying theory or clearly articulate the core components of DST as a pedagogical approach in language learning. Comparable concerns have been reported in flipped learning, gamified learning, and broader educational technology research, where studies often lack explicit theoretical frameworks, provide insufficient detail on pedagogical challenges, or fail to align theory with research questions, design, and analysis (Antonenko, 2014; ElGamal & Zawacki-Richter, 2025; Hew et al., 2019; Karabulut-Ilgu et al., 2018; Sancar-Tokmak & Dagli, 2025; Schad et al., 2021). This matters because theoretical frameworks provide the conceptual structure through which findings are analysed, interpreted, and contextualised (Kivunja, 2018). Theory-informed reviews can also generate more coherent and practically useful evidence by explaining not only whether DST works, but also why, how, and under what conditions it works (Noyes et al., 2020).

The limited integration of pedagogical design frameworks further constrains the interpretability of DST outcomes. DST is not a single uniform intervention; rather, its effects may vary depending on how activities are designed, scaffolded, sequenced, and assessed. For example, Smeda et al. (2014) highlight the need for comprehensive frameworks to guide students through different stages of digital storytelling and propose the e-Learning Digital Storytelling framework as a constructivist model structured around storytelling elements and progressive levels of complexity. Similar concerns have been reported in related educational technology fields, where pedagogical approaches are often insufficiently theorised, inconsistently operationalised, or weakly aligned with learning processes (Alvarez et al., 2022; Hew et al., 2021; Sancar-Tokmak & Dagli, 2025). Evidence from other fields further demonstrates the value of theory-informed synthesis. For instance, Gajendran and Harrison (2007) adopt a theory-driven design in their meta-analysis of telecommuting, Kruis et al. (2020) use Akers’ Social Learning Theory to evaluate whether empirical findings align with theoretical predictions, and Li et al. (2025) employ Kolcaba’s Comfort Theory to structure patient experiences across multiple dimensions. In addition, evidence from learning sciences suggests that pedagogical design variations can influence learning effectiveness. The ICAP framework, for instance, indicates that outcomes differ depending on whether learners engage passively, actively, constructively, or interactively (M. T. Chi & Wylie, 2014). However, the current DST review corpus provides limited evidence on whether such theoretical and pedagogical differences moderate outcomes. Future reviews, particularly meta-analytic and mixed-method syntheses, should therefore examine theoretical grounding and instructional design as analytical categories or moderator variables.

A further methodological concern is the insufficient reporting of digital tools and the limited analysis of how tool affordances relate to reported outcomes. This is important because DST technologies differ substantially in the forms of interaction, collaboration, creativity, feedback, and reflection they support. Psomos and Kordaki (2012), for example, show that platforms such as Toontastic, Kodu, Storytelling Alice, and Scratch vary in their capacity to foster collaborative learning, engagement, cognitive effort, goal orientation, and metacognitive development. However, the reviews included in this umbrella review rarely examined whether specific technological features contributed to particular cognitive, affective, linguistic, or social outcomes. This weakens the field’s ability to determine which tools are most appropriate for particular pedagogical purposes.

Given the different affordances of DST platforms, future reviews should move beyond simply naming digital tools and should instead examine how tool features align with instructional objectives and learner needs. Nasir et al. (2024) similarly emphasise that educators and instructional designers should evaluate digital tools in relation to intended learning outcomes rather than treating them only as content-delivery mechanisms. Arzeman et al. (2025) identify interactivity, usability, immediate feedback, and personalisation as core features that support learner engagement, while Navas-Bonilla et al. (2025) underline the importance of tool functionalities for inclusive educational technologies. These perspectives suggest that tool-related reporting should become a more explicit component of DST evidence synthesis.

The final methodological finding concerns the limited use of standardised measurement instruments and the insufficient reporting of their psychometric properties. Measurement quality is central to the credibility of evidence synthesis because review conclusions depend on the validity, reliability, and comparability of the instruments used in primary studies. This issue has already been noted in the DST literature. Lim et al. (2022) report that approximately 90% of studies in their dataset did not provide reliability information for instruments assessing learning outcomes. Similar measurement concerns are evident in other fields, where inconsistent definitions, reliance on self-report measures, and limited psychometric validation reduce cross-study comparability and weaken confidence in reported outcomes (Doran et al., 2025; Lawrence et al., 2026; Mettert et al., 2020; Taveras et al., 2025; Wang et al., 2026).

The implications of weak measurement are substantial. Abrami et al. (2015) demonstrate that non-standardised measures may yield larger effect sizes than standardised instruments, suggesting that measurement quality can influence conclusions about intervention effectiveness. Similarly, Anmarkrud et al. (2019) show that limited psychometric rigour and inconsistent construct use can make it difficult to interpret findings in cognitive load research. In the present umbrella review, the lack of measurement-level analysis across reviews represents an important methodological gap. Future DST syntheses should therefore report whether instruments are standardised, whether reliability and validity evidence is available, and how measurement quality may influence outcome interpretation.

Overall, this umbrella review indicates that the credibility of DST evidence depends on the combined consideration of five interrelated methodological indicators: quality assessment, theoretical frameworks, pedagogical design, technological tools, and measurement instruments. These indicators should not be treated as separate or optional elements, because each shapes the interpretation and strength of review conclusions. Strengthening future DST evidence syntheses will require more systematic quality appraisal, stronger theoretical and pedagogical grounding, clearer reporting of technological affordances, and more rigorous attention to measurement quality. Addressing these methodological priorities will support the development of a more robust, transparent, and practically useful evidence base for DST research and implementation.

6. Recommendations

The following recommendations are offered as a prioritised methodological framework for strengthening the rigour, transparency, and global relevance of future DST review research.

First, improving quality appraisal practices is the most foundational step. Regardless of review type, future DST syntheses should integrate standardised critical appraisal tools appropriate to their methodological design. In disciplines where formal appraisal is less common, such as general education and language learning, established frameworks from the health sciences, including JBI checklists, MMAT, and CASP, can be adapted while accounting for discipline-specific conventions (Aromataris et al., 2015; Cumpston et al., 2019). Meta-analytic reviews in particular must conduct quality appraisal, as pooling data from methodologically flawed studies can yield statistically precise but substantively misleading effect estimates (Cheungpasitporn et al., 2025).

Second, theoretical frameworks and pedagogical designs should be reported and applied more systematically. Their use should be integrated as analytical lenses throughout the review process, from research question formulation to data extraction, coding, and synthesis, rather than being confined to background or discussion sections (Newman & Gough, 2020; Schad et al., 2021). Reviews that attribute DST effectiveness to inherent characteristics such as multimodality, narrative construction, learner agency, or interactivity should ground such claims in established learning theories to provide more robust explanations. In addition, when sufficient primary-level detail is available, meta-analytic and mixed-method reviews should examine theoretical grounding and pedagogical design variation as moderator or explanatory variables. The field would also benefit from developing a cross-disciplinary theoretical framework that bridges health, education, and language domains, enabling more coherent synthesis and advancing understanding of the mechanisms through which DST may influence outcomes.

Third, future reviews should systematically extract and classify technological tool characteristics rather than listing them incidentally. Reviewers should report the software, applications, platforms, devices, and multimodal affordances involved in DST interventions, including features such as audio, video, image, text integration, collaboration functions, accessibility requirements, and online or offline use. Explicit analysis of tool–outcome relationships is needed to clarify whether observed outcomes are associated with technological affordances, pedagogical design, or the interaction between the two. Where consistent reporting is available across primary studies, meta-analytic reviews should examine tool type or tool characteristics as moderator variables.

Fourth, reviewers should report measurement instruments at the individual level. This includes instrument names, constructs measured, target populations, scoring procedures, and psychometric properties such as reliability and validity evidence. Such reporting is essential for interpreting whether outcomes across studies are conceptually and methodologically comparable. Meta-analytic studies should also evaluate instrument quality, as variation in measurement tools may confound observed effect sizes. Where psychometric information is absent or insufficiently reported in primary studies, this limitation should be explicitly acknowledged, and findings should be interpreted with appropriate caution.

Fifth, future DST evidence syntheses should prioritise meta-analytic and mixed-method designs when the available evidence allows. Meta-analyses can help estimate the magnitude of DST effects and examine moderators such as learner age, educational level, domain, intervention duration, theoretical grounding, pedagogical design, tool type, and measurement instrument quality. Mixed-method syntheses can complement this by integrating quantitative effect estimates with qualitative insights into implementation processes, learner experiences, contextual conditions, and mechanisms of change. Future reviews should also extend beyond micro-level classroom or intervention practices to examine DST at institutional and policy levels, including institutional adoption processes, policy integration strategies, teacher professional development, and large-scale implementation conditions.

Finally, the geographical and linguistic scope of future DST reviews should be deliberately broadened. Searches should incorporate multilingual strategies, including but not limited to Spanish, Portuguese, French, Arabic, and other languages relevant to underrepresented regions. Reviewers should also include regional databases, institutional repositories, and local indexing systems to identify DST research that may not be indexed in major international databases. For example, SciELO may support retrieval of research from Latin America, while regionally relevant repositories and indexing platforms may help identify research from African, Arab, and Asian contexts. International review teams and partnerships with researchers from underrepresented regions should be encouraged, as they can improve language coverage, contextual interpretation, and access to locally indexed or grey literature. In addition, advocacy for indexing regionally produced DST research in major international databases such as Scopus and Web of Science would help reduce structural barriers to more globally inclusive evidence synthesis.

7. Limitations

This umbrella review has five main limitations. First, the corpus is restricted to 19 reviews published in English and Turkish, which may have excluded relevant syntheses published in other languages. English was included because it is the dominant language of international academic publishing in educational technology and health sciences, while Turkish was included to capture regional research output from Türkiye. However, because no systematic search or screening was conducted in other major languages, including Arabic, Chinese, Spanish, French, and Portuguese, language bias cannot be ruled out. Reviews published in these languages may contain evidence that could have influenced the findings reported in this umbrella review.

Second, a related limitation concerns the potential overlap of primary studies across the 19 included reviews. In umbrella reviews, the same primary study may be included in multiple constituent reviews, which lead to false conclusions (Pieper et al., 2014). In this umbrella review, a complete formal assessment of primary-study overlap could not be conducted because not all included DST reviews provided explicit lists or sufficient citation details of their primary studies. Consequently, it was not possible to construct a reliable review-by-primary-study citation matrix across the full corpus.

Third, as an umbrella review, this study synthesises evidence at the review level rather than analysing primary studies directly. Consequently, when included reviews omit key methodological details, such as theoretical frameworks, pedagogical design features, technological tool characteristics, or measurement instruments, it is not possible to determine whether these elements were absent from the primary studies themselves or simply unreported at the review level. This constraint is inherent to umbrella review methodology and limits the extent to which conclusions can be drawn about methodological practices in the broader DST primary literature.

Fourth, the predominance of configurative reviews limits the examination of DST effectiveness. The limited availability of meta-analyses made robust meta-meta-analysis infeasible, reflecting the current state of the DST review literature rather than a methodological choice in this study. Relatedly, the heterogeneity of review types complicates uniform quality comparison, as systematic reviews, scoping reviews, narrative reviews, and meta-analyses follow different methodological conventions. This challenge was addressed through the application of the JBI umbrella review checklist, which accommodates diverse review types, and by interpreting review quality relative to each review’s stated purpose rather than applying a single uniform standard.

Fifth, although the search covered four major databases and was supplemented by backward reference tracking, relevant reviews indexed exclusively in specialised or discipline-specific databases may have been missed. This limitation is particularly important given the interdisciplinary scope of the present review, which includes DST research in education, health, psychology, and language learning. For example, PsycINFO may include psychological and psychotherapeutic applications of DST, CINAHL may index nursing, allied health, and clinical intervention literature, and LLBA may provide additional coverage of linguistics and language education research. Reviews published only in journals indexed by these specialised databases, but not in Web of Science, Scopus, Academic Search Ultimate, or ERIC, may therefore represent a gap in the retrieved corpus, potentially affecting the completeness of health-related, psychological, and language-focused DST evidence.

Despite these limitations, this umbrella review provides a cross-disciplinary synthesis of methodological indicators across the available DST review literature. By identifying patterns in quality appraisal, theoretical and pedagogical reporting, technological tool classification, and measurement instrument reporting, it offers a foundation for strengthening the rigour, transparency, and methodological consistency of future DST evidence syntheses.