Mapping the Use of Bibliometric Software and Methodological Transparency in Literature Review Studies: A Comparative Analysis of China-Affiliated and Non-China-Affiliated Research Communities (2015–2024)

Abstract

The growing use of bibliometric methods in literature reviews has intensified concerns about methodological transparency and consistency. This study compares English-language reviews authored by China-affiliated and non-China-affiliated researchers between 2015 and 2024. Through bibliometric content analysis and co-word network mapping, it evaluates the following: (1) the use and purposes of bibliometric software; (2) the clarity of methodological reporting, including software versions, threshold settings, data preprocessing, and database selection; (3) the extent to which limitations are acknowledged and recommendations proposed; and (4) the dominant conceptual themes shaping research practices. The analysis covers 50 highly cited reviews (25 per group) and 4000 additional papers for thematic mapping. Findings show both convergence and divergence: while tools such as VOSviewer, CiteSpace, Gephi, and Bibliometrix are widely adopted, non-China-affiliated studies exhibit greater transparency and reflexivity, whereas China-affiliated research often emphasizes output metrics and underreports methodological challenges. These contrasts reflect broader epistemological norms and research cultures. This study underscores the need for unified reporting standards and contributes to meta-research by offering practical guidance to improve the transparency, comparability, and rigor of bibliometric-supported literature reviews.

1. Introduction

The exponential growth of scientific literature has ushered in an era where researchers are increasingly reliant on advanced tools and analytical frameworks to make sense of the vast volumes of published research. In this context, bibliometric analysis has emerged as a pivotal method for quantitatively assessing scientific production, mapping knowledge structures, and identifying influential themes, sources, and contributors across disciplines (Aria & Cuccurullo, 2017; Cobo et al., 2011; Donthu et al., 2021). Bibliometric techniques—especially when integrated into literature reviews—have not only enhanced the rigor and transparency of synthesis efforts but also contributed to a deeper understanding of the intellectual, social, and conceptual landscapes of scientific inquiry (Moral-Muñoz et al., 2020; van Eck & Waltman, 2010; Zupic & Čater, 2014).

Driven by this surge in interest, an increasing number of review studies now incorporate bibliometric software tools such as VOSviewer (van Eck & Waltman, 2010), CiteSpace (C. Chen, 2006), and Bibliometrix (Aria & Cuccurullo, 2017) to perform co-word, citation, co-citation, and co-authorship analyses, as well as bibliographic coupling, to identify thematic evolution and visualize scientific networks. These tools have enabled researchers to explore topics ranging from artificial intelligence (Di Vaio et al., 2020; Tlili et al., 2022), environmental sustainability (Geissdoerfer et al., 2017; Zhang et al., 2019), the impact of climate change on tea cultivation (Ali Abaker Omer et al., 2025), and healthcare innovation (Faust et al., 2018), to social media (Leung et al., 2017), urban resilience (Meerow et al., 2016), agrivoltaic systems (Omer et al., 2025), and educational reform (Bızel, 2023; Tlili et al., 2022). Despite these methodological advancements, recent studies have raised concerns about inconsistencies in the application and reporting of bibliometric methods (Linnenluecke et al., 2020; Mukherjee et al., 2022). Core issues include the omission of critical details such as software versions, threshold settings, preprocessing steps, and database selection criteria—factors essential for the reproducibility and reliability of bibliometric research (Moral-Muñoz et al., 2020; Pranckutė, 2021).

Importantly, these inconsistencies are not merely methodological oversights—they also reflect more profound structural and contextual differences among research communities worldwide. Over the past decade, China has emerged as a global leader in scientific output, becoming the largest producer of peer-reviewed publications across many fields (Cao et al., 2020; Owens, 2024; Tang et al., 2021). This rise is mirrored in the proliferation of bibliometric-supported reviews by China-affiliated researchers, many of which are published in high-impact journals and address issues of national priority such as green energy (Zhao et al., 2020), public health (Zyoud & Al-Jabi, 2020), and digital transformation (X. Ding & Yang, 2022). Simultaneously, researchers from Europe, North America, and other regions—often referred to as non-China-affiliated in global analyses—continue to drive innovation in bibliometric methodologies, promoting frameworks for best practices, reproducibility, and theoretical advancement (Donthu et al., 2021; Linnenluecke et al., 2020; Verma et al., 2021a).

In selecting China-affiliated and non-China-affiliated research communities for comparison, this study focused on the most active and rapidly expanding contributors to bibliometric-supported literature reviews—China, contrasted with the broader international research landscape. China has become a leading producer of bibliometric studies, yet concerns have been raised in the literature about differences in methodological transparency and reporting standards compared with global practices. Framing China as a case study against non-China outputs allows us to examine whether these differences are systematic, while also situating the findings within the wider international context. This approach provides both depth and breadth: depth by focusing on China as a distinctive contributor, and breadth by assessing how its practices align with or diverge from global trends.

These parallel trajectories raise critical questions about how bibliometric tools are being employed and reported across different national and institutional contexts. Are Chinese researchers more likely to adopt specific tools like CiteSpace due to language accessibility and training availability (Liang et al., 2017)? Do non-China-affiliated researchers place greater emphasis on methodological transparency due to editorial policies or disciplinary norms (Mukherjee et al., 2022)? Moreover, how do the thematic priorities of these groups, whether focused on technological innovation, environmental resilience, or social transformation, influence the selection and application of bibliometric methods (Mora et al., 2017; Zhang et al., 2019)?

To date, few studies have systematically compared the methodological behaviors and conceptual orientations of China-affiliated and non-China-affiliated research communities in the domain of bibliometric-supported literature reviews. While some prior research has evaluated bibliometric methods in specific disciplines (Linnenluecke et al., 2020; Moral-Muñoz et al., 2020) or assessed the performance of software tools (Cobo et al., 2011; Verma et al., 2021a), there is a lack of comparative analyses that examine how national affiliation might shape both methodological practices and thematic structures. Given the increasing use of bibliometric analysis to inform evidence-based policymaking, research evaluation, and funding strategies, such an inquiry is both timely and necessary.

This study aims to address the identified gap by conducting a comprehensive comparative bibliometric content analysis of English-language literature review publications authored by researchers affiliated with institutions in China and those affiliated with institutions outside China, covering the period from 2015 to 2024. By focusing on the most highly cited studies from each group (specifically, the top 25 most-cited publications per group), the research seeks to identify differences in the use of bibliometric software tools, the clarity and completeness of methodological reporting, and the acknowledgment of analytical challenges. In addition, the study examines the broader conceptual frameworks of each research community through a co-word network analysis of 4000 highly cited publications (2000 from each group), offering insights into the thematic landscapes that may influence methodological behaviors. The following four core research questions (RQs) guide the investigation:

RQ1: What bibliometric software tools are most frequently used in English-language literature review publications authored by China-affiliated versus non-China-affiliated researchers, and for what analytical purposes?

RQ2: How do the two groups differ in the clarity and completeness of methodological reporting related to bibliometric analysis (e.g., reporting of software versions, threshold settings, inclusion and exclusion criteria, data preprocessing procedures, and bibliographic database used)?

RQ3: What methodological limitations and challenges are most frequently acknowledged in studies from each group, and what recommendations do the authors provide for improving bibliometric methods?

RQ4: What are the dominant conceptual themes in literature reviews authored by China-affiliated versus non-China-affiliated researchers, and how might these thematic priorities help explain differences in methodological practices?

By addressing these questions, this study makes three key contributions. First, it offers a rare comparative lens on bibliometric practices, highlighting similarities and divergences between two globally significant research communities. Second, it contributes to the growing literature on research transparency by identifying methodological gaps and proposing areas for improved reporting standards. Third, it adds a contextual dimension by linking methodological behavior to thematic priorities, offering a more nuanced understanding of how research agendas, disciplinary traditions, and institutional environments influence bibliometric analysis.

Ultimately, this work aligns with broader calls for reflexivity, rigor, and reproducibility in bibliometric scholarship (Bornmann & Leydesdorff, 2014; Zupic & Čater, 2014). As the field of bibliometric-supported literature reviews continues to expand, particularly in the context of meta-research and science of science studies, there is a growing need to understand not only what tools are used but also how and why they are used differently across regions. This study seeks to advance that understanding and to inform future efforts to develop more standardized, inclusive, and context-sensitive bibliometric methodologies.

2. Methodology

2.1. Study Design

This study adopts a comparative bibliometric content analysis to examine the use and reporting of bibliometric software tools in English-language literature review publications. It distinguishes between two groups: studies authored by researchers affiliated with institutions in China and those written by researchers affiliated with institutions outside China, based on the affiliation of the first author(s). This comparative framework facilitates a detailed examination of potential differences in software usage patterns, the transparency of methodological reporting, the acknowledgement of methodological challenges, and the underlying conceptual structures within each research community.

To achieve both analytical depth and thematic breadth, the study employs a two-phase design. In Phase 1, a detailed content analysis was conducted on a curated sample of 50 high-impact publications—specifically, the 25 most-cited literature review articles from each group, published between 2015 and 2024. These influential publications were selected to enable a close examination of widely referenced methodological practices. In Phase 2, a broader conceptual analysis was conducted by performing co-word network mapping on the 2000 most-cited literature review studies from each group. This phase aimed to identify dominant thematic clusters and conceptual orientations, providing a contextual foundation for interpreting the methodological practices observed in Phase 1. Together, this dual-phase design supports the investigation of four core research questions related to (1) tool usage, (2) methodological reporting standards, (3) commonly acknowledged challenges, and (4) conceptual priorities shaping bibliometric-supported literature reviews.

2.2. Data Source and Search Strategy

All data for this study were retrieved from the Scopus database, a widely recognized and multidisciplinary citation index known for its comprehensive coverage of peer-reviewed literature. Scopus was selected because it indexes over 25,000 active journals and provides detailed metadata across various fields, including author affiliations, document types, and language, allowing for precise filtering of both China-affiliated and non-China-affiliated publications (Elsevier, 2023). Compared to other indexing platforms, such as Web of Science, Scopus offers broader coverage of Asian journals and more advanced search capabilities, making it particularly well-suited for bibliometric studies focused on global comparisons (Mongeon & Paul-Hus, 2016). Additionally, Scopus enables the export of structured data in formats compatible with widely used bibliometric software tools, including VOSviewer, CiteSpace, and Bibliometrix, thereby supporting data interoperability and reproducibility (Donthu et al., 2021). Given the aim of evaluating how bibliometric software tools are employed in literature reviews, Scopus provided the necessary breadth and functionality to construct two comparable and high-quality corpora for analysis. This ensured the robustness of both the sampling and the methodological conclusions drawn from the data.

2.3. Search Strategy, Screening, and Final Sample Selection

To build a rigorous and comparable dataset for examining the use and reporting of bibliometric software tools in literature reviews, a structured and reproducible search strategy was implemented on 18 July 2025. The Scopus database was selected for this purpose due to its comprehensive indexing of peer-reviewed research and robust metadata functionalities. Two parallel queries were conducted to generate two distinct but comparable corpora: one consisting of studies authored by researchers affiliated with institutions in China, and the other comprising studies from all other countries. Identical keyword strings were applied to both searches to ensure thematic alignment. The search targeted publications that explicitly used bibliometric analysis within the context of literature reviews and were published between 2015 and 2024 in English. Document types were limited to journal articles, reviews, and conference papers.

Query 1—China-Affiliated Publications: This search filtered for documents where at least the first author was affiliated with a Chinese institution. It returned 17,727 records. After applying filters for English language and document type, 3715 records met the inclusion criteria. From this pool, the top 2000 most-cited studies were selected to support the conceptual structure analysis (RQ4), and the top 25 most-cited were retained for full-text content analysis (RQ1–RQ3). Search string: TITLE-ABS-KEY ((“bibliometric analysis”) AND (“literature review” OR review)) AND PUBYEAR > 2014 AND PUBYEAR < 2025 AND (LIMIT-TO (DOCTYPE, “ar”) OR LIMIT-TO (DOCTYPE, “re”) OR LIMIT-TO (DOCTYPE, “cp”)) AND (LIMIT-TO (AFFILCOUNTRY, “China”)) AND (LIMIT-TO (LANGUAGE, “English”)).

Query 2—Non-China-Affiliated Publications: This query excluded studies conducted within China. It also yielded 17,727 records, of which 9202 remained after filtering by language and document type. As with the first group, the top 2000 most-cited studies were selected for co-word analysis, and the top 25 were used for qualitative content analysis. Search string: TITLE-ABS-KEY ((“bibliometric analysis”) AND (“literature review” OR review)) AND PUBYEAR > 2014 AND PUBYEAR < 2025 AND (LIMIT-TO (DOCTYPE, “ar”) OR LIMIT-TO (DOCTYPE, “re”) OR LIMIT-TO (DOCTYPE, “cp”)) AND (EXCLUDE (AFFILCOUNTRY, “China”)) AND (LIMIT-TO (LANGUAGE, “English”)).

This tiered selection approach served two core analytical purposes: Regarding conceptual structure mapping (RQ4), the top 2000 most-cited studies from each group were used for co-word analysis to explore dominant research themes and intellectual structures, and the results are presented here (Ali Abaker Omer & Dong, 2025). Then, for qualitative content analysis (RQ1–RQ3), the top 25 most-cited studies from each group were reviewed in full to extract detailed information on software tools, methodological practices, and reported limitations.

For RQ1–RQ3, we focused on the top 25 most-cited studies in each group. This threshold was chosen to enable a detailed, qualitative assessment of methodological reporting practices, which requires a manageable sample size for close reading and comparison. By contrast, for RQ4, the study expanded the dataset to the top 2000 most-cited studies per group to ensure statistical robustness and thematic breadth in the co-word analysis. This dual-threshold strategy thus balances the need for in-depth qualitative insight with the scale required for reliable conceptual mapping.

This dual-phase strategy ensured both analytical depth, through an intensive review of high-impact papers, and conceptual breadth, by mapping the broader thematic landscape. The citation-based sampling adds methodological rigor by focusing on influential studies that have demonstrably shaped the field. A PRISMA-style summary of this search and selection process is presented in

Table 1. PRISMA-style summary of the search and screening process.

Stage	China-Affiliated Studies	Non-China-Affiliated Studies	Description
Identification	17,727 records retrieved	17,727 records retrieved	Initial records were identified from the Scopus database using the same keyword query, focused on literature reviews employing bibliometric analysis: (“bibliometric analysis”) and (“literature review” OR “review”).
Screening	3715 studies retained	9202 studies retained	Records were filtered to include only English-language journal articles, reviews, and conference papers. Editorials, letters, notes, and non-peer-reviewed documents were excluded. China includes (China-affiliated) and excludes (non-China-affiliated).
Eligibility—citation ranking	Top 2000 most-cited studies	Top 2000 most-cited studies	For conceptual structure analysis (RQ4), the 2000 most-cited studies in each group were selected based on total citations.
Inclusion—content analysis	Top 25 most-cited studies	Top 25 most-cited studies	For qualitative content analysis (RQ1–RQ3), the top 25 most-cited studies in each group were reviewed in full text to extract data on software tools, methodological reporting, and methodological limitations and recommendations.

to ensure transparency and reproducibility (Page et al., 2021).

2.4. Comparative Analysis of Methodological Practices (RQ1–RQ3)

To address RQs 1 through 3, this study conducted a structured comparative content analysis of the 50 most-cited literature review studies, 25 authored by China-affiliated researchers and 25 by non-China-affiliated researchers. The analysis focused on three key dimensions of methodological practice: the bibliometric tools employed and their stated analytical purposes (RQ1); the transparency and completeness of methodological reporting (RQ2); and the extent to which studies acknowledged limitations or offered recommendations for improving future bibliometric research (RQ3).

For RQ1, the study recorded which bibliometric software tools (e.g., VOSviewer, CiteSpace, or Bibliometrix) were used in each study, along with the specific analytical purposes they served, such as co-authorship mapping, keyword co-occurrence, citation network analysis, or trend evolution.

For RQ2, the study evaluated the clarity and rigor of methodological reporting across five indicators:

• Whether the software version was explicitly reported;
• Whether threshold parameters (e.g., minimum citation counts and keyword frequency) were defined;
• Whether the inclusion and exclusion criteria for source documents were clearly articulated;
• Whether data preprocessing procedures—such as name unification, abbreviation handling, or data cleaning—were described;
• Whether the source database (e.g., Scopus or Web of Science) was identified and justified.

These indicators enabled the assessment of the degree of transparency and reproducibility in methodological design, which is essential for advancing best practices in bibliometric-supported literature reviews.

For RQ3, the study examined whether studies explicitly acknowledged methodological limitations, categorizing them into four broad areas:

• Software functionality constraints (e.g., limited clustering algorithms or visualization tools);
• Data limitations (e.g., incomplete citation coverage or disciplinary bias);
• Analytical subjectivity (e.g., author decisions in parameter selection);
• Issues related to citation normalization or field variance (e.g., differing citation behaviors across disciplines).

The study also recorded any forward-looking recommendations made by the authors for improving bibliometric methods, enhancing transparency, or addressing known limitations. This structured approach to comparing China-affiliated and non-China-affiliated studies provides insights into both the standard practices and methodological disparities that shape the current landscape of bibliometric literature reviews. These findings are presented in detail in Section 3.

3. Results

This section presents the results obtained from the analysis of 50 highly cited literature review publications—25 authored by researchers affiliated with Chinese institutions and 25 by those from institutions outside China. At the same time, the analysis includes 4000 studies, 2000 for each group, as presented in (Ali Abaker Omer & Dong, 2025). The findings are structured according to the four guiding research questions (RQ1–RQ4), focusing on the usage of bibliometric tools, methodological practices, reported limitations, and the conceptual structure.

3.1. Most-Cited Studies by China-Affiliated and Non-China-Affiliated Authors

To establish a foundational understanding of influential contributions in the field of bibliometric-supported literature reviews, this study first identified the 25 most-cited publications authored by researchers affiliated with Chinese institutions and the 25 most-cited publications authored by researchers affiliated with institutions outside China. Citation data were extracted from the Scopus database as of 18 July 2025. The selection was guided by total citation count—a proxy for scholarly impact—within the defined timeframe (2015–2024). For each study, bibliographic information was documented, including its rank, total citations, title, and corresponding reference. This ranking provides a lens through which to interpret prevailing trends, research foci, and the institutional contexts shaping methodological norms in bibliometric analyses.

Table 2. Top 25 most-cited literature review studies authored by China-affiliated and non-China-affiliated researchers (2015–2024). The table provides a side-by-side comparison of the 25 most-cited literature review publications from each group. Rankings, total citations (TC), study titles, and references are included to highlight key differences in citation impact and thematic focus between the two research communities.

Top 25 Most-Cited Studies Authored by China-Affiliated Researchers				Top 25 Most-Cited Studies Authored by Non-China-Affiliated Researchers
Rank	TC 1	Study Title	Reference	Rank	TC 1	Study Title	Reference
1	725	Comprehensive utilization and environmental risks of coal gangue: A review	(Li & Wang, 2019)	1	6538	How to conduct a bibliometric analysis: An overview and guidelines	(Donthu et al., 2021)
2	560	Supply chain finance: A systematic literature review and bibliometric analysis	(X. Xu et al., 2018)	2	5200	The Circular Economy—A new sustainability paradigm?	(Geissdoerfer et al., 2017)
3	552	A comprehensive review on food waste anaerobic digestion: Research updates and tendencies	(Ren et al., 2018)	3	1955	Defining urban resilience: A review	(Meerow et al., 2016)
4	461	A bibliometric analysis on green finance: Current status, development, and future directions	(Zhang et al., 2019)	4	1558	Green supply chain management: A review and bibliometric analysis	(Fahimnia et al., 2015)
5	425	Supply chain collaboration for Sustainability: A literature review and future research agenda	(L. Chen et al., 2017)	5	1375	Web of Science (WoS) and Scopus: the titans of bibliographic information in today’s academic world	(Pranckutė, 2021)
6	413	Bibliometric analysis of safety culture research	(van Nunen et al., 2018)	6	1108	Software tools for conducting bibliometric analysis in science: An up-to-date review	(Moral-Muñoz et al., 2020)
7	410	Research advances of magnesium and magnesium alloys worldwide in 2021	(Song et al., 2022)	7	1004	Internet of things and supply chain management: a literature review	(Ben-Daya et al., 2019)
8	399	Corporate social responsibility for supply chain management: A literature review and bibliometric analysis	(Feng et al., 2017)	8	976	Conducting systematic literature reviews and bibliometric analyses	(Linnenluecke et al., 2020)
9	396	Is Metaverse in education a blessing or a curse: a combined content and bibliometric analysis	(Tlili et al., 2022)	9	822	Deep learning for healthcare applications based on physiological signals: A review	(Faust et al., 2018)
10	387	Bibliometric studies in tourism	(Koseoglu et al., 2016)	10	738	Artificial intelligence and business models in the sustainable development goals perspective: A systematic literature review	(Di Vaio et al., 2020)
11	384	Two Decades of Artificial Intelligence in Education: Contributors, Collaborations, Research Topics, Challenges, and Future Directions	(X. Chen et al., 2022)	11	671	Financial literacy: A systematic review and bibliometric analysis	(Goyal & Kumar, 2021)
12	366	Smart Manufacturing and Intelligent Manufacturing: A Comparative Review	(B. Wang et al., 2021)	12	655	Guidelines for advancing theory and practice through bibliometric research	(Mukherjee et al., 2022)
13	357	Knowledge mapping of platform research: a visual analysis using VOSviewer and CiteSpace	(X. Ding & Yang, 2022)	13	583	Social innovation research: An emerging area of innovation studies?	(van der Have & Rubalcaba, 2016)
14	332	Application of artificial intelligence to wastewater treatment: A bibliometric analysis and systematic review of technology, economy, management, and wastewater reuse	(Zhao et al., 2020)	14	582	A Comprehensive Review on NSGA-II for Multi-Objective Combinatorial Optimization Problems	(Verma et al., 2021a)
15	293	Disruption risks in supply chain management: a literature review based on bibliometric analysis	(S. Xu et al., 2020)	15	574	Conducting systematic literature review in operations management	(Thomé et al., 2016)
16	291	Anaerobic digestion: An alternative resource treatment option for food waste in China	(Jin et al., 2021)	16	553	What is the bioeconomy? A review of the literature	(Bugge et al., 2016)
17	291	Integration of BIM and GIS in sustainable built environment: A review and bibliometric analysis	(H. Wang et al., 2019)	17	493	Freshwater plastic pollution: Recognizing research biases and identifying knowledge gaps	(Blettler et al., 2018)
18	278	Nanomaterials for treating emerging contaminants in water by adsorption and photocatalysis: Systematic review and bibliometric analysis	(Zhao et al., 2018)	18	491	The First Two Decades of Smart-City Research: A Bibliometric Analysis	(Mora et al., 2017)
19	263	Smart logistics based on the internet of things technology: an overview	(Y. Ding et al., 2021)	19	480	A review on Environmental Kuznets Curve hypothesis using bibliometric and meta-analysis	(Sarkodie & Strezov, 2019)
20	256	Integrating blockchain technology into the energy sector—From theory of blockchain to research and application of energy blockchain	(Q. Wang & Su, 2020)	20	459	Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis	(Goodell et al., 2021)
21	255	Study of acupuncture for low back pain in recent 20 years: A bibliometric analysis via CiteSpace	(Liang et al., 2017)	21	445	Artificial intelligence in marketing: Systematic review and future research direction	(Verma et al., 2021b)
22	255	Research advances of magnesium and magnesium alloys worldwide in 2022	(Yang et al., 2023)	22	426	A bibliometric analysis of board diversity: Current status, development, and future research directions	(Kent Baker et al., 2020)
23	249	Reviewing two decades of cleaner alternative marine fuels: Towards IMO’s decarbonization of the maritime transport sector	(Ampah et al., 2021)	23	416	Bibliometrics of social media research: A co-citation and co-word analysis	(Leung et al., 2017)
24	247	12 years roadmap of the sulfur cathode for lithium sulfur batteries (2009–2020)	(Liu et al., 2020)	24	377	Drones in agriculture: A review and bibliometric analysis	(Rejeb et al., 2022)
25	240	Research on biomass energy and environment from the past to the future: A bibliometric analysis	(Mao et al., 2018)	25	377	Hydrogen energy storage integrated hybrid renewable energy systems: A review analysis for future research directions	(Arsad et al., 2022)

¹ Total citations.

(left) presents the ranked list of China-affiliated studies. These publications span a diverse array of topics, reflecting the interdisciplinary application of bibliometric tools in emerging and rapidly evolving fields. Thematically, these studies cover green finance, smart manufacturing, safety culture, sustainable supply chains, artificial intelligence in education, digital twins, and blockchain integration. The top-ranked study, cited 725 times, provides a comprehensive review of coal gangue utilization and associated environmental risks, highlighting how environmental and industrial topics dominate high-impact bibliometric research within the China-affiliated cohort.

Table 2 (right) displays the 25 most-cited studies authored by non-China-affiliated researchers. These works demonstrate considerable scholarly influence, with citation counts in the top tier far exceeding those of the China-affiliated group. The most-cited publication—“How to conduct a bibliometric analysis: An overview and guidelines”—has been cited over 6500 times and is frequently referenced as a foundational guide for researchers employing bibliometric methods. Other highly cited studies address key global topics such as the circular economy, urban resilience, smart cities, hydrogen energy, and bibliometric software itself. This diversity highlights the centrality of bibliometric-supported reviews in informing strategic planning, interdisciplinary innovation, and evidence-based policy.

A comparative analysis of the two groups reveals significant differences. Citation counts among the non-China-affiliated publications are significantly higher at the upper end of the distribution, indicating earlier publication dates and longer citation windows in many cases. For example, several of the top-cited non-China studies were published between 2015 and 2018, offering ample time for citation accumulation. By contrast, many China-affiliated studies have been published more recently (since 2020), suggesting a rapidly growing engagement with bibliometric analysis methods among Chinese scholars, especially in the post-COVID research era.

Beyond citation metrics, these ranked lists function as critical reference points for the methodological review conducted in later sections of this study. They not only identify widely recognized contributions but also illuminate recurring thematic priorities, research networks, and the software tools most adopted in each scholarly community. Furthermore, they provide a basis for evaluating differences in methodological transparency, tool usage practices, and the articulation of analytical challenges across institutional contexts.

3.2. Comparative Use of Bibliometric Software Tools and Their Analytical Purposes (RQ1)

This section presents a comprehensive comparative analysis addressing RQ1. To answer this question, the study examined the top 25 most-cited literature review studies from each group between 2015 and 2024. The analysis focused on both the frequency of software tool usage and the stated or implied analytical functions these tools performed. The results reveal significant trends that reflect broader epistemic cultures, methodological preferences, and infrastructure differences between the two communities.

3.2.1. Bibliometric Software Use in China-Affiliated Studies

Among the China-affiliated studies, several tools emerged as dominant, with VOSviewer (n = 10) and CiteSpace (n = 7) occupying the top positions. These tools were often used in conjunction with or in tandem with complementary software, such as Gephi (n = 6), BibExcel (n = 4), and Excel (n = 4). A notable feature of this group is the multi-tool approach, wherein researchers integrated multiple platforms across the bibliometric pipeline—from data preprocessing and extraction to visualization and trend interpretation (Figure 1a).

VOSviewer was widely adopted for its robust visualization capabilities in co-word and co-authorship mapping. Studies employed it to generate network diagrams reflecting thematic clusters, intellectual structures, and keyword co-occurrence matrices. Its strength in semantic mapping, coupled with density and overlay visualizations, made it particularly suitable for constructing the conceptual landscape of rapidly evolving fields (X. Ding & Yang, 2022; Feng et al., 2017; Liang et al., 2017; Liu et al., 2020; Song et al., 2022; Tlili et al., 2022; van Nunen et al., 2018; B. Wang et al., 2021; H. Wang et al., 2019; S. Xu et al., 2020). CiteSpace was valued for its ability to detect temporal patterns and emerging hotspots in citation networks. China-affiliated authors used it to trace the evolution of research fronts, identify citation bursts, and cluster references based on similarity measures and modularity (L. Chen et al., 2017; X. Ding & Yang, 2022; Liu et al., 2020; Q. Wang & Su, 2020; S. Xu et al., 2020; Yang et al., 2023; Zhang et al., 2019). The software’s timeline and dual-map overlay features facilitated the exploration of interdisciplinary linkages and intellectual transitions, especially in policy-driven and high-growth research domains. Gephi, though less frequently reported, played a significant role in visualizing large, complex networks such as institutional collaborations or international co-authorships. The tool’s layout algorithms, such as ForceAtlas2 and Yifan Hu, allowed researchers to represent network centrality, density, and community structures. In several cases, Gephi was used after BibExcel preprocessing, which highlights the importance of a structured workflow (Ampah et al., 2021; X. Chen et al., 2022; Feng et al., 2017; X. Xu et al., 2018; Zhao et al., 2020, 2018).

BibExcel served primarily as a preprocessing tool to clean and reformat data for compatibility with other software. While it does not generate visualizations independently, its utility lies in enabling citation analysis, keyword filtering, and frequency counts before network construction (Feng et al., 2017; Koseoglu et al., 2016; Mao et al., 2018; X. Xu et al., 2018). Moreover, a broader suite of tools—including Pajek, UCINET, Python, IBM SPSS Statistics (23), SCI2, and SATI—was used sporadically but significantly. These were often adopted for advanced clustering, statistical validation, or topic modeling, signaling a growing interest in combining bibliometric mapping with quantitative and qualitative validation techniques. An interesting trend in this group is the frequent use of custom or regional tools like HitSite™ or DDA, which are rarely cited in international publications but may reflect regional training environments or institutional preferences.

Additionally, two studies (Y. Ding et al., 2021; Li & Wang, 2019) conducted bibliometric analyses manually, without citing any specific software. This practice highlights the transitional nature of methodological literacy in certain research circles. Overall, the China-affiliated studies demonstrated a methodologically hybrid and tool-integrated approach, reflecting both the influence of domestic bibliometric traditions and an increasing awareness of global methodological standards.

3.2.2. Bibliometric Software Use in Non-China-Affiliated Studies

In contrast, the non-China-affiliated studies showed a distinct pattern in tool adoption and reporting. The most frequently used software was Gephi (n = 11), followed by VOSviewer (n = 10), BibExcel (n = 7), and Bibliometrix (R package) (n = 5). Other tools, including Pajek, Sci2, Leximancer, and CiteSpace, appeared less frequently and were often applied for specific analytical tasks (Figure 1b).

Gephi’s prominence in this group underscores its central role in network visualization and exploratory data analysis. Researchers employed it to construct and analyze co-citation networks, author collaboration networks, and conceptual clusters, often applying modularity-based clustering to uncover latent thematic structures (Ben-Daya et al., 2019; Donthu et al., 2021; Fahimnia et al., 2015; Goodell et al., 2021; Goyal & Kumar, 2021; Kent Baker et al., 2020; Linnenluecke et al., 2020; Meerow et al., 2016; Mora et al., 2017; Rejeb et al., 2022; van der Have & Rubalcaba, 2016). Several studies leveraged Gephi’s metrics (e.g., degree centrality, betweenness, and PageRank) to interpret the influence of nodes (authors, keywords, or publications) within the network. VOSviewer, while equally common, was often used for a narrower set of functions—mainly co-word analysis, keyword co-occurrence visualization, and bibliographic coupling. Its ease of use, downloadable outputs, and built-in clustering algorithms made it a preferred choice for visualizing large bibliometric datasets without advanced programming skills (Faust et al., 2018; Goodell et al., 2021; Goyal & Kumar, 2021; Kent Baker et al., 2020; Leung et al., 2017; Meerow et al., 2016; Mukherjee et al., 2022; Rejeb et al., 2022; Sarkodie & Strezov, 2019; Verma et al., 2021b). BibExcel again served as a foundational tool for data formatting and generating citation matrices. Its ability to preprocess large volumes of RIS or BibTeX data made it indispensable for studies relying on external visualization tools like Gephi or Pajek (Ben-Daya et al., 2019; Fahimnia et al., 2015; Kent Baker et al., 2020; Leung et al., 2017; Meerow et al., 2016; Rejeb et al., 2022). Bibliometrix, uniquely, was employed as both an analytic and a visualization tool. Unlike standalone software, this R-based package allowed researchers to conduct performance analysis, science mapping, and trend analysis within a single computational framework. The integration with Biblioshiny provided an accessible interface for non-programmers, expanding the methodological repertoire of researchers who sought reproducibility and custom analysis pipelines (Blettler et al., 2018; Assunta Di Vaio et al., 2020; Goodell et al., 2021; Moral-Muñoz et al., 2020; Verma et al., 2021b).

Notably, non-China-affiliated studies also demonstrated thematic experimentation with tools such as Leximancer, CRExplorer, and HistCite—each used for specialized content analyses, citation disambiguation, or historiographic mapping. Sci² and CiteSpace, while less common in this group, were used in a few studies for burst detection and longitudinal network exploration (Moral-Muñoz et al., 2020; van der Have & Rubalcaba, 2016). A distinguishing characteristic of the non-China-affiliated group is the greater methodological transparency in tool reporting. Most studies explicitly stated the software version, provided clear justifications for tool selection, and often outlined limitations related to software functionality or citation normalization. Furthermore, these studies tended to adopt modular tool workflows, separating preprocessing (BibExcel and Excel), analysis (Gephi and Bibliometrix), and visualization (VOSviewer) into discrete stages, often guided by reproducible code or scripts.

3.2.3. Cross-Group Comparison and Reflections

Comparing the two groups reveals both convergence and divergence in the usage of bibliometric tools. Convergent trends include a reliance on a core set of tools—Gephi, VOSviewer, BibExcel, and Bibliometrix—each used for network-based and performance-based bibliometric analyses. However, the divergences are more revealing:

China-affiliated researchers tend to adopt multi-tool ecosystems with a strong emphasis on CiteSpace and hybrid visualization strategies, often integrating 4–5 tools in layered pipelines. Their workflows emphasize network dynamics, temporal clustering, and the detection of research frontiers.

Non-China-affiliated researchers, by contrast, emphasize modular and transparent workflows, relying heavily on Gephi and Bibliometrix for structured analysis. They are also more likely to acknowledge tool limitations and justify methodological choices explicitly.

Structural, linguistic, and institutional factors may shape these patterns. For instance, the popularity of CiteSpace among Chinese researchers may reflect the availability of Chinese-language documentation and training. On the other hand, the global appeal of Bibliometrix, with its open-source ethos and R-based architecture, aligns with the transparency and reproducibility movement that characterizes much of Western scientific practice. Ultimately, the comparative analysis highlights the growing methodological sophistication across both communities and underscores the importance of aligning tool selection with research objectives, data characteristics, and transparency norms.

3.3. Methodological Reporting and Transparency (RQ2)

This section presents a comparative analysis of methodological transparency in China-affiliated and non-China-affiliated literature review studies, with a specific focus on five key indicators: (1) whether the software version used was explicitly reported, (2) whether threshold settings were specified, (3) whether inclusion and exclusion criteria were clearly defined, (4) whether data preprocessing procedures were described, and (5) whether the bibliographic database was identified. These indicators were selected not only because they are critical for methodological rigor but also because they reflect foundational elements of reproducibility in bibliometric research.

3.3.1. Clarity and Transparency in Methodological Reporting in China-Affiliated Studies

This section assesses the clarity and transparency with which methodological details are reported in China-affiliated literature review studies that employ bibliometric analysis.

Table 3. (a) Summary of methodological transparency in China-affiliated publications (n = 25). (b) Summary of methodological transparency in non-China-affiliated publications (n = 25). (c) Comparative summary of methodological transparency between the two groups (n = 25). (d) Comparative summary of methodological dimensions in China- and non-China-affiliated studies.

(a)
Reporting Item	Yes (Count, %)	No (Count, %)
Software version explicitly reported	6 (24%)	19 (76%)
Threshold settings specified	15 (60%)	10 (40%)
Inclusion and exclusion criteria are defined	25 (100%)	0 (0%)
Data preprocessing described	25 (100%)	0 (0%)
Bibliographic database identified	25 (100%)	0 (0%)
(b)
Reporting Indicator	Yes (Count, %)	No (Count, %)
Software version explicitly reported	5 (20%)	20 (80%)
Threshold settings specified	19 (76%)	6 (24%)
Inclusion and exclusion criteria are defined	25 (100%)	0 (0%)
Data preprocessing procedures are described	23 (92%)	2 (8%)
Bibliographic database identified	25 (100%)	0 (0%)
(c)
Indicator	China-Affiliated (Yes %)	Non-China-Affiliated (Yes %)
Software version explicitly reported	24% (6/25)	20% (5/25)
Threshold settings specified	60% (15/25)	76% (19/25)
Inclusion and exclusion criteria are defined	100% (25/25)	100% (25/25)
Data preprocessing procedures are described	100% (25/25)	92% (23/25)
Bibliographic database identified	100% (25/25)	100% (25/25)
(d)
Dimension	China-Affiliated Studies	Non-China-Affiliated Studies
Most-used tools	Predominantly VOSviewer, CiteSpace, and occasionally Gephi	Primarily Gephi, VOSviewer, and Bibliometrix, with more frequent use of hybrid tools
Reporting transparency	Strong in inclusion/exclusion criteria and preprocessing, but weak on software versioning and thresholds	Strong emphasis on threshold settings and justification of data sources, but weak on software versioning
Limitations acknowledged	Focused on data scope (e.g., database limitations) and publication types; limited attention to software constraints or interpretive bias	Broad coverage of limitations, including software functionality, data quality, analytical subjectivity, and field-normalization issues
Recommendations	Emphasis on expanding content scope and database coverage; relatively fewer methodological suggestions	Advocates for tool triangulation, mixed-method approaches, and improvements in citation normalization
Conceptual themes	Dominated by AI applications, sustainability, public health, and traditional Chinese medicine	Focused on Sustainable Development Goals (SDGs), digital transformation, innovation, and education systems

a shows the summary of results of clarity and transparency in methodological reporting for China-affiliated publications.

• Reporting of software version: Among the 25 China-affiliated studies examined, a significant number failed to report the version of the bibliometric software employed explicitly. Only 6 out of 25 studies (24%) mentioned the version used, which includes the explicit reporting of Microsoft Excel 2016 (Ren et al., 2018) and 2018 (Jin et al., 2021), CiteSpace v3.8.R5 (L. Chen et al., 2017), IBM SPSS Statistics (23) (van Nunen et al., 2018), and Gephi 0.8.2 (Zhang et al., 2019; Zhao et al., 2020). In most studies (76%), the absence of version reporting raises concerns about reproducibility, particularly when different versions of tools like VOSviewer or CiteSpace can yield varying visualizations or clustering outputs due to algorithmic updates.
• Specification of threshold settings: A more encouraging pattern emerges regarding threshold reporting. Fifteen studies (60%) specified threshold parameters relevant to their analyses. These thresholds included citation count cutoffs for co-citation analysis (Ampah et al., 2021; X. Ding & Yang, 2022; Feng et al., 2017; van Nunen et al., 2018; S. Xu et al., 2020; X. Xu et al., 2018), and statistical analysis of countries and regions, as well as organizations, based on a minimum threshold of published papers (Song et al., 2022). For term selection in topic analysis, a term frequency–inverse document frequency (TF-IDF) with a threshold of 0.05 (X. Chen et al., 2022), frequency filters for co-word or keyword analyses (Ampah et al., 2021; X. Ding & Yang, 2022; B. Wang et al., 2021; H. Wang et al., 2019; S. Xu et al., 2020), and co-authorship analysis (Liu et al., 2020) were used. For the statistical analysis of countries and regions, a threshold of “at least 5 Mg publications in 2022” was applied (Liang et al., 2017), and parameters for clustering or term pruning were used (Yang et al., 2023). Such detail provides critical transparency for understanding how data was filtered or visualized. However, in 10 studies (40%), threshold settings were either omitted or vaguely referenced without quantitative justification (Y. Ding et al., 2021; Jin et al., 2021; Koseoglu et al., 2016; Li & Wang, 2019; Mao et al., 2018; Ren et al., 2018; Tlili et al., 2022; Q. Wang & Su, 2020; Zhang et al., 2019; Zhao et al., 2020).
• Definition of inclusion and exclusion criteria: There is strong consistency across the sample in defining inclusion and exclusion criteria. All 25 studies (100%) clearly articulated the parameters for selecting relevant literature. Typical inclusion criteria encompassed publication date ranges, document types (e.g., peer-reviewed articles and reviews), database scope (e.g., Web of Science Core Collection), and topical relevance (e.g., the presence of keywords in titles, abstracts, or the full text). Exclusion criteria often targeted conference papers, non-English texts, irrelevant disciplines, or duplicate records (Koseoglu et al., 2016; Liu et al., 2020; X. Xu et al., 2018; Zhang et al., 2019).
• Description of data preprocessing procedures: Similarly, all 25 studies (100%) offered some description of their data preprocessing workflows. These descriptions ranged from simple keyword merging and duplicate removal (Ren et al., 2018; H. Wang et al., 2019) to more sophisticated processes involving multi-stage screening, Python-based data transformation, or normalization of institutional names and terms (Ampah et al., 2021; X. Chen et al., 2022; X. Xu et al., 2018). While not all studies elaborated on every step of preprocessing (e.g., stop-word removal and stemming), most outlined how raw bibliographic data were transformed into analyzable formats, enhancing procedural transparency.
• Identification of bibliographic databases: Every study in the China-affiliated group identified the source(s) of bibliographic data, fulfilling this reporting criterion. The WoS emerged as the most frequently used database, cited in nearly all cases. Some studies employed CNKI (Li & Wang, 2019), Scopus (X. Xu et al., 2018), or ERIC (X. Chen et al., 2022) as complementary sources. The clarity in database selection allows readers to assess the scope of coverage, disciplinary bias, and indexing limitations of the analyses.

These findings indicate that while China-affiliated bibliometric studies consistently excel in defining data selection boundaries and preprocessing steps, they often underreport software version details—a critical factor for methodological transparency and reproducibility. Moreover, although most studies specify threshold settings, there remains room for improved justification and standardization. To enhance clarity and methodological rigor in bibliometric-supported reviews, future studies should adopt best practices in software reporting, including version control and parameter logs, as part of their supplementary materials or appendices.

3.3.2. Clarity and Transparency in Methodological Reporting in Non-China-Affiliated Studies

This analysis investigates the transparency and rigor with which non-China-affiliated researchers report their bibliometric methodologies in literature review studies. Table 3b shows the summary of results of clarity and transparency in methodological reporting for non-China-affiliated publications.

• Software version reporting: In contrast to best practices for reproducibility, only 5 out of 25 studies (20%) explicitly stated the version of the bibliometric software or tools used. Examples include studies that report the use of the analyzed version for each software tool. For instance, CRExplorer 1.9 (2018), Publish or Perish 7 (2019), ScientoPyUI 1.4.0 (2019), BibExcel 2017 (2017), Biblioshiny (2019), BiblioMaps 3.2 (2018), CiteSpace 5.5.R2 (2019), CitNetExplorer 1.0.0 (2014), SciMAT 1.1.04 (2016), Sci² Tool 1.3 (2018), and VOSviewer 1.6.13 (2019) are listed. Similarly, library versions like Bibliometrix R-3.6.1 (2019) are provided (Moral-Muñoz et al., 2020). The version of the Shiny package is reported as 1.2.0. For R, the study implicitly refers to ‘R Core Team, 2019’ and ‘RStudio Team, 2019’ for RStudio, suggesting the latest stable versions available at the time of writing (Linnenluecke et al., 2020). VOSviewer (last accessed 11 December 2017) from http://www.vosviewer.com/ (Faust et al., 2018), VOSviewer 1.6.16 (Rejeb et al., 2022), and UCINET 6 (Bugge et al., 2016) were also mentioned. The vast majority—20 studies (80%)—referred to tools such as VOSviewer, Gephi, or Bibliometrix without detailing their version, making it difficult for future researchers to replicate results with precision. The lack of versioning documentation remains a significant barrier to reproducibility, especially considering software updates often alter algorithmic behavior and visual output.
• Threshold settings specification: A strong performance is observed regarding threshold transparency: 19 studies (76%) specified threshold settings used in their bibliometric analyses. These thresholds include minimum citation counts for co-citation inclusion, keyword co-occurrence frequencies, clustering or visualization cutoffs, etc. Despite some variation in how thresholds are justified, their explicit reporting enhances transparency and allows others to understand the basis for thematic and structural delineations. Only six studies (24%) omitted threshold details entirely (Ben-Daya et al., 2019; Di Vaio et al., 2020; Donthu et al., 2021; Geissdoerfer et al., 2017; Moral-Muñoz et al., 2020; Mukherjee et al., 2022).
• Inclusion and exclusion criteria: The definition of inclusion and exclusion criteria was particularly robust. All 25 studies (100%) offered clearly defined criteria, often outlining publication timeframes, language filters, source types (e.g., peer-reviewed journals), and domain relevance. These criteria were frequently detailed in either the methodology sections or supplementary tables. A few studies applied more complex criteria, such as snowballing techniques or multi-stage filtering, reinforcing methodological thoroughness.
• Data preprocessing procedures: A large majority of studies, 23 out of 25 (92%), described their data preprocessing steps, such as duplicate removal, keyword normalization, author disambiguation, and transformation of RIS/BibTeX files into analyzable formats. In some cases, studies also detailed the creation of thesaurus files or the use of stemming methods to harmonize terminology before keyword co-occurrence analyses. Only two studies (8%) failed to describe preprocessing steps beyond general references to data “cleaning” or “filtering” (Geissdoerfer et al., 2017; Pranckutė, 2021).
• Bibliographic database identification: All 25 studies (100%) explicitly identified the bibliographic databases used for data collection, demonstrating full compliance with this fundamental reporting standard. The most frequently cited databases included Web of Science, Scopus, and occasionally Google Scholar or ERIC, depending on the domain, etc. Several studies justified their database selection based on indexing scope, citation quality, or disciplinary relevance.

Non-China-affiliated studies consistently demonstrate high methodological transparency, particularly in defining inclusion/exclusion criteria, as well as in reporting bibliographic databases. Moreover, most studies provide detailed threshold settings and preprocessing descriptions, supporting reproducibility and interpretability. However, as with the China-affiliated group, the lack of consistent software version reporting remains a methodological weakness that future bibliometric research must address. Explicit version tracking—especially for tools with frequent updates—is essential for scholarly rigor. Together, these findings highlight a global need for standardized methodological reporting protocols in bibliometrically supported literature reviews. While non-China-affiliated studies generally perform well in this regard, there remains room for improvement in reproducibility practices, particularly concerning tool documentation.

3.3.3. Comparative Analysis of Methodological Transparency in China-Affiliated and Non-China-Affiliated Studies

Methodological transparency is fundamental to the credibility and reproducibility of bibliometric-supported literature reviews. To assess this transparency, this study evaluated and compared 25 China-affiliated and 25 non-China-affiliated publications across five core indicators. This comparison reveals both convergences and marked divergences in reporting practices between the two research communities. Table 3c shows a summary of the comparative clarity and transparency in methodological reporting of the two groups.

• Software version reporting: One of the most striking differences between the two groups lies in the reporting of software versions. Only 24% (6/25) of the China-affiliated studies explicitly reported the version of bibliometric software used, compared to an even lower 20% (5/25) in the non-China-affiliated group. Although both groups fall short of best practices, the slight edge for China-affiliated studies suggests some awareness of the importance of version control, albeit inconsistently applied. In both cohorts, the widespread omission of version reporting reflects a significant barrier to replicability, especially when bibliometric software tools (e.g., CiteSpace, VOSviewer, and BibExcel) are regularly updated with altered algorithms and interface changes.
• Threshold settings specification: Threshold transparency appears more robust in the non-China group, with 76% (19/25) of studies specifying threshold settings, compared to 60% (15/25) in the China group. These thresholds—such as minimum citation counts for inclusion in co-citation networks, frequency filters in keyword co-occurrence, or conditions for cluster inclusion—are essential to understanding and reproducing the analytical logic of the study. The higher reporting rate among non-China-affiliated researchers may reflect greater adherence to emerging standards in science mapping, or it may suggest more frequent engagement with advanced tools that require parameter customization.
• Inclusion and exclusion criteria: Encouragingly, both groups demonstrate exemplary performance in defining inclusion and exclusion criteria, with 100% of studies in both cohorts clearly articulating how publications were selected and filtered. This uniformity underscores a shared recognition of the importance of delimiting the study corpus—an essential step for reducing bias and ensuring thematic coherence. Criteria typically included publication type (e.g., peer-reviewed articles), language restrictions (primarily English), topical relevance, and timeframes. In more advanced studies, snowballing techniques and journal rankings (e.g., AJG and ABDC) were also employed as inclusion mechanisms.
• Data preprocessing procedures: Data preprocessing was also robustly described in both groups, though the non-China-affiliated studies exhibited a slight advantage: 92% (23/25) versus 100% (25/25) in the China group. Both groups frequently detailed steps such as duplicate removal, keyword normalization, author name disambiguation, and conversion of RIS/BibTeX formats for input into tools like Gephi or VOSviewer. However, China-affiliated studies often provided more procedural specificity—for example, outlining multi-stage screening strategies or manually validating abstracts. In contrast, a few non-China-affiliated studies mentioned data cleaning only in general terms.
• Bibliographic database identification: On this indicator, both groups again performed flawlessly, with 100% of studies identifying their data sources. Web of Science was the most frequently used database in both groups, followed by Scopus. Some studies in the non-China group also drew on Google Scholar, ERIC, or subject-specific repositories, whereas China-affiliated studies occasionally incorporated domestic databases such as CNKI. The consistency across both groups indicates a solid understanding of the need to document data provenance, a crucial aspect of methodological rigor.

Overall, both research communities demonstrate strong performance in several key areas of methodological transparency, particularly in defining study inclusion and identifying data sources. However, noticeable differences persist in the level of detail in technical reporting. Non-China-affiliated studies appear more consistent in disclosing threshold settings, which can be critical for understanding visual outputs and cluster stability in science mapping exercises. On the other hand, China-affiliated publications often include more detailed explanations of preprocessing steps, particularly in multi-phase study designs. The weakest point in both groups remains the reporting of software versions—a critical lapse that undermines the reproducibility of findings. This signals an urgent need for journal editors and reviewers to enforce stricter standards regarding software documentation in bibliometric and scientometric studies.

As bibliometric analysis continues to mature as a methodological cornerstone in literature reviews and knowledge synthesis, transparency in reporting must evolve in tandem. Both China-affiliated and non-China-affiliated research communities contribute meaningfully to the global bibliometric discourse; however, both would benefit from a more uniform adoption of best practices—particularly in the detailed reporting of software parameters and data handling protocols. Future guidelines should emphasize not only what was done, but also how, with what tools, and under what exact conditions.

3.4. Methodological Limitations and Recommendations

As bibliometric-supported literature reviews continue to gain prominence across disciplines, the methodological transparency of such studies becomes increasingly critical. While bibliometric techniques offer powerful tools for mapping scientific knowledge and identifying research trends, their credibility and analytical value depend on how researchers engage with the limitations of these methods and the rigor with which they document their analytical decisions.

This section critically evaluates how methodological limitations are acknowledged and addressed in the most-cited literature review publications authored by researchers affiliated with China-based institutions and those affiliated with institutions outside of China. The assessment is structured around four recurring domains of methodological concern: (1) software functionality, (2) data availability and quality, (3) analytical subjectivity or researcher bias, and (4) citation normalization or disciplinary field differences. Beyond identifying reported challenges, this section also synthesizes the methodological recommendations proposed to improve the robustness, transparency, and applicability of bibliometric analyses. By juxtaposing the practices of China-affiliated and non-China-affiliated research communities, this section highlights differences in methodological reflexivity, depth of critique, and the nature of proposed solutions—offering valuable insights into the evolving standards and expectations shaping global bibliometric research.

3.4.1. Methodological Limitations and Recommendations in China-Affiliated Studies

An in-depth review of China-affiliated literature review publications revealed a range of methodological limitations and challenges acknowledged by the authors, as well as varying degrees of recommendations for improving future bibliometric studies. These findings are grouped under four key dimensions: software functionality, data availability and quality, analytical subjectivity or researcher bias, and citation normalization or disciplinary field differences.

• Software functionality: Most China-affiliated studies did not explicitly acknowledge limitations related to bibliometric software functionality. Many papers applied bibliometric methods without specifying the tools used or evaluating their constraints (L. Chen et al., 2017; Li & Wang, 2019; Ren et al., 2018; S. Xu et al., 2020). However, a few studies offered implicit or explicit reflections. One study (Feng et al., 2017) highlighted the limitations of keyword-based maps and recommended broader content inclusion. Another study (Koseoglu et al., 2016) underscored the underuse of relational statistical tools due to a lack of researcher familiarity. Another study (X. Chen et al., 2022) criticized the inefficiency of manual content analysis in handling large datasets. Technical integration challenges between BIM and GIS systems were reported by (H. Wang et al., 2019), highlighting interoperability issues. More direct critiques came from (Yang et al., 2023) and (Ampah et al., 2021), which described the incompatibility of HistCite with Scopus, as well as the complexity of BibExcel and CiteSpace’s merging limitations. These insights collectively call for the development of more accessible, interoperable, and scalable bibliometric software to enhance analytical precision and usability in future research.
• Data availability and quality: A considerable number of studies (Ampah et al., 2021; L. Chen et al., 2017; Y. Ding et al., 2021; Koseoglu et al., 2016; Li & Wang, 2019; Liang et al., 2017; Liu et al., 2020; Mao et al., 2018; Song et al., 2022; Tlili et al., 2022; van Nunen et al., 2018; Q. Wang & Su, 2020; X. Xu et al., 2018; Yang et al., 2023; Zhang et al., 2019; Zhao et al., 2020) explicitly acknowledged limitations about data sources and quality. Common issues included the following: database restriction—most studies relied solely on the Web of Science Core Collection, leading to potential exclusion of relevant documents from Scopus, CNKI, or non-English literature (L. Chen et al., 2017; Tlili et al., 2022; S. Xu et al., 2020; Yang et al., 2023); incomplete indexing— several authors noted delays in database updates, causing recent articles (e.g., from 2021–2022) to be underrepresented at the time of data collection (Song et al., 2022; van Nunen et al., 2018; Q. Wang & Su, 2020); keyword and scope constraints—limitations in search strategies, including narrow keyword scopes, were commonly cited as contributing to possible omissions of relevant studies (L. Chen et al., 2017; Feng et al., 2017; X. Xu et al., 2018; Zhang et al., 2019); and data noise and errors—a few studies reported challenges with duplicate records, author disambiguation, and inconsistent metadata (Y. Ding et al., 2021; van Nunen et al., 2018).
• Analytical subjectivity and researcher bias: Although fewer in number, some studies openly addressed concerns about subjective judgment in coding or interpretation: For instance, (X. Xu et al., 2018) and (X. Ding & Yang, 2022) acknowledged that keyword selection, thematic clustering, and coding processes might introduce bias, despite efforts to ensure objectivity through expert input or consensus. Other studies implied subjectivity by noting that bibliometric indicators alone cannot reveal underlying qualitative nuances, thus encouraging the use of complementary content analysis (X. Chen et al., 2022; Tlili et al., 2022; van Nunen et al., 2018).
• Citation normalization and disciplinary differences: Only a minority of studies explicitly addressed limitations related to citation normalization or disciplinary variance. One study (van Nunen et al., 2018) highlighted that older publications naturally accumulate more citations, potentially biasing impact analyses. The study (Koseoglu et al., 2016) emphasized that interdisciplinary topics—such as tourism, marketing, and psychology—have divergent citation behaviors, complicating normalization efforts. Other studies (Mao et al., 2018; B. Wang et al., 2021) suggested that conceptual overlaps or translation ambiguities (e.g., between “smart” and “intelligent”) can also distort thematic interpretation across fields and regions.

Recommendations for Improving Future Bibliometric Studies in China-Affiliated Studies

Despite the limited number of studies offering detailed methodological reflections, several recurring themes emerged in the recommendations for improving future bibliometric studies in China-affiliated studies:

• Expand and diversify data sources: Many studies (L. Chen et al., 2017; Yang et al., 2023; Zhao et al., 2020) called for the inclusion of multiple databases (e.g., Scopus, CNKI, PubMed, and Google Scholar), and broader document types (e.g., books, conference papers, and dissertations) to enhance coverage.
• Improve keyword strategy and taxonomy development: Several authors (van Nunen et al., 2018; X. Xu et al., 2018; Zhang et al., 2019) suggested the need for more robust keyword taxonomies and iterative refinement of search terms to improve comprehensiveness and replicability.
• Promote transparency and reporting standards: Studies (X. Ding & Yang, 2022; Koseoglu et al., 2016; Yang et al., 2023) emphasized the importance of better documentation of software tools, database search strategies, and inclusion/exclusion criteria to ensure reproducibility and cross-study comparability.
• Incorporate complementary methods: Several studies recommended combining bibliometric approaches with qualitative content analysis, expert interviews, or case-based reasoning to deepen insights (Tlili et al., 2022; van Nunen et al., 2018; S. Xu et al., 2020).
• Enhance interdisciplinary and cross-cultural collaboration: Encouraging multinational and multidisciplinary research collaborations was highlighted in several studies (Mao et al., 2018; Q. Wang & Su, 2020; Yang et al., 2023) to bridge knowledge gaps and enrich citation networks. Addressing author and institutional disambiguation, the adoption of researcher identifiers, such as ORCID, and organizational standardization is recommended to improve accuracy in author-level metrics (van Nunen et al., 2018).

Overall, the findings suggest that while China-affiliated bibliometric studies frequently acknowledge challenges related to data quality and scope, there remains a limited explicit reflection on software functionality and disciplinary citation differences. Recommendations, where present, predominantly emphasize expanding data inclusion, promoting transparency, and integrating complementary methodologies.

3.4.2. Methodological Limitations and Recommendations in Non-China-Affiliated Studies

An analysis of non-China-affiliated literature review studies revealed broader and more reflective engagement with methodological limitations across multiple domains. These studies frequently demonstrated greater transparency in discussing the challenges inherent in bibliometric analysis and offered detailed recommendations for enhancing methodological rigor in future research. The limitations and suggestions were grouped into four major dimensions: software functionality, data availability and quality, analytical subjectivity or researcher bias, and citation normalization or disciplinary field differences.

• Software functionality: Several studies explicitly acknowledged software-related limitations. For example, (Donthu et al., 2021; Fahimnia et al., 2015; Leung et al., 2017; Moral-Muñoz et al., 2020; Mukherjee et al., 2022; Pranckutė, 2021) described constraints associated with widely used tools such as VOSviewer, HistCite, Pajek, Gephi, and Bibliometrix. Common issues included difficulties with merging synonymous keywords, limited compatibility with specific databases (e.g., HistCite’s inability to support Scopus), file format constraints (e.g., Pajek), and steep learning curves that may deter novice users. Additionally, the studies (Donthu et al., 2021; Goyal & Kumar, 2021; Moral-Muñoz et al., 2020) cautioned against over-reliance on software-generated outputs without deeper theoretical engagement, warning that such reliance may not meet the standards of high-impact journals. Several studies (Arsad et al., 2022; Faust et al., 2018; Verma et al., 2021a) also noted performance limitations in algorithmic tools such as PSO and ANFIS due to convergence speed and computational expense.
• Data availability and quality: Data-related limitations were among the most acknowledged. Many studies (Blettler et al., 2018; Donthu et al., 2021; Fahimnia et al., 2015; Geissdoerfer et al., 2017; Goodell et al., 2021; Meerow et al., 2016; Mora et al., 2017; Moral-Muñoz et al., 2020; Pranckutė, 2021; Rejeb et al., 2022; van der Have & Rubalcaba, 2016; Verma et al., 2021b) pointed to incomplete or biased bibliographic databases as a key issue. Frequent challenges included the following: limited access to non-English or gray literature (Leung et al., 2017; Meerow et al., 2016; van der Have & Rubalcaba, 2016); errors in metadata, duplicate records, and inconsistencies in indexing (Blettler et al., 2018; Goodell et al., 2021; Pranckutė, 2021); search date limitations causing underrepresentation of recent publications (Arsad et al., 2022; Fahimnia et al., 2015); and structural limitations in export capacity from platforms like Scopus and Web of Science (Moral-Muñoz et al., 2020; Pranckutė, 2021). These issues raised concerns about comprehensiveness, particularly in emerging or multidisciplinary fields such as sustainability, circular economy, and IoT.
• Analytical subjectivity and researcher bias: A majority of the reviewed studies acknowledged the inherent subjectivity involved in bibliometric interpretation. For example, (Donthu et al., 2021; Fahimnia et al., 2015; Geissdoerfer et al., 2017; Linnenluecke et al., 2020; Meerow et al., 2016; Mukherjee et al., 2022) pointed to selection bias introduced through keyword choices, manual coding, or interpretation of thematic clusters. The role of researcher judgment in defining inclusion/exclusion criteria, coding conceptual frameworks, and interpreting network visualizations was often emphasized. While some studies (Donthu et al., 2021; Fahimnia et al., 2015) suggested that advanced software tools can mitigate subjectivity, most agreed that qualitative insight remains essential to meaningful interpretation of bibliometric results.
• Citation normalization and disciplinary field differences: Compared to China-affiliated studies, non-China-affiliated publications more frequently addressed citation normalization issues. Several studies (Fahimnia et al., 2015; Goyal & Kumar, 2021; Linnenluecke et al., 2020; Mora et al., 2017; Moral-Muñoz et al., 2020; Mukherjee et al., 2022; Pranckutė, 2021) acknowledged that citation behaviors vary significantly across disciplines, affecting impact measures like the h-index and Local vs. Global Citation Scores. Studies also emphasized that citation-based indicators often disadvantage newer publications and disciplines that value book chapters or non-journal outputs. Some researchers (Goyal & Kumar, 2021; Sarkodie & Strezov, 2019) argued for the development of context-specific metrics or multi-metric frameworks to account for disciplinary diversity.

Recommendations for Improving Future Bibliometric Studies in Non-China-Affiliated Studies

Across the reviewed non-China-affiliated studies, a rich set of recommendations emerged. These include the following:

• Software integration and methodological triangulation: Several studies (Donthu et al., 2021; Fahimnia et al., 2015; Moral-Muñoz et al., 2020) advocated combining tools such as VOSviewer, Gephi, and Bibliometrix to balance strengths and overcome individual limitations. Additionally, authors encouraged integrating bibliometric methods with systematic reviews, content analysis, or theory-driven frameworks to enhance depth and validity (Ben-Daya et al., 2019; Donthu et al., 2021; Goyal & Kumar, 2021).
• Enhanced data cleaning and preprocessing: The importance of meticulous data cleaning was repeatedly emphasized (Donthu et al., 2021; Goodell et al., 2021; Moral-Muñoz et al., 2020; Pranckutė, 2021), including removing duplicates, resolving author disambiguation issues, and verifying metadata accuracy.
• Improved database strategies: To reduce selection bias and broaden the scope of analysis, researchers recommended using multiple databases (e.g., WoS, Scopus, SSRN, and Google Scholar) and incorporating gray literature and non-English sources (Meerow et al., 2016; Pranckutė, 2021; Rejeb et al., 2022; van der Have & Rubalcaba, 2016).
• Disciplinary sensitivity and normalization techniques: Scholars urged caution when comparing indicators across fields, suggesting the use of field-normalized citation metrics and the inclusion of contextual metadata (Moral-Muñoz et al., 2020; Mukherjee et al., 2022; Pranckutė, 2021).
• Capacity building and accessibility: Recognizing steep learning curves, some studies (Faust et al., 2018; Moral-Muñoz et al., 2020) proposed developing user-friendly bibliometric platforms and training resources to support broader adoption among novice researchers.
• Ethical and inclusive research practices: Some studies, especially those focusing on sustainability and social sciences (Kent Baker et al., 2020; Meerow et al., 2016; van der Have & Rubalcaba, 2016), highlighted the need for more inclusive, participatory, and ethically grounded bibliometric research that addresses underrepresented regions, languages, and communities.

In sum, non-China-affiliated studies demonstrated a more systematic engagement with methodological limitations and offered forward-looking, practical recommendations. Their openness to combining quantitative and qualitative approaches, refining tool use, and broadening data sources reflects a maturing understanding of the strengths and limitations of bibliometric methodologies in scholarly research.

3.4.3. Comparative Analysis of Methodological Limitations and Recommendations

The comparative analysis between China-affiliated and non-China-affiliated literature review studies reveals notable contrasts in how each research group addresses methodological limitations and articulates improvement strategies in bibliometric-supported reviews. This divergence reflects broader differences in research transparency, engagement with bibliometric tools, and commitment to methodological rigor.

• Software functionality: China-affiliated studies vastly underreport software-related limitations. Many did not identify the tools used or engage with their operational constraints, suggesting a general lack of reflection on how software choice might shape findings (L. Chen et al., 2017; Li & Wang, 2019; Ren et al., 2018; S. Xu et al., 2020). Where mentioned, limitations were often peripheral or implied, with only a few studies acknowledging tool-related barriers such as interface complexity or challenges in data integration (Feng et al., 2017; H. Wang et al., 2019; Yang et al., 2023). In contrast, non-China-affiliated studies demonstrated a deeper engagement with software functionality. Several studies provided explicit critiques of widely used tools such as HistCite, VOSviewer, Pajek, and Bibliometrix, acknowledging limitations in merging keyword variants, database compatibility, scalability, and learning curves (Donthu et al., 2021; Fahimnia et al., 2015; Moral-Muñoz et al., 2020; Mukherjee et al., 2022). Furthermore, multiple studies emphasized that bibliometric outputs alone are insufficient and must be accompanied by theoretical insight and content analysis. There was also a clear trend toward combining tools to compensate for individual limitations, a practice scarcely observed in China-affiliated work.
• Data availability and quality: Both groups acknowledged limitations in data quality and availability, but the depth and specificity differed. China-affiliated studies often pointed to database restrictions (e.g., reliance on WoS or CNKI), delayed indexing, or narrow keyword selection. However, these acknowledgments were usually brief and seldom accompanied by corrective strategies. Non-China-affiliated studies, by contrast, offered more comprehensive critiques. They highlighted metadata inconsistencies, limitations in export capacity, exclusion of non-English and gray literature, and underrepresentation of Social Sciences and Humanities in mainstream databases (Moral-Muñoz et al., 2020; Pranckutė, 2021; Rejeb et al., 2022; van der Have & Rubalcaba, 2016). Importantly, these studies frequently provided targeted recommendations—such as multi-database use, inclusion of non-journal sources, and metadata validation protocols—to address these issues. This signals a more proactive and systematic approach to data reliability.
• Analytical subjectivity and researcher bias: Subjectivity in coding and interpretation was more frequently acknowledged in non-China-affiliated studies. Researchers openly discussed how thematic clustering, keyword selection, or classification of conceptual tensions involve human judgment (Geissdoerfer et al., 2017; Goyal & Kumar, 2021; Linnenluecke et al., 2020; Meerow et al., 2016). Several authors addressed this limitation by proposing collaborative coding, triangulation methods, and integration with qualitative techniques such as content analysis or expert interviews. In China-affiliated studies, analytical subjectivity was less frequently acknowledged and rarely problematized. When mentioned, it was often framed as a byproduct of broader research limitations rather than a specific concern of bibliometric analysis. This suggests a less developed awareness of the interpretive dimensions of bibliometric methodology.
• Citation normalization and disciplinary differences: Non-China-affiliated studies were more attuned to the challenges of comparing citation metrics across disciplines. Several papers explicitly noted that metrics like the h-index or citation counts are not normalized and may be skewed by disciplinary norms, publication age, or source type (Fahimnia et al., 2015; Goyal & Kumar, 2021; Linnenluecke et al., 2020; Mora et al., 2017; Moral-Muñoz et al., 2020; Mukherjee et al., 2022; Pranckutė, 2021). Recommendations included using field-normalized indicators, triangulating metrics, and acknowledging citation biases against novel or interdisciplinary work. Conversely, China-affiliated studies infrequently addressed citation normalization. Only a few studies noted differences in citation accumulation or scope across research areas, and these were generally discussed in passing rather than as core methodological concerns. This indicates a gap in the analytical sensitivity to bibliometric disparities rooted in disciplinary cultures.

Recommendations for Methodological Improvement in Both China-Affiliated and Non-China-Affiliated Research Communities

The most striking divergence lies in the depth and specificity of improvement recommendations. Non-China-affiliated studies offered a wide array of actionable suggestions: from expanding database coverage, refining keyword strategies, and combining bibliometric with qualitative methods, to promoting open-access metrics, interdisciplinary collaboration, and transparent methodological protocols (Donthu et al., 2021; Fahimnia et al., 2015; Goyal & Kumar, 2021; Moral-Muñoz et al., 2020; van der Have & Rubalcaba, 2016). By comparison, China-affiliated studies were more conservative in their recommendations, often focusing on future content areas rather than methodological advancement. In many cases, recommendations were directed toward the subject matter (e.g., green finance or smart cities) rather than the bibliometric approach itself. Only a handful of papers proposed expanding keyword scope or using more inclusive databases, and even fewer addressed tool improvement or validation strategies. Overall, non-China-affiliated bibliometric literature reviews tend to be more methodologically transparent, self-critical, and proactive in addressing the limitations of their tools, data sources, and interpretive frameworks. These studies often view bibliometric analysis not as an end in itself, but as part of a broader methodological ecosystem that requires thoughtful integration with qualitative insights and theoretical rigor. China-affiliated studies, while often robust in data extraction and trend mapping, exhibit a more limited engagement with methodological challenges and offer fewer systematic recommendations for improving bibliometric practice. Addressing this gap could significantly enhance the credibility, replicability, and theoretical contribution of bibliometric reviews within China-affiliated research communities.

To synthesize the comparative insights discussed in the preceding sections, Table 3d provides a consolidated snapshot of the key methodological and conceptual dimensions that distinguish China-affiliated and non-China-affiliated literature review studies. This high-level summary highlights both shared characteristics—such as a common reliance on tools like VOSviewer and Gephi—and notable divergences in areas such as reporting transparency, acknowledgment of methodological limitations, and the nature of improvement recommendations. It also illustrates distinct thematic orientations in the types of research topics emphasized by each group. Taken together, these contrasts underscore deeper epistemological and institutional differences that shape how bibliometric analyses are conducted and communicated across global research communities.

For completeness, a contextual co-word analysis of the 4000 most-cited studies (RQ4) was conducted using VOSviewer (version 1.6.20) to examine the conceptual structures of China- and non-China-affiliated research communities. As this analysis extends beyond the methodological scope of the present paper, the full results—including visual maps, cluster reports, and supporting datasets—are available in a public repository managed by the authors (Ali Abaker Omer and Dong, 2025), ensuring transparency and accessibility for interested readers.

4. Discussion

This study aimed to conduct a comparative bibliometric content analysis of methodological practices in literature review studies authored by researchers affiliated with China and those not affiliated with China. By analyzing the top 25 most-cited publications from each group, alongside a large-scale co-word analysis of 2000 studies per group, the findings reveal meaningful differences in software tool adoption, methodological reporting, engagement with analytical limitations, and practical implications and research directions.

4.1. Tool Preferences Reflect Regional Infrastructures and Methodological Cultures

The observed divergence in software usage (Section 3.2) indicates that institutional and linguistic ecosystems influence the selection of bibliometric tools. Researchers affiliated with China demonstrated a strong preference for CiteSpace and VOSviewer, often integrating 3–5 tools into complex workflows. This suggests a methodologically hybrid culture emphasizing network visualization and temporal analysis. In contrast, non-China-affiliated researchers preferred Gephi, Bibliometrix, and BibExcel, frequently deploying them in modular sequences with more precise documentation.

These patterns may reflect differences in training environments, software accessibility, or disciplinary conventions. For example, CiteSpace’s Chinese-language documentation and user forums may encourage uptake among researchers based in China. At the same time, the open-source, R-based architecture of Bibliometrix aligns with the reproducibility movement in Western academia. This confirms earlier findings by Aria and Cuccurullo (Aria & Cuccurullo, 2017), who advocated for increased tool transparency to foster methodological comparability.

Another key distinction concerns user-friendliness. GUI-based tools such as VOSviewer are relatively straightforward to operate and require little to no programming knowledge, which makes them accessible to a wide range of researchers, including those without advanced technical training. In contrast, script-based approaches such as Bibliometrix in R or Python libraries demand greater technical expertise but offer enhanced flexibility and customization. These differences in usability and accessibility help to explain the adoption patterns observed across China-affiliated and non-China-affiliated studies, with the former often favoring GUI-driven tools and the latter showing greater engagement with script-based platforms. Furthermore, we clarify that widely used platforms such as VOSviewer, CiteSpace, and Bibliometrix are open source, while tools such as HistCite™ represent commercial options.

4.2. Transparency Gaps Undermine Reproducibility

Despite a growing emphasis on methodological rigor, both groups exhibited deficiencies in reporting the versions of bibliometric software—a critical concern for replicability (Section 3.3). While inclusion/exclusion criteria and data preprocessing steps were widely reported, versioning and threshold settings were inconsistently documented. These findings align with critiques from Sweileh (Sweileh, 2021) and Bellucci et al. (Bellucci et al., 2021), who have also found that highly cited bibliometric papers often omit core methodological details.

Non-China-affiliated studies exhibited greater clarity in threshold specification (76% vs. 60%) and offered more comprehensive preprocessing descriptions. Meanwhile, China-affiliated studies demonstrated strength in multi-stage data preparation and scope definition but showed more variability in justifying parameter settings. These discrepancies point to the need for community-wide reporting standards—potentially akin to PRISMA for systematic reviews—that ensure transparency in bibliometric methods.

While the harmonization of bibliometric practices depends in part on authors’ methodological awareness, reviewers and editors also play a crucial role. Reviewers act as methodological gatekeepers by ensuring that studies include clear details on software versions, threshold settings, database coverage, and preprocessing steps. Their feedback provides an important safeguard against underreporting and helps align individual studies with broader community standards. Editors, likewise, can reinforce good practice by encouraging adherence to transparent reporting guidelines in their journals. Strengthening these practices on the reviewer–editor side is therefore essential to achieving greater consistency and reproducibility across research communities.

4.3. Reflexivity and Limitations: A Divided Practice

One of the most pronounced divergences between the two research communities lies in the degree of methodological reflexivity. As highlighted in Section 3.4, non-China-affiliated researchers more frequently and explicitly acknowledged limitations related to software functionality, database coverage, analytical subjectivity, and disciplinary citation disparities. These studies also offered concrete methodological recommendations, such as integrating multiple tools to overcome individual software constraints (Moral-Muñoz et al., 2020), combining bibliometric analysis with qualitative content analysis to enrich interpretation (Zupic & Čater, 2014), and employing field-normalized citation metrics to improve cross-disciplinary comparability (Waltman & van Eck, 2015). By contrast, China-affiliated studies seldom addressed software-related challenges or the interpretive biases that may arise from parameter settings, clustering techniques, or database restrictions. When recommendations were made, they were primarily oriented toward expanding content coverage—such as incorporating Scopus, CNKI, or additional document types—rather than advancing methodological frameworks or analytical strategies. This pattern suggests a predominantly instrumental use of bibliometric tools focused on topic mapping, in contrast to the more theory-informed, reflexively critical stance observed among non-China-affiliated researchers.

This divergence aligns with earlier insights by Donthu et al. (Donthu et al., 2021), who emphasized the importance of methodological self-awareness in bibliometric studies. They argued that to contribute meaningfully to the science of science, bibliometric research must go beyond technical application and engage more deeply with questions of validity, transparency, and epistemological rigor. The findings of this study affirm this and highlight the need for broader adoption of reflexive practices in bibliometric-supported literature reviews, particularly within rapidly expanding research systems.

4.4. Conceptual Orientations Shape Methodological Choices

The co-word analysis (Ali Abaker Omer & Dong, 2025) reveals how thematic priorities may influence methodological behaviors. China-affiliated studies clustered around applied domains such as artificial intelligence, traditional Chinese medicine, and wastewater treatment—topics aligned with national priorities and industrial applications. In contrast, non-China-affiliated studies demonstrated a broader engagement with the SDGs, educational reform, and digital transformation, often supported by theory-driven frameworks. These differences are not merely topical but structural: applied topics may favor rapid, tool-driven analysis, while policy-relevant or interdisciplinary inquiries may demand greater methodological depth. Thus, conceptual orientations shape the form, function, and transparency of bibliometric methods. This finding aligns with Knorr-Cetina’s notion of epistemic cultures (Cetina, 1999)—that the way researchers construct and validate knowledge varies across communities and institutional settings.

4.5. Toward Methodological Convergence and Future Standards

This comparative investigation suggests an urgent need for harmonizing bibliometric practices across global research communities. Editors, reviewers, and institutions can play a pivotal role by applying the following standards:

• Requiring explicit reporting of software versions and threshold settings.
• Promoting the use of standardized methodological checklists.
• Encouraging integration of bibliometric and qualitative methods.
• Supporting training in open-source tools to reduce regional inequalities.

By converging toward shared standards while remaining sensitive to contextual differences, bibliometric-supported literature reviews can achieve greater reproducibility, inclusivity, and theoretical sophistication. These findings add to ongoing debates about methodological pluralism and the future of science research (Fortunato et al., 2018).

4.6. Limitations of the Study

While this study offers valuable insights into the methodological practices of bibliometric-supported literature reviews across China-affiliated and non-China-affiliated research communities, several limitations should be acknowledged.

First, the analysis was restricted to English-language publications indexed in the Scopus database. This linguistic and database constraint may have excluded important contributions published in other languages, particularly Chinese, and in other indexing platforms, such as Web of Science, CNKI, or Dimensions. As a result, the findings may not fully capture the diversity of bibliometric practices employed globally, especially within non-English academic contexts.

Second, the sampling strategy focused on the top 25 most-cited literature review studies from each group between 2015 and 2024 for content analysis, and the top 2000 most-cited documents per group for co-word network analysis. While this approach ensured a focus on highly influential work, it may have introduced citation bias by excluding less-cited but methodologically innovative studies. Citation counts may also reflect visibility rather than methodological quality, potentially skewing the representation of best practices.

Third, although the study examined multiple indicators of methodological transparency, including software version reporting, threshold settings, data preprocessing, and databases used, some of these elements were inconsistently reported across studies. As such, certain inferences were based on the absence of reporting rather than confirmed methodological omission, which may limit the precision of the analysis.

Fourth, the categorization of studies as “China-affiliated” or “non-China-affiliated” was based on the institutional affiliation of the first author(s). This binary classification, while practical, may overlook the nuances of international collaboration, multicultural teams, or the influence of institutional research norms that transcend national boundaries.

Finally, the study employed a content coding framework that, while grounded in the prior literature and validated through inter-coder agreement, may still involve interpretive subjectivity. Although care was taken to ensure consistency, some degree of bias in interpretation cannot be entirely ruled out.

Despite these limitations, the study provides a foundational and comparative perspective on how methodological practices are evolving within bibliometrically supported literature reviews. Future research should build upon these findings by incorporating non-English publications, broadening the range of data sources, and exploring longitudinal shifts in methodological rigor and reporting standards.

4.7. Practical Implications and Research Directions

This study offers several practical implications for researchers, journal editors, and policymakers involved in the production, evaluation, and dissemination of bibliometric-supported literature reviews.

For researchers, the findings underscore the importance of adhering to clear and transparent methodological reporting standards. The consistent documentation of software versions, threshold settings, data preprocessing procedures, and database selection is critical not only for enhancing the credibility of bibliometric analyses but also for facilitating reproducibility and critical assessment. The observed disparities in reporting practices between China-affiliated and non-China-affiliated researchers highlight the need for greater awareness of international expectations and best practices in meta-research.

For journal editors and reviewers, the results signal the need to establish and enforce minimum methodological reporting requirements for bibliometric studies. As bibliometric methods become increasingly integral to research evaluation and scientific mapping, editorial policies must evolve to ensure that studies meet basic criteria for transparency, rigor, and reproducibility. Clearer review guidelines and standardized checklists for reporting bibliometric procedures could help bridge current reporting gaps and improve overall research quality.

For institutions and policy stakeholders, the study points to the broader structural and epistemological influences that shape bibliometric practices. Research training programs should incorporate modules on bibliometric methodology that not only cover the technical use of tools but also emphasize responsible research practices, ethical interpretation, and critical reflexivity. Institutions should also encourage cross-cultural and interdisciplinary collaboration to harmonize methodological standards and reduce regional disparities in practice.

Looking Ahead, Several Avenues for Future Research Emerge from This Study

First, expanding the analysis to include non-English publications, particularly those in languages such as Chinese, Spanish, Portuguese, and Russian, would provide a more comprehensive understanding of global bibliometric practices.

Second, future studies could adopt longitudinal approaches to assess how reporting behaviors and tool preferences evolve, especially in response to changes in editorial policies, software developments, or shifts in research evaluation frameworks.

Third, comparative analyses across disciplines—such as natural sciences, social sciences, and humanities—could uncover field-specific methodological trends and help tailor best practices accordingly.

Finally, integrating qualitative interviews or surveys with authors, editors, or methodologists could provide deeper insights into the motivations, challenges, and contextual factors influencing bibliometric reporting behaviors. Such mixed-methods research would enrich the current findings and contribute to a more comprehensive and actionable roadmap for advancing methodological integrity in bibliometric scholarship.

5. Conclusions

This study provided a comparative analysis of bibliometric software use and methodological reporting in English-language literature reviews authored by China-affiliated and non-China-affiliated researchers between 2015 and 2024. Combining bibliometric mapping, content analysis, and co-word analysis, it highlighted both similarities and differences in tool preferences, reporting practices, reflexivity, and thematic orientations.

Both groups frequently used widely adopted tools such as VOSviewer, CiteSpace, Gephi, and Bibliometrix. However, non-China-affiliated studies more consistently reported critical details—software versions, threshold settings, preprocessing protocols, and database sources—demonstrating stronger adherence to transparency and reproducibility. By contrast, many China-affiliated studies prioritized output metrics while underreporting methodological choices.

Differences also emerged in acknowledging limitations. Non-China-affiliated researchers more often discussed software constraints, citation biases, and interpretive subjectivity, whereas China-affiliated studies focused on scope and coverage with limited methodological critique. These contrasts reflect broader epistemological and institutional influences shaping bibliometric practice.

This study emphasizes the need for unified reporting standards and greater reflexivity across the field. Transparent methods not only strengthen reproducibility but also enhance the credibility of bibliometric insights, which increasingly inform evaluation, policy, and funding decisions.

Future research should broaden the scope to other regions, disciplines, and languages, especially non-English outputs, and encourage interdisciplinary collaboration. Advancing shared guidelines and context-sensitive practices will help foster a more cohesive, reflective, and ethically grounded bibliometric community that values both analytical outcomes and methodological integrity.