Next Article in Journal
Development and Validation of a Novel Score for Predicting Paroxysmal Atrial Fibrillation in Acute Ischemic Stroke
Previous Article in Journal
Factors Predicting Training Delays and Attrition of Recruits during Basic Military Training
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 19
Issue 12
10.3390/ijerph19127273
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Study Protocol

Development of a Guideline to Enhance the Reporting of Concept Mapping Research: Study Protocol

by
Sandesh Pantha
1,*,
Martin Jones
1,2,3 and
Richard Gray
1
1
School of Nursing and Midwifery, La Trobe University, Bundoora, Melbourne, VIC 3086, Australia
2
Department of Rural Health, University of South Australia, Whyalla Campus, Whyalla Norrie, SA 5608, Australia
3
IIMPACT in Health, University of South Australia, Adelaide, SA 5000, Australia
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(12), 7273; https://doi.org/10.3390/ijerph19127273
Submission received: 20 April 2022 / Revised: 7 June 2022 / Accepted: 9 June 2022 / Published: 14 June 2022
Review Reports Versions Notes

Abstract

:
Reporting guidelines are intended to enhance the clarity and transparency of research publications. Concept mapping, a mixed-methods design, has been widely used in health research. Current reporting guidelines for mixed-methods studies are not relevant for concept mapping research. The aim of this study is to develop a reporting guideline for concept mapping research following the EQUATOR network toolkit. Guideline development is in three stages: 1. A systematic review to identify key components of reporting concept mapping research, 2. A concept mapping study involving the key stakeholder groups—researchers, methodologists, peer reviewers, journal editors, statisticians, and people who have participated in concept mapping research—to identify candidate items to include in a reporting guideline, and 3. Development of a draft reporting guideline for concept mapping research. The outcome of the research will be a reporting guideline for concept mapping research.

1. Introduction

Concept mapping is a type of mixed-methods study design where participants generate and prioritize ideas to develop an understanding of a complex phenomenon [1,2]. Concept mapping research has six phases—preparation, brainstorming (idea generation), structuring of the statements (prioritization and ranking), representation of statements, interpretation of maps, and utilization of the map. Concept mapping has been extensively used in public health, education, and social science research [2,3,4,5,6,7,8,9,10]. For example, Grewal et al., (2021) used concept mapping to understand barriers to self-management for people with diabetes [7]. Concept mapping has also been used to inform the development of patient-reported outcome measures such as quality of care in diabetic patients [11]. Rosas and Ridings (2017) reported that at least 23 measurement tools and scales have been developed using the concept mapping approach [12]. The number of concept mapping studies that have been published in the scientific literature has increased over the past 20 years [3,13]. Donnelly (2017) reported a 400% increase in the number of doctoral dissertations using concept mapping research between 2000 and 2014 [13].
Two systematic reviews have examined the quality of reporting concept mapping research [3,13]. Donnelly (2017) reviewed 104 concept mapping dissertations and reported frequent omission of important information across the different phases of concept mapping research [13]. Rosas and Kane (2012) reported a review of the quality and rigor of 69 concept mapping studies [3]. The review authors note that studies frequently omitted important information about concept mapping. Both reviewers conclude that there is a need for guideline to enhance the quality of reporting of concept mapping research.
A reporting guideline is a checklist of core information to include in a manuscript or research report [14]. Authors have demonstrated that reporting guidelines enhance the quality, clarity, and transparency of a research report [15,16,17,18]. For example, a review by Nawijn et al., (2019) showed that the implementation of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analysis) reporting guideline led to a marked improvement in the comprehensiveness of the reporting of 112 systematic reviews published in five Emergency Medical Journals.
The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network is a repository of 486 reporting guidelines covering a broad range of research methodologies [14]. To date, there are no reporting guidelines for concept mapping research listed on the EQUATOR network. However, Group Wisdom®,—a software package specifically designed to support concept mapping research—has suggested ten essential components to describe group concept mapping data collection and results [19]. There are several limitations to this checklist: 1. The checklist can only be applied to studies using the Group Wisdom® software package, 2. It is unclear how the checklist was developed, and 3. There are important omissions in the checklist including details of how the research question was developed and reporting the number of statements generated and filtered during the process of reduction.
We identified two concept mapping studies where the authors adapted reporting guidelines from observational [20] and mixed-methods research [21]. Dopp et al. (2020) used Good Reporting of a Mixed-Methods Study (GRAMMS) to report the methodological rigor of a concept mapping manuscript [21]. The GRAMMS, however, does not address many of the key methodological components of concept mapping such as prioritization and clustering.
The aim of this study is to develop a comprehensive reporting guideline for concept mapping research adhering to the recommendations made by the EQUATOR network for reporting guideline development. Our guideline will support researchers in reporting the important parameters of concept mapping research. It may further contribute to enhancing the quality of reporting, provide transparency, and aid replicability of concept mapping research. In this protocol, we set out a detailed description of the guideline development methodology.

2. Methods

There are three elements to our study:
  • A systematic review of previous concept mapping studies to identify candidate items to include in a reporting guideline.
  • A concept mapping study to identify key items that should be included in the guidelines.
  • Drafting of reporting guidelines for concept mapping research.
We have prospectively registered our study with 1. Open Science Framework (OSF) on 17 July 2021 (https://osf.io/nrvjh/, accessed on 24 February 2022) and 2. EQUATOR network on 29 March 2022.

2.1. Stage 1: Systematic Review

Our review of previous health-related concept mapping research has three aims: 1. Identify candidate items to include in the reporting guideline, 2. Identify inconsistencies in reporting, and 3. Identify the potential participants (researchers, journals, editors, and peer-reviewers previously involved in concept mapping studies) with a view to inviting them to participate in stage 2 of this project. Our review methodology adheres to the PRISMA-P statement (Supplementary Material, Table S1) [22]. As our review does not focus on a health-reported outcome, the review is not eligible for registration with the PROSPERO (Prospective Register of Systematic Review) [23].

2.1.1. Eligibility Criteria

We will include any primary concept mapping study that meets the following criteria-1. The focus of the research is a health topic (because EQUATOR guidelines only relate to this field of research) and 2. Published in a peer-reviewed journal. No restriction will be placed based on the date of publication. We will exclude studies not in English as there is evidence to suggest that the exclusion of studies in other languages has little or no influence on the strength of evidence generated from a systematic review [24,25,26].

2.1.2. Information Sources

We will conduct a systematic search of three online databases: the Medical Literature Analysis and Retrieval System Online (MEDLINE), PsycInfo, and Cumulative Index to Nursing and Allied Health Literature (CINAHL). MEDLINE, PsycInfo, and CINAHL databases capture the majority of research conducted in health-related disciplines. MEDLINE and PsycInfo will be accessed via Ovid. We will access CINAHL through the EBSCOhost platform.

2.1.3. Search Strategy

We followed the Peer Review of Electronic Search Strategies (PRESS) for developing the search strategy [27]. PRESS guidelines suggest developing a search strategy using the keywords (full and various truncations) and indexing terms (for example, Medical Subject Headings (MeSH)) that can then be combined using the bullion operators (OR) [28]. We adapted a search strategy previously used by Donnelly (2017) for a systematic review of concept mapping studies [13]. Our search terms include “concept map *”, “structured conceptualization”, “Ariadne”, and “concept systems”. The search strategy was initially developed for MEDLINE (Supplementary Material, Table S2) and subsequently customized for other databases (PsycInfo and CINAHL).

2.1.4. Study Records

Data Management

Covidence, an online systematic review management package, will be used [29]. Results from searches of each database will be exported to Endnote X9.3 as a .enl file. The citations will then be combined and exported to Covidence as a .xml file. Duplicate citations will be identified and removed in Covidence.

Selection Process

Screening of the articles will be carried out in two steps against our eligibility criteria (Section 2.1.1): 1. Title and abstract and 2. Full-text screening. Two reviewers will independently screen the articles. Any conflicts during the title and abstract screening will be resolved by mutual agreement. Conflicts that arise during full-text screening will be resolved after a discussion of the information available on the manuscript. If the agreement cannot be reached among the reviewers, a third researcher will provide the final decision on the inclusion or exclusion of the article.

Data Collection Process

We have developed a draft data extraction table (Supplementary Material, Table S3). To test the extraction table, we will extract data for the first five studies, then meet—as a research team—and review and amend as necessary. Data will then be extracted for the rest of the included studies. Two researchers will independently undertake data extraction. Any discrepancies in data extraction will be resolved by discussion of information available in the manuscript.

2.1.5. Data Items

We will extract the following from included studies: 1. Citation (surname and initial of corresponding author, title of the manuscript, year of publication), 2. Corresponding author, 3. Detail of information reported at each phase of concept mapping, and 4. Participant characteristics.
In addition, we will extract the following information that will be used in phase two: 1. Email address for the corresponding author, 2. Names (last name and initial) of all authors 2. Title of the journal, 3. Name and email address of handling editor, 4. Name and email address of the reviewer (if available).

2.1.6. Risk of Bias

Our primary aim is to identify the pattern in reporting concept mapping manuscripts. Therefore, we do not intend to undertake a risk of bias assessment.

2.1.7. Stage 1—Summary

In summary, our systematic review will identify how existing concept mapping studies are reported, identify potential candidate reporting items, and gather information to inform stage 2 of this study.

2.2. Stage 2: Concept Mapping

The second stage of guideline development will be a concept mapping study. We acknowledge possible confusion to readers that we are using concept mapping methodology to develop a reporting guideline for concept mapping. Concept mapping is potentially an appropriate methodology for generating and prioritizing candidate items for a reporting guideline [30]. Moher et al., (2010)—describing the EQUATOR network methodology for guideline development—recommends using the Delphi methodology for identifying candidate items to include in the guideline [28]. That said, we argue that concept mapping has several important advantages over the Delphi methodology: 1. Participants generate candidate items to include in the guideline and 2. The relative importance of candidate items can be ranked (meaning that potentially less important items can be removed).
We will follow the six phases of a conventional concept mapping study as described by Kane and Trochim (2007) [30].

2.2.1. Phase 1: Preparation

The first phase of concept mapping is to determine the focus of the study and decide who the participants will be. The research team has identified the focus as “What are the items that need to be included in a reporting guideline for concept mapping research”?

Participants

Consistent with other guideline development research [31,32,33,34,35,36,37,38], participants in our concept mapping study will be: 1. authors of concept mapping studies (researchers), 2. people involved in at least five concept mapping studies (that we consider concept mapping or methodological experts), 3. journal editors (that have published concept mapping research), 4. academic peer-reviewers (that have reviewed at least one concept mapping study), 5. Statisticians (who have conducted a statistical analysis of concept mapping research), and 6. people that have previously participated in a concept mapping study (consumers).
Study participants have not traditionally been involved in guideline development; we think this is an important omission. By engaging people that have previously participated in concept mapping, we consider that we are extending the methodological rigor of the process of guideline development.

Sample Size

There is no specific number of participants recommended for a concept mapping study. Kane & Trochim (2007) suggest at least five participants can produce meaningful data [30]. We reviewed eight manuscripts detailing the development of reporting guidelines for health research. Between 12 and 133 people (researchers, methodological experts, statisticians, and journal editors) participated in the guideline development process [31,32,33,34,35,36,37,38]. Six out of eight studies included between 25 and 33 participants [32,34,35,36,37,38]. The EQUATOR-network methodology (for guideline development) states that at least 30 participants—that should include researchers, journal editors, academic reviewers, methodological experts, and statisticians—contribute to the development of a guideline [39]. We, therefore, plan to recruit approximately 37 participants to the concept mapping phase of guideline development.
We will recruit the following number of participants from different stakeholder groups: 1. Researchers (that have previously undertaken a concept mapping study) (n = 15), 2. Journal editors (n = 5), 3. Academic reviewers (n = 5), 4. Methodologists (n = 5), 5. Consumers (people who have participated in a concept mapping study) (n = 5), and 6. Statisticians (n = 2).

Demographic Information

Participant demographic information will be collected using REDCap research data capture software prior to the brainstorming task (phase 2). We will collect the following demographic information: 1. Age (in years), 2. Gender (coded as male, female, other), 3. Country of current residence, 4. Highest academic qualification (coded as no degree, baccalaureate, post-graduate, doctoral), and 5. Stakeholder group (coded researchers, methodological experts, journal editors, peer-reviewers, statisticians, consumers).

2.2.2. Phase 2: Brainstorming

Individual interviews will be conducted remotely using a password-protected videoconferencing platform. Participants will be asked to respond to the focus question “What are the items that need to be included in the report of a concept mapping study?”. Interviews will be audio recorded and transcribed.
From interview transcripts, we will extract a list of statements that are singular, specific, and not overlapping. Duplicate statements will be removed. If we generate more than 98 statements (the maximum number of statements that can be handled by the software package), we will combine similar statements until the total number is below this threshold, this is a standard part of concept mapping methodology [40,41].

2.2.3. Phase 3: Structuring of Statements

We will use the online software package “Ariadne” to complete this task [40,41]. We obtained written permission from the software developer to use “Ariadne” in our research. In this phase, participants will be given a pack of statements and asked to undertake two tasks: prioritization and clustering.

Prioritization

In this task, statements are ranked in order of importance ((1 = least important) through (5 = most important)). Participants are instructed to ensure that each point on the scale has an equal number of statements (for example, if there are 50 statements, participants would need to place 10 statements on each point on the scale).

Clustering

In this task, participants are told to sort an identical list of statements into clusters (groups of statements that seem to the participant to belong together). They are then asked to generate potential labels for these clusters.

Instructions for Completing the Tasks

A link to the “Ariadne” website will be emailed to participants where they can undertake the prioritizing and clustering tasks. A step-by-step guide on how to complete this task (DOI: 10.26181/20037188) will be attached to the email.
Once the prioritizing and clustering tasks have been completed, we will ask participants if they would like to take part in the final interpretation phase (phase 5) of the study.

2.2.4. Phase 4: Representation of Statements

Data analysis will be undertaken using the “Ariadne” software package [40]. “Ariadne” produces a binary coding for the statements based on how they are grouped. Each pair of statements grouped together will be assigned “1” and “0” will be assigned for pairs that are not grouped together. A principal component analysis is computed to generate a two-dimensional concept map. Individual statements will be represented on the map as dots. The distance between two statements is proportionate to the frequency of them being grouped together by the participants. A hierarchical cluster analysis will then be conducted to produce candidate cluster solutions. “Ariadne” will generate up to 16 separate cluster solutions, each of which needs to be reviewed and interpreted [41].

2.2.5. Phase 5: Interpretation of Map

Research participants will be engaged in the interpretation of the candidate maps. This phase of the study will be conducted using a video conferencing platform. Participants who agree to be a part of the interpretation workshop will be emailed, two weeks in advance, to determine if they will be available to attend. We will send a meeting link to the first six participants who reply to our email request. We will acknowledge all other participants, whom we will not be able to accommodate in the interpretation workshop.
The research team will facilitate a discussion about the strengths and limitations of each cluster solution produced during the data analysis phase. We will then ask the participants to review and interpret each of the candidate concept maps with a view to inform the structure and items to include in the reporting guidelines. The group will select the map that, in their view, offers the most robust way of understanding the data.

2.2.6. Phase 6: Utilization of Concept Maps

Statement and clusters from our concept map will be considered as candidate items (and groups) for inclusion in our reporting guideline for concept mapping research.

2.3. Stage 3: Development of a Draft Reporting Guideline

We will synthesize findings from phase 1 (systematic review) and phase 2 (concept mapping study) to produce a draft reporting guideline. The concept map will be elaborated using information obtained during the brainstorming sessions and how these items have been reported in existing concept mapping studies (based on the findings from the systematic review—phase 1). For this phase of the study, we will form an expert panel comprising the research team plus three concept mapping experts. We will ask concept mapping experts that participated in the study if they would like to join the expert panel. A draft reporting guideline will then be developed. The expert panel will review the preliminary draft of the reporting guideline. Feedback will be incorporated into a final draft of the reporting items for concept mapping research. The guideline will be submitted to the EQUATOR network for consideration for inclusion.

3. Conclusions

The outcome of the study will be a draft reporting guideline for concept mapping research. By developing a reporting guideline, we seek to enhance the rigor and transparency of reporting of concept mapping research. Reporting guidelines will help to benefit populations that are the focus of the research. Prospective registration of our study methodology with the OSF and the website of the EQUATOR network will contribute to enhancing the transparency of this guideline development process. To facilitate the use of this guideline by researchers across the globe, the guideline will be published in an open-access journal and on the website of the EQUATOR network.
By using concept mapping to inform guidelines development, we are testing a different methodology that is traditionally used (Delphi) in guidelines development and we consider that it will be informative to other research groups developing reporting guidelines.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ijerph19127273/s1, Table S1: PRISMA-P statement, Table S2: MEDLINE search strategy, and Table S3: Pilot data extraction table.

Author Contributions

All three authors were involved in the conceptualization and design of the study. S.P. prepared the first draft of the manuscript. All authors are equally involved in manuscript revision for intellectual content. The final manuscript has been reviewed and approved by all authors. All authors have read and agreed to the published version of the manuscript.

Funding

The corresponding author is supported by a La Trobe University Post Graduate Research Scholarship (LTUPRS) and La Trobe University Full Fee Research Scholarship (LTUFFRS). Additional internal or external funding was not available for the study and for the manuscript publication.

Institutional Review Board Statement

Ethical approval is not required for stage 1 of the guideline development process (systematic review). The human research ethics committee at La Trobe University approved stage 2 and stage 3 of this protocol (Ethics approval number: HEC 21286; Date of Approval: 9 September 2021).

Informed Consent Statement

Written informed consent will be obtained from the participants. Additionally, verbal consent will be obtained for the audio recording of the interview. Participants will be informed that they can withdraw from the study at any point within four weeks of data collection. A withdrawal form will be provided to the participants along with the participant information and consent form.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare that they have no conflict of interest.

Abbreviations

PRISMAPreferred Reporting Guidelines for Systematic Reviews and Meta-Analysis
EQUATOREnhancing the QUAlity and Transparency Of health Research
GRAMMSGood Reporting of a Mixed-Methods Study
OSFOpen Science Framework
MEDLINEMedical Literature Analysis and Retrieval System Online
CINAHLCumulative Index to Nursing and Allied Health Literature
PRESSPeer Review of Electronic Search Strategies
MeSHMedical Subject Headings

References

  1. Trochim, W.; Kane, M. Concept mapping: An introduction to structured conceptualization in health care. Int. J. Qual. Health Care 2005, 17, 187–191. [Google Scholar] [CrossRef] [PubMed]
  2. Daley, B.J.; Torre, D.M. Concept maps in medical education: An analytical literature review. Med. Educ. 2010, 44, 440–448. [Google Scholar] [CrossRef] [PubMed]
  3. Rosas, S.R.; Kane, M. Quality and rigor of the concept mapping methodology: A pooled study analysis. Eval. Program Plan. 2012, 35, 236–245. [Google Scholar] [CrossRef] [PubMed]
  4. Ayala, E.E.; Almond, A.L. Self-care of women enrolled in health service psychology programs: A concept mapping approach. Prof. Psychol. Res. Pract. 2018, 49, 177–184. [Google Scholar] [CrossRef]
  5. Brown, E.; Topping, A.; Cheston, R. What are the barriers to accessing psychological therapy in Qatar: A concept mapping study. Couns. Psychother. Res. 2019, 19, 441–454. [Google Scholar] [CrossRef]
  6. Moattari, M.; Soleimani, S.; Moghaddam, N.J.; Mehbodi, F. Clinical concept mapping: Does it improve critical thinking of nursing students? Iran. J. Nurs. Midwifery Res. 2014, 19, 70–76. [Google Scholar] [PubMed]
  7. Grewal, E.K.; Campbell, R.B.; Booth, G.L.; McBrien, K.A.; Hwang, S.W.; O’Campo, P.; Campbell, D.J. Using concept mapping to prioritize barriers to diabetes care and self-management for those who experience homelessness. Int. J. Equity Health 2021, 20, 158. [Google Scholar] [CrossRef]
  8. Rosas, S.R.; Camphausen, L.C. The use of concept mapping for scale development and validation in evaluation. Eval. Program Plan. 2007, 30, 125–135. [Google Scholar] [CrossRef]
  9. Osborne, R.H.; Elsworth, G.R.; Whitfield, K. The Health Education Impact Questionnaire (heiQ): An outcomes and evaluation measure for patient education and self-management interventions for people with chronic conditions. Patient Educ. Couns. 2007, 66, 192–201. [Google Scholar] [CrossRef]
  10. Wallace, C.; Elliott, M.; Pontin, D. Identifying Quality Indicators for Health Visiting Using Group Concept Mapping. 2020. Available online: https://ssrn.com/abstract=3566179 (accessed on 18 January 2022).
  11. Rising, K.L.; LaNoue, M.; Gentsch, A.T.; Doty, A.; Cunningham, A.; Carr, B.G.; Hollander, J.E.; Latimer, L.; Loebell, L.; Weingarten, G. The power of the group: Comparison of interviews and group concept mapping for identifying patient-important outcomes of care. BMC Med. Res. Methodol. 2019, 19, 1–9. [Google Scholar] [CrossRef] [Green Version]
  12. Rosas, S.R.; Ridings, J.W. The use of concept mapping in measurement development and evaluation: Application and future directions. Eval. Program Plan. 2017, 60, 265–276. [Google Scholar] [CrossRef] [PubMed]
  13. Donnelly, J.P. A systematic review of concept mapping dissertations. Eval. Program Plan. 2017, 60, 186–193. [Google Scholar] [CrossRef] [PubMed]
  14. Equator-Network. Reporting Guidelines. Available online: https://www.equator-network.org/?post_type=eq_guidelines&eq_guidelines_study_design=mixed-methods-studies&eq_guidelines_clinical_specialty=0&eq_guidelines_report_section=0&s= (accessed on 16 June 2021).
  15. Simera, I.; Altman, D.G. Writing a research article that is “fit for purpose”: EQUATOR Network and reporting guidelines. BMJ Evid. Based Med. 2009, 14, 132–134. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. More, S.J. Improving the quality of reporting in veterinary journals: How far do we need to go with reporting guidelines? Vet. J. 2010, 184, 249–250. [Google Scholar] [CrossRef] [PubMed]
  17. Simera, I.; Moher, D.; Hirst, A.; Hoey, J.; Schulz, K.F.; Altman, D.G. Transparent and accurate reporting increases reliability, utility, and impact of your research: Reporting guidelines and the EQUATOR Network. BMC Med. 2010, 8, 24. [Google Scholar] [CrossRef] [Green Version]
  18. Nawijn, F.; Ham, W.H.; Houwert, R.M.; Groenwold, R.H.; Hietbrink, F.; Smeeing, D.P. Quality of reporting of systematic reviews and meta-analyses in emergency medicine based on the PRISMA statement. BMC Emerg. Med. 2019, 19, 19. [Google Scholar] [CrossRef]
  19. Group Wisdom. Group Concept Mapping Resource Guide. Available online: https://groupwisdom.com/gcmrg (accessed on 20 June 2021).
  20. Cardwell, R.; McKenna, L.; Davis, J.; Gray, R. How is clinical credibility defined in nursing? A concept mapping study. J. Clin. Nurs. 2021, 30, 2441–2452. [Google Scholar] [CrossRef]
  21. Dopp, A.R.; Parisi, K.E.; Munson, S.A.; Lyon, A.R. Aligning implementation and user-centered design strategies to enhance the impact of health services: Results from a concept mapping study. Implement. Sci. Commun. 2020, 1, 17. [Google Scholar] [CrossRef] [Green Version]
  22. Moher, D.; Shamseer, L.; Clarke, M.; Ghersi, D.; Liberati, A.; Petticrew, M.; Shekelle, P.; Stewart, L.A.; Group, P.-P. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 2015, 4, 1. [Google Scholar] [CrossRef] [Green Version]
  23. Page, M.J.; Shamseer, L.; Tricco, A.C. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst. Rev. 2018, 7, 32. [Google Scholar] [CrossRef]
  24. Jüni, P.; Holenstein, F.; Sterne, J.; Bartlett, C.; Egger, M. Direction and impact of language bias in meta-analyses of controlled trials: Empirical study. Int. J. Epidemiol. 2002, 31, 115–123. [Google Scholar] [CrossRef] [PubMed]
  25. Morrison, A.; Polisena, J.; Husereau, D.; Moulton, K.; Clark, M.; Fiander, M.; Mierzwinski-Urban, M.; Clifford, T.; Hutton, B.; Rabb, D. The effect of English-language restriction on systematic review-based meta-analyses: A systematic review of empirical studies. Int. J. Technol. Assess. Health Care 2012, 28, 138–144. [Google Scholar] [CrossRef] [PubMed]
  26. Moher, D.; Pham, B.; Lawson, M.L.; Klassen, T.P. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health Technol. Assess. 2003, 7, 1–90. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. McGowan, J.; Sampson, M.; Salzwedel, D.M.; Cogo, E.; Foerster, V.; Lefebvre, C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J. Clin. Epidemiol. 2016, 75, 40–46. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Moher, D.; Schulz, K.F.; Simera, I.; Altman, D.G. Guidance for developers of health research reporting guidelines. PloS Med. 2010, 7, e1000217. [Google Scholar] [CrossRef] [Green Version]
  29. Babineau, J. Product review: Covidence (systematic review software). J. Can. Health Libr. Assoc. 2014, 35, 68–71. [Google Scholar] [CrossRef]
  30. Kane, M.; Trochim, W. Concept Mapping for Planning and Evaluation; Sage Publications: Thousand Oaks, CA, USA, 2007; Volume 50, p. 217. [Google Scholar]
  31. Begg, C.; Cho, M.; Eastwood, S.; Horton, R.; Moher, D.; Olkin, I.; Pitkin, R.; Rennie, D.; Schulz, K.F.; Simel, D.; et al. Improving the quality of reporting of randomized controlled trials. JAMA 1996, 276, 637–639. [Google Scholar] [CrossRef]
  32. Bossuyt, P.M.; Reitsma, J.B.; Bruns, D.E.; Gatsonis, C.A.; Glasziou, P.P.; Irwig, L.M.; Lijmer, J.G.; Moher, D.; Rennie, D.; De Vet, H.C. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. Radiology 2003, 226, 24–28. [Google Scholar] [CrossRef] [Green Version]
  33. Chan, A.-W.; Tetzlaff, J.; Altman, D.G.; Laupacis, A.; Gøtzsche, P.C.; Krleža-Jerić, K.; Hróbjartsson, A.; Mann, H.; Dickersin, K.; Berlin, J.A.; et al. SPIRIT 2013 statement: Defining standard protocol items for clinical trials. Ann. Intern. Med. 2013, 158, 200–207. [Google Scholar] [CrossRef] [Green Version]
  34. Gagnier, J.J.; Kienle, G.; Altman, D.G.; Moher, D.; Sox, H.; Riley, D.; Group, C. The CARE guidelines: Consensus-based clinical case report guideline development. J. Clin. Epidemiol. 2014, 67, 46–51. [Google Scholar] [CrossRef]
  35. Moher, D.; Cook, D.J.; Eastwood, S.; Olkin, I.; Rennie, D.; Stroup, D.F. Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. Oncol. Res. Treat. 2000, 23, 597–602. [Google Scholar] [CrossRef] [PubMed]
  36. Stevens, A.; Garritty, C.; Moher, D. Developing PRISMA-RR, a Reporting Guideline for Rapid Reviews of Primary Studies (Protocol). EQUATOR Network. 2018. Available online: https://www.equator-network.org/wp-content/uploads/2018/02/PRISMA-RR-protocol.pdf (accessed on 20 July 2021).
  37. Stroup, D.F.; Berlin, J.A.; Morton, S.C.; Olkin, I.; Williamson, G.D.; Rennie, D.; Moher, D.; Becker, B.J.; Sipe, T.A.; Thacker, S.B. Meta-analysis of observational studies in epidemiology: A proposal for reporting. JAMA 2000, 283, 2008–2012. [Google Scholar] [CrossRef] [PubMed]
  38. von Elm, E.; Altman, D.G.; Egger, M.; Pocock, S.J.; Gotzsche, P.C.; Vandenbroucke, J.P.; Initiative, S. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Bull. World Health Organ. 2007, 85, 867–872. [Google Scholar] [CrossRef] [PubMed]
  39. Equator-Network. Toolkits. Available online: https://www.equator-network.org/toolkits/ (accessed on 16 June 2021).
  40. Severens, P. Handbook Concept Mapping; Linde: Utrecht, The Netherlands, 1995. [Google Scholar]
  41. Severans, P. Manual Ariadne 3.0. 2015. Available online: http://www.minds21.org (accessed on 10 February 2020).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Pantha, S.; Jones, M.; Gray, R. Development of a Guideline to Enhance the Reporting of Concept Mapping Research: Study Protocol. Int. J. Environ. Res. Public Health 2022, 19, 7273. https://doi.org/10.3390/ijerph19127273

AMA Style

Pantha S, Jones M, Gray R. Development of a Guideline to Enhance the Reporting of Concept Mapping Research: Study Protocol. International Journal of Environmental Research and Public Health. 2022; 19(12):7273. https://doi.org/10.3390/ijerph19127273

Chicago/Turabian Style

Pantha, Sandesh, Martin Jones, and Richard Gray. 2022. "Development of a Guideline to Enhance the Reporting of Concept Mapping Research: Study Protocol" International Journal of Environmental Research and Public Health 19, no. 12: 7273. https://doi.org/10.3390/ijerph19127273

APA Style

Pantha, S., Jones, M., & Gray, R. (2022). Development of a Guideline to Enhance the Reporting of Concept Mapping Research: Study Protocol. International Journal of Environmental Research and Public Health, 19(12), 7273. https://doi.org/10.3390/ijerph19127273

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop