Abstract
The National Research Mentoring Network (NRMN) Strategic Empowerment Tailored for Health Equity Investigators (SETH) study evaluates the value of adding Developmental Network to Coaching in the career advancement of diverse Early-Stage Investigators (ESIs). Focused NIH-formatted Mock Reviewing Sessions (MRS) prior to the submission of grants can significantly enhance the scientific merits of an ESI’s grant application. We evaluated the most prevalent design, analysis-related factors, and the likelihood of grant submissions and awards associated with going through MRS, using descriptive statistics, Chi-square, and logistic regression methods. A total of 62 out of 234 applications went through the MRS. There were 69.4% that pursued R grants, 22.6% career development (K) awards, and 8.0% other grant mechanisms. Comparing applications that underwent MRS versus those that did not (N = 172), 67.7% vs. 38.4% were submitted for funding (i.e., unadjusted difference of 29.3%; OR = 4.8, 95% CI = (2.4, 9.8), p-value < 0.0001). This indicates that, relative to those who did not undergo MRS, ESIs who did, were 4.8 times as likely to submit an application for funding. Also, ESIs in earlier cohorts (1–2) (a period that coincided with the pre COVID-19 era) as compared to those who were recruited at later cohorts (3–4) (i.e., during the peak of COVID-19 period) were 3.8 times as likely to submit grants (p-value < 0.0001). The most prevalent issues that were identified included insufficient statistical design considerations and plans (75%), conceptual framework (28.3%), specific aims (11.7%), evidence of significance (3.3%), and innovation (3.3%). MRS potentially enhances grant submissions for extramural funding and offers constructive feedback allowing for modifications that enhance the scientific merits of research grants.
1. Introduction
Early-Stage Investigators’ (ESIs) research skills development is a paramount mission of many academic institutions in the USA [1,2,3,4,5,6,7,8,9]. A pivotal strategy to achieve such mission is through mentoring and training [4,5,6,7,8,9]. Nevertheless, previous studies indicated that researchers from less privileged/underrepresented societies have less access to quality traditional dyadic mentoring [10,11]. Ginther et al. also reported an alarming differences in award rates for research grant (R01) applications between White and Black applicants, albeit adjusting for several confounders [10]. Recently, Ransdell et al., 2021 [1] reported results based on 46 PsychINFO, CINAHL, and PubMed published papers between 2010 and 2020, impediments to research development among ESIs from underrepresented minority faculty that included bias, discrimination, isolation, an institutional lack of mentors, and lack of appreciation of experience and/or expertise. Per the NIH, “an ESI is an investigator on a research track to becoming a Program Director or a Principal Investigator (PD/PI) after completing his/her terminal research degree or end of post-graduate clinical training (whichever date is later) within the past 10 years and who has not previously successfully competed as a PD/PI for a substantial NIH independent research award (Investigator Career Stage Benefits NIH Center for Scientific Review”. In our NRMN SETH study, ESIs are investigators on a research career track but have not yet received independent NIH R01 or equivalent funding.
Our primary hypothesis is that developmental networks (DN) when added to grant-writing coaching can significantly enhance ESIs’ research careers [4,12]. Our NIH-sponsored Seth nested cluster-randomized study [5,6,7,8,9,12] was launched to specifically determine the magnitude and significance of adding DN to structured coaching, among diverse ESIs. Notably, the ESIs’role in the study is to work (within their institutions) closely with their study-assigned coaches, mentors and, if randomized to the DN group, their network study [5,6,7,8,9,12] assigned developer, to prepare and submit, within 12 months of recruitment into the study, grant application(s) for the NIH K, R, U and/or minority supplement awards or other comparable federal funding mechanisms.
1.1. Mock Review Sessions (MRS)
To address the mission of enhancing the research experience of the NRMN SETH’s ESIs, we added to our curricula of training a focused NIH-formatted MRS emphasizing NIH criteria of scientific rigor and reproducibility.
1.1.1. NIH Mock Study Section [13]
Since 2014, the Institute of Clinical and Translational Sciences (ICTS) at NIH offers NIH Mock Study Section aimed at Improving K, R, and F series grant submissions. The sessions are aimed at simulating an actual NIH study section through the Research Development Program. The review sessions for clinical and translational K, R, and F series grant applications are offered twice annually, in Spring and Winter. According to NIH, the purpose of this program is to “increase the likelihood of grant funding success by providing comprehensive, study section-like feedback to applicants on their complete grant application prior to grant submission”.
1.1.2. MRS at US Institutions’ Health Science and Clinical Schools vis-à-vis MRS at NRMN SETH Program
The model of MRS has been conceptually implemented at several U.S. institutions, academic medical centers, and schools of medicine to help applicants for extramural funding to enhance their applications and maximize their chances to obtain favorable reviews that may result in funding their research (Table 1). It should be emphasized that these institutions’ MRS mechanisms are aimed at servicing their own intrinsic goals that are targeted towards their ESI scholars with no emphasis on minority subgroup inclusion. Our NRMN/SETH program is nationally focused with centralized MRS services targeted towards ESIs from smaller institutions such as the Research Centers in Minority Institutions (RCMI) which serve historically underserved students and generally have limited resources to provide NIH format and quality of MRS review of RCMI applications for funding [14,15].
Table 1.
Sample of Mock Review Session (MRS) models at US Institutions.
1.2. Adding Focused NIH-Formatted MRS to NRMN SETH Training Curriculum
During the conduct of the NRMN SETH program and to enhance the ESIs’ scientific research capabilities that could potentially result in successful grant submissions, we added NIH-tailored MRS scheduled prior to the submission date of the grant applications.
The aims of this manuscript are to (1) characterize the role of the MRS in terms of structure and utilization, (2) evaluate the association between going through an MRS and submissions and awards of extramural research grants and (3) present the most prevalent design, conduct and analysis—specific issues deemed deficient by the mock reviewers.
2. Materials and Methods
Our study has thus far recruited 210 ESI participants since 14 December 2019, through four cohorts sequentially recruited over time. ESIs are required to work with their coaches and developers (for those randomized to the coaching plus DN group) and submit at least one NIH extramural application for funding within the first 12 months of their enrollment in the program. MRS were optional for cohorts 1–2 and became mandatory starting July 2020 for subsequent cohorts. Figure 1 illustrates the interfacing of the flow of the study design with the inclusion of the MRS process.
Figure 1.
Flowchart of the interfacing of the design of the randomized NRMN SETH Study with the mock reviewing process.
2.1. Composition of the MRS Reviewers and Criteria of Review
Two to three seasoned members of the NRMN SETH study team with prior NIH reviewing experience were assigned to conduct the mock reviews. Applicants were asked to submit their completed draft applications 2–3 weeks before convening the MRS. Applications were then sent to at least two reviewers, who convened a 30-min MRS session.
The MRS were tailored after the NIH format which focuses on the scientific rigor and reproducibility [22], and feasibility of conducting and successfully completing the planned research. The NIH criteria focuses on (a) significance of the proposed research topic in terms of the pre-existing gaps in the scientific, clinical and public health literature, (b) the suitability of the investigator(s) background, experience, training and research/scientific abilities to successfully conduct and achieve the proposed research aims, (c) innovation of the methodology and approaches underlying the design and conduct of the proposed research topic, (d) appropriateness and reasonability of the proposed overall approach, strategy, methodology, and analysis to accomplishing the specific aims of the project, as well as protection of human subjects, inclusion of minorities, the inclusion of biological sex as a variable, and the inclusion of children, (e) alignment of institutional environment and available resources with implementation needs of the proposed research topic, so that the successful completion of the project is highly likely and (f) justification of human subjects inclusion and plans for their protection when they participate in the proposed research. Based on these criteria, we computed the frequency distributions of the most prevalent design, conduct and analysis-related factors that were deemed deficient and/or insufficient.
2.2. Conduct of the MRS
After introductions and procedure review, the applicant and coaches are muted. Each reviewer states an initial 1–9 NIH-tailored score. Consequently, each of the two reviewers gives a 10–12 min detailed NIH-style review of the application stating its strengths and weaknesses, with further discussion followed by restating priority scores. If time allows, the applicant can unmute for a few questions at the end. In 1–2 days following the MRS, feedback is sent to the applicant through live MRS video recording of the session for later review, along with the reviewers’ written critiques in NIH Summary Statement format. The intent is for the applicant to use the critiques in revising the application prior to final submission to NIH. It should also be noted that the MRS members provide scientifically-based suggestions to the respective ESI to assist in formulating remedies to address the cited critiques and enhance the scientific rigor of the application.
2.3. Quantitative Data Analysis Plan
2.3.1. Study Variables
The primary outcomes included (1) submission and (2) award for funding of an NIH or other extramural grant application. The potential predicting covariates we considered for these analyses included (1) whether the ESI’s application underwent a mock review session (Yes/no), (2) type of sought mechanism of funding, i.e., R-series research grants, career development (K series) grants and others (Support of Competitive Research (SCORE) and/or National Science Foundation grants), and (3) Cohort effect, as a continious covariate (1, 2, 3, 4) and in separate models, to potentially measure COVID-19 effects (4) Early cohorts (1–2) (Pre-COVID19 era) vs. late cohorts (3–4) (when COVID-19 pandemic peaked). MRS were optional for cohorts 1–2 and became mandatory starting July 2020 starting with cohort 3. The unit of analysis was the ESI’s prepared grant application (whether or not submitted). There were (n = 234) prepared applications, produced by (n = 210) ESIs. Several ESIs had multiple applications, either through different mechanisms or resubmissions (Table 2).
Table 2.
Number, percentage and cumulative frequency of prepared grant applications for submission by ESIs in the NRMN-SETH Study.
2.3.2. Data Analysis
Summary of the data is presented using frequencies and percentages of grant submissions and awards vis-à-vis MRS sought mechanism of funding, recruitment cohorts, and MRS periods (before and after it became mandatory) and bar chart graphics. We initially tested separately in a univariate manner using Chi-Square or Fisher Exact tests [23] the independent association of each of the grant submissions and awards with (a) going through an MRS (yes/no), (b) sought mechanism of funding, (c) early or late cohort recruitment and (d) MRS periods. Multivariate testing was pursued using unconditional independent (as well as interaction-based models) logistic regression [24] for assessing independent associations of these factors with the likelihood of grant submission or award.
In a mutually exclusive iterative process, we also tested cohort effects and MRS period (optional vs. mandatory). To accommodate the effects of multiple applications per ESI, additional analyses used conditional logistic regression principles to test for these effects using ESI as a cluster variable. Results are visually demonstrated using odds ratio graphs. Generally, for these graphs (c.f., Figure 2 and Figure 3 in the results section), each solid line with whiskers at the end and a dot in the middle represents the point and interval estimate (magnitude) of the Odds Ratio(s) (OR) of the outcome when comparing two subgroups of a particular independent/predictive factor (in Figure 2 for example, the dot in the middle of the top solid line represents the estimated (from the data) odds (likelihood) of applying for extramural funding for the subgroup that underwent MRS when it was optional (cohorts 1–2) divided by the corresponding odds (likelihood) of those who underwent MRS after it became mandatory (cohorts 3 + i.e., the reference subgroup). The length of the solid line in those graphs for each comparison shows the width of the 95% confidence interval of the “population” odds ratios (i.e., the range that the OR can take in the underlying population of such data). As long as the solid line does not include the value “1” (which represents equality of the two compared odds) that means the ratio of the two odds are not the same (i.e., statistically significantly different from “1” at a 0.05 significance level). The overall significance level for the test of hypotheses was set at 0.05 with provision for multiple comparisons adjustment when needed. Statistical analyses were conducted using SAS version 9.4 and R version 4.2.2
Figure 2.
Modeling probability of applying for extramural funding with MRS period (MRS was optional for cohorts 1–2 and became mandatory for cohorts 3 and subsequent cohorts) *. * See the Methods Section for quantitative data analyses for interpretation of these graphs.
Figure 3.
Distribution of grants submitted by mock review status.
3. Results
Through cohorts 1–4, our study has thus far recruited 210 ESI participants that are deemed eligible to submit applications for funding. The 210 ESIs produced 234 applications for extramural funding. By October 2022, of these applications 62 (26.5%) went through an MRS with proposed mechanisms of funding distributed as follows: 43 (69.4%) R-series research grants, 14 (22.6%) career development (K series) grants and 5 (8.1%) others (Support of Competitive Research (SCORE) and National Science Foundation grants). Before and after the mock review became mandatory, 32 (51.6%) and 30 (48.4%) MRS were, respectively, held. Table 3 provides additional frequency distributions, overall, by mock review period and going through an MRS (yes/no).
Table 3.
NRMN SETH study frequency distributions of grant submissions and awards by mock review periods, status sought mechanism of funding and recruitment cohorts up to August, 2022.
3.1. Submission and Awards vis-à-vis MRS
Table 3 and Figure 3 indicate that when comparing applications that went through MRS (n = 62) versus those that did not (n = 172), 42 (67.7%) vs. 66 (38.4%), respectively, were submitted for extramural funding. The corresponding comparisons for applications that were awarded funding were 19.35% vs. 24.42% (8 (8.6%) of the submitted applications are still pending decision outcomes from their funding agencies/entities). Table 3 further gives frequency distributions of submitted/awarded applications by MRS period, sought mechanism of funding and recruitment cohorts.
3.2. Multivariate Analyses-Based Findings
Upon adjusting for sought mechanism of funding, cohort effects and (separately) MRS period, our results indicated that going through MRS (yes vs. no) was significantly associated with submitting an application for extramural funding (those who went through MRS were at least 4.8 times as likely to submit an application (OR (95% CI) and p-values were 4.8 (2.1, 8.7) p-value < 0.0001, Table 4 and Figure 4). Additionally, those who were recruited in later cohorts (cohorts 2, 3, 4 vs. cohort 1) were on average 49% less likely to submit an application for extramural funding (OR (95% CI) and p-value were 0.51 (0.38, 0.70) p-value < 0.0001 (Table 3 and Figure 3). Additionally, based on separate modeling iterations, ESIs in earlier cohorts (1–2) (a period that coincided with the pre COVID-19 era) as compared to those who were recruited at later cohorts (3–4) (i.e., during the peak of COVID-19 period), were 3.8 times as likely to submit grants (p-value < 0.0001) (Table 3 and Figure 2). Furthermore, as compared to those seeking K-series funding, R-series funding and other mechanisms of funding applicants were, respectively, 2.3 (p-value 0.042) and 5.8 (p-value = 0.0002) times as likely to submit for extramural funding.
Table 4.
NRMN SETH Study frequency distributions of grant submissions and awards by mock review periods, status sought mechanism of funding and recruitment cohorts up to August, 2022.
Figure 4.
Modeling Probability of applying for NIH/Extramural Funding with cohort effect (without the covariate MRS period) *. *: See the Methods Section for quantitative data analyses for interpretation of these graphs.
3.3. MRS Findings by Type of Application-Based Deficient Issues
The most prevalent issues (not necessarily mutually exclusive, i.e., some may overlap with each other) that were detected by the MRS were distributed as follows: 45 (75%) lacking and/or insufficient statistical design considerations and statistical analyses plan (SAP) (e.g., missing/generic sample size and statistical power justifications, and lack of clarity identifying outcome measures and covariates/confounders/predictors of outcome), 17 (28.3%) lacking conceptual model/framework, 7 (11.7%) with overlapping specific aims, 2 (3.3%) with insufficient justification/evidence of significance, and 2 (3.3%) with insufficient justification/evidence of innovation (Figure 5).
Figure 5.
Most Prevalent deficiencies/issues (Total Number of MRS-Reviewed = 62 Applications).
4. Discussion
The NRMN SETH randomized study aimed to test the effectiveness of adding DN to structured coaching in enhancing the research capabilities of ESIs to support career advancement to independent investigators.
At the beginning of this project we added NIH-type MRS to enhance the scientific rigor of ESIs’ applications for funding.
In this manuscript, we evaluated the association between going through an MRS and the likelihood of (1) submitting an application for extramural funding and (2) being awarded funding. All submitted applications, as we speak, were included in these analyses. However, to evaluate the impact of MRS on the likelihood of being awarded funding, i.e., to properly determine the association between going through an MRS and getting funded, we needed to allow for the ”lag” time between submission and decision and consider several other predictive/mediating factors like the time-to-submission, whether for an initial application or a resubmission, the ESI’s background discipline, pre-existing self-established readiness (measured for example by number of lead-authored publications in high tier journals), existence of a mentor and existence of within-institution supporting development/incentivizing programs/resources. These factors will be considered in future analyses when we report the final findings of the NRMN SETH randomized study.
Our results showed a significant association (that can also be potentially construed as an impact) between going through an MRS and the likelihood of submitting an application for extramural funding. ESIs who went through an MRS (versus those who did not) were at least 4.8 times as likely to submit for funding (95% confidence interval = (2.8, 9.4); p-value < 0.0001). We intentionally focused on evaluating the potential impact of an ESI’s application going through an MRS on the likelihood of submitting an application for funding. Future analyses will investigate the association of going through an MRS vis-à-vis being awarded funding which will focus on including covariates like time-to-submission, time-to getting awarded funding and the intensity of submission/awards, i.e., multiple submissions by the same ESI.
Our analyses also revealed that earlier cohorts i.e., 1–2 versus later cohorts, i.e., 3–4, were more likely to submit an application for funding. Cohorts 1–2 were 3.8 times as likely to submit an application for funding (95% confidence interval = (2.1, 6.9); p-value < 0.0001). This finding was also confirmed when we portioned MRS into two periods; optional, which spanned cohorts 1–2, and mandatory, when cohorts 3–4 coincided with the MRS becoming mandatory. These findings have a very interesting implication as they point to the impact of COVID-19 on hampering ESIs’ efforts and disrupting continuity of ESIs to engage with their coaches and developers to pursue successful completion of their applications.
Multivariate analyses also indicated that R-series and other mechanisms of funding seekers (versus K-series mechanism) were, respectively, 2.3 and 5.8 times as likely to submit an application for funding (95% confidence intervals and p-values were, respectively, (1.03, 5.3), (2.3, 14.6) and 0.042 and 0.0002). This speaks to the composition of the ESIs in terms of their career passage and points to the fact that the risk of not submitting a K-type application is higher, likely due to the lack of strong institutional commitment, uncertain career future for postdocs, and challenges in identifying mentors with active NIH funding.
Our study also determined the most prevalent deficient issues deemed by MRS experts consequent to reviewing ESIs’ applications prior to their submission dates. Lack of or insufficient components of statistical analyses plans, like missing/generic sample size and statistical power justifications, and lack of (a) clarity identifying outcome measures and (b) covariates/confounders/predictors of outcome, were most prevalent, comprising 75% of the issues. Second to the statistical consideration issues was lack of conceptual model/framework which comprised 28.3% of the deficiencies. Other issues that were also deemed deficient were overlapping specific aims, insufficient justification/evidence of significance, and insufficient justification/evidence of innovation.
4.1. Strengths
To our determination, no published reports presented evaluation of MRS role vis-à-vis the likelihood of submitting for and/or being awarded extramural funding while adjusting for cohort effect and sought mechanism of funding in a structured nested cluster randomization study. The objective nature of analyses’ outcomes also bestowed rigor and reproducibility of results. Finally, analysis methodologies used for evaluating associations were statistically and mathematically sound.
The report presented here provides a rigorous base for theoretical propositions and practical suggestions. Conceptually, this study suggests that a mock study section is a unique strategy for academic development that mediates success in submitting a grant proposal. Future studies can explore the extent to which MRS promotes research self-efficacy, grant writer identity, and sense of belonging in the academic community. Practically, this study shows that since there is evidence of positive contribution of the MRS to the ESI career development progress, participation in MRS-like sessions needs to be included as a required component in grant writing programs. MRS enforces an earlier deadline for preparing a full proposal, which in turn increases the overall chances for funding for ESIs.
4.2. Limitations
This was a cross-sectional evaluation that did not account for the repeated nature of submitting applications for funding. Future analyses will account for repeated time-to-submission and/or getting awarded funding and the intensity of submission/awards, i.e., multiple submissions by the same ESI. Other conceivably impacting factors on the likelihood of submitting an application for funding and/or getting awarded funding, such as the ESI’s background discipline, pre-existing self-established readiness and within-institution mentoring and supporting incentivizing environments, were not included.
5. Conclusions
NRMN-SETH MRS offer timely, specific, and constructive feedback to ESIs to allow for potential modifications/incorporation of suggestions into their applications to help in enhancing the scientific merits of their intended research. We hope that the identified list of potential scientific deficiencies helps in articulating training curricula and protocols to enhance the scientific merits of ESIs’ applications for extramural funding. We would also like to emphasize the data-evidenced significant association and potential impact of MRS on the likelihood of pursuing extramural funding.
Author Contributions
Conceptualization, E.O., P.P., T.P., A.B., J.K.S., M.S.S., W.E.T., K.L., A.Q. and M.M.; methodology, M.M., E.O., A.Q. and P.P.; formal analysis, M.M., K.L. and J.H.; data curation, K.L., J.H. and M.Y.I.; writing—original draft preparation, M.M., E.O., T.P., A.B., M.S.S., L.S.C. and Y.S.; writing—review and editing, All study members; project administration, K.L. and J.H.; funding acquisition, E.O., P.P., J.K.S., M.M. and A.B. All authors have read and agreed to the published version of the manuscript.
Funding
Supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number 1U01GM132771.”|Georgia CTSA support: “Supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1TR002378. |RCMI Support: “Supported by the National Institute of Minority Health and Health Disparities of the National Institutes of Health under Award Number 1U24MD015970.” We also acknowledge the support of the 2019 RCMI Program National Conference, which was made possible, in part, by grant U13MD014961. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Institutional Review Board Statement
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Morehouse School of Medicine.
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from the patient(s) to publish this paper.
Data Availability Statement
Data available on request due to restrictions e.g., privacy or ethical.
Acknowledgments
The authors of this work would like to acknowledge the review, editing, and support of Jessica Faupel-Badger, National Institutes of Health.
Conflicts of Interest
None of the study investigators/personnel have disclosed any conflict of interest associated with being a member in this project.
List of Acronyms
| NRMN | National Research Mentoring Network |
| ESI | Early-Stage Investigator |
| SETH | Strategic Empowerment Tailored for Health Equity |
| NIH | National Institute of Health |
| DN | Developmental Networking |
| MRS | Mock Review Session |
References
- Ransdell, L.B.; Lane, T.S.; Schwartz, A.L.; Wayment, H.A.; Baldwin, J.A. Mentoring New and Early-Stage Investigators and Underrepresented Minority Faculty for Research Success in Health-Related Fields: An Integrative Literature Review (2010–2020). Int. J. Environ. Res. Public Health 2021, 18, 432. [Google Scholar] [CrossRef] [PubMed]
- Gibbs, K.D.; McGready, J.; Griffin, K. Career Development among American Biomedical Postdocs. CBE Life Sci. Educ. 2015, 14, ar44. [Google Scholar] [CrossRef] [PubMed]
- Urizar, G.G.; Henriques, L.; Chun, C.-A.; Buonora, P.; Vu, K.-P.L.; Galvez, G.; Kingsford, L. Advancing Research Opportunities and Promoting Pathways in Graduate Education: A Systemic Approach to BUILD Training at California State University, Long Beach (CSULB). BMC Proc. 2017, 11, 26. [Google Scholar] [CrossRef] [PubMed]
- Yip, J.; Kram, K.E. Developmental Networks: Enhancing the Science and Practice of Mentoring. In The SAGE Handbook of Mentoring; SAGE Publications Ltd.: London, UK, 2017; pp. 88–104. ISBN 978-1-4129-6253-7. [Google Scholar]
- Weber-Main, A.M.; McGee, R.; Boman, K.E.; Hemming, J.; Hall, M.; Unold, T.; Harwood, E.M.; Risner, L.E.; Smith, A.; Lawson, K.; et al. Grant Application Outcomes for Biomedical Researchers Who Participated in the National Research Mentoring Network’s Grant Writing Coaching Programs. PLoS ONE 2020, 15, e0241851. [Google Scholar] [CrossRef] [PubMed]
- Hall, M.; Engler, J.; Hemming, J.; Alema-Mensah, E.; Baez, A.; Lawson, K.; Quarshie, A.; Stiles, J.; Pemu, P.; Thompson, W.; et al. Using a Virtual Community (the Health Equity Learning Collaboratory) to Support Early-Stage Investigators Pursuing Grant Funding. Int. J. Environ. Res. Public Health 2018, 15, 2408. [Google Scholar] [CrossRef] [PubMed]
- Johnson Hemming, J.; Baez, A.; Hall, M.; Thompson, W.; Stiles, J.; Ofili, E. Advancing Health Equity through Organizational Mentoring Policies at Minority-Serving Institutions. Ethn. Dis. 2019, 29, 371–376. [Google Scholar] [CrossRef] [PubMed]
- Sorkness, C.A.; Pfund, C.; Ofili, E.O.; Okuyemi, K.S.; Vishwanatha, J.K.; NRMN team; Zavala, M.E.; Pesavento, T.; Fernandez, M.; Tissera, A.; et al. A New Approach to Mentoring for Research Careers: The National Research Mentoring Network. BMC Proc. 2017, 11, 22. [Google Scholar] [CrossRef] [PubMed]
- Estape, E.S.; Quarshie, A.; Segarra, B.; San Martin, M.; Ríos, R.; Martínez, K.; Ali, J.; Nwagwu, U.; Ofili, E.; Pemu, P. Promoting Diversity in the Clinical and Translational Research Workforce. J. Natl. Med. Assoc. 2018, 110, 598–605. [Google Scholar] [CrossRef] [PubMed]
- Ginther, D.K.; Schaffer, W.T.; Schnell, J.; Masimore, B.; Liu, F.; Haak, L.L.; Kington, R. Race, Ethnicity, and NIH Research Awards. Science 2011, 333, 1015–1019. [Google Scholar] [CrossRef] [PubMed]
- McCreath, H.E.; Norris, K.C.; Calderόn, N.E.; Purnell, D.L.; Maccalla, N.M.G.; Seeman, T.E. Evaluating Efforts to Diversify the Biomedical Workforce: The Role and Function of the Coordination and Evaluation Center of the Diversity Program Consortium. BMC Proc. 2017, 11, 27. [Google Scholar] [CrossRef] [PubMed]
- Mubasher, M.; Lawson, K.; Pemu, P.; Pearson, T.; Engler, J.; Baez, A.; Stiles, J.K.; Salazar, M.S.; Caplan, L.S.; Green, K.; et al. Randomized Controlled Study to Test the Effectiveness of Developmental Network Coaching in the Career Advancement of Diverse Early-Stage Investigators (ESIs): Implementation Challenges and Lessons Learned. Int. J. Environ. Res. Public Health 2021, 18, 12003. [Google Scholar] [CrossRef] [PubMed]
- Ofili, E.O.; Sarpong, D.; Yanagihara, R.; Tchounwou, P.B.; Fernández-Repollet, E.; Malouhi, M.; Idris, M.Y.; Lawson, K.; Spring, N.H.; Rivers, B.M. The Research Centers in Minority Institutions (RCMI) Consortium: A Blueprint for Inclusive Excellence. Int. J. Environ. Res. Public Health 2021, 18, 6848. [Google Scholar] [CrossRef] [PubMed]
- Ofili, E.O.; Tchounwou, P.B.; Fernandez-Repollet, E.; Yanagihara, R.; Akintobi, T.H.; Lee, J.E.; Malouhi, M.; Garner, S.T.; Hayes, T.T.; Baker, A.R.; et al. The Research Centers in Minority Institutions (RCMI) Translational Research Network: Building and Sustaining Capacity for Multi-Site Basic Biomedical, Clinical and Behavioral Research. Ethn. Dis. 2019, 29, 135–144. [Google Scholar] [CrossRef] [PubMed]
- Mock Review. Available online: https://tracs.unc.edu/index.php/services/education/mock-review (accessed on 30 December 2022).
- Grant Mock Review|SC CTSI. Available online: https://sc-ctsi.org/resources/grant-mock-review (accessed on 30 December 2022).
- Mock Review of Grants (MoRe) Program. Penn State College of Medicine Research. Available online: https://research.med.psu.edu/education/mock-review-of-grants/#:~:text=Penn%20State%20College%20of%20Medicine’s,to%20help%20strengthen%20the%20proposal (accessed on 30 December 2022).
- ORD’s Mock Review Panel Program: Making Proposals More Competitive|FSU Office of Research. Available online: https://www.research.fsu.edu/research-offices/ord/about-ord-and-outreachmarketing/digest-home/july-2021-ord-digest/ords-mock-review-panel-program-making-proposals-more-competitive/ (accessed on 30 December 2022).
- NIH Training Grant Mock Study Section | Clinical Research Training Center|Washington University in St. Louis. Available online: https://crtc.wustl.edu/resources/otg/nih-mock-study-section/ (accessed on 30 December 2022).
- Duke|Log. Available online: https://shib.oit.duke.edu/idp/authn/external?conversation=e1s1 (accessed on 30 December 2022).
- Available online: https://icts.wustl.edu/research-services/research-development-program/nih-mock-study-section/ (accessed on 30 December 2022).
- Rigor-and-Reproducibility-Chart-508. Available online: http://grants.nih.gov/grants/RigorandReproducibilityChart508.pdf (accessed on 30 December 2022).
- Agresti, A. An Introduction to Categorical Data Analysis, 2nd ed.; Wiley series in probability and mathematical statistics; Wiley-Interscience: Hoboken, NJ, USA, 2007; ISBN 978-0-471-22618-5. [Google Scholar]
- Hosmer, D.W.; Lemeshow, S. Applied Logistic Regression, 2nd ed.; Wiley series in probability and statistics; Wiley: New York, NY, USA, 2000; ISBN 978-0-471-35632-5. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).