Rotator cuff (RC) tears are among the most frequent injuries of the tendon-bone junction in the human population [1
]. RC repair is one of the most common surgical operations for shoulder disorders [2
] or may be the most common of all shoulder surgeries [3
Randomized controlled trials (RCTs) can be useful to improve the health of patients affected by RC tears. Randomization provides one single important advantage over all the other methodologies: it equipoises known and unknown prognostic elements among treatment groups. For this reason, it is very important to draft an accurate report of the results to make the analysis scientifically creditable [4
Evidence based medicine (EBM) aims to optimize clinical care with the use of published literature, evaluating not only medicine studies, but also other disciplines such as nursing, dentistry, public health, and health policy [5
RCTs must be conducted with careful attention because there are a lot of known and unknown confounding factors. It is useful to implement factors such as the allocation of randomization and to perform an intent-to-treat analysis of the patients minimizing the bias effects [6
However, the quality of RCT reports was improved when CONSORT (consolidated standards of reporting trials) was introduced to reduce the problems arising from the inadequate non-pharmacologic treatments (NPTs) such as surgery, rehabilitation, or physiotherapy RCT reports [7
Our systematic analysis aims to widely evaluate the quality of reporting of RCTs linked to RC tears. In particular, our research investigated potential factors linked to the quality of the RCTs and reports quality trends over time.
2. Materials and Methods
2.1. Search Strategy and Study Eligibility
A systematic analysis of the literature was made following the preferred reported items for systematic review and meta-analysis statement (PRISMA). RCTs and prospective cohort studies on the topic of RC surgery published until 1 March 2020 were included. The following combinations of keywords were used: “rotator cuff”; “rotator cuff tear”; “rotator cuff repair”; “pain after rotator cuff repair”; “rotator cuff physiotherapy”; and “rotator cuff surgery”. All articles were initially screened for relevance by title and abstract excluding articles without an abstract and obtaining the full-text article if the abstract did not allow the investigators to assess the defined inclusion and exclusion criteria. A cross-reference search of the selected articles was also performed to obtain other relevant articles for the study. Inclusion criteria were: level of evidence I or II according to the Oxford Center of EBM; studies on human patients; English-language; and focus on RC tears. Exclusion criteria were: review articles; animal and cadaveric studies; commentary reports; pilot studies; meta-analysis; and conference papers. The articles (titles, abstracts and full-text) were screened independently by two researchers: two investigators (G.S and C.D.N) separately performed a careful reading of all papers and extracted data. In this review, the CONSORT checklist was used independently to assess each trial by the two reviewers, with particular attention to those items that were not addressed in the Coleman score. The CONSORT checklist is a series of 25 items and 37 checklist items focused on reporting how the trial was designed, analyzed, and interpreted. For each item, a yes/no answer, depending on the completeness of the information reported in the study is given. The flow diagram just displays the progress of all participants through the trial and is one of the items in the CONSORT checklist.
2.2. Data Abstraction
The data extracted were recorded using Word and Excel (Microsoft). Full name of the first author, year of publication, number of patients enrolled, number of patients lost at follow-up, level of evidence of the study, location of the study, age of patients, sex of patients, financial support, number of centers involved, the presence of CONSORT flow diagram, sample size, trial type and topic, mean follow-up, type of rehabilitation, and journal of publication were recorded.
2.3. Methodological Quality Assessment
The methodological quality of the included articles was independently evaluated by two reviewers using the modified Coleman methodology score (MCMS). The MCMS is characterized by the presence of eleven criteria evaluated through a scoring system between 0 and 100, where 100 indicates a high-quality study without chances, bias, and other confounding factors [8
]. The MCMS was used for the statistical analysis to evaluate the correlation between the variables involved.
2.4. Assessment of Agreement
The data from the full-text selected articles were extracted independently by two authors (G.S., C.D.N.) and checked by a third author (G.F.). To ascertain the validity of the included studies, two reviewers independently evaluated the methodological quality and reliability of the findings through MCMS.
2.5. Statistical Analysis
To compare the MCMS of articles written before 2010 and articles written after 2010 and to find statistically significant differences in terms of MCMS, between studies containing a CONSORT diagram and studies without the flow-chart of the independent samples t-test was used. The inter-rater reliability of grading the Coleman score and the association between the number of checklist items and the Coleman score for each article with Pearson’s correlation were evaluated. The inter-rater reliability of grading the CONSORT checklist with percent agreement between raters was evaluated.
Rotator cuff disease is among the most common musculoskeletal disorders [184
]. It is a disabling condition with high prevalence rate [185
], causing high direct and indirect costs [186
]. The appropriate treatment for rotator cuff disease is debated. Rotator cuff repair is an option for patients with chronic, symptomatic full-thickness rotator cuff tear, but the quality of evidence is unconvincing. There is also little compelling evidence for conservative treatment [187
]. RCTs are considered to be the most reliable form of scientific evidence in the hierarchy of evidence that influences healthcare policy and practice because RCTs reduce spurious causality and bias. Well done RCTs are the only way to find strong evidence on rotator cuff disease and treatment. This systematic review aimed to analyze factors connected to the quality of the RCTs and trends in the quality of reporting through time. The use of a CONSORT flow diagram, CONSORT checklist, and the MCMS have been found to be the most important predictive factor for a good quality RCT.
Rotator cuff repair can be performed with several techniques: transosseous simple suture, single-row suture anchor, double-row suture anchor, and transosseous-equivalent techniques. Current evidence has failed to prove a significant difference between single-row and double-row techniques in terms of clinical outcomes. The ideal rotator cuff repair technique should provide a good restoration of the footprint contact area and compression of the tendon on the footprint. Moreover, rotator cuff repair is not the only procedure undertaken to manage rotator cuff tears. Acromioplasty, long head of biceps tendon tenotomy, or tenodesis are surgical procedures often performed with rotator cuff repair. Acromioplasty is an arthroscopic surgical procedure made to prevent the impingement of the rotator cuff and consists of the decompression of subacromial space. Stem cell therapy is an emerging treatment for tendon disorders and can be used during arthroscopic rotator cuff repair, after rotator cuff repair, or instead of rotator cuff repair. The real efficacy of stem cell therapy for rotator cuff tears is nowadays debated. The 183 studies analyzed focused on this major topic concerning rotator cuffs. Pain resulted in being the most macro-area studied with 54 articles (29.5% of total RCTs), followed by physiotherapy (31 articles, 17% of total RCTs). Another popular topic is the use of stem cell therapy (27 articles, 15%).
The number of RCTs has increased over the last few years thanks to higher attention to the process of randomization and the prospective calculation of sample size. On the other hand, factors such as the description of concealment of patient allocation and the intention-to-treat analysis did not receive the same attention.
The MCMS, CONSORT flow diagram, and CONSORT checklist have been used to assess the quality of RCTs previously identified. This review showed a significant improvement in the quality in RCTs in the long term. This was confirmed by the studies completed until the year 2010 that had an average MCMS of 68.54 compared to the studies completed between 2011 and 2020 with an average of MCMS of 72.12. A statistically significant difference in MCMS between articles written before 2010 and articles written after 2010 was found. The main limitation of MCMS is its capacity to evaluate only the quality of reporting and not the quality of the study itself; hence high-quality studies that are reported poorly would receive a low score.
On the other hand, the CONSORT checklist is a series of 25 items focused on reporting how the trial was designed, analyzed, and interpreted. The CONSORT checklist was introduced to reduce the problems arising from the inadequate NPTs and to improve the quality of RCT reports [7
]. The CONSORT statement is an evidence-based minimum set of recommendations including a checklist and flow diagram for reporting RCTs to improve the complete reporting of trials [57
]. In 2008, the CONSORT Group developed an extension to the original statement that addressed methodological issues specific to trials of NPTs such as surgery, rehabilitation, or psychotherapy [59
]. Thus, a modified checklist, aimed specifically at trials with non-pharmacologic interventions, was created, with a detailed explanation published in 2008 [73
]. The factors outlined in this extension included descriptions of how such trials should outline the randomization procedure used, calculate sample sizes, report the blinding status, and outline the flow of participants, with flow diagrams strongly recommended [79
The average number of CONSORT checklist items for each article across all analyzed RCTs was 21.67. The 37 studies completed until the year 2010 averaged a number of checklist items of 19.97 and the studies completed between 2011 and 2019 averaged a number of checklist items of 22.10. A statistically significant difference in the number of checklist items between articles written before 2010 and articles written after 2010 was found. Articles written after 2010 had on average more checklist items. However, low correlation (0.26) between the number of checklist items for each article and the respective Coleman score was found. On the other hand, articles with a CONSORT diagram had a significant high Coleman score.
Taking into consideration only the studies completed between 1996 and 2010, three included a CONSORT flow diagram. The remaining 44 studies in which we found a flow diagram were completed between the years 2011 and 2020. All of the analyzed studies completed up to 2000 did not include a CONSORT flow diagram.
There are several limitations related to the use of MCMS to measure the quality of RCTs. This index assesses the quality of reporting of the trials rather than the actual quality of the trials themselves. In fact, some trials may have used higher methodological safeguards that were not reported in the study. Moreover, certain factors that compose the MCMS can be only addressed to surgical trials (e.g., appropriate description of surgical technique and description of postoperative protocol). Therefore, by definition, non-surgical studies could never reach the maximum score of 100.
This study included only English-language trial and those indexed in PubMed and Ovid (MEDLINE); thus, our analysis did not address RCTs on RC surgery in other languages or those that were not indexed in these databases. Moreover, RCTs focused on shoulder pain with intact rotator cuff tendons were excluded [188
]. High-certainty evidence shows that subacromial decompression does not provide clinically important benefits over placebo in pain, function, or health-related quality of life [189
In this context, despite the “new” updated checklist specifics to evaluate a report of a non-pharmacological trial, it is often impossible to perform sham interventions or to blind patients and care providers.
Future research might investigate the impact that RCTs have had on clinical practices in RC surgery over time including the effect of specific methodological factors. It is also important to evaluate the quality of reporting in other areas of orthopedics to compare areas that require improvements.