A Historical Analysis of Randomized Controlled Trials in Rotator Cuff Tears

Background and objectives: Our research aimed to evaluate the quality of reporting of randomized controlled trials (RCTs) linked to rotator cuff (RC) tears. The present study analyzed factors connected to the quality of the RCTs and trends in the quality of reporting through time. Materials and Methods: The online databases used to search all RCTs on the topic of RC surgery completed until March 2020 were PubMed and Ovid (MEDLINE). The quality of reporting was evaluated using the modified Coleman methodology score (MCMS) and the consolidated standards of reporting trials (CONSORT). Results: The online search found 957 articles. Finally, 183 studies were included in the quantitative synthesis. A total of 97 (53%) of 183 studies had a level of evidence I and 86 (47%) of 183 studies had a level of evidence II, according to the Oxford Center of Evidence Based Medicine (EBM). A statistically significant difference in MCMS between articles written before 2010 and articles written after 2010 was found. Articles written after 2010 had, on average, the highest Coleman score. The average number of CONSORT checklist items for each article across all analyzed RCTs was 21.67. The 37 studies completed up to 2010 averaged a number of checklist items of 19.97 and the studies completed between 2011 and 2019 averaged a number of checklist items of 22.10. A statistically significant difference in the number of checklist items between articles written before 2010 and articles written after 2010 was found. Articles written after 2010 had on average more checklist items. However, low correlation (0.26) between the number of checklist items for each article and the respective Coleman score was found. On the other hand, articles with the CONSORT diagram had a significantly high Coleman score. Conclusions: An improvement in the quantity and quality of RCTs relating to RC surgery over the analyzed period was found.


Introduction
Rotator cuff (RC) tears are among the most frequent injuries of the tendon-bone junction in the human population [1]. RC repair is one of the most common surgical operations for shoulder disorders [2] or may be the most common of all shoulder surgeries [3]. of eleven criteria evaluated through a scoring system between 0 and 100, where 100 indicates a high-quality study without chances, bias, and other confounding factors [8]. The MCMS was used for the statistical analysis to evaluate the correlation between the variables involved.

Assessment of Agreement
The data from the full-text selected articles were extracted independently by two authors (G.S., C.D.N.) and checked by a third author (G.F.). To ascertain the validity of the included studies, two reviewers independently evaluated the methodological quality and reliability of the findings through MCMS.

Statistical Analysis
To compare the MCMS of articles written before 2010 and articles written after 2010 and to find statistically significant differences in terms of MCMS, between studies containing a CONSORT diagram and studies without the flow-chart of the independent samples t-test was used. The inter-rater reliability of grading the Coleman score and the association between the number of checklist items and the Coleman score for each article with Pearson's correlation were evaluated. The inter-rater reliability of grading the CONSORT checklist with percent agreement between raters was evaluated.
Most of the included studies were published in the following three major journals: Arthroscopy (34 studies; 18.6%), The American Journal of Sports Medicine (30 studies; 16.4% of the total), and J Shoulder Elbow Surg (23 studies; 12.6%).

Topic
Among the 183 identified articles, six were focused on acromioplasty, 27 on the use of platelet rich plasma, platelet-rich fibrin matrix, leucocyte and platelet-rich fibrin and augmentation, 13 on single-row versus double-row repair, 54 on pain, five on conservative versus surgical repair, one on trauma-related rotator cuff tears, 31 on physiotherapy, six on the treatment of the long head of the biceps tendon disorder, four on mini-open versus arthroscopic repair, five on subacromial decompression, 10 on surgical techniques, four on post-operative immobilization, and 17 on other items linked to rotator cuff pathology (miscellaneous) (Supplementary Materials Tables S1-S13).

Modified Coleman Methodology Score (MCMS)
The inter-rater reliability of grading the Coleman score was 98.6%. The average MCMS across all analyzed RCTs was 71.40. The 37 studies completed up to 2010 averaged a MCMS of 68.54 and the studies completed between 2011 and 2019 averaged a MCMS of 72.12. (Table 1). A statistically significant difference in MCMS between articles written before 2010 and articles written after 2010 was found (Table 2). Articles with CONSORT diagram had the highest Coleman score (Table 3). Statistically significant differences in terms of MCMS between studies containing a CONSORT diagram or not were found (Table 4).

Trends
The majority of the identified studies had been produced since 2014. From about 2012, the number of RCTs published has increased significantly when compared to previous years ( Figure 2).

Trends
The majority of the identified studies had been produced since 2014. From about 2012, the number of RCTs published has increased significantly when compared to previous years ( Figure 2).

Other Methodological Factors on the Consolidated Standards of Reporting Trials (CONSORT) Checklist
The inter-rater reliability of grading the CONSORT checklist was 97.4%. The rate of missed checklist items of the CONSORT checklist for each trial, calculated as the ratio of number of checklist items with incomplete information to total checklist items for each trial, is shown in Supplementary Materials Table S14.
Items and checklist items most frequently lost were analyzed. Items "3b. Important changes to methods after trial commencement (such as eligibility criteria), with reasons" (182 missed, Trial Design item), "6b. Any changes to trial outcomes after the trial commenced, with reasons" (180 missed, Outcomes item), "7b. When applicable, explanation of any interim analyses and stopping guidelines" (181 missed, Sample Size item), "14b. Why the trial ended or was stopped" (183 missed, Recruitment item), "17b. For binary outcomes, presentation of both absolute and relative effect sizes is recommended" (178 missed, outcomes, and estimation item), and "18. Results of any other analyses performed including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory" (181 missed, Ancillary analyses item) were the most frequently lost.
Low correlation (0.26) between the number of checklist items for each article and the respective Coleman score was found.

Other Methodological Factors on the Consolidated Standards of Reporting Trials (CONSORT) Checklist
The inter-rater reliability of grading the CONSORT checklist was 97.4%. The rate of missed checklist items of the CONSORT checklist for each trial, calculated as the ratio of number of checklist items with incomplete information to total checklist items for each trial, is shown in Supplementary Materials Table S14.
Items and checklist items most frequently lost were analyzed. Items "3b. Important changes to methods after trial commencement (such as eligibility criteria), with reasons" (182 missed, Trial Design item), "6b. Any changes to trial outcomes after the trial commenced, with reasons" (180 missed, Outcomes item), "7b. When applicable, explanation of any interim analyses and stopping guidelines" (181 missed, Sample Size item), "14b. Why the trial ended or was stopped" (183 missed, Recruitment item), "17b. For binary outcomes, presentation of both absolute and relative effect sizes is recommended" (178 missed, outcomes, and estimation item), and "18. Results of any other analyses performed including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory" (181 missed, Ancillary analyses item) were the most frequently lost.
Low correlation (0.26) between the number of checklist items for each article and the respective Coleman score was found.
The average number of checklist items for each article across all analyzed RCTs was 21.67. The 37 studies completed until 2010 averaged a number of checklist items of 19.97 and the studies completed between 2011 and 2019 averaged a number of checklist items of 22.10 (Table 5). A statistically significant difference in the number of checklist items between articles written before 2010 and articles written after 2010 was found (Table 6). Articles written after 2010 have on average more checklist items. A CONSORT flow diagram that outlines the inclusion and exclusion of patients for the trial as well as follow-up rate was included in 47 studies (25.6% of total studies analyzed). Taking into consideration only the studies completed between 1996 and 2010, three included a CONSORT flow diagram. Next, the remaining 44 studies in which a flow diagram was found were completed between the years 2011 and 2019. All the analyzed studies completed until 2000 did not include a CONSORT flow diagram.

Discussion
Rotator cuff disease is among the most common musculoskeletal disorders [184]. It is a disabling condition with high prevalence rate [185], causing high direct and indirect costs [186]. The appropriate treatment for rotator cuff disease is debated. Rotator cuff repair is an option for patients with chronic, symptomatic full-thickness rotator cuff tear, but the quality of evidence is unconvincing. There is also little compelling evidence for conservative treatment [187]. RCTs are considered to be the most reliable form of scientific evidence in the hierarchy of evidence that influences healthcare policy and practice because RCTs reduce spurious causality and bias. Well done RCTs are the only way to find strong evidence on rotator cuff disease and treatment. This systematic review aimed to analyze factors connected to the quality of the RCTs and trends in the quality of reporting through time. The use of a CONSORT flow diagram, CONSORT checklist, and the MCMS have been found to be the most important predictive factor for a good quality RCT.
Rotator cuff repair can be performed with several techniques: transosseous simple suture, single-row suture anchor, double-row suture anchor, and transosseous-equivalent techniques. Current evidence has failed to prove a significant difference between single-row and double-row techniques in terms of clinical outcomes. The ideal rotator cuff repair technique should provide a good restoration of the footprint contact area and compression of the tendon on the footprint. Moreover, rotator cuff repair is not the only procedure undertaken to manage rotator cuff tears. Acromioplasty, long head of biceps tendon tenotomy, or tenodesis are surgical procedures often performed with rotator cuff repair. Acromioplasty is an arthroscopic surgical procedure made to prevent the impingement of the rotator cuff and consists of the decompression of subacromial space. Stem cell therapy is an emerging treatment for tendon disorders and can be used during arthroscopic rotator cuff repair, after rotator cuff repair, or instead of rotator cuff repair. The real efficacy of stem cell therapy for rotator cuff tears is nowadays debated. The 183 studies analyzed focused on this major topic concerning rotator cuffs. Pain resulted in being the most macro-area studied with 54 articles (29.5% of total RCTs), followed by physiotherapy (31 articles, 17% of total RCTs). Another popular topic is the use of stem cell therapy (27 articles, 15%).
The number of RCTs has increased over the last few years thanks to higher attention to the process of randomization and the prospective calculation of sample size. On the other hand, factors such as the description of concealment of patient allocation and the intention-to-treat analysis did not receive the same attention.
The MCMS, CONSORT flow diagram, and CONSORT checklist have been used to assess the quality of RCTs previously identified. This review showed a significant improvement in the quality in RCTs in the long term. This was confirmed by the studies completed until the year 2010 that had an average MCMS of 68.54 compared to the studies completed between 2011 and 2020 with an average of MCMS of 72.12. A statistically significant difference in MCMS between articles written before 2010 and articles written after 2010 was found. The main limitation of MCMS is its capacity to evaluate only the quality of reporting and not the quality of the study itself; hence high-quality studies that are reported poorly would receive a low score.
On the other hand, the CONSORT checklist is a series of 25 items focused on reporting how the trial was designed, analyzed, and interpreted. The CONSORT checklist was introduced to reduce the problems arising from the inadequate NPTs and to improve the quality of RCT reports [7]. The CONSORT statement is an evidence-based minimum set of recommendations including a checklist and flow diagram for reporting RCTs to improve the complete reporting of trials [57]. In 2008, the CONSORT Group developed an extension to the original statement that addressed methodological issues specific to trials of NPTs such as surgery, rehabilitation, or psychotherapy [59]. Thus, a modified checklist, aimed specifically at trials with non-pharmacologic interventions, was created, with a detailed explanation published in 2008 [73]. The factors outlined in this extension included descriptions of how such trials should outline the randomization procedure used, calculate sample sizes, report the blinding status, and outline the flow of participants, with flow diagrams strongly recommended [79].
The There are several limitations related to the use of MCMS to measure the quality of RCTs. This index assesses the quality of reporting of the trials rather than the actual quality of the trials themselves. In fact, some trials may have used higher methodological safeguards that were not reported in the study. Moreover, certain factors that compose the MCMS can be only addressed to surgical trials (e.g., appropriate description of surgical technique and description of postoperative protocol). Therefore, by definition, non-surgical studies could never reach the maximum score of 100.
This study included only English-language trial and those indexed in PubMed and Ovid (MEDLINE); thus, our analysis did not address RCTs on RC surgery in other languages or those that were not indexed in these databases. Moreover, RCTs focused on shoulder pain with intact rotator cuff tendons were excluded [188]. High-certainty evidence shows that subacromial decompression does not provide clinically important benefits over placebo in pain, function, or health-related quality of life [189].
In this context, despite the "new" updated checklist specifics to evaluate a report of a non-pharmacological trial, it is often impossible to perform sham interventions or to blind patients and care providers.
Future research might investigate the impact that RCTs have had on clinical practices in RC surgery over time including the effect of specific methodological factors. It is also important to evaluate the quality of reporting in other areas of orthopedics to compare areas that require improvements.

Conclusions
The last decade was characterized by a significant increase in the quantity of RCTs evaluating RC repairs. The quality of RCT reporting (assessed through MCMS and other quality indexes) has shown a steady increase in results after 2010 compared to previous years. In particular, significant improvements in the reporting of the randomization process as well as prospective sample size calculation were noted. Statistically significant differences, in terms of MCMS, between studies containing a CONSORT diagram or not were found. Articles with a CONSORT diagram had the highest Coleman score. A low correlation between the number of checklist items for each article and the respective Coleman score was found.