Quality of Surgical Outcome Reporting in Randomised Clinical Trials of Multimodal Rectal Cancer Treatment: A Systematic Review

Simple Summary Reporting surgical outcome and complication data in RCTs on rectal cancer is important because it is the basis for judging whether the results of a study warrant a change in clinical practice. In this review, we systematically analysed the quality of reporting in RCTs and the completeness rate of reporting of surgical outcomes and complication data. We found that only 2% (N = 7) of the RCTs met all 14 reporting criteria, and nearly half (N = 168, 49%) completed the procedure-specific quality criteria noted in the article. The most underreported criteria included complication severity (15% of articles). Abstract Introduction: Randomised controlled trials (RCTs) continue to provide the best evidence for treatment options, but the quality of reporting in RCTs and the completeness rate of reporting of surgical outcomes and complication data vary widely. The aim of this study was to measure the quality of reporting of the surgical outcome and complication data in RCTs of rectal cancer treatment and whether this quality has changed over time. Methods: Eligible articles with the keywords (“rectal cancer” OR “rectal carcinoma”) AND (“radiation” OR “radiotherapy”) that were RCTs and published in the English, German, Polish, or Italian language were identified by reviewing all abstracts published from 1982 through 2022. Two authors independently screened and analysed all studies. The quality of the surgical outcome and complication data was assessed based on fourteen criteria, and the quality of RCTs was evaluated based on a modified Jadad scale. The primary outcome was the quality of reporting in RCTs and the completeness rate of reporting of surgical results and complication data. Results: A total of 340 articles reporting multimodal therapy outcomes for 143,576 rectal cancer patients were analysed. A total of 7 articles (2%) met all 14 reporting criteria, 13 met 13 criteria, 27 met from 11 to 12 criteria, 36 met from 9 to 10 criteria, 76 met from 7 to 8 criteria, and most articles met fewer than 7 criteria (mean 5.5 criteria). Commonly underreported criteria included complication severity (15% of articles), macroscopic integrity of mesorectal excision (17% of articles), length of stay (18% of articles), number of lymph nodes (21% of articles), distance between the tumour and circumferential resection margin (CRM) (26% of articles), surgical radicality according to the site of the primary tumour (R0 vs. R1 + R2) (29% of articles), and CRM status (38% of articles). Conclusion: Inconsistent surgical outcome and complication data reporting in multimodal rectal cancer treatment RCTs is standard. Standardised reporting of clinical and oncological outcomes should be established to facilitate comparing studies and results of related research topics.


Introduction
Only little is known about the quality of reporting of surgical outcomes and complication data in randomised controlled trials (RCTs) on the multimodal treatment regimens for rectal cancer.The results of RCTs may have a substantial impact on future treatment regimens.In multimodal treatment regimens, the quality of surgical treatments remains of the utmost prognostic relevance to patients with rectal cancer.Over the last 40 years, rectal cancer treatment has made impressive progress by developing and implementing differentiated quality criteria for surgical treatment, including multimodal therapy concepts [1].Heald et al. [2] proposed the notion of total mesorectal excision (TME).This has become the surgical standard for cancer of the middle and lower third of the rectum [2][3][4].A microscopically negative circumferential resection margin (CRM), sufficient lymph node retrieval, and the integrity of the mesorectal fascia are all associated with lower local and distant recurrence rates and better long-term survival [5,6].
In studies, RCTs lower the risk of selection bias.However, this study design is not necessarily associated with high practical relevance.Reporting surgical outcomes and complication data in RCTs is essential because it is the basis for judging whether the results of a study warrant a change in clinical practice.On the contrary, low-quality data reporting or omitting critical data relevant to the overall outcome may result in misleading reports from studies.Different scales and checklists for assessing the quality of reporting in RCTs have been developed [7][8][9], including the Jadad scale, which was developed using standard techniques of scale development [10].
Another critical aspect of evaluating the efficacy of rectal cancer treatment is the need for standardised reporting of surgical outcome and complication data due to the mostly elderly population of rectal cancer patients and their associated medical comorbidities.In addition, the success of multimodal therapies for treating rectal cancer often relies on the timing of chemotherapy or radiotherapy with surgical intervention, and whether these therapies are considered successful can be affected by surgical outcome criteria and perioperative complications.Perioperative complication grading systems [11,12] and, most recently, standard criteria for reporting surgical outcomes have been established [13][14][15].
We hypothesise that the quality of reporting of surgical outcome and complication data in RCTs is high.To address this hypothesis, we conducted this systematic review.This study aimed to measure the quality of the reporting of surgical outcome and complication data in RCTs of rectal cancer treatment and to analyse the change in the reporting quality over 40 years.

Materials and Methods
This systematic review complied with the PRISMA guidelines [16].This review was prospectively registered in an international prospective registry for systematic reviews under the study number review registry 1682.

Literature Search
To identify studies, we performed a systematic literature search of EMBASE, MED-LINE (via PubMed), and Cochrane Central Register of Controlled Trials using the following strings: (("rectal cancer" OR "rectal carcinoma") AND (radiation OR radiotherapy)) (Figure 1).The results were limited to from January 1982 to 10 February 2022.Two of the authors (JJ and KU) independently checked whether the full-text articles were eligible.Disagreements were clarified through a one-by-one discussion.
sights in rectal cancer treatment Heald promoted at that time [2,17].A subgroup analysis was performed to compare publications published before and after the year 2000, as the implementation of the TME concept and related procedure-specific criteria occurred primarily on a global scale in the new millennium.The year 2001 was chosen as the cut-off point because this was the time when these procedures were being performed-specific criteria were already in place or were recommended in many guidelines.

Inclusion and Exclusion Criteria
Articles that met the following criteria were selected for inclusion: (1) population-patients with rectal cancer; (2) treatment-radiotherapy as part of the regimen; (3) outcomes-treatment methods; (4) study design-RCTs; (5) language of search results-limited to the English, German, Polish, or Italian languages.
The excluded articles were non-RCTs, literature reviews, observational epidemiological studies (cohort and case-control designs), case series, case reports, articles describing other forms of rectal cancer treatment without radiotherapy in the treatment regimen, and studies including cancers other than rectal cancer and articles are written in a language different than the above mentioned.All duplicate articles were removed, and database limits were utilised to exclude paediatric and anatomical/cadaver articles.The time interval from 1982 to 2022 was chosen because RCTs were infrequently published in the journals studied before this period and because of the ground-breaking insights in rectal cancer treatment Heald promoted at that time [2,17].A subgroup analysis was performed to compare publications published before and after the year 2000, as the implementation of the TME concept and related procedure-specific criteria occurred primarily on a global scale in the new millennium.The year 2001 was chosen as the cut-off point because this was the time when these procedures were being performed-specific criteria were already in place or were recommended in many guidelines.

Inclusion and Exclusion Criteria
Articles that met the following criteria were selected for inclusion: (1) population-patients with rectal cancer; (2) treatment-radiotherapy as part of the regimen; (3) outcomes-treatment methods; (4) study design-RCTs; (5) language of search results-limited to the English, German, Polish, or Italian languages.
The excluded articles were non-RCTs, literature reviews, observational epidemiological studies (cohort and case-control designs), case series, case reports, articles describing other forms of rectal cancer treatment without radiotherapy in the treatment regimen, and studies including cancers other than rectal cancer and articles are written in a language different than the above mentioned.All duplicate articles were removed, and database limits were utilised to exclude paediatric and anatomical/cadaver articles.

Outcome Parameters
The primary outcome was the quality of reporting in RCTs and the completeness rate of reporting of surgical outcome and complication data; this outcome was assessed based on fourteen criteria (Table 1).Evidence of risk stratification and method used indicated in the study.+1 No information on risk stratification or the method used is given in the study.0 Notes: a R0 vs. R1 + R2 vs. unresected patients.b The non-peritonealized surface of the fresh specimen is examined circumferentially, and the completeness of the mesorectum is scored as follows: complete vs. near complete vs. incomplete vs. cannot be determined.Adapted from Martin et al. [13].
The following secondary outcomes were also collected and assessed: study period, study design, the population size of rectal cancer patients, clinical data, type of treatment, country of origin, publication year, the absolute number of citations, single or multicentre study design, sample size, study intervention and control groups, study endpoints, the Cancers 2024, 16, 26 5 of 21 conclusion of the study, Quirke grade as a marker of the quality of surgical resection, the field of medical speciality, Scientific SCImago Journal Ranking, journal impact factor (IF) according to the Web of Science Group, and the type of surgical procedures performed.
An additional outcome measure was to objectively evaluate changes in procedurespecific quality criteria over time, particularly with respect to their initial consultation in previous literature.

Quality of Reporting
We modified previously published criteria related to the completeness of surgical outcomes and complication reporting to rate the articles [13].Fourteen critical components were evaluated (Table 1).Providing information on surgical interventions using the 14 components allows a complete understanding of the quality of outcomes and complication reporting.As initially, the 14 criteria did not address oncological radicality in rectal cancer surgery, this was assessed separately.In particular, the category "Procedure-specific quality" was adjusted, as this outcome is highly relevant in rectal cancer surgery because they are all associated with lower local and distant recurrence rates and better long-term survival.If the items described were mentioned in the assessed articles, 1 point was awarded for each item.If an item was not noted, no point was given.
In turn, the category "quality of reporting in RCTs" was created, as these outcomes are often of interest in RCTs.The Jadad scale [10], shown previously as reliable and valid, was used.The Jadad scale is a 3-item scale covering the randomisation method, the blinding method, and withdrawals/dropouts (Supplementary Table S1).For each item, 1 point was assigned to a study if it was described as randomised or double-blinded or had discontinuations/failures. If the described randomisation or blinding method was judged appropriate, 1 point was assigned for that item.
In contrast, no point was awarded for this item if the described randomisation or blinding method was deemed inadequate.In the original Jadad scale, only double blinding was considered.The scale was modified to include single blinding because double blinding is not always possible in surgical procedures.The blinding method was considered appropriate if the item indicated who was involved in blinding and, depending on the type of intervention, possible additional measures to ensure blinding.The final quality score for each article ranged from 0 (lowest quality) to 5 points (highest quality).
We searched (in August 2023) the official relevant websites for information regarding the main characteristics of the journal IF, according to the SCI Journal website, [18] and the SJR indicator, provided by the SCImago journal and country rank.[19] Journals were classified by the IF, with an IF of 10 or more considered excellent, a value between 3 and 10 considered very good, a value between 1 and 3 considered good, and a value below 1 considered moderately good.

Statistical Analysis
Summary statistics were tabulated via established norms.Baseline characteristics were summarized using counts and percentages.Weighted overall rates were calculated for dichotomous data.Means were determined using normally distributed continuous data.Microsoft Excel was used to analyse the data.A p-value less than 0.05 (2-sided) was considered to be statistically significant.
Median and range scores for the quality were calculated for all articles and each journal.Student's t-test was used to compare sample distributions.Differences were statistically significant if the p-value was less than 0.05.

Ethical Statement
The Ethical Committee of the Cantonal Hospital St. Gallen reviewed this study.Because all data used are publicly available, the study was exempt from further oversight and requirements concerning informed consent.The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines were applied [20].

Results
We initially identified 529 articles (Figure 1).After excluding 27 duplicates, titles and abstracts were assessed based on the inclusion criteria.A full-text review was performed on all selected articles, after which 340 were further analysed.
One hundred fifty-nine articles (47%) reported one or more complications or at least indicated the duration of follow-up for the assessment of complications.One hundred eighty-one (51%) of the articles reported morbidity and mortality rates including the complications leading to death.The death rates occurred most frequently in reports of postoperative morbidity and mortality (19 articles) and least frequently in descriptions of the predictive value of different proteins (0 articles) or radiological features (0 articles).Morbidity risk factor stratification based on factors such as the American Society of Anesthesia classification, body mass index, age, and comorbid diseases for characterising the patient population was described in 128 articles (38%).
Abbreviations: DFS: disease-free survival; LRR: locoregional recurrence rate; MoMo: postoperative morbidity and mortality; OS: overall survival; pCR: pathological complete response; CRM: Circumferential resection margin.Notes: a R0 vs. R1 + R2 vs. unresected patients.b The non-peritonealized surface of the fresh specimen is examined circumferentially, and the completeness of the mesorectum is scored as follows: complete vs. near complete vs. incomplete vs. cannot be determined.Adapted from Martin et al. [13].
The median value of the criteria met for RCTs published in oncology journals was 6 (range 2-14), and it was 7 (range 2-14) for those in surgical journals, 4.5 (range 2-11) for those in medical journals, and 5 (range 0-13) for those in radiology journals (Table 5).Abbreviations: CRM: Circumferential resection margin.Notes: a R0 vs. R1 + R2 vs. unresected patients; b The non-peritonealized surface of the fresh specimen is examined circumferentially, and the completeness of the mesorectum is scored as follows: complete vs. near complete vs. incomplete vs. cannot be determined.Adapted from Martin et al. [13].
Abbreviations: DFS: disease-free survival; LRR: locoregional recurrence rate; MoMo: postoperative morbidity and mortality; OS: overall survival; pCR: pathological complete response; CRM: Circumferential resection margin.Notes: a R0 vs. R1 + R2 vs. unresected patients.b The non-peritonealized surface of the fresh specimen is examined circumferentially, and the completeness of the mesorectum is scored as follows: complete vs. near complete vs. incomplete vs. cannot be determined.Adapted from Martin et al. [13].

Discussion
In this critical review of the multimodal rectal cancer treatment literature from over 40 years, we demonstrate considerable inconsistency in the quality of reporting of surgical outcome and complication data.The quality of surgical outcomes and reporting of complication data in articles on RCTs on rectal cancer are essential in the assessment of outcomes and areas for improving the quality of surgical care.An accurate appraisal of surgical quality requires consistency and reliability in the reporting of outcomes, data on specific complications, and procedure-specific quality.
In the study presented here, only 44% of the RCTs related to multimodal rectal cancer treatment reported on a minimum of 7 out of 14 critical surgical reporting criteria.Further, only half of the articles (N = 168, 49%) met the procedure-specific quality criteria reported in the article.Interestingly, surgical articles did not have a higher rate of better compliance regarding reporting procedure-specific quality criteria compared to oncology journals.Only 53% of the articles adequately defined the complications reported in the study.There was substantial variability in the published articles regarding what constituted a complication.The reporting of the complication severity using grading criteria was very limited (15%), and appropriate consideration of risk factors for surgical complications was described in only 38% of the articles.This lack of surgical quality reporting poses a significant challenge to synthesising and analysing the data for surgical interventions and multimodal treatment and potentially confounds the comparison of outcomes across studies.
Consistency in surgical quality criteria reporting would facilitate data comparison across studies and enable authors to perform better meta-analyses.
Although significant progress has been made in improving the reporting of surgical outcomes, our study indicates that further efforts are needed.Our results support the work of others [13,134] who have demonstrated the limitations in how surgical quality outcomes and complications are reported in the surgical literature.
The current study emphasises the need for the use of standard grading systems [11,12] to characterise the severity of complications after rectal cancer surgery.In the surgical literature, there are two areas that have received substantial attention regarding complication reporting: measuring the severity of complications and developing standardised outcome measures and clear definitions of complications.For example, the term "surgical site infection" provides a broad spectrum of different diagnoses.Data on "surgical site infection" are, therefore, only comparable between different studies or further computed in meta-analyses with caution.
The consistent reporting of standardised procedure-specific quality criteria is of equal importance to reporting the severity of complications.Curative surgical therapy of rectal cancer usually requires partial or complete excision of the mesorectum and, thus, the regional lymphatic drainage area and resection of the primary tumour in healthy tissue.Surgical therapy should include the following principles: complete mesorectal excision, CRM status, R0 status depending on the location of the primary tumour, a distance between the tumour and CRM of >1 mm, sufficient lymph nodes removed, and complete resection of the mesorectum.All these procedure-specific quality criteria influence OS, DFS, and LLR.However, our study shows that current strategies for reporting these data complicate the comparison of different multimodal therapy regimens and, thus, could be better.For example, if an attempt is made to compare two neoadjuvant therapy protocols wherein the surgical quality is described in one study but not in the other, it is difficult to determine which effect is from the neoadjuvant therapy and which is from the surgery.
When evaluating the procedure-specific quality criteria, it is important to note that the treatment of rectal cancer has changed dramatically over the past 40 years; nevertheless, it must be assumed that years or even decades are required before a novel surgical technique becomes an accepted standard.TME was introduced in 1982 [2], and the distance between the tumour and circumferential resection margin was introduced in 1986 [135].It took several years to gain acceptance, so TME has only been considered the procedure of choice since 2004 [136].It can be assumed that studies published before 2004 describe many fewer Cancers 2024, 16, 26 13 of 21 procedure-specific quality criteria than studies published after 2004 because the criteria did not yet exist.Interestingly, we found that in studies published after 2004 and even after 2020, the rate of reporting procedure-specific quality criteria remained low, at 15-45%, depending on the criterion.
The limitation of the study was that only randomized control trials were used.Selection bias may limit the generalizability of our findings.More recent articles may reflect improvements in complication reporting not captured in this study.A further limitation is the evaluation criteria used.The criteria used here were previously published and utilised in several studies to evaluate complication data.However, we acknowledge that these criteria are not universally accepted.Unfortunately, while checklists such as the STROBE statement [20] exist for evaluating what items should be reported in different studies, there are no universally accepted guidelines for assessing the quality of reporting on specific complications.Thus, we chose to use these criteria and modify them because they represent a practical and easy-to-understand strategy for characterising the quality of complication reporting.We further acknowledge that some criteria are not ubiquitous in their relevance.For example, it is not always possible to account for complication risk factors if the relationship to the outcome is not established.Another possible limitation is the time period that was chosen.The introduction of three procedure-specific quality criteria after 1982 may have had an effect on the results.To counteract this, we performed a subgroup analysis.The time periods were as follows: 1984-2001 and 2002-2022.The year 2001 was chosen as the reference period because this was the time when these procedures were performed-specific criteria already existed or were recommended in many guidelines.We also wanted to analyse how a surgical quality criterion became established in RCTs over time, so we chose the time when a method was first described.
This study demonstrates inconsistent quality in the reporting of surgical outcome and complication data in RCTs of multimodal rectal cancer treatment.The variability in reporting these data complicates the comparison of surgical outcomes between trials and potentially dilutes its importance for the further clinical development of the treatment.

Conclusions
The future development of guidelines for the accurate, comparable, and reproducible collection and reporting of quality surgical data is essential for future RCTs on multimodal rectal cancer treatment.

Supplementary Materials:
The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/cancers16010026/s1, Figure S1.Analysis of mean quality criteria met from 1984 to 2022.The figure demonstrates no significant linear trend between the quality of articles; Figure S2.Analysis of Procedure-specific quality criteria met from 1984 to 2022.The figure demonstrates no significant linear trend between the quality of articles; Table S1.Jadad scale for assessing the quality of reporting in randomised clinical trials of rectal cancer treatments (modified from Jadad et al.); Table S2.Characteristics and publication output of RCTs of multimodal rectal cancer treatment; Table S3.Baseline characteristics of the included studies.
Author Contributions: J.J.: Study design, data analysis, interpretation of the data analysis, literature search, figures, manuscript writing, and manuscript review.K.U.: Study design, data analysis, interpretation of the data analysis, literature search, figures, manuscript writing, and manuscript review.S.B.: Interpretation of the data analysis and manuscript review.M.T.: Interpretation of the data analysis and manuscript review.P.C.M.: Interpretation of the data and review of the manuscript.T.S.: Study design, interpretation of the data analysis, literature search, manuscript writing, and manuscript review.All authors have read and agreed to the published version of the manuscript.

Funding:
The Cancer League of Eastern Switzerland (KLOCH) funded part of this research.Thomas Steffen holds a corresponding grant.

Figure 1 .
Figure 1.PRISMA flowchart of the literature research.

Figure 1 .
Figure 1.PRISMA flowchart of the literature research.

Figure 2 .
Figure 2. Completeness of surgical outcomes and complication data reporting in a convenience sample of the rectal treatment literature.

Figure 2 .
Figure 2. Completeness of surgical outcomes and complication data reporting in a convenience sample of the rectal treatment literature.

Table 1 .
[13]eria utilised to evaluate surgical quality reporting in randomised clinical trials of the treatment of rectal cancer (modified from Martin RCG)[13].

Table 3 .
Specific criteria and reporting compliance according to the primary endpoint.

Table 4 .
Specific criteria and reporting compliance according to time.

Table 5 .
Specific criteria and reporting compliance according to the type of journal.