Evaluation of Cell-Free DNA Long Fragments in the Triage of FIT+ Patients Enrolled in a Colorectal Cancer Screening Program: An Italian Prospective, Cross-Sectional Study

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

While the idea of use the cell free DNA as a diagnostic tool generally is neither novel nor original, I feel the potential of the proposed article in the clear practical outcome for a routine clinical application. The data presented clearly demonstrate the increase of the sensitivity of the standard test procedure. This is of a great benefit for patient. I consider the proposal quite informative, with the exception of the graphical part. I´d recommend to revise the format and inclusion/exclusion of the figures. First, the quality of every figure is poor – the letters are blurry, sometimes barely readable! Second, please consider the relevance of the 1st and 2nd figure – the fig. 1. Brings no new information to reader as the experimental design is described in the text with all the important information. Fig. 2.  Is absolutely irrelevant as it presents only example of general calibration curve – here even without the axes description! Figure 3. Is the same situation like the fig. 1 actually – no new information, hence the figure is of low importance. The figure 4 is quite important as it is the part of the results. It deserves the proper description! All the used abbreviation should be explained in the text; the statistic is missing! The sensitivity should be explained as well! Please consider the inclusion of any form of chart/figure that will compare the sensitivity in both the tested arrangements – AAP and SOC! Regarding the tables – table number 1 is irrelevant containing far more 0 rows than the informative one! Please consider the importance of the graphical presentation of such an information! Table 2 – adjust the format! Why the p-value is not presented? Within the text – please consider over-usage of abbreviations – it makes the text poorly understood sometimes. I strongly suggest the careful corrections at the minimal rate indicated above. Overall, I suggest to publish the article after the revisions.

Author Response

While the idea of use the cell free DNA as a diagnostic tool generally is neither novel nor original, I feel the potential of the proposed article in the clear practical outcome for a routine clinical application. The data presented clearly demonstrate the increase of the sensitivity of the standard test procedure. This is of a great benefit for patient. I consider the proposal quite informative, with the exception of the graphical part. I´d recommend to revise the format and inclusion/exclusion of the figures. First, the quality of every figure is poor – the letters are blurry, sometimes barely readable! Second, please consider the relevance of the 1st and 2nd figure – the fig. 1. Brings no new information to reader as the experimental design is described in the text with all the important information. Fig. 2.  Is absolutely irrelevant as it presents only example of general calibration curve – here even without the axes description! Figure 3. Is the same situation like the fig. 1 actually – no new information, hence the figure is of low importance. The figure 4 is quite important as it is the part of the results. It deserves the proper description!

• We really thank the reviewer for this kind suggestion. We have modified the latest version of the manuscript accordingly.

All the used abbreviation should be explained in the text; the statistic is missing!

• We really thank the reviewer for this kind suggestion. We have modified the latest version of the manuscript accordingly. We included a glossary of terms, showing explanation of statistics and other terms.

 

The sensitivity should be explained as well! Please consider the inclusion of any form of chart/figure that will compare the sensitivity in both the tested arrangements – AAP and SOC! Regarding the tables – table number 1 is irrelevant containing far more 0 rows than the informative one! Please consider the importance of the graphical presentation of such an information! Table 2 – adjust the format! Why the p-value is not presented?

• We really thank the reviewer for this kind suggestion. As far as the comparison of AAP and SOC, please see the non-inferiority analysis in Figure 2, showing the non-inferiority graphs, along with Table 2, which shows values. We are comparing the Detection Rates of AAP and SOC, which is a more commonly used indicator within Colorectal Cancer screening programs in Europe. Sensitivity findings are nevertheless shown for AAP in the new Table 3. We removed ll the rows and columns including 0 values.

 

Within the text – please consider over-usage of abbreviations – it makes the text poorly understood sometimes.

We really thank the reviewer for this kind suggestion. We have modified the latest version of the manuscript accordingly.

 

I strongly suggest the careful corrections at the minimal rate indicated above. Overall, I suggest to publish the article after the revisions.

Reviewer 2 Report

Comments and Suggestions for Authors

In this study, the authors presented the accuracy of a cross-sectional study of cell free DNA (cfDNA) in addition to fecal immunochemical test (FIT) for colorectal neoplasia screening. I think this is an interesting topic with a substaintial impact. On the other hand, I have several comments below:

1. What is the definition of "colorectal neoplasia (CN)"? Did it include hyperplastic polyp or sessile serrated lesions? Previous studies have suggested FIT is relatively insensitive to such lesions. While cell free DNA might reduce colonoscopy burden, the two-step approach is also possible to exclude such patients into colonoscopy exam.

2. In the legend of Figure 4, the size of lesions would affect the sensitivity, which was not described in the main text. Actually, the description of figure 4 in the main text seemed unrelevant. Furthermore, why smaller size was associated with higher sensitivity in the figure? It should be further clarified.

3. The presentation was unclear in several places. For example, in page 7, line 219, the "clinical performance section", the authors stated: "The odds ratio was 1.76 (95% C.I. 1.15, 2.69 p= 0.009). Hence, a subject with a positive test had 76.0% greater odds of disease than one with a negative test."

I'm somewhat confused that what is the test here? Dose it means the FIT, or FIT plus cfDNA? Nevertheless, what refers to a negative test here? Dose it mean negative FIT, or positive FIT but negative cfDNA? I think these points should be carefully addressed to improve readibility and soundness.

4. I did not found Table 3. although the authors stated: "and CRC. PPV and NPV for CRC were 4.5% and 96.9% respectively (Table 3). There is an 228 expected 33.4% decrease in the need for colonoscopies. (Table 2)..." in Page 7.

5. In table 2, the authors used CRC to calculated sensitivity, but used the total cases (CN+AA+CRC) to calculated specificity. To me it was a somewhat wierd approach. I suggest the calculation may be based on advanced adenoma (+/- serrated lesions >= 1cm) to be sensible for clinical practice.

6. Overall, I believe a revision for more easily understandable and consistent content is necessary before publication. Look forward to the responses from the authors. 

Author Response

Dear Reviewer,

 

Thank you very much indeed for your comments that will undoubtedly help us improve the quality of this paper.

 

Please see our responses and actions as follows:

 

In this study, the authors presented the accuracy of a cross-sectional study of cell free DNA (cfDNA) in addition to fecal immunochemical test (FIT) for colorectal neoplasia screening. I think this is an interesting topic with a substaintial impact. On the other hand, I have several comments below:

1. What is the definition of "colorectal neoplasia (CN)"? Did it include hyperplastic polyp or sessile serrated lesions? Previous studies have suggested FIT is relatively insensitive to such lesions. While cell free DNA might reduce colonoscopy burden, the two-step approach is also possible to exclude such patients into colonoscopy exam.
2. We really thank the reviewer for this kind suggestion. We have modified the latest version of the manuscript accordingly. We have now included the definition of Colorectal Neoplasia (CN) in the key words at the beginning of the paper, as per your suggestion. By the way, the term includes both Advanced adenomas and Colorectal Cancers. Hyperplastic polyps are not included, while sessile serrated lesions are. Please note that the study is performed on a population of FIT positive patients only, hence the performance of cfDNA on FIT negative subjects has not been investigated, because the purpose was to explore the potential of cfDNA for the triage of FIT positive subjects.

 

2. In the legend of Figure 4, the size of lesions would affect the sensitivity, which was not described in the main text. Actually, the description of figure 4 in the main text seemed unrelevant. Furthermore, why smaller size was associated with higher sensitivity in the figure? It should be further clarified.

We really thank the reviewer for this kind suggestion. We have modified the latest version of the manuscript accordingly.  Thanks to your comment we were able to catch a mismatch of figures and text. In fact, figure 4 refers to lesion size and sensitivity, while the text actually refers to figure 2, which instead shows the calibration curve. We have corrected the error. We also corrected the text for fig.4, describing sensitivities by size for AAs in the Study Population section. As to your question about the association of smaller size with higher sensitivity, we clarified that 4 lesions of size 0-9 mm had features of AAs and were all detected by AAP, thus making 100% sensitivity, although on a small number.

3. 
   
   The presentation was unclear in several places. For example, in page 7, line 219, the "clinical performance section", the authors stated: "The odds ratio was 1.76 (95% C.I. 1.15, 2.69 p= 0.009). Hence, a subject with a positive test had 76.0% greater odds of disease than one with a negative test.” I'm somewhat confused that what is the test here? Dose it means the FIT, or FIT plus cfDNA? Nevertheless, what refers to a negative test here? Dose it mean negative FIT, or positive FIT but negative cfDNA? I think these points should be carefully addressed to improve readibility and soundness.

We really thank the reviewer for this kind suggestion. As previously advised, the study has been performed on FIT positive patients only. This is clearly stated both in the abstract and the introduction and also in other sections of the article. Therefore, the odds ratio explanation refers precisely to a 76% greater odds of disease for a subject with a positive cfDNA test, compared with one with a negative cfDNA test. Although the features of the study population has been clearly explained, we accept your invitation for a greater clarity, specifying in the text that the test being referred to in the odds ratio calculation is the cfDNA.

4. 
   
   I did not found Table 3. although the authors stated: "and CRC. PPV and NPV for CRC were 4.5% and 96.9% respectively (Table 3). There is an 228 expected 33.4% decrease in the need for colonoscopies. (Table 2)..." in Page 7.

We really thank the reviewer for this kind suggestion. We have modified the latest version of the manuscript accordingly:  table 3 refers to table 2, which was labelled 3 in a previous version of the manuscript. It is now reported as table 4, since we added some tables, as described below: Table 2 is referred to the non-inferiority analysis on detection rate, Table 3 refers to the Accuracy parameters and Table 4 is for the accuracy parameters for CRC only.

5. 
   
   In table 2, the authors used CRC to calculated sensitivity, but used the total cases (CN+AA+CRC) to calculated specificity. To me it was a somewhat wierd approach. I suggest the calculation may be based on advanced adenoma (+/- serrated lesions >= 1cm) to be sensible for clinical practice.

We really thank the reviewer for this kind suggestion. We have modified the latest version of the manuscript accordingly. We have included accuracy tables as described above in point 4 and erased the specificity line from calculations for CRC, which may indeed prove confusing.

6. 
   
   Overall, I believe a revision for more easily understandable and consistent content is necessary before publication. Look forward to the responses from the authors.

 

 

In conclusion, we hope to have addressed all of your useful comments and that this work has improved the quality of the paper.

 

Thank you very much indeed,

 

The authors team.

 

 

Reviewer 3 Report

Comments and Suggestions for Authors

The article: Evaluation of Cell-free DNA Long Fragments in the triage of FIT+ patients enrolled in a colorectal cancer screening program: 3 an Italian prospective, cross-sectional study is very practical and well thought out study. It highlights the importance of rationalizing screening using tests like QuantiDNA with good results 

The length of the article and layout is clear and concise 

Author Response

We really thank the reviewer for the kind appreciation

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

I have read the the response and revision thoroughly. The authors have addressed all the questions I raised. However, there are still some comments for the work:

1. The authors explained sessile serrated lesions (SSL) were counted as colorectal neoplasia in the response. However, SSL by definition was different from advanced adenomas. Actually, the cancer risk for a small SSL without dysplasia may be much lower than advanced adenomas. Moreover, the statement in maintext did not include SSL in the colorectal neoplasia.

If SSL are not intended to be included as colorectal neoplasia in the study, I recommend further clarification and discussion. If SSL are included, I would like to see this part of data and please revise the defition as well.

2. The Table legend was somewhat confusing. For example the Table 2, the legend was "Table 2. Non-Inferiority Analysis on Detection Rate (DR) by Cancer Type (Fragment Efficacy Analysis Set)" Do we have multiple types of cancers in the study?

3. Similarly, the font format for table legends was a little bit messy. In addition, there were many references in the table legend making them redundant. The references, if necessary, should be included in the material and method part. 

4. For Table 4, why there was no specificity for CRC?

5. I recommend a table summarize the main clinical findings including CN, AA, CRC based on the cfDNA results.

6. I could not see Figures in the revised maintext.

7. Although the authors have done the non-inferiority test, the results were different for AA and CRC. For CRC, the result was clearly non-inferior to standard care. However, for AA, the 95% CI of risk difference was below 0, respresenting a "non-inferior and inferior" status (N Engl J Med 2017;377:1357-1367. DOI: 10.1056/NEJMra1510063) I would recommend the conclusion should be revised that CRC detection was non-inferior but not including AAs. 

8. Last point, I did not see any other reviewer's response. I strongly encourage multiple reviewers for the peer-review process to improve the work.

Comments on the Quality of English Language

Nil

Author Response

Dear Reviewer,

 

Thank you very much indeed for your comments that will undoubtedly help us improve the quality of this paper.

 

Please see our responses and actions as follows:

 

I have read the the response and revision thoroughly. The authors have addressed all the questions I raised. However, there are still some comments for the work:

1. The authors explained sessile serrated lesions (SSL) were counted as colorectal neoplasia in the response. However, SSL by definition was different from advanced adenomas. Actually, the cancer risk for a small SSL without dysplasia may be much lower than advanced adenomas. Moreover, the statement in maintext did not include SSL in the colorectal neoplasia.
   
   If SSL are not intended to be included as colorectal neoplasia in the study, I recommend further clarification and discussion. If SSL are included, I would like to see this part of data and please revise the defition as well.

• Thank you for your comment. Please see the table here below, which will be now included in the supplements (see table 26):

Value	Parameter	Model 1	Model 2	Model 3
		Total N	n	Estimate	Lower 95% CL	Upper 95% CL	Total N	n	Estimate	Lower 95% CL	Upper 95% CL	Total N	n	Estimate	Lower 95% CL	Upper 95% CL
Adenoma-villous growth pattern (>=25%)	Sensitivity	47	35	74.5	59.7	86.1	49	8	16.3	7.3	29.7	47	38	80.9	66.7	90.9
	PPV	35	35	100.0	90.0	100.0	8	8	100.0	63.1	100.0	38	38	100.0	90.7	100.0
Serrated lesion >/= 10mm	Sensitivity	26	21	80.8	60.6	93.4	27	2	7.4	0.9	24.3	26	21	80.8	60.6	93.4
	PPV	21	21	100.0	83.9	100.0	2	2	100.0	15.8	100.0	21	21	100.0	83.9	100.0
Tubular adenoma >/= 10mm	Sensitivity	42	32	76.2	60.5	87.9	45	5	11.1	3.7	24.1	42	34	81.0	65.9	91.4
	PPV	32	32	100.0	89.1	100.0	5	5	100.0	47.8	100.0	34	34	100.0	89.7	100.0

 

Sessile serrated lesions referred to here are those at greater risk to progress to CRC, i.e. either dysplastic or non-small. As per your suggestion, we have included the SSL in the definition of CN in the glossary of terms.

2. The Table legend was somewhat confusing. For example the Table 2, the legend was "Table 2. Non-Inferiority Analysis on Detection Rate (DR) by Cancer Type (Fragment Efficacy Analysis Set)" Do we have multiple types of cancers in the study?

• Thank you for your observation. There was a typo that we promptly corrected. You will now find “Lesion type” instead of “Cancer type”.
  
  Similarly, the font format for table legends was a little bit messy. In addition, there were many references in the table legend making them redundant. The references, if necessary, should be included in the material and method part. 
• Thank you for your comment, we removed the references from the tables in question and placed them in the materials and methods section.

4. For Table 4, why there was no specificity for CRC?

• Thank you for your comment .We did not find the specificity for CRC to be relevant to the purpose of the study, since all samples testing as positive will be referred to colonoscopy anyway.
  
  I recommend a table summarize the main clinical findings including CN, AA, CRC based on the cfDNA results.
• Thank you for your comment. We arranged a summary table for CN, AA and CRC, here below:

CN	Parameter	Total N	True Outcome	Estimate (%)	Lower 95% CL	Upper 95% CL	P-Value
	Sensitivity	141	107	75.9	68.0	82.7
	Specificity	530	190	35.8	31.8	40.1
	PPV	447	107	23.9	21.9	26.0
	NPV	224	190	84.8	80.3	88.4
	DR	671	107	15.9	13.3	18.9
	PR	671	447	66.6	62.9	70.2
	NLR			0.67	0.49	0.92
	PLR			1.18	1.06	1.32
	OR			1.76	1.15	2.69	0.009
	Youden's J statistic			0.117
AA	Sensitivity	114	87	76.3	67.4	83.8
	Specificity	557	197	35.4	31.4	39.5
	PPV	447	87	19.5	17.7	21.4
	NPV	224	197	87.9	83.7	91.2
	DR	671	87	13.0	10.5	15.7
	PR	671	447	66.6	62.9	70.2
	NLR			0.67	0.47	0.95
	PLR			1.18	1.05	1.33
	OR			1.76	1.11	2.81	0.017
	Youden's J statistic			0.117
CRC	Sensitivity	27	20	74.1	53.7	88.9
	Specificity	644	217	33.7	30.0	37.5
	PPV	447	20	4.5	3.6	5.6
	NPV	224	217	96.9	94.2	98.3
	DR	671	20	3.0	1.8	4.6
	PR	671	447	66.6	62.9	70.2
	NLR			0.77	0.40	1.47
	PLR			1.12	0.89	1.41
	OR			1.45	0.60	3.49	0.404
	Youden's J statistic			0.078

• 
  
  I could not see Figures in the revised maintext.
• We sent them separately, but we will include them in the present reply.

7. 
   Although the authors have done the non-inferiority test, the results were different for AA and CRC. For CRC, the result was clearly non-inferior to standard care. However, for AA, the 95% CI of risk difference was below 0, respresenting a "non-inferior and inferior" status (N Engl J Med2017;377:1357-1367. DOI: 10.1056/NEJMra1510063) I would recommend the conclusion should be revised that CRC detection was non-inferior but not including AAs. 

• Thank you for your comments. In a non-inferiority analysis the starting point is usually the difference between treatment and control (in our case AAP and SOC). Now, if we let T be the treatment and C the control, we can express this difference in two ways: T-C or C-T. Both are acceptable as long as it is clearly stated. In our case, we chose to use T-C and this is reflected in the following table, where you can see that the non-inferiority margin is negative.

Cancer Type	Non-inferiority Margin (%)	Risk Difference (%)	Lower 95% CL	Upper 95% CL	P-Value
CN	-10	-5.07	-9.23	-0.90	0.010
AA	-10	-4.02	-7.89	-0.16	0.001
CRC	-3.8	-1.04	-3.16	1.07	0.005

• 
  
  The rule in non-inferiority analysis is that the point estimate and the lower bound of the 95% CI have to seat entirely within the margin and this is indeed the case for our assay, as one can even better see from the following graph.
  Once again, having we chosen the form T-C, the reader should pay attention to the left side of the graph. Furthermore, the associated p-value of 0.001 confirms statistical significance and non-inferiority for AA.
  However, we take your recommendation for better clarity and specify this choice of calculation in the materials and methods section.
  
  Last point, I did not see any other reviewer's response. I strongly encourage

• 
  multiple reviewers for the peer-review process to improve the work.

 

In conclusion, we hope to have addressed all of your useful comments and that this work has improved the quality of the paper.

 

Thank you very much indeed,

 

The authors team.

Author Response File: Author Response.pdf

Round 3

Reviewer 2 Report

Comments and Suggestions for Authors

I appreciate the delegent revision from the authors. On the other hand, I would like to suggest at least another reviewer to improve this article in terms of readibility.