Guidance for Clinical Evaluation under the Medical Device Regulation through Automated Scoping Searches
Round 1
Reviewer 1 Report
- There are several databases, why did you choose only 2 and why Pubmed and clinical trial?
- Line 227: explain what PubMedBERT is, we don't quite understand
- Line 237: how many people were involved in reading and selecting the articles? please specify and note
- Line 262: why did you do 30 min sessions?
- Line 299: what criteria were used to assess the relevance of the MESH?
- How to explain the differences between the 2 SMEs presented in Table 2
- Figure 5: It is not the accuracy that changes between the first and second search (25-65 and 65%-80%) but the search time (Publications of the last 10 years and no filter), is this correct?
- What is the mAP score? detail it and present it
- What do you mean by accuracy? How was it evaluated?
- The big limitation of this work is that not all articles are studied, only the top 10, how do you justify this and do you think this could have an impact on the results?
- For innovative medical devices there is not always an associated MESH term? did you evaluate this bias?
Author Response
Dear Reviewer,
Thank you very much for providing us the review on our manuscript ‘Guidance for Clinical Evaluation under the Medical Device Regulation through Automated Scoping Searches’.
We are very pleased that you consider our work for publication in the Applied Science Journal. We carefully addressed the mentioned points in the revised version of our manuscript und used the MS WORD Track Change function to highlight all revisions.
Please find our responses to the comments in the uploaded Response_FirstReviewer_15June2023.docx document, as well as an overview of all the changes made.
We will be happy to comply with any further suggestion for improvement raised by you.
Yours sincerely,
Kim Tang and Robert Farkas
Author Response File: 
Author Response.docx
Reviewer 2 Report
The paper is well written and understandable and it presents two search methods on the data from pubmed and clinicaltrials.gov.
As an example the topic was searching information about medical devices.
comment 1: L85-86 apart from searching clinical data on medical devices the search strategies have not been investigated for myriad of other domains why it is important for those strategies?
comment 2: L414-415 It is clear that the task of extracting mesh-terms from description document has helped, but there is no comparison that the overall effort to get to the same/similar results was lower compared to standard keyword search.
Author Response
Dear Reviewer,
Thank you very much for providing us the review on our manuscript ‘Guidance for Clinical Evaluation under the Medical Device Regulation through Automated Scoping Searches’.
We are very pleased that you consider our work for publication in the Applied Science Journal. We carefully addressed the mentioned points in the revised version of our manuscript und used the MS WORD Track Change function to highlight all revisions.
Please find below our responses to the comments in the uploaded document Response_SecondReviewer_15June2023.docx, as well as an overview of all the changes made.
We will be happy to comply with any further suggestion for improvement raised by you.
Yours sincerely,
Kim Tang and Robert Farkas
Author Response File: Author Response.docx
