Diagnosis and Management of Sexually Transmitted Infections Using Artificial Intelligence Applications Among Key and General Populations in Sub-Saharan Africa: A Systematic Review and Meta-Analysis

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This is a well-written manuscript on an important topic following best practices for systematic reviews including protocol registration. My main comment is that although authors are assessing AI methods, the scope of the review is not on 4IR. The term 4IR was coined by the World Economic Forum and encompasses several technologies including AI but also robotics, nanotechnology, quantum computing, biotechnology, the internet of things, etc. Since not all these technologies are part of the scope of the review, it’s incorrect to say that this is a review on 4IR technologies, it’s a review of AI methods. AI methods are commonly used in the healthcare literature and will correctly communicate what is the scope of the review. I suggest removing the term 4IR throughout and use artificial intelligence instead. 4IR can be mentioned in the introduction referring to AI, but the review is not on 4IR.

 

Major points:

1.       One of the main problems with AI methods is that they tend to be 1) context specific and hard to generalize and 2) they are commonly designed for methodological purposes with limited implementation and reach. Authors do not assess the purpose of the study from an implementation perspective, and without knowing whether the study was methodological only (designed and/or tested an AI method in simulation setting) or was implemented (implemented and tested in real clinical settings), it’s hard to determine the actual contribution and potential of the technology. If authors find that most studies were methodological only, they can claim that from a methodological standpoint, AI methods have the potential to contribute to diagnosis and management of STIs “when they reach the implementation stage.” Without this distinction it’s hard to truly determine the stage of this technology and its real contribution. Thus, I suggest authors revisit the 20 studies and classify them for implementation: methodological vs. implementation study).  Please also provider a short paragraph in the results and/or discussion describing this finding.

2.       Authors have mentioned in the Methods section that CASP, Cochrane Risk of Bias Tool, and Newcastle-Ottawa Scale were used for quality and bias assessment, but the quality and bias of the studies is not reported. Please add a table with the quality and bias assessment of each study following the methodology adopted.

3.       While the review is focused on SSA countries, the discussion shouldn’t be. Authors can improve the relevance of their findings by comparing what has been found in SSA with what has been produced in other regions, if discrepancies are found, they should be noted in the discussion by comparing the findings of this review with AI methods used in other regions. Further, most methods used are dated and literature on more advanced methods (e.g., large language models [LLMs] and transformers) must be added to the Discussion to contract methods found in SSA with methods used somewhere else. Fore insurance, there currently are studies using LLMs applied to similar topics as the ones reported in this review? If so (it’s very likely that there are), how they compare and what could they add to SSA countries? Should those be recommended for future research?

4.       Also, with most studies reporting similar outcomes (e.g., precision, recall, F-score), why the results across studies were not quantitatively synthesized (meta-analysis)? Even if this is done only for a smaller subset analyzing similar outcomes it’s also valuable? Please clarify why this was/was not done.  

 

Minor points:

Introduction:

·         Page 2 lines 55-56: This sentence is redundant as it communicates the same information from previous paragraph, please consider revising it.

·         Page 2 lines 71-73: I assume by Goal 3 you refer to the UN 2030 agenda, if so, please specify that in the text. Also, citations 22 and 23 seem to be unrelated to this statement, please revise.

·         Page 2: lines 81-91: the research questions 1 and 2 seem to be inverted, you first identify what methods are used and then you assess their effectiveness. Also, please consider moving the research questions to the Methods section as they are more related to the approach than to background.

Methods:

·         Page 3 - 2.2 Search strategy and database searches: The search strategy takes too much space, please add examples of the terms searched in Section 2.2 and move the complete search strategy to the supplement, readers interested in more details can access it online.

·         Page 6 - 2.4 Selection and screening of documents: This section needs to be further clarified. Please add the initials of each author mentioned as they are cited in the text, and clarify the steps for study selection and who did what (e.g., first one author [XYZ] reviewed the tiles of all studies, next two other authors [XYZ, ABC] reviewed the abstracts of eligible studies, then the full-text was reviewed…)

Results:

·         Figure 3: West Africa is shown as if it was in the east, please adjust it to be on the left side of the map.

·         Tables 3 and 4: These tables have some redundant information and would be more informative if they were merged into a single one.

o   Subject matter in Table 3 seems to be redundant with approach and main findings in Table 4

o   It’s not clear what authors mean by “Comparator” in table 4, if this refers to a gold standard for comparison with an AI model, then just add it to the column Study Design or Approaches.

o   Experimental study is not the design is the type of design, please specify if these are randomized controlled trial and what type of trial

o   Add a collum for Implementation (see my main comments).

·         Page 8: The last two paragraphs of the results section actually belong in the discussion as they start to make links of the findings with the literature on the topic.

Discussion:

Overall, needs to be improved with comparison of findings and methods used in other regions, and how 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

AI success requires access to large databases that can be used to generate predictions (supervised modeling). This is lacking in SSA.

All 3 research questions reflect the degree to which AI algorithms can be applied. This is a function of infrastructure, computational support, the ability to apply the AI algorithms and the underlying requirement of curated databases.

Since this is essentially a systematic review paper, the 3 research questions should be adjusted to include a mention of this approach. Currently they read as if actual numerical values from each site are going to be addressed.

The systematic review details are clearly presented later in the paper.

The 20 studies: While the approaches are mentioned this paper does not attempt to summarize the actual results of each study, especially in relation to the predictive accuracy of the A.I. methods used. If unsupervised methods (clustering of patients) or supervised methods (predictive classification methods) were used this should be reported. Any summary findings (AUC values, predictive accuracy values, clustering summaries, correlations etc.) should also be reported. The table presented seems superficial and incomplete. This additional information would be very useful and should be made available.

Related to this, there should be an overall assessment of the accuracy of the A.I. methods used.

As well, some details on the type and nature of databases used to support the A.I. applications. For example, how similar were they, how large, can they be compared or are they very different.

If there are comments in the papers, some summary of how A.I. is to be further extended in the future would be useful, perhaps breaking this down by country.

Author Response

Comments 1: AI success requires access to large databases that can be used to generate predictions (supervised modeling). This is lacking in SSA.
Response 1: Thank you for pointing this out. We agree with this comment. We have taken it as a compliment as it has been discussed in several sections in the review from the introduction to the end of the manuscript.

Comments 2: All 3 research questions reflect the degree to which AI algorithms can be applied. This is a function of infrastructure, computational support, the ability to apply the AI algorithms and the underlying requirement of curated databases. The systematic review details are clearly presented later in the paper.
Response 2: Agree. Thank you for the insightful feedback. We have, accordingly, revised and modified the research questions to ensure they align with the systematic review details and address the infrastructure, computational support, and curated database requirements for AI algorithm application. See Page 2, lines 80 – 93. The questions have been revised as follows:

Primary Question:
What evidence exists regarding the effectiveness of AI technologies in the diagnosis and management of STIs among KPs and general populations in SSA, based on available studies and curated databases?
Secondary Questions:
What AI technologies are currently documented in the literature for use in diagnosing and managing STIs among KPs and general populations in SSA, and what are their reported applications and limitations?
What barriers and opportunities, including infrastructure, computational support, and database requirements, are identified in the implementation of 4IR technologies for STI diagnosis and management among KPs and general populations in SSA?

Comments 3: The 20 studies: While the approaches are mentioned this paper does not attempt to summarize the actual results of each study, especially in relation to the predictive accuracy of the A.I. methods used. If unsupervised methods (clustering of patients) or supervised methods (predictive classification methods) were used this should be reported. Any summary findings (AUC values, predictive accuracy values, clustering summaries, correlations etc.) should also be reported. The table presented seems superficial and incomplete. This additional information would be very useful and should be made available.

Related to this, there should be an overall assessment of the accuracy of the A.I. methods used.

As well, some details on the type and nature of databases used to support the A.I. applications. For example, how similar were they, how large, can they be compared or are they very different.

If there are comments in the papers, some summary of how A.I. is to be further extended in the future would be useful, perhaps breaking this down by country.
Response 3: Thank you so much for highlighting this. We agreed with these comments, and we conducted the meta-analysis, see lines 395 – 590 to complement the qualitative review that was done. There is section 3.5 on meta-analysis that was added after the qualitative review.

4. Response to Comments on the Quality of English Language
Point 1: The quality of English does not limit my understanding of the research.
Response 1: Thank you so much for such a comment. It is encouraging. Revisions and adjustments were done where the authors saw a need, otherwise, nothing much was changed.
5. Additional clarifications
We have also added some information to complement and strengthen the available information and we have highlighted everything in the document.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript has significantly improved, as I suspected, all studies are methodological and so the real impact at the point of care is still unknown. My only suggestion is that this limitation (that all studies are methodological and lack real-worl implementations/evaluations) should be explicitly stated in the abstract.

Author Response

Comments 1: The manuscript has significantly improved, as I suspected, all studies are methodological and so the real impact at the point of care is still unknown. My only suggestion is that this limitation (that all studies are methodological and lack real-world implementations/evaluations) should be explicitly stated in the abstract.
Response 1: Thank you for pointing this out. We agree with this comment. Therefore, we have added a statement in the abstract to address the comment.
See page 1 lines 22 – 24. However, it is important to note that all the reviewed studies are methodological, and the real-world implementation and impact of these approaches at the point of care remain largely unknown. 

4. Response to Comments on the Quality of English Language
Point 1 The English is fine and does not require any improvement.
Response 1: Thank you for the comment, we take it as a complement. We appreciate it.
5. Additional clarifications
Some Tables have been moved to the Supplement Material section to reduce the length of the manuscript. Table 2: Quality and bias assessment has been moved to SM 8 Quality and bias assessment and Table 5 Extracted data for meta-analysis has been moved to SM 9 Extracted data for meta-analysis.

 

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

The comments and justifications in the paper have been improved.

The paper has been extended to include a meta-analysis based on sample size and F1 score. The F1 score has some drawbacks, but seems to be a useful summary statistic here. The meta-analysis seems reasonable for such limited data and the comparison of a wide variety of methods.

The only issue is the length of the paper. I recommend accepting the paper, but suggesting most of the tables be placed in an appendix or supplementary material. 

Author Response

Comments 1: The paper has been extended to include a meta-analysis based on sample size and F1 score. The F1 score has some drawbacks but seems to be a useful summary statistic here. The meta-analysis seems reasonable for such limited data and the comparison of a wide variety of methods.

The only issue is the length of the paper. I recommend accepting the paper but suggesting most of the tables be placed in an appendix or supplementary material. 

Response 1: Thank you so much for this comment. We agreed and we moved some Tables to the Supplement Material section. Table 2: Quality and bias assessment has been moved to SM 8 Quality and bias assessment and Table 5 Extracted data for meta-analysis has been moved to SM 9 Extracted data for meta-analysis.

4. Response to Comments on the Quality of English Language
Point 1 The English is fine and does not require any improvement.
Response 1: Thank you for the comment, we take it as a complement. We appreciate it.  
5. Additional clarifications
We have added a statement to the abstract explicitly highlighting the lack of real-world implementation of AI approaches at the point of care

Author Response File: Author Response.docx