The Ifakara Ambient Chamber Test (I-ACT) for Evaluation of Indoor Residual Sprays: A Non-Inferiority Test of Sylando^® 240SC and SumiShield^® 50WG

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Machange et al present a very clear and useful study, which shows non-inferiority of a new chlorfenapyr-based IRS compared to an established IRS product, and even more interestingly demonstrate a new testing methodology which is suitable for pro-insecticide-based products which require metabolic activation to kill mosquitoes. This method is presented in sufficient detail for others to adopt it, and the advantages over existing methodologies are well laid out. Addressing the suggestions below would help to clarify some details of the methods and significance of the results.

Some additional discussion of elements of the results would be useful so that the full implications of the data are made clear:

• Pooling all 12 months hides the variability shown in the supplementary time-series. The authors state that both products were evaluated “over 12 months”, but none of the main figures present monthly data and the primary non-inferiority analysis uses the pooled dataset. As written, it implies stable residual efficacy across the year, which is not supported by the supplementary plots. How do the authors account for the wide differences between individual months in mortality? (seen in the supp figures)

• The authors include temperature and humidity data in the supplementary but do not discuss it. From month 2 to 6 the RH in the holding room appears lower than earlier months despite stable temperature. The temperature inside the IACT is consistently higher during months 3 to 7, overlapping somewhat with the same drop in RH. When looking at the monthly mortality data, especially for mud, declines in mortality often occur in the same period. It is hard to judge the significance of this is from the analysis, but it should be acknowledged.
• There is no discussion of species-specific patterns.
• For An. gambiae, Sylando performs very similarly to SumiShield at 168 h across substrates.
• For An. arabiensis, Sylando is slightly weaker on mud and concrete but still broadly acceptable.
• For An. funestus, Sylando consistently underperforms on mud and, to a lesser degree, concrete, often at 40 to 70 percent mortality compared with 80 to 100 percent for SumiShield. This is a clear biological difference that could be addressed in the Discussion.
• The paper states that “both products exhibited comparable performance” and that “mortality was almost identical between products on wood and mud”, but for An. funestus Sylando showed consistently lower mortality on concrete relative to Sumishield which should be addressed, particularly given the growing significance of funestus as a malaria vector.
• Further discussion of the Culex and Aedes data would also be informative given the observed differences. There appears to be more variation between the treated substrates for the Aedes experiments, compared to the Anopheles strains. Why do the authors think this is?

• The results suggest that the main driver for mortality is the long 168h follow up time. Without measuring the amount of mosquito movement in the MEH, how can the authors be certain of the contribution of increased metabolic activity due to flight?

Data analysis and presentation is good, but some further clarification would be useful:

• A 7 percent non-inferiority margin is applied, which is a generally accepted value for non-inferiority, but the authors do not explain where this comes from.
• The WHO AMS IRS annex gives no numerical guidance.
• The WHO non-inferiority protocol for IRS and ITNs recommends a margin on the odds-ratio scale with OR = 0.7 as the consensus value for entomological endpoints.
• If I have read their analysis correctly, they have instead assessed non-inferiority using percentage differences. An OR of 0.7 is not equivalent to a 7 percent difference. They also cite Piaggio et al., which explicitly states that the margin should be justified on biological or clinical grounds. They need to be explicit about the source and rationale for the 7 percent threshold and cite the correct WHO guidance.
• The non-inferiority figure also needs a clear explanation of what the coloured regions represent.

Minor comments and requests for clarification or further detail:

• Fig 2: The captions are out of order with the images (d, e and f)
• What was the plywood used (indoor, outdoor, marine?), how thick was it? did it have any sealant applied or sanding before spraying?
• Lines 44-45 – perhaps include a sentence on why free flight is important, for those unfamiliar with chlorfenapyr
• Line 102 – useful to specify that a modified I-ACT method with a host was used
• Section 2.3 – the text says that panels were designed to be treated with a track sprayer, but Figure 2 shows manual spraying, maybe clarify what was done
• Figure 4 – Clarify in the legend that this shows (I think) mortality in assays done 12m after treatment of the panels, rather than an average over the 12m
• The Methods state that blood feeding was measured, but this data is not presented in the Results
• Lines 289-303 – this paragraph is interesting, but not relevant to this paper, which is looking at product efficacy not resistance monitoring. If kept, the last sentence could be adapted to say that this highlights the need for appropriate methods when testing efficacy or (or susceptibility to) a pro-insecticide such as chlorfenapyr
• Line 311 – reconsider this mention of the use of chlorfenapyr in ITNs – this is an argument against the introduction of a new IRS based on chlorfenapyr, from a resistance management point of view
• Table 2 – might be interesting to add in the WHO bottle bioassay, since it is discussed in the text.
• The conclusion does not mention the comparison between IRS products, which is included in the title as a goal of the study
• Table S1 – what does ‘-‘ signify? Not done, or 0%, or...?

Author Response

Thank you very much for taking the time to review this manuscript. Please find the detailed responses attached and the corresponding revisions highlighted in yellow in the re-submitted files.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This manuscript presents a comprehensive and well-designed 12-month semi-field evaluation of two WHO-prequalified indoor residual spray (IRS) products, chlorfenapyr (Sylando® 240SC) and clothianidin (SumiShield® 50WG), using a novel bioassay platform—the Ifakara Ambient Chamber Test (I-ACT) with Miniature Experimental Huts (MEH). The core finding that Sylando® 240SC is non-inferior to SumiShield® 50WG against key malaria vectors on multiple substrates is significant for vector control portfolios. Furthermore, the development and validation of the I-ACT/MEH assay to evaluate pro-insecticides like chlorfenapyr under free-flight, host-present conditions addresses a critical methodological gap in the field. The study is generally well-executed and reported. However, several major clarifications and methodological justifications are required before the manuscript can be considered for publication.

Major Concerns:

The choice of a 7% non-inferiority margin is pivotal to the study's conclusion. The manuscript states it was used but does not justify why this margin was selected. Was it based on regulatory guidelines, historical data from experimental hut trials, or a consensus on a minimally important efficacy difference for public health impact? A clear, referenced justification must be provided in the Methods section (2.10).

While the I-ACT is described, more details are needed to allow for potential replication and to assess its representativeness. A schematic diagram of the MEH within the I-ACT chamber, showing dimensions, entry/exit points, and host placement, would be invaluable.

The authors state the surface area-to-volume ratio is ~3.3, double that of standard experimental huts. Please discuss the potential implications of this. Could a higher ratio lead to an overestimation of contact and efficacy compared to a real-world room? Some discussion on how MEH results are expected to correlate with full-scale hut data is necessary.

The manuscript would be strengthened by a direct comparison of mortality results for a standard insecticide (e.g., a pyrethroid) obtained via the I-ACT/MEH assay versus a standard cone test or a historical hut trial, to "benchmark" the new system.

The primary non-inferiority analysis pools data from An. gambiae (susceptible), An. arabiensis (resistant), and An. funestus (resistant). While this increases statistical power, it may obscure important biologically and operationally relevant differences. It is strongly recommended to present the non-inferiority analysis for each malaria vector species separately (at least in the supplement). The efficacy pattern appears different for An. funestus (Fig. 4), and pooling may not be fully justified.

Relatedly, the results for culicine species (Ae. aegypti, Cx. quinquefasciatus) are presented but not included in the main non-inferiority claim. The abstract states the aim was to measure efficacy against "vectors of malaria and dengue," but the conclusion focuses on malaria. Please clarify the objective regarding non-malaria vectors or adjust the title/abstract accordingly. A brief discussion on the implications of the lower efficacy against culicines is warranted.

Chemical analysis confirmed the initial application dose (Table 1). However, for a 12-month study, it is important to know how the surface concentration of the active ingredient declined over time, particularly for chlorfenapyr, which may have different environmental persistence than clothianidin. Were substrate samples analyzed at multiple time points? If not, this is a significant limitation that should be explicitly acknowledged in the Discussion. Statements about "12-month residual efficacy" are based on bioassay results but lack chemical residue data to support them.

The comparison of bioassay methods (Table 2) is useful. However, the discussion should more explicitly state the key limitations of the I-ACT/MEH: (1) It uses laboratory colonies, not wild, free-ranging mosquitoes with natural host-seeking histories. (2) It does not measure community-level effects like deterrence or excito-repellency that huts can capture.

The statement in the Conclusion (lines 354-355) that I-ACT bridges the gap between lab and field studies should be tempered. It is a semi-field tool that sits between the two, offering significant advantages over lab tests but not fully replacing the need for ultimate field validation in experimental huts.

Minor Suggestions:

Abstract/Simple Summary: The Simple Summary mentions dengue vectors, but the primary analysis does not include them. Ensure consistency.

Line 64 (Keywords): "and SumiShield®" appears duplicated and should be cleaned up.

Figure 3/4: Consider using consistent color schemes for the two insecticides across all graphs to improve readability.

Statistical Model: In section 2.10, clarify if "chamber and day" were treated as random effects in the mixed model, as the wording "fixed effects" is used.

References: Ensure all in-text citations have corresponding entries in the reference list.

Author Response

Thank you very much for taking the time to review this manuscript. Please find the detailed responses attached and the corresponding revisions highlighted in yellow in the re-submitted files.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors have provided a thorough and detailed response to the reviewer's previous comments. All concerns raised in the initial review have been addressed in a satisfactory manner. The revisions have substantially improved the quality and clarity of the manuscript. No further technical or scientific issues remain.