Developing Consensus Standard Operating Procedures (SOPs) to Evaluate New Types of Insecticide-Treated Nets

Simple Summary Malaria control relies on insecticide-based tools which target the mosquito vector. Predominantly, a group of insecticides called pyrethroids are used in these tools. Globally, however, mosquitoes are increasingly developing resistance to pyrethroids. Subsequently, new products, such as insecticide-treated nets (ITNs), which contain combinations of insecticides from different classes, or chemicals that work synergistically with pyrethroids, are being developed. Several of these new net types are being rolled out for testing and use. However, standardized methods to measure how long these nets remain active against mosquitoes are lacking, which makes evaluating the long-term efficacy of these products challenging. In this publication, we propose a pipeline used to collate and interrogate several different methods to produce a singular ‘consensus standard operating procedure (SOP)’, for monitoring the residual efficacy of three new net types: pyrethroid + piperonyl butoxide (PBO), pyrethroid + pyriproxyfen (PPF), and pyrethroid + chlorfenapyr (CFP). Abstract In response to growing concerns over the sustained effectiveness of pyrethroid-only based control tools, new products are being developed and evaluated. Some examples of these are dual-active ingredient (AI) insecticide-treated nets (ITNs) which contain secondary insecticides, or synergist ITNs which contain insecticide synergist, both in combination with a pyrethroid. These net types are often termed ‘next-generation’ insecticide-treated nets. Several of these new types of ITNs are being evaluated in large-scale randomized control trials (RCTs) and pilot deployment schemes at a country level. However, no methods for measuring the biological durability of the AIs or synergists on these products are currently recommended. In this publication, we describe a pipeline used to collate and interrogate several different methods to produce a singular ‘consensus standard operating procedure (SOP)’, for monitoring the biological durability of three new types of ITNs: pyrethroid + piperonyl butoxide (PBO), pyrethroid + pyriproxyfen (PPF), and pyrethroid + chlorfenapyr (CFP). This process, convened under the auspices of the Innovation to Impact programme, sought to align methodologies used for conducting durability monitoring activities of next-generation ITNs.


Introduction
Globally, malaria control progress is plateauing, and, in some instances, case numbers are rising [1]. Although the reasons for this are multifaceted, an increasing and intense resistance to pyrethroids in Anopheles vectors is almost certainly a contributing factor. Insecticide-treated nets (ITNs) have significantly contributed to the control of malaria over the past two decades [2]. However, currently, all WHO-prequalified ITNs contain pyrethroids [3], and pyrethroid resistance is widespread in all major malaria vectors [4,5].
In response to growing concerns over the sustained effectiveness of solely pyrethroidbased control tools, new products are being developed and evaluated. Examples of these are dual-active ingredient (AI) ITNs containing an additional insecticide, or synergist ITNs which contain an insecticide synergist, in combination with a pyrethroid. These net types are often termed 'next-generation' insecticide-treated nets. The second AIs have a different mode of action (MoA) from their partner pyrethroid, to improve the control of resistant vector populations.
The current methods for measuring ITN durability [6] were developed for pyrethroidonly nets, which cause rapid knockdown and death in susceptible mosquitoes. Consequently, the different MoAs of the new insecticides necessitate the need for new protocols to reliably measure net durability. In nets with the synergist piperonyl butoxide (PBO), the PBO works by improving the efficacy of the pyrethroid it is paired with, in populations with pyrethroid resistance due to increases in oxidase activity, and is itself generally noninsecticidal. Without suitable mosquito strains or net controls, it is difficult to determine if the synergist component of the net is long-lasting using the currently recommended methods. For other AIs, such as chlorfeniapyr, which targets the insect mitochondria, or pyriproxyfen, which is a juvenile hormone analogue, 'non-standard' endpoints such as delayed mortality and insect fertility and fecundity need to be measured to assess biological durability (bioefficacy, measured through direct impact on mosquitoes).
Several of these new types of ITN are being evaluated in large-scale randomized control trials (RCTs) and pilot deployment schemes. These trials are expected to demonstrate the biological durability, attrition, and fabric integrity of these new net types when under long-term household use. Measuring the biological durability of the ITNs involves assessing the insecticidal activity of a sub-sample of randomly selected nets withdrawn from the field. There is an urgent need for methods to reliably measure the bioefficacy of these nets, to collect baseline data, and to subsequently measure the durability of biological efficacy of nets collected from the field after fixed periods of use. This has resulted in methods for measuring net bioefficacy and biological durability being developed and utilized by multiple programme teams, which makes comparing the results of these studies complex. A better approach would be for programme teams to adopt a single, standardized method validated using a multi-site approach.
In this publication, we demonstrate the process used to collate and interrogate several different methods to produce a singular 'consensus standard operating procedure (SOP)', for evaluating the biological efficacy of new net types, suitable for durability monitoring. Our objective was to create procedures that build on the experience from studies already underway. We also considered the feasibility of conducting these methods in as many sites as possible, accounting for factors such as throughput of mosquito colonies and space, which can preclude the use of certain methods and inform choices about sample sizes and replicate numbers.
This project forms part of a package of work to improve entomological methods in vector control and is supported by Innovation to Impact (I2I) at the Liverpool School of Tropical Medicine (LSTM). Three new types [7] of ITN are used as case studies: pyrethroid + Insects 2022, 13, 7 3 of 27 piperonyl butoxide (PBO), pyrethroid + pyriproxyfen (PPF), and pyrethroid + chlorfenapyr (CFP). The final consensus SOPs for measuring the biological durability of these net types are included in Additional Files 2-4 (Supplementary materials).

Materials and Methods
For each net type, a collaborative process of method development and iterative drafting was conducted to produce a consensus SOP (Figure 1). Initially, a group of stakeholders was formed. Inclusion in these groups was based on having (1) a research interest in the development or deployment of new net types, (2) experience in the development or testing of new net types, or (3) an involvement in ongoing trials or deployment schemes of new net types. Available methods for measuring the biological durability of each net type were then identified through consultations with stakeholder groups and literature searches. This was not a systematic process, and for each net type, several historical procedures exist which were not considered here. Rather, the focus was to identify SOPs currently being developed or utilized which evaluated the biological durability of new net types and to use them to align the methods on points of difference. For each net type, the experimental parameters of the method were established (i.e., exposure method, controls used, population, replicates, endpoints). Values for each parameter were extracted from all accessible methods and compared before a 'consensus value' was suggested for each experimental element. Other methodological questions were identified for discussion. At this stage, the method development document was shared with the stakeholder group for comment, and further discussed on a group call. The feedback on the method development was then used to prepare a draft consensus SOP. The draft was distributed with the group for a second round of comments and discussion. Following the incorporation of this feedback, a final consensus SOP was produced and submitted to the group for approval. and space, which can preclude the use of certain methods and inform choices about sample sizes and replicate numbers. This project forms part of a package of work to improve entomological methods in vector control and is supported by Innovation to Impact (I2I) at the Liverpool School of Tropical Medicine (LSTM). Three new types [7] of ITN are used as case studies: pyrethroid + piperonyl butoxide (PBO), pyrethroid + pyriproxyfen (PPF), and pyrethroid + chlorfenapyr (CFP). The final consensus SOPs for measuring the biological durability of these net types are included in Additional Files 2-4 (Supplementary materials).

Materials and Methods
For each net type, a collaborative process of method development and iterative drafting was conducted to produce a consensus SOP (Figure 1). Initially, a group of stakeholders was formed. Inclusion in these groups was based on having (1) a research interest in the development or deployment of new net types, (2) experience in the development or testing of new net types, or (3) an involvement in ongoing trials or deployment schemes of new net types. Available methods for measuring the biological durability of each net type were then identified through consultations with stakeholder groups and literature searches. This was not a systematic process, and for each net type, several historical procedures exist which were not considered here. Rather, the focus was to identify SOPs currently being developed or utilized which evaluated the biological durability of new net types and to use them to align the methods on points of difference. For each net type, the experimental parameters of the method were established (i.e., exposure method, controls used, population, replicates, endpoints). Values for each parameter were extracted from all accessible methods and compared before a 'consensus value' was suggested for each experimental element. Other methodological questions were identified for discussion. At this stage, the method development document was shared with the stakeholder group for comment, and further discussed on a group call. The feedback on the method development was then used to prepare a draft consensus SOP. The draft was distributed with the group for a second round of comments and discussion. Following the incorporation of this feedback, a final consensus SOP was produced and submitted to the group for approval.

Case Study 1: ITNs Containing Pyrethroid plus Piperonyl Butoxide (Pyrethroid + PBO Nets)
Currently, six pyrethroid + PBO nets are prequalified by the WHO (DuraNet Plus, VEERALIN, PermaNet 3.0, Tsara Boost, Tsara Plus, Olyset Plus) [3]. These vary in several specifications (Additional File 1: Table S1) such as pyrethroid AI, PBO concentration, and location of PBO on the net (roof only or on all panels). A conventional cone test, followed by a tunnel test for those nets which fail to reach cone bioassay thresholds [8], is suitable for exposing mosquitoes to pyrethroid + PBO nets and monitoring mortality. Certain methodological parameters of the WHO cone test, such as replicate number and control nets, vary depending on if the assay is being used for WHOPES (the precursor to WHO prequalification) phase I, II, or III testing. The WHO guidance states "candidate LNs (nets) treated with insecticides with effects on mosquitoes that differ from those of pyrethroids may require proof of principle and new assays" [8]; however, guidance or thresholds on how to interpret PBO-synergism for biological durability monitoring is not available.
Nine methodologies that measure pyrethroid + PBO net biological durability were identified through searching the literature and contacting key stakeholders (Table 1). Of these, methods were accessible for six of them (published or provided on request). Of the remaining three, one study had not yet finalized its methods (ID = 7), one confirmed it was not conducting biological durability monitoring (ID = 8), and one did not have biological durability monitoring listed as an intervention endpoint on its clinical trial registry; the authors were contacted to confirm this, but they did not respond (ID = 9). Values for each methodological parameter were extracted from the accessible SOPs and a 'consensus' value suggested for each parameter ( Table 2). It was established that one method (ID = 2) was an updated version of another (ID = 1), so study #2 was later excluded. Nets and mosquitoes should be acclimatized to the temperature and humidity of the testing room for a minimum of 1 h before testing. This is critical if nets have been stored in a refrigerator or cold room. • For mosquitoes collected as larvae from the field, details on the collection procedure, such as the number and distribution of collection sites, and mosquito-rearing conditions, should be recorded. • Some pyrethroid + PBO nets have different pyrethroid concentrations on the sides and the roof and this should be considered in the data recording and interpretation. Therefore, it is important that net pieces are well labelled to establish if the sample is from the roof or sides, and data should be recorded per net piece. Though analysis should be pooled for each net for interpretation, having the data disaggregated in this way will allow for further interrogation of the data if required.  This aligns with the other new net type SOPs, and with the standard WHO biological durability testing where (post-baseline) 4 pieces of net are tested [6]. The decision to take equal pieces from the roof is due to greater mosquito activity observed here [9][10][11] and because some nets only have PBO on the roof.
During their manufacture, roof panels can come from different net runs than side panels [12].
Replicate tests per piece of net 2 cones per net piece; PBO all over: n = 40; PBO roof only: n = 60.

1.
It was decided that it was clearer to structure the SOP based on net panel type (i.e., a pyrethroid-only net panel), rather than describe testing based on nets with 'PBO all over' vs. 'PBO mosaic net' (PBO on the roof only). This structuring should allow adaptation to ITNs that may be developed in the future with different net panel configurations.

2.
Number of pieces sampled from each net: WHO biological durability monitoring [6] for pyrethroid-only nets recommended sampling one piece from the net roof and three-four pieces from the sides (four-five total). Our original proposal for pyrethroid + PBO nets was to sample three pieces from the roof and three from the sides (six total). The decision to test more roof samples was based on research which has shown greater mosquito activity on the net roof [9][10][11], the acknowledgement that some pyrethroid + PBO nets have different physio-chemical properties on the net roof, and that, during their manufacture, roof panels come from different net runs than side panels [12]. However, weighing up the benefits of a more precise measurement of intra-net heterogeneity by using six replicates per net against the challenge of evaluating large cohorts of ITNs with high numbers of mosquitoes per net, it was decided that the key measurement was the estimated bioefficacy of a cohort of ITNs. Therefore, it is important to be able to evaluate as many ITNs as possible (as nets have a high degree of heterogeneity due to different variability in use and care) while balancing this against the requirement for mosquitoes. Four samples per net (two from the roof, two from the sides) will allow the maximal numbers of samples to be tested without putting undue strain on testing facilities.

3.
Replicates: The original proposal was four replicates per net sample based on the WHOPES recommendations for pyrethroid-only nets [6]. However, this made the required mosquito numbers unfeasible. The consensus was that two replicates per net sample was sufficient. If mosquito numbers are abundant, testing should prioritize testing more nets (if available), as this will provide more precision. If additional nets are not available, surplus mosquitoes could be used to conduct more test replicates. After the consensus SOP was developed, a pre-print was published [13], which contained additional methods for the planned evaluation of the biological durability of PBO nets. The methods published in that report were compared to the draft consensus SOP and, methodologically, these were found to be largely the same, with some variability in sampling position and number of net samples/replicates.

4.
Testing should primarily use the WHO cone method specified in the consensus SOP (Additional File 2). A tunnel test may be used as a second test when nets fail to meet WHO thresholds (<95% 60-min knockdown or <80% 24-h mortality in a susceptible strain [6]), although this is not preferred. Currently, there are no recommended thresholds for resistant mosquito strains.
Following feedback from stakeholders, a final consensus SOP was produced and approved by the group (Additional File 2: I2I-SOP-001: Methods for monitoring the biological durability of insecticide-treated nets containing a pyrethroid plus piperonyl butoxide (PBO)).

Case Study 2: ITNs Containing Pyrethroid plus Pyriproxyfen (Pyrethroid + PPF Nets)
Royal Guard, developed by Disease Control Technologies, is currently the only WHO prequalification listed pyrethroid + PPF net (Additional File 1: Table S2). The WHO cone test is a suitable method for exposing mosquitoes to pyrethroid + pyriproxyfen (PPF) nets for measuring the nets' biological durability, but different endpoints are needed for each active ingredient. Knockdown and mortality can be used to assess the bio-efficacy of the pyrethroid but the most suitable endpoints for PPF, a juvenile hormone analogue that affects fertility and fecundity in mosquitoes, need to be defined.
Seven documents detailing methods for evaluating pyrethroid + PPF nets were provided by stakeholders (Table 3). One of these (ID = 1) did not measure fertility endpoints. Of the remaining documents, four detailed methods for oviposition observations, and two detailed methods for ovary dissection. To reach a consensus SOP for both methods, methodological parameter values were extracted from available SOPs and a 'consensus' value was proposed for each one (Oviposition: Table 4; Dissections: Table 5). Methods for both oviposition and dissection are included, as discussions showed differences in preference between labs for one or the other method ( Figure 2).      Fed in the hour before exposure.
3-9 h before net exposure Blood fed using method of feeding standard for the test population (e.g., Hemotek membrane feeding system, arm feed, animal fed to repletion).
There is little data available and some contradiction on the impact of time of blood feeding, and this could be validated. Consensus was that this was a suitable and logistically possible method.  The cone test has been used in several studies to evaluate PPF nets and seems to be a suitable method of exposure.

Exposure time Not included in SOP 3 min 3 min
This is the standard exposure time used in WHO cone bioassays [6]. Preliminary validation testing will be conducted to look at effect of exposure time. Females 'freshly blood fed' for exposure.
3-9 h before net exposure. Blood fed using method of feeding standard for the test population (e.g., Hemotek membrane feeding system, arm feed, animal feed).
There is little data available and some contradiction on the impact of time of blood feeding, and this could be validated. Consensus was that this was a suitable, and logistically possible, method.
Mosquitoes per replicate N/A 5 5 per cone This is the standard number used in WHO cone bioassays [6].  This is a well-established method for scoring fertility Changes Made to the Proposed Methods following Stakeholder Discussions

1.
The option to score oviposition and then dissect those that did not lay was discounted. This would have meant dissections were being conducted on non-standardized days, making results incomparable to data collected using the standard dissection method, and likely resulting in a small sample size for that subset. For similar reasons, those which died before oviposition counts should not be dissected and scored.

2.
As we do not expect the pyrethroid to impact fertility, and we are using a pyrethroidresistant strain, the untreated net is a useful negative control, and oviposition inhibition can be compared to this. Therefore, the decision was made not to include a pyrethroid-only net.

3.
Questions remain regarding the 'net effectiveness threshold' for sterility endpoints. For pyrethroid only nets, a net is considered effective if KD 60 is >95% or 24-h mortality is >80% [6]. We do not yet know what an operationally meaningful level of sterility is, i.e., what level of sterility in a cone test means the net is controlling mosquitoes in the field. Hence, it is not yet possible to set a threshold for biological durability monitoring, and the best approach is to simply monitor for a reduction in sterilizing effect over time. However, this question is critical and should be considered as data is generated.

4.
When analyzing the results, the untreated net and the test net should be paired, i.e., a single control for the day acts as the benchmark for all tests on that day, and inhibition is calculated against that day's control. Inhibition can be calculated by odds ratio using regressions.

5.
Following the development of the consensus SOP, a pre-print was published, which contained additional methods planned for evaluating biological durability of PPF nets [13]). These methods were compared to the drafted consensus SOP and found to be methodologically the same, apart from some variability in sampling position and number of net samples/replicates.
Following feedback from stakeholders, a final consensus SOP was prepared and approved by the group (Additional File 3: I2I-SOP-002: Methods for monitoring the biological durability of insecticide-treated nets containing a pyrethroid plus pyriproxyfen (PPF)).

Case Study 3: ITNs Containing Pyrethroid plus Chlorfenapyr (Pyrethroid + CFP Nets)
Interceptor G2 (IG2), developed by BASF, is currently the only WHO prequalification listed pyrethroid + CFP net (Additional File 1: Table S3). The cone test has been shown to be ineffective in reliably measuring the bioefficacy of the chlorfenapyr component of IG2 nets [16], and so an alternative bioassay is needed. There is a growing consensus around the WHO tunnel test as being the best method to assess IG2 bioefficacy. This should be run in parallel with a standard WHO cone test [6], which assesses the biological durability of the alpha-cypermethrin component of the net. The SOP discussed and included (Additional File 4) here is related to assessing the biological durability of the CFP component.
Eight documents, detailing methods used for evaluating pyrethroid + CFP nets, were provided by stakeholders (Table 6). Of these, three were generic SOPs for conducting the 'net in tube' cylinder assay (ID = 6) or tunnel test (ID = 7, 8), and did not contain specific experimental parameters for testing CFP nets, and, therefore, information was not extracted from them for comparison. Methodological parameters were extracted from the available SOPs, compared, and used to propose a 'consensus' value for each (Table 7). Table 6. List of identified methods/trials measuring pyrethroid + CFP net biological durability.    In the standard WHO tunnel test, one net piece is used [6]. The increase allows a 2nd piece from the roof to be tested. During their manufacture, roof panels can come from different net runs than side panels [12].  Changes Made to the Proposed Pyrethroid + CFP Methods following Stakeholder Discussions

1.
Where tunnel testing is not possible, it would be beneficial to have an additional method available. It was established that S. Moore will be validating the I-ACT method [18] for IG2 testing, and K. Gleave will be validating the 'Net in Tube' (cylinder) test. When complete, we will include these SOPs with the tunnel-test methodology on the I2I website (https://innovationtoimpact.org/workstreams/methodsvalidation/). Accessed on 20 December 2021.

2.
Following a preliminary discussion with all stakeholders, a sub-group was formed with key individuals to start a draft proposal for the CFP methodology. In the initial meeting, representatives of BASF joined to share information on Interceptor G2. Following on from these discussions, a draft method development with methodological parameters for the tunnel test was shared with the sub-group, and this was refined before sharing with the full stakeholder group for approval. There is a lack of data on how mortality in tunnel tests changes with mosquito numbers (the standard is 100 mosquitoes in a tunnel). Reducing the sample to 50 mosquitoes per tunnel allows us to increase the sample pieces tested per net without increasing mosquito numbers. However, this also increases the risk of having to disregard testing results if high control mortality is observed-control mortality would still be based on 100 mosquitoes, but over two net replicates.
a. Data comparing the use of 50 vs. 100 mosquitoes in tunnels with pyrethroid nets are available (Moore, Personal communication), and these data were considered to confirm the number of mosquitoes tested. b.
Further to this, preliminary work to compare 50 vs. 100 mosquitoes in tunnels using Interceptor net and Interceptor G2 nets was conducted, and found no significant difference in these two numbers (Kamande, Personal communication).

5.
The number of mosquitoes required must be balanced against the number of replicates, since maximizing the number of nets, to measure efficacy of the ITN population, is key. There was some disagreement over which was the best balance. It is likely that the capacity to test more mosquitoes per net will be related to mosquito availability in the testing sites. Therefore, it is suggested we validate with the lower number to make the SOP less onerous for testing sites. We are interested in measuring the biological durability of the ITN population-not individual nets, which could be highly variable. Currently, the WHO recommends 30 nets per time point, but increasing this will provide better data. Thirty nets should be seen as the minimum. Reducing the number of mosquitoes may allow increases in replication to be possible. 6.
Control thresholds: blood-feeding must be >50% on the untreated control net. Mortality will be measured up to 72 h, due to the slow-acting nature of chlorfenapyr. Mortality in the untreated control must be <10% after 24 h and <20% at 72 h (both must be true for the test to be valid).
Following feedback from stakeholders, a final consensus SOP was produced and approved by the group (Additional File 4: I2I-SOP-003: Methods for monitoring the biological durability of insecticide-treated nets containing chlorfenapyr).

Discussion
Methodological consistency is crucially important when monitoring the durability of new net types, due to there not being validated methods to assess these tools. Even small differences in testing methods may lead to additional sources of variation in endpoints, making results difficult to interpret between countries, studies, and test facilities. The use of standardized testing methods streamlines the process of product evaluation, leads to a more rapid generation of consistent performance data across studies, and subsequently speeds up product uptake. In vector control, methods for new tools with novel modes of action are often developed in one site or by one group in response to a specific product or research question. This can narrow the applicability of that method, make it challenging to adopt it at other sites, or it may not be applicable to all products within a particular product class.
Developing evaluation methods in a collaborative group ('consensus' SOPs) allows the process to benefit from the collective knowledge and experience of a diverse set of stakeholders, and maximizes the chances for a specific methodology that will be widely relevant. However, developing a consensus SOP is just one of the first steps in the methodvalidation pipeline. Defining and improving the robustness of a method can be viewed as an incremental process which follows a stepwise progression from singular SOPs to consensus SOPs, to consensus SOPs that are experimentally validated at one site, and finally, to consensus SOPs that are validated at multiple sites. In this publication, we have defined the desired endpoints, and designed and refined methodologies for evaluating the biological durability of three new net types. The next steps in this process will be to (1) quantify inherent errors in the methods, (2) evaluate the ability of the methods to accurately characterize the vector control product, and (3) validate these results in multiple facilities. The scope of this would include assessing the methods' ability to measure the biological durability of different products within the class of nets, and against different vector species. More information is gathered when a method is in operational use, which can help to improve or refine the method. At this stage, it is imperative to ascertain that the methods can be implemented and used successfully within research teams, and identify training needs, if required. This is to ensure that data collected using these methods are as transferable and comparable as possible.
The agreement on key entomological endpoints to be measured, followed by the use of standardized and validated methods to measure them, needs to be partnered with an acceptance of the need for flexibility in product evaluation. For instance, the SOPs developed here have been formulated based on nets that are currently in development/on the market and therefore may be unsuitable for new formulations or designs within the same product classes. However, it should be noted that this is the way that previous ITN guidelines were developed-in response to new technologies coming to market [6]. It is challenging to 'future-proof' methods from the outset, especially in a rapidly evolving landscape which must be sensitive to the pressures of evolving and emerging insecticide resistance. Therefore, the process cannot be averse to change or updates in the future, which would lead to stagnation in innovation and delayed decision making-such has been the situation with non-pyrethroid products being evaluated with tests designed for pyrethroids. Regular updates of guidance based on consensus among key stakeholders will harmonize data collection procedures and, ultimately, hasten progress towards the goal of bringing new vector control products to market more rapidly, using robust data-driven decision making.
To take this further, the dissemination of up-to-date methods is crucial to ensure relevant data are being collected whenever possible. This process, convened under the auspices of the Innovation to Impact programme, sought to align methodologies used by those conducting durability monitoring activities of new net types (so-called 'nextgeneration ITNs'). While this objective was largely achieved through the engagement and insight of those involved, it is important to recognize that even though this process involved the key stakeholders in designing and implementing durability monitoring, the current durability monitoring guidelines [6] for these products may differ or simply do not exist. There is a clear need for further engagement with normative (WHO and control