Next Article in Journal
How Rearing Systems for Various Species of Flies Benefit Humanity
Previous Article in Journal
Investigating Photo-Degradation as a Potential Pheromone Production Pathway in Spotted Lanternfly, Lycorma delicatula
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Ovary Dissection Is a Sensitive Measure of Sterility in Anopheles gambiae Exposed to the Insect Growth Regulator Pyriproxyfen

1
Department of Disease Control, London School of Hygiene & Tropical Medicine, London WC1E 7HT, UK
2
Department of Environmental and Occupational Health, School of Public Health, University of Nevada, Las Vegas, NV 89557, USA
3
PMI VectorLink Project, Abt Associates, 6130 Executive Blvd., Rockville, MD 20852, USA
4
Pan African Malaria Vector Research Consortium, Kilimanjaro Christian Medical University College, Moshi P.O. Box 2240, Tanzania
5
National Institute for Medical Research Tanzania, Mwanza P.O. Box 9653, Tanzania
*
Author to whom correspondence should be addressed.
Insects 2023, 14(6), 552; https://doi.org/10.3390/insects14060552
Submission received: 18 May 2023 / Revised: 5 June 2023 / Accepted: 8 June 2023 / Published: 14 June 2023
(This article belongs to the Section Insect Pest and Vector Management)

Abstract

:

Simple Summary

Malaria is transmitted by Anopheles mosquitoes. An important control tool against malaria is insecticidal bed nets containing a pyrethroid insecticide and another active ingredient, such as pyriproxyfen (PPF), which sterilizes adult mosquitoes by disrupting egg development. To evaluate the impact of PPF-treated bed nets, there are two potential techniques available to measure mosquito sterility: (1) observing egg-laying rates and (2) ovarial dissection. Our aim was to show which technique was most effective by exposing adult mosquitoes to PPF-treated or untreated nets and comparing sterility outcomes. When identifying the true percentage of mosquitoes sterilized from PPF exposure (i.e., true positives), both techniques showed similar sensitivity (egg-laying: 99% vs. dissection: 100%). For identifying non-exposed mosquitoes in the untreated group (i.e., true negatives), dissection had a higher specificity than egg-laying (53% vs. 19%). To show that ovary dissection remains effective under different conditions, we exposed mosquitoes to insecticides using two different exposure methods, including nets treated with pyrethroids, and blinded the investigator to reduce bias when scoring. The investigator predicted the exposure status of dissected females across all conditions with >90% accuracy. We conclude that ovary dissection is a more sensitive and specific tool for measuring sterility in Anopheles mosquitoes exposed to PPF.

Abstract

Pyriproxyfen (PPF) is an insect growth regulator used in the co-treatment of long-lasting insecticidal nets for its ability to sterilize female mosquitoes. To evaluate the efficacy of PPF-treated nets on mosquito reproductivity, most studies observe oviposition (egg-laying) rates in the laboratory. This technique has several technical disadvantages. Our study assessed if ovarial dissection could serve as an effective proxy for evaluating sterility in Anopheles gambiae mosquitoes. Blood-fed females were exposed to untreated or PPF-treated nets in cylinder assays and followed over several days to observe oviposition rates or egg development by dissection. For identifying PPF-exposed mosquitoes, both techniques demonstrated high sensitivity (oviposition: 99.1%; dissection: 100.0%), but for identifying non-exposed mosquitoes, specificity was significantly higher in the dissection group (52.5% vs. 18.9%). To assess whether dissection could be applied to nets treated with a pyrethroid or co-treated with a pyrethroid and PPF in tunnel tests, a blinded investigator performed dissections to predict the PPF exposure status across different treatment groups. The exposure status of dissected females was predicted with >90% accuracy. We report that dissection is a sensitive technique to assess sterility in female Anopheles gambiae mosquitoes and can be used as a predictor of PPF exposure.

1. Introduction

Malaria is transmitted to humans by the bites of female Anopheles mosquitoes. In the reproductive cycle of mosquitoes, the females must blood-feed to obtain the necessary nutrients for egg development. For this reason, long-lasting insecticidal nets (LLINs), which intercept host-seeking mosquitoes, are a successful control tool against malaria. LLINs were responsible for considerable reductions in malaria transmission until their success was threatened by widespread insecticide resistance in Anopheles (An.) mosquitoes [1]. To counteract pyrethroid resistance, novel vector control tools, particularly LLINs, containing two active ingredients (AIs) have been developed. The insect growth regulator pyriproxyfen (PPF) is a strong candidate for dual-AI LLINs because it is a potent sterilant: three minutes of tarsal exposure to PPF can induce lifelong sterilization in adult female mosquitoes [2]. The principle behind LLINs co-treated with PPF and a pyrethroid in resistance management is that the pyrethroid component may kill or knock down susceptible mosquitoes, while PPF sterilizes any pyrethroid-resistant survivors, preventing these individuals from producing viable offspring [3]. Two commercially manufactured dual-AI LLINS are Olyset® Duo and Royal Guard®, impregnated with a combination of PPF and permethrin or alpha-cypermethrin, respectively.
In 2022, the WHO published a guideline to assess the sterilizing effect of pyriproxyfen in the framework of insecticide resistance management [4], but there are no specific WHO guidelines for evaluating the bio-efficacy of LLINs containing growth regulators [5]. Most laboratory studies have evaluated the impact of PPF on adult sterility by measuring egg development (fecundity) and egg hatching (fertility) outcomes [2,6,7,8,9,10,11]. This is typically performed by exposing adult female mosquitoes to PPF before or after a bloodmeal, providing them with an egg-laying (oviposition) substrate, and recording the number of eggs laid and the proportion of emerged larvae, pupae, and F1 adults. These measurements provide estimates of reductions in fecundity, fertility, and reproductive rates caused by PPF exposure. Unfortunately, a disadvantage often observed with this method is that wild-caught females do not readily oviposit under the artificial confines of the laboratory, resulting in low oviposition rates and an underestimated impact of PPF exposure [12]. Additionally, this method requires suitable equipment and rearing facilities with potentially long follow-up times [13].
Dissecting gravid females to measure egg development may be an alternate technique to evaluate PPF-induced sterility. Because PPF causes arrested development in eggs, this effect can be observed microscopically. Evidently, studies have shown clear differences in the development of An. gambiae eggs after exposure to PPF [2,14]. The aims of our study were: (1) to evaluate if ovarial dissection can be used to measure sterility outcomes (as proxies for oviposition studies) in female An. gambiae exposed to PPF and pyrethroid LLINs, and (2) to test whether dissection can be used to predict the exposure status (i.e., PPF-exposed or non-exposed) of female An. gambiae mosquitoes.

2. Materials and Methods

2.1. Mosquitoes

Two pyrethroid-resistant Anopheles gambiae sensu stricto (s.s.) strains were used in this study: the RSP strain with the knockdown resistance mutation VGSC-1014S, and the Muleba-Kis strain with established VGSC-1014S and metabolic resistance mediated by overexpression of Cyp6 cytochrome p450 enzymes. Mosquitoes were obtained from the insectary at the Pan African Malaria Vector Research Consortium (PAMVERC) in Moshi, Tanzania.
For blood-feeding mosquitoes, guinea pigs were sedated using xylazine and placed over a mesh cage to allow mosquitoes to probe through and take blood from the underside of the guinea pig. The mosquitoes were allowed to feed for 1 h either the same morning or the evening prior to insecticide exposure in cylinder assays. Mosquitoes were sugar-starved for at least 1 h before blood-feeding.

2.2. Test LLINs

The following LLINs obtained from Sumitomo Chemical Co. Ltd., Tokyo, Japan were tested in cylinder assays and tunnel tests:
(1)
Olyset® Duo (2% permethrin and 1% PPF), unwashed;
(2)
Olyset® Duo (2% permethrin and 1% PPF) with 20 washes (20W);
(3)
Olyset® Net (2% permethrin), unwashed;
(4)
Long-lasting PPF Net (1% PPF), unwashed;
(5)
Long-lasting PPF Net (1% PPF), 20W.
An Untreated net was included as a control (polyester net, A to Z Textile Mills Ltd., Arusha, Tanzania). Olyset® Duo 20W and PPF Net 20W were washed following WHO guidelines [5].

2.3. Bioassay Procedures

Cylinder assays were conducted using the An. gambiae s.s. RSP and Muleba-Kis strains. Net pieces (12 cm × 15 cm) were stapled onto a blank piece of paper and inserted into the exposure tubes. A total of 10 to 20 recently blood-fed 2–5 day old females were aspirated into holding tubes and left to acclimatize for 1 h, then transferred to exposure tubes for 3 min. Following exposure, mosquitoes from each replicate were placed into individual cups and provided with 10% glucose solution on cotton pads ad libitum. Mortality was recorded at 24, 48, and 72 h.
Tunnel tests were carried out using the An. gambiae s.s. Muleba-Kis strain, following standard WHO guidelines [5]. The tunnels consisted of a square glass cylinder (60 cm × 25 cm × 25 cm) divided into two sections with a framed net piece held between each section. The net piece was deliberately holed for mosquitoes to pass through. A guinea pig was placed behind the net piece and mosquitoes were released into the opposite section. Mosquitoes were able to fly freely toward the guinea pig and cross the net piece for a bloodmeal. Between 30 and 50 unfed 3–6 day old female An. gambiae s.s. Muleba-Kis mosquitoes were released into the tunnels and given 14 h of overnight exposure. Live blood-fed females were collected and provided with 10% glucose solution on cotton pads ad libitum. Mortality was recorded at 24 and 72 h.

2.4. Oviposition Method

Following LLIN exposure in the cylinder assay, an equal number of individual mosquitoes per treatment group were randomly allocated to paper cups containing either an oviposition substrate (oviposition group) or nothing (dissection group).
Two types of oviposition substrates were tested in the paper cups: (1) a single piece of cotton placed at the bottom of a paper cup was soaked in distilled water and overlaid with a piece of filter paper (for females to oviposit directly on the filter paper); or (2) a piece of filter paper was spread along the bottom edges of the cup and distilled water was added to fill one-third of the cup (for females to oviposit directly on the water). The proportion of ovipositing females did not significantly differ between the two oviposition substrates in terms of the proportion of females that laid eggs (odds ratio; OR: 0.76, p = 0.481); therefore, both methods were used and combined for analysis.
The cups were checked for eggs every 24 h for up to 6 days. If eggs were found, the filter paper was removed, and the eggs were counted twice under a stereoscopic microscope. Any eggs that remained on the side of the cup were counted twice under the naked eye with a magnifying glass. Females that laid eggs were transferred to a new cup containing an oviposition substrate to provide the opportunity to lay additional eggs. After counting, eggs were washed in distilled water and transferred to small bowls or returned to the original cup, and thereafter kept in an incubator maintained at 27 °C. Larvae were checked daily and counted twice using pipettes. Eggs and larvae were recounted a third time if there was a difference greater than 5% between the two counts.

2.5. Dissection Procedure

Female mosquitoes were dissected 72 h after insecticide exposure and 76–88 h after blood-feeding (Figure 1). Preliminary tests were conducted to determine the timing of dissection in relation to the bloodmeal: it was observed that the stages of egg development were more distinctive at 72 h post-exposure than at 48 h between PPF-exposed and unexposed females. Therefore, a three-day minimum interval between blood-feeding and dissection was deemed optimal to allow sufficient time for bloodmeal assimilation and egg development. For females in the oviposition group, survivors were dissected 144 h post-exposure to allow sufficient time to lay eggs.
Prior to dissection, females were killed using ethyl acetate exposure for 10 min. Shortly after mosquito death, each female was placed on a glass slide along with a drop of 1X phosphate-buffered saline (PBS) solution under a stereoscopic microscope. Two needles were used to make an incision between the sixth and seventh tarsal segments of the abdomen to extract the eggs into PBS solution. Each female was scored based on the stage of egg development (Christophers’ stages) from I to V [15]. The number of eggs per female were counted twice, and a recount was performed if there was at least a 5% disparity between the two counts. The absence of eggs was also recorded, as some females were blood-fed but did not produce eggs.
After insecticide exposure, females were sorted randomly into individual cups with unique labels placed at the bottom of the cup. The investigator was blinded during dissections to reduce bias when scoring fecundity outcomes between treatment groups.

2.6. Outcomes

The fecundity outcomes calculated in the oviposition and dissection groups are described in Table 1. Females classified as “fecund” are those which laid eggs in the oviposition group or were observed to carry mature (stage V) eggs when dissected at 72 h. Those classified as “sterile” did not oviposit in the oviposition group or carried underdeveloped eggs (stages I–IV) in the dissection group.

2.7. Data Analysis

Induced sterility, reduction in fecundity, and reduction in reproductive rate were calculated using the equation below. Induced sterility was calculated using the proportion (%) of fecund females, reduction in fecundity used mean mature eggs (n) per female, and reduction in reproductive rate used mean larvae (n) per female.
I n h i b i t i o n = 1 t r e a t e d g r o u p u n t r e a t e d g r o u p × 100
To assess the sensitivity and specificity of the oviposition and dissection methods, females were grouped into two categories: sterile or fecund. Females were considered fecund if they laid eggs (oviposition group) or had stage V (mature/developed) eggs when dissected (dissection group). Females were considered sterile if they did not lay eggs (oviposition group) or had stages I–IV (underdeveloped) eggs when dissected (dissection group). Sensitivity (true ‘sterile’ rate), specificity (true ‘fecund’ rate), positive predictive value (PPV; rate of ‘sterile’ results that are true ‘sterile’), negative predictive value (NPV; rate of ‘fecund’ results that are true ‘fecund’) were calculated using the equations in Table 2.
For analysis of blinded dissections, a correct prediction (%) was calculated as the proportion of true positives (sensitivity) predicted correctly in each treatment group. For Untreated net and Olyset® net, this was calculated as the number of female mosquitoes predicted to be unexposed or fecund over the total number of females dissected in that treatment group. For the treatment groups exposed to PPF Net, PPF Net 20W, or Olyset® Duo 20W, this was calculated as the number of female mosquitoes predicted to be PPF-exposed or sterile over the total number of females dissected in that treatment group.
Statistical analyses were performed using STATA 14 (StataCorp, College Station, TX, USA). Comparisons of proportions were analyzed using logistic regression reporting the odds ratio (OR), and comparisons of count data were performed with Poisson regression, reporting the incidence rate ratio (IRR). Linear regression was used to compare means. Graphs were made using GraphPad Prism 9 (GraphPad Software, Boston, MA, USA).

3. Results

3.1. Comparison of Oviposition and Dissection Methods

A total of 707 blood-fed An. gambiae RSP females were exposed to LLINs in cylinder assays. In both the dissection and oviposition groups, female survival at 72 h post-exposure was 69.8% (95% CI: 65.1–74.2; n = 388) in the untreated arm and 57.6% (95% CI: 51.9–63.1; n = 302) in the PPF arm. At 144 h post-exposure in the oviposition group, survival was 40.2% (95% CI: 34.3–46.5; n = 246) in the untreated arm and lower (22.2%, 95% CI: 16.7–28.8; n = 185) in the PPF arm. Female survival was significantly different between the two treatment arms at both 72 h (OR: 0.59, p = 0.001) and 144 h (OR: 0.42, p < 0.001) post-exposure.
A total of 134 blood-fed females were dissected at 72 h post-exposure, and 278 females were followed up for egg-laying in the oviposition group up to 144 h post-exposure. The dissection method classified a greater proportion of untreated females as fecund than the oviposition method (71.2% (95% CI: 58.0–81.5) and 17.0% (95% CI: 12.2–23.3), respectively), indicating that not all bloodmeals led to a batch of eggs, and that the cues for oviposition were suboptimal under laboratory conditions (Table 3). Induced sterility comparing the PPF arm to the untreated arm was 100.0% in the dissection group and 96.8% in the oviposition group. The fecundity (mean mature eggs per female) of the untreated females was higher in the dissection group than in the oviposition group (29.9 and 10.8 eggs, respectively) because all eggs were accounted for in the dissection group but not all females laid eggs in the oviposition group. Reduction in fecundity was higher in PPF-exposed mosquitoes in the dissection group (100.0%, (0/29.9)) compared to the oviposition group (86.1%, (1.5/10.8)); this must have been by chance as both oviposition and dissection groups were exposed to PPF the same way.
Eggs laid by females in the oviposition group were followed up to measure fertility outcomes. In the untreated group, females laid eggs an average of 3.5 days following insecticide exposure, with a range of 3 to 6 days. One female in the PPF group laid eggs at 4 days post-exposure. Ovipositing females produced a total of 1023 larvae in the control group but only 17 larvae in the PPF group. The fertility (egg hatch rate) was 50.0% (95% CI: 34.4–65.7) and 22.8% in the untreated and PPF groups, respectively. This results in a reduction in fertility induced by PPF of only 54%. The reproductive rate (mean larvae per female) was 33 (95% CI: 20–46) and 17 in the untreated and PPF groups, respectively. The reduction in reproductive rate was 48.5%. By contrast, the PPF-induced sterility was 96.8–100%, making a larger contribution than fertility. This is most likely because PPF-induced sterility is due to reductions in fecundity and not fertility.
To confirm the development stage of the retained eggs in the non-ovipositing females, all females in the oviposition group that survived up to six days post-exposure were dissected (n = 135). The majority of the untreated (68.4%, n = 95) and PPF-exposed females (82.5%, n = 40) had retained eggs. Most eggs belonging to the untreated females (78.5%, 95% CI: 66.5–87.0; n = 65) were mature, whilst 100.0% of PPF-exposed females (n = 32) had underdeveloped eggs.

3.2. Sensitivity & Specificity Analysis

The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of each method (Figure 2) was calculated based on the number of sterile or fecund females (Table S1). The oviposition method correctly identified almost all sterilized females exposed to PPF (99.1% sensitivity) and most females classified as fecund with this method were truly unexposed to PPF (96.9% NPV). However, based on the exposure, this method underestimated the number of unexposed females that should be fecund (18.9% specificity), as these mosquitoes failed to produce eggs after blood-feeding, and overestimated the proportion of females classified as sterile (45.9% PPV). The dissection method had 100.0% sensitivity and NPV, as all PPF-exposed females with underdeveloped eggs were classified as sterile and all unexposed females with mature eggs as fecund. The dissection technique underestimated approximately half the true number of unexposed females (52.5% specificity) but had more than two-fold the specificity as the oviposition method. The dissection method overestimated the true proportion of sterile females but had a higher positive predictive value (58.7%) than the oviposition method.

3.3. Dissections after PPF Exposure in Cylinder Assays and Tunnel Tests

A total of 768 blood-fed Muleba-Kis mosquitoes were exposed in 3 min cylinder bioassays to untreated net, PPF net, Olyset® net, Olyset® Duo, and Olyset® Duo 20W (Table 4). There were no survivors following Olyset® Duo exposure in cylinder assays (n = 95). Mortality after 24 h in cylinder assays ranged between 5.2% (95% CI: 2.7–9.7) in the untreated group and 11.6% (95% CI: 7.8–17.1) in the PPF group. There was no evidence for delayed mortality between 24 h and 72 h of holding.
In overnight tunnel tests, a total of 730 unfed Muleba-Kis mosquitoes were exposed to the same treatments, including a PPF 20W Net. Of those exposed, 60.5% (n = 205), 67.3% (n = 196), 100.0% (n = 50), 23.2% (n = 164), and 29.6% (n = 115) had bloodmeals in tunnels containing untreated net, PPF net, PPF 20W, Olyset® net, and Olyset® Duo 20W, respectively. Mortality after 24 h ranged from 8.8% (95% CI: 2.8–24.4) in the Olyset® Duo 20W group to 18.2% (95% CI: 9.3–32.6) in the PPF 20W group. There was no evidence for delayed mortality between 24 h and 72 h after the tunnel test.
The majority (81.0–96.4%) of dissected females from both types of bioassays presented with eggs, while the rest did not undergo oogenesis. The proportion of females with mature eggs was the highest in the untreated net and Olyset® net groups with no significant difference in both cylinder assays (OR: 1.02, 95% CI: 0.45–2.33; p = 0.963) or tunnel tests (OR: 3.14, 95% CI: 0.38–26.12; p = 0.291). On the other hand, PPF net exposure resulted in a significantly lower proportion of females with mature eggs compared to the untreated group from cylinder assays (OR: 0.002, 95% CI: 0.0005–0.01; p < 0.001) and tunnel tests (OR: 0.01, 95% CI: 0.004–0.03; p < 0.001). In the Olyset® Duo 20W group, the proportion of females with mature eggs was also significantly lower in cylinder assays (OR: 0.29, 95% CI: 0.13–0.61; p = 0.001) and tunnel tests (OR: 0.07, 95% CI: 0.02–0.21; p < 0.001). No females exposed to PPF 20W in tunnel tests developed mature eggs.
Induced sterility (a proxy term for oviposition inhibition) was 97.9% in females exposed to PPF Net and 20.4% in those exposed to Olyset® Duo 20W in cylinder assays. In tunnel tests, induced sterility was 91.6%, 100.0%, and 60.7% in females exposed to PPF net, PPF 20W, and Olyset® Duo 20W, respectively. There was no induced sterility in females exposed to Olyset® net in either cylinder assays or tunnel tests.
In cylinder assays, the mean number of mature eggs did not differ between the untreated and Olyset® net groups (IRR: 1.02, p = 0.027; 95% CI: 1.0–1.03) but differed significantly between other treatment groups in reference to the control group: untreated net vs. PPF net (IRR: 0.44, p < 0.001; 95% CI: 0.35–0.56) and untreated net vs. Olyset® Duo 20W (IRR: 1.04, p < 0.001; 95% CI: 1.03–1.05). In tunnel tests, the mean number of mature eggs was significantly different across all treatment groups in reference to the control group: untreated net vs. Olyset® net (IRR: 1.05, p < 0.001; 95% CI: 1.02–1.08), untreated net vs. PPF net (IRR: 1.22, p < 0.001; 95% CI: 1.13–1.32), and untreated net vs. Olyset® Duo 20W (IRR: 0.96, p = 0.002; 95% CI: 0.93–0.99).
Fecundity (mean mature eggs per female) from cylinder assays was 57.2, 0.6, 64.0, and 55.9 for the untreated, PPF net, Olyset® net and Olyset® Duo 20W groups, respectively. Reduction in fecundity was 99.0% in females exposed to PPF net and 2.3% for those exposed to Olyset® Duo 20W in cylinders. In tunnel tests, fecundity was 59.9, 7.1, 0.0, 82.6, and 21.6 in the untreated net, PPF net, PPF 20W, Olyset® net and Olyset® Duo 20W groups, respectively. Reduction in fecundity was 88.1%, 100.0%, and 63.9% in females exposed to PPF net, PPF 20W, and Olyset® Duo 20W, respectively. There was no reduction in fecundity for females exposed to Olyset® net from either bioassay.

3.4. Blinded Dissections

To test the applicability of ovary dissection in practice, gravid females were dissected and a prediction of PPF exposure status (i.e., exposed to PPF or unexposed) was made by a blinded investigator based on the stages of egg development. Females with only mature eggs were predicted to be fecund or unexposed, and those with underdeveloped eggs were predicted to be sterile or PPF-exposed (Table S2). Females containing eggs with both mature and underdeveloped eggs (mixed stages) were excluded from the predictions.
Based on the stage of egg development, 93.8%, 97.1%, 93.6%, and 18.9% of females exposed in cylinders to untreated net, PPF net, Olyset® net and Olyset® Duo 20W, respectively, had the correct exposure status predicted (Figure 3). In tunnel tests, correct predictions were 91.9%, 90.1%, and 95.8% for females exposed to untreated net, PPF net and Olyset® net, respectively. All dissected females exposed to PPF 20W and Olyset® Duo 20W were predicted correctly.
In cylinder assays, 2.6%, 1.9%, 1.6%, and 11.9% of those exposed to untreated net, PPF net, Olyset® net and Olyset® Duo 20W, respectively, had mixed stages of eggs and were therefore not given a prediction. In tunnel tests, 1.3%, 3.2%, and 14.8% of females had mixed stages of eggs when exposed to untreated net, PPF net and Olyset® Duo 20W, respectively. There were no females with mixed stages when exposed to PPF 20W or Olyset® net in tunnels.

4. Discussion

In this paper, upon comparing dissection and oviposition to assess the efficacy of PPF on fecundity outcomes, dissection was an effective technique for observing sterility in An. gambiae s.s. exposed to PPF. Two key fecundity outcomes (induced sterility and reduction in fecundity) were measured using oviposition and dissection, which produced similar results due to the strong sterilizing impact of PPF and the high sensitivities (oviposition: 99.1%; dissection: 100.0%) of both techniques. However, the proportion of fecund females in the untreated group was significantly higher when measured by dissection versus oviposition (71.2% vs. 17.0%), owing to the low oviposition rates of the mosquito colony. It was therefore more effective to directly measure fecundity by dissection.
Both techniques showed low specificity (oviposition: 18.9%; dissection: 52.5%). This is in part due to some untreated females being falsely classified as sterile (i.e., false positives) because they had underdeveloped eggs when dissected (oviposition: 31.6%; dissection: 28.4%). Possible reasons for this phenomenon are discussed in the Appendix (Appendix A). In the oviposition group, only a small proportion of untreated females laid eggs, despite the majority (68.4%) presenting with mature eggs when dissected six days post-exposure. Thus, dissection showed greater specificity, owing again to the low oviposition rates of the colony.
Next, we dissected An. gambiae females exposed to dual-AI nets impregnated with PPF and a pyrethroid. In cylinder assays, mosquitoes were exposed to net pieces for 3 min, and in tunnel tests, the exposure times were ad libitum. The holed net pieces create a natural barrier that host-seeking mosquitoes must cross to feed on a host (i.e., guinea pig), providing more realistic exposure to the AIs as they would encounter in real-life settings. In both assays, the PPF unwashed and 20W groups had the highest reductions in fecundity compared to all other treatment groups, as most eggs from PPF-exposed females arrested at an early stage of development (typically, Christophers’ stage III). In the Olyset® Duo 20W group in both assays, the proportion of fecund females was higher than the PPF-exposed groups, but significantly lower than the untreated group, suggesting that there was still enough PPF to sterilize some mosquitoes, but not all. The proportion of fecund females did not differ between the untreated and Olyset® net groups, indicating that permethrin had no impact on egg development. Similarly, another study found that eggs were 100% abnormal in An. gambiae females exposed to PPF net and 60% abnormal for those exposed to Olyset® Duo, and they observed 0% abnormalities in those exposed to untreated net or Olyset® net [14]. Moreover, the dissections were performed in this study by a blinded investigator to reduce bias when scoring sterility between treatment groups and test if predictions of prior PPF exposure could be made accurately. Correct predictions ranged from 90 to 100% across all treatment groups, confirming the high sensitivity of dissection.
PPF inhibits the action of juvenile hormone secreted by the corpora allata after a bloodmeal [2,16,17,18,19]. Eggs that would have normally reached follicular maturation are arrested at an early stage of development, resulting in underdeveloped eggs and sterility. PPF is dose-dependent and observed to be most effective in An. gambiae mosquitoes when encountered within a 24 h period before or after a bloodmeal [2,6,9]. Regardless of whether the female is parous or nulliparous, after a single exposure to PPF, oviposition inhibition may be lifelong [2,6,8]. In this study, the stages of egg development were clear between PPF-exposed (Christophers’ stage III) and unexposed (stage V) females dissected 72 h post-exposure (see Supplementary Figure S1). Thus, dissecting gravid females is a feasible and practical technique to measure sterility.
In comparison to measuring oviposition rates, dissection would be ideal to measure PPF-induced sterility in wild mosquitoes exposed to PPF LLINs in field or semi-field studies, such as phase II experimental hut trials or phase III cluster randomized controlled trials (RCTs) [5]. Dissection is more operationally feasible and timely, as sterility can be rapidly assessed by observing the stages of egg development and it bypasses the need for wild mosquitoes to oviposit in the laboratory. The artificial conditions of the laboratory, the removal of the need to fly to search for breeding sites, and insufficient oviposition stimuli may inhibit the mosquito’s natural urge to oviposit [20]. In some studies, An. gambiae colonies adapted to insectary conditions produced oviposition rates as low as 25% (Kisumu strain), 34% (VKPER strain), 49% (RSP strain), and 0–100% (Dieldrin Dlr-s strain) [2,6,7,21]. Overall, dissection as a substitute for oviposition would circumvent the need for forced oviposition under laboratory conditions and shorten the follow-up time to when females are expected to have fully developed eggs, thus minimizing the time, resources, and labor that are required.

5. Conclusions

This study demonstrated that clear morphological differences could be observed between the eggs of PPF-exposed and unexposed or permethrin-exposed blood-fed An. gambiae s.s. when dissected 3 days post-exposure. During blinded dissections, the correct exposure group was predicted with >90% accuracy. We hereby demonstrated that ovary dissection is a sensitive tool to measure sterility in An. gambiae females exposed to PPF-treated nets.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/insects14060552/s1. Table S1. Sterile and fecund An. gambiae RSP females by method and treatment group with the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Table S2. Prediction of PPF exposure to An. gambiae females during blinded dissections. Figure S1. Image taken from stereoscopic microscope comparing dissected eggs at Christophers’ stage III (underdeveloped) and stage V (developed).

Author Contributions

Conceptualization, N.P., F.W.M. and M.R.; methodology, N.P., M.R. and M.K.; software, A.S.; formal analysis, A.S.; investigation, A.S.; resources, F.W.M. and M.K.; data curation, A.S.; writing—original draft preparation, A.S.; writing—review and editing, L.A.M., M.R., F.W.M., A.M., and N.P. supervision, M.K; validation: A.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Joint Global Health Trials scheme (UK Foreign, Commonwealth and Development Office; UK Medical Research Council; Wellcome Trust, UK; UK Department of Health and Social Care).

Institutional Review Board Statement

Ethical approval for the study was obtained from the London School of Hygiene and Tropical Medicine (LSHTM; ref 6551), the LSHTM MSc Research Ethics Committee (ref: 15483), and the National Institute for Medical Research (NIMR) in Tanzania (Vol.IX/1803; Vol.IX/2743). Approval from the Animal Welfare and Ethical Review Board of LSHTM (ref. no. 2019-14) for use of animals for mosquito maintenance was obtained. Study procedures and reporting are in accordance with the Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines.

Data Availability Statement

The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Acknowledgments

Special thanks to Janneke Snetselaar for her support and to Agnes Benson for her invaluable assistance on this project. We also thank the editors and reviewers for their time.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

There are several possible explanations for why some untreated females had underdeveloped eggs. Firstly, the bloodmeal quantity may have been insufficient for complete egg development, as some Anopheles mosquitoes require multiple bloodmeals to overcome the limitations of a small midgut volume [22,23]. Egg development is directly related to bloodmeal quantity, and a first bloodmeal may initiate oogenesis from stage I to the pre-vitellogenic resting stage II, but a second may be required to propel oogenesis further [24,25]. Secondly, the resources from the bloodmeal may have been directed toward self-development and/or the female may not have been inseminated. Smaller An. gambiae females tend to produce fewer eggs and are more likely to blood-feed as virgins [16]. Insufficient metabolic reserves from the larval stage may have caused nutritional resources from the bloodmeal to reallocate from oogenesis to other energy-demanding processes, such as flight [16]. Thirdly, there may have been insufficient bloodmeal uptake due to excess sugar-feeding. Females in this study were given at least 1 h of sugar-fasting before blood-feeding, which may not have been enough time to fully digest a sugar meal. In that case, an undigested sugar meal may have limited their capacity to take a bloodmeal with enough protein for lipid synthesis in oocytes [25]. Finally, there may have been PPF contamination in the laboratory where this study was conducted. We tested this theory by dissecting a small subset of blood-fed An. gambiae s.s. females kept in a separate testing room where PPF was not used. They were not subjected to any bioassay and kept separately from any environment or surface, which may be potentially contaminated with PPF. Upon dissection of these mosquitoes, the majority (76.5%, n = 13/17) had mature eggs, 17.6% (n = 3/17) had underdeveloped eggs, and 23.5% (n = 4/17) had no eggs similar to what was observed in this study. Therefore, we exclude PPF contamination as a reason for blood-fed females presenting with underdeveloped eggs.

References

  1. WHO. World Malaria Report 2017; WHO: Geneva, Switzerland, 2017. [Google Scholar]
  2. Koama, B.; Namountougou, M.; Sanou, R.; Ndo, S.; Ouattara, A.; Dabiré, R.K.; Malone, D.; Diabaté, A. The sterilizing effect of pyriproxyfen on the malaria vector Anopheles gambiae: Physiological impact on ovaries development. Malar. J. 2015, 14, 101. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. White, M.T.; Lwetoijera, D.; Marshall, J.; Caron-Lormier, G.; Bohan, D.A.; Denholm, I.; Devine, G.J. Negative cross resistance mediated by co-treated bed nets: A potential means of restoring pyrethroid-susceptibility to malaria vectors. PLoS ONE 2014, 9, e95640. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. World Health Organization. Standard Operating Procedure for Evaluating the Sterilizing Properties of Pyriproxyfen in Adult Female Mosquitoes in WHO Bottle Bioassays; World Health Organization: Geneva, Switzerland, 2022. [Google Scholar]
  5. WHO. Guidelines for Laboratory and Field-Testing of Long-Lasting Insecticidal Nets, 2013th ed; World Health Organization Press: Geneva, Switzerland, 2013. [Google Scholar]
  6. Jaffer, A.; Protopopoff, N.; Mosha, F.W.; Malone, D.; Rowland, M.W.; Oxborough, R.M. Evaluating the sterilizing effect of pyriproxyfen treated mosquito nets against Anopheles gambiae at different blood-feeding intervals. Acta Trop. 2015, 150, 131–135. [Google Scholar] [CrossRef] [PubMed]
  7. Ngufor, C.; N’Guessan, R.; Fagbohoun, J.; Odjo, A.; Malone, D.; Akogbeto, M.; Rowland, M. Olyset Duo® (a pyriproxyfen and permethrin mixture net): An experimental hut trial against pyre-throid resistant Anopheles gambiae and Culex quinquefasciatus in southern Benin. PLoS ONE 2014, 9, e93603. [Google Scholar] [CrossRef] [PubMed]
  8. Ohashi, K.; Nakada, K.; Ishiwatari, T.; Miyaguchi, J.I.; Shono, Y.; Lucas, J.R.; Mito, N. Efficacy of Pyriproxyfen-Treated Nets in Sterilizing and Shortening the Longevity of Anopheles gambiae (Diptera: Culicidae). J. Med. Entomol. 2012, 49, 1052–1058. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Harris, C.; Lwetoijera, D.W.; Dongus, S.; Matowo, N.S.; Lorenz, L.M.; Devine, G.J.; Majambere, S. Sterilising effects of pyriproxyfen on Anopheles arabiensis and its potential use in malaria control. Parasites Vectors 2013, 6, 144–148. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Grisales, N.; Lees, R.S.; Maas, J.; Morgan, J.C.; Wangrawa, D.W.; Guelbeogo, W.M.; N’fale, S.; Lindsay, S.W.; McCall, P.J.; Ranson, H. Pyriproxyfen-treated bed nets reduce reproductive fitness and longevity of pyrethroid-resistant Anopheles gambiae under laboratory and field conditions. Malar. J. 2021, 20, 273. [Google Scholar] [CrossRef] [PubMed]
  11. Toé, K.H.; Mechan, F.; Tangena, J.-A.A.; Morris, M.; Solino, J.; Tchicaya, E.F.S.; Traoré, A.; Ismail, H.; Maas, J.; Lissenden, N.; et al. Assessing the impact of the addition of pyriproxyfen on the durability of permethrin-treated bed nets in Burkina Faso: A compound-randomized controlled trial. Malar. J. 2019, 18, 355. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Koffi, A.A. Efficacy of Olyset ® Duo, a permethrin and pyriproxyfen mixture net against wild pyrethroid-resistant Anopheles gambiae s.s. from Côte d’Ivoire: An experimental hut trial. Parasite 2015, 22, 28. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Kawada, H.; Dida, G.O.; Ohashi, K.; Kawashima, E.; Sonye, G.; Njenga, S.M.; Mwandawiro, C.; Minakawa, N. A Small-Scale Field Trial of Pyriproxyfen-Impregnated Bed Nets against Pyrethroid-Resistant Anopheles gambiae s.s. in Western Kenya. PLoS ONE 2014, 9, e111195. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Yunta, C.; Grisales, N.; Nász, S.; Hemmings, K.; Pignatelli, P.; Voice, M.; Ranson, H.; Paine, M.J. Pyriproxyfen is metabolized by P450s associated with pyrethroid resistance in An. gambiae. Insect Biochem. Mol. Biol. 2016, 78, 50–57. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. World Health Organization. Division of Malaria and other Parasitic Diseases. In Manual on Practical Entomology in Malaria; Part II Methods and Techniques; World Health Organization: Geneva, Switzerland, 1975. [Google Scholar]
  16. Baldini, F.; Gabrieli, P.; South, A.; Valim, C.; Mancini, F.; Catteruccia, F. The Interaction between a Sexually Trans-ferred Steroid Hormone and a Female Protein Regulates Oogenesis in the Malaria Mosquito Anopheles gambiae. PLoS Biol. 2013, 11, e1001695. [Google Scholar] [CrossRef] [PubMed]
  17. Gulia-Nuss, M.; Eum, J.-H.; Strand, M.R.; Brown, M.R. Ovary ecdysteroidogenic hormone activates egg maturation in the mosquito, Georgecraigius atropalpus, after adult eclosion or a blood meal. J. Exp. Biol. 2012, 215 Pt 21, 3758–3767. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Li, Y.; Hernández-Martínez, S.; Unnithan, G.C.; Feyereisen, R.; Noriega, F.G. Activity of the corpora allata of adult female Aedes aegypti: Effects of mating and feeding. Insect Biochem. Mol. Biol. 2003, 33, 1307–1315. [Google Scholar] [CrossRef] [PubMed]
  19. Detinova, T. On the Influence of Glands of internal Secretion upon the Ripening of the Gonads and the imaginai Diapause in Anopheles maculipennis. Zool. Zhurnal. 1945, 5, 291–298. [Google Scholar]
  20. Gillies, M.T. The duration of the gonotrophic cycle in Anopheles gambiae and Anopheles funestus, with a note on the efficiency of hand catching. East Afr. Med. J. 1953, 30, 129–135. [Google Scholar]
  21. Rowland, M. Behaviour and fitness of γHCH/dieldrin resistant and susceptible female Anopheles gambiae and An.stephensi mosquitoes in the absence of insecticide. Med. Vet. Entomol. 1991, 5, 193–206. [Google Scholar] [CrossRef]
  22. Fernandes, L.; Briegel, H. Reproductive physiology of Anopheles gambiae and Anopheles atroparvus. J. Vector Ecol. 2005, 30, 11–26. [Google Scholar] [PubMed]
  23. Takken, W.; Klowden, M.J.; Chambers, G.M. Effect of body size on host seeking and blood meal utilization in Anopheles gambiae sensu stricto (Diptera: Culicidae): The disadvantage of being small. J. Med. Èntomol. 1998, 35, 639–645. [Google Scholar] [CrossRef]
  24. Detinova, T. Age-Grouping Methods in Diptera of Medical Importance with Special Reference to Some Vectors of Malaria; World Health Organization: Geneva, Switzerland, 1962. [Google Scholar]
  25. Clements, A.N. The Biology of Mosquitoes: Development, Nutrition and Reproduction; Chapman & Hall: London, UK, 1992. [Google Scholar]
Figure 1. Schematic of bioassays, treatment groups, and timeline of mosquito blood-feeding, net exposure and dissection. The green treatments and arrows refer to the first experiment comparing the oviposition and dissection methods; the blue treatments and arrows refer to the remaining experiments comparing the bioassays and treatment groups. The red bracket indicates the period of blood-feeding prior to net exposure at 0 h. Mosquitoes in the dissection and oviposition groups were dissected at 72 h and 144 h post-exposure, respectively.
Figure 1. Schematic of bioassays, treatment groups, and timeline of mosquito blood-feeding, net exposure and dissection. The green treatments and arrows refer to the first experiment comparing the oviposition and dissection methods; the blue treatments and arrows refer to the remaining experiments comparing the bioassays and treatment groups. The red bracket indicates the period of blood-feeding prior to net exposure at 0 h. Mosquitoes in the dissection and oviposition groups were dissected at 72 h and 144 h post-exposure, respectively.
Insects 14 00552 g001
Figure 2. The percentage (%) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the oviposition and dissection methods.
Figure 2. The percentage (%) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the oviposition and dissection methods.
Insects 14 00552 g002
Figure 3. Correct prediction (%) of PPF exposure to An. gambiae females during blinded dissections in cylinder assays and tunnel tests.
Figure 3. Correct prediction (%) of PPF exposure to An. gambiae females during blinded dissections in cylinder assays and tunnel tests.
Insects 14 00552 g003
Table 1. Reproductive outcomes described for the oviposition and dissection methods. Outcomes are expressed as a mean number or as a proportion (%).
Table 1. Reproductive outcomes described for the oviposition and dissection methods. Outcomes are expressed as a mean number or as a proportion (%).
OutcomeOviposition GroupDissection Group
% Fecund femalesProportion of females that laid eggs over the total number of blood-fed females alive at 3 days post-exposure.Proportion of dissected females with mature eggs over the total number of blood-fed females dissected at 3 days post-exposure.
% Induced sterility (proxy for oviposition inhibition)Reduction in the proportion of egg-laying females caused by PPF exposure.Reduction in the proportion of dissected females with mature eggs caused by PPF exposure.
FecundityMean number of eggs laid per female, calculated as the total number of eggs laid divided by the total number of blood-fed females.Mean number of mature eggs per female, calculated as the total number of mature eggs divided by the total number of blood-fed females.
% Reduction in fecundityReduction in the proportion of eggs laid per female caused by PPF exposure.Reduction in the proportion of mature eggs dissected per female caused by PPF exposure.
Fertility Proportion of laid eggs that hatched into larvae.-
Reproductive rateMean number of larvae that hatched per blood-fed female alive at 3 days post-exposure.-
% Reduction in reproductive rateReduction in the proportion of larvae that hatched per female caused by PPF exposure.-
Table 2. Calculations for the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of each method based on PPF exposure status.
Table 2. Calculations for the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of each method based on PPF exposure status.
PPF GroupUntreated GroupCalculation
SterileABPPV = A/(A + B)
FecundCDNPV = D/(C + D)
Calculation Sensitivity = A/(A + C)Specificity = D/(B + D)
Table 3. Fecundity and fertility results of An. gambiae RSP females in the oviposition and dissection groups.
Table 3. Fecundity and fertility results of An. gambiae RSP females in the oviposition and dissection groups.
OvipositionDissection
UntreatedPPFUntreatedPPF
N Survivors at 3 days post-exposure1641148054
N Survivors at 6 days post-exposure9941--
N Dissected females with eggs--5936
N Fecund females311420
% Fecund females (95% CI)17.0 (12.2–23.3)0.7 (0.1–4.9)71.2 (58.0–81.5)0.0
% Induced sterility-96.8-100.0
N Underdeveloped (stages II–IV) eggs 0 (not laid)0 (not laid)9183510
N Developed (stage V) eggs17787523930.0
N Mean eggs per female10.81.541.465.0
Fecundity: N Mean mature eggs per female10.81.529.90.0
% Reduction in fecundity-86.1-100.0
Table 4. Mortality and reproductive outcomes An. gambiae s.s. Muleba-Kis females dissected 3 days post-exposure to untreated net, PPF net, PPF 20W, Olyset® net, and Olyset® Duo 20W in cylinder assays and/or tunnel tests.
Table 4. Mortality and reproductive outcomes An. gambiae s.s. Muleba-Kis females dissected 3 days post-exposure to untreated net, PPF net, PPF 20W, Olyset® net, and Olyset® Duo 20W in cylinder assays and/or tunnel tests.
Cylinder AssayTunnel Test
UntreatedPPFOlyset® NetOlyset® Duo 20WUntreatedPPFPPF 20WOlyset® NetOlyset® Duo 20W
N Exposed17418918112920519650164115
N Blood-fed (tunnel tests)----12413250 3834
% Mortality 24 h (95% CI)5.2 (2.7–9.7)11.6 (7.8–17.1)9.4 (5.9–14.6)9.3 (5.3–15.7)13.7 (8.7–21.0)9.4 (5.4–15.8)18.2 (9.3–32.6)8.6 (2.7–23.8)8.8 (2.8–24.4)
N Females dissected142113145939098282530
% Females with eggs, all stages (95% CI)81.0 (73.6–86.7)93.8 (87.5–97.0)89.0 (82.7–93.1)90.3 (82.4–94.9)83.3 (74.1–89.7)95.9 (89.5–98.5)96.4 (77.8–99.5)96.0 (75.5–99.5)86.7 (68.9–95.0)
N Females with mature eggs1052116606670239
% Females with mature eggs (95% CI)89.7 (82.7–94.1)1.9 * (0.5–7.3)89.9 (83.4–94.1)71.4 * (60.8–80.1)88.0 (78.4–93.7)7.4 * (3.6–14.9)0.0 *95.8 (74.7–99.4)34.6 * (18.8–54.8)
% Induced sterility -97.90.020.4-91.6100.00.060.7
N Eggs, all stages (95% CI)8422 (7658–9186)8399 (7965–8832)9705 (8897–10,512)6598 (5848–7347)5926 (5347–6505)7582 (7094–8070)1974 (1764–2183)2176 (1928–2424)1590 (1310–1869)
N Mature eggs dissected (n, 95% CI)8127 (7391–8863)68 (0–159)9285 (8511–10,059)5202 (4542–5862)5392 (4828–5956)699 (481–916)02065 (1818–2311)648 (421–875)
N Mean eggs dissected, all stages (SD)75.2 (36.7)81.5 (21.7)77.6 (36.8)78.5 (41.6)79.0 (34.0)80.7 (25.5)73.1 (20.5)90.7 (25.7)61.1 (27.8)
N Mean mature eggs dissected (SD)77.4 (36.5)34.0 * (32.5) 80.0 (36.5)86.7 * (43.3)81.7 (35.0)99.8 * (41.5)0.089.8 * (25.9)72.0 * (38.2)
Fecundity: N Mean mature eggs per female57.20.664.055.959.97.10.082.621.6
% Reduction in fecundity-99.00.02.3-88.1100.00.063.9
* Significantly different results compared to untreated net group by logistic regression or linear regression (p < 0.05).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Soto, A.; Rowland, M.; Messenger, L.A.; Kirby, M.; Mosha, F.W.; Manjurano, A.; Protopopoff, N. Ovary Dissection Is a Sensitive Measure of Sterility in Anopheles gambiae Exposed to the Insect Growth Regulator Pyriproxyfen. Insects 2023, 14, 552. https://doi.org/10.3390/insects14060552

AMA Style

Soto A, Rowland M, Messenger LA, Kirby M, Mosha FW, Manjurano A, Protopopoff N. Ovary Dissection Is a Sensitive Measure of Sterility in Anopheles gambiae Exposed to the Insect Growth Regulator Pyriproxyfen. Insects. 2023; 14(6):552. https://doi.org/10.3390/insects14060552

Chicago/Turabian Style

Soto, Alina, Mark Rowland, Louisa A. Messenger, Mathew Kirby, Franklin W. Mosha, Alphaxard Manjurano, and Natacha Protopopoff. 2023. "Ovary Dissection Is a Sensitive Measure of Sterility in Anopheles gambiae Exposed to the Insect Growth Regulator Pyriproxyfen" Insects 14, no. 6: 552. https://doi.org/10.3390/insects14060552

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop