Next Article in Journal
Diving Deep into Short-Term Electricity Load Forecasting: Comparative Analysis and a Novel Framework
Next Article in Special Issue
Mathematical and Statistical Analysis of Doubling Times to Investigate the Early Spread of Epidemics: Application to the COVID-19 Pandemic
Previous Article in Journal
Flipped-OCN Method in Mathematics Learning to Analyze the Attitudes of Pre-Service Teachers
Previous Article in Special Issue
On a Discrete SEIR Epidemic Model with Exposed Infectivity, Feedback Vaccination and Partial Delayed Re-Susceptibility
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Mathematical Model of Contact Tracing during the 2014–2016 West African Ebola Outbreak

by
Danielle Burton
1,*,
Suzanne Lenhart
2,
Christina J. Edholm
3,
Benjamin Levy
4,
Michael L. Washington
5,
Bradford R. Greening, Jr.
5,
K. A. Jane White
6,
Edward Lungu
7,
Obias Chimbola
7,
Moatlhodi Kgosimore
8,
Faraimunashe Chirove
9,
Marilyn Ronoh
10 and
M. Helen Machingauta
7
1
Department of Mathematics, University of Wisconsin, Madison, WI 53706, USA
2
Department of Mathematics, University of Tennessee, Knoxville, TN 37996, USA
3
Mathematics Department, Scripps College, Claremont, CA 91711, USA
4
Department of Mathematics, Fitchburg State University, Fitchburg, MA 01420, USA
5
Centers for Disease Control and Prevention, Atlanta, GA 30333, USA
6
Department of Mathematical Sciences, University of Bath, Bath BA2 7AY, UK
7
Department of Mathematical & Statistical Sciences, Botswana International University of Science & Technology, Palapye, Botswana
8
Department of Biometry & Mathematics, Botswana University of Agriculture and Natural Sciences, Gaborone, Botswana
9
Department of Applied Mathematics, University of Johannesburg, Johannesburg 2092, South Africa
10
School of Mathematics, University of Nairobi, Nairobi, Kenya
*
Author to whom correspondence should be addressed.
Mathematics 2021, 9(6), 608; https://doi.org/10.3390/math9060608
Submission received: 2 February 2021 / Revised: 4 March 2021 / Accepted: 8 March 2021 / Published: 12 March 2021
(This article belongs to the Special Issue Mathematical Biology: Developments in Epidemic and Endemic Models)

Abstract

:
The 2014–2016 West African outbreak of Ebola Virus Disease (EVD) was the largest and most deadly to date. Contact tracing, following up those who may have been infected through contact with an infected individual to prevent secondary spread, plays a vital role in controlling such outbreaks. Our aim in this work was to mechanistically represent the contact tracing process to illustrate potential areas of improvement in managing contact tracing efforts. We also explored the role contact tracing played in eventually ending the outbreak. We present a system of ordinary differential equations to model contact tracing in Sierra Leonne during the outbreak. Using data on cumulative cases and deaths, we estimate most of the parameters in our model. We include the novel features of counting the total number of people being traced and tying this directly to the number of tracers doing this work. Our work highlights the importance of incorporating changing behavior into one’s model as needed when indicated by the data and reported trends. Our results show that a larger contact tracing program would have reduced the death toll of the outbreak. Counting the total number of people being traced and including changes in behavior in our model led to better understanding of disease management.

1. Introduction

In March 2014, the most deadly outbreak to date of Ebola virus disease (EVD), a hemorrhagic fever, began in Guinea and rapidly spread to Liberia, Nigeria, Senegal, and Sierra Leone [1]. In October 2014, the World Health Organization (WHO) Ebola Response Team estimated an overall case fatality rate of 70.8 % and basic reproduction numbers ( 0 ) of 1.71 for Guinea, 1.83 for Libera and 1.38 for Sierra Leone [2]. Concern that Ebola might spread globally via airline travel led to recommendations for health assessments at airports in the affected countries [3]. A review and meta-analysis of 31 reports found that the main methods of spread were direct contact with an infected individual and contact with deceased loved ones during traditional funeral practices [4]. In the 2014–2016 outbreak in Sierra Leone, among individuals confirmed to have EVD, 47.9 % reported that they had had contact with someone suspected of having EVD, and 25.5 % reported having attended a funeral [5]. These transmission pathways are further indicated as important by mathematical models and by statistical models [6,7,8,9]. Ebola can survive on some surfaces for up to 192 h unless they are properly disinfected [10]. This might be one of the reasons why so many health care workers became infected [11]. Outcomes for individuals who contracted EVD during the outbreak varied based on location, time of infection, and whether the individual was hospitalized [12].
Contact tracing, sometimes called partner notification, is often used in the fight against the spread of HIV (Human Immunodeficiency Virus) [13,14,15]. Contact tracing for Ebola is quite different, though, because it does not focus primarily on sexual partners but rather on people who have been in some kind of close contact with the infected or deceased individual. The goal of contact tracing is to identify secondary infections and to isolate them in order to stop disease transmission. Throughout the outbreak, the Centers for Disease Control and Prevention’s Morbidity and Mortality Weekly Report detailed the progress of the disease as well as some information about contact tracing efforts. The ideal process for contact tracing is now described, though in some cases it was altered due to constraints of geography, resource limitations, or testing availability. Contacts were traced for 21 days after their last known exposure to a confirmed, probable, or suspected case [16]. All contacts being traced were instructed to remain isolated from the general population. If a contact showed symptoms of EVD, they were moved to a suspected case isolation ward and tested. If the test was positive, that individual was moved to the confirmed case ward. If the test was negative, the individual was sent home to be traced for another 21 days.
Webb and Browne and their collaborators built two models using data from Sierra Leone and Guinea [17,18]. In their SEIR (Susceptible–Exposed–Infectious–Recovered) model [17], they incorporated contact tracing by building separate compartments for Exposed individuals and Infectious individuals being traced. Their model did not include spread within hospitals and spread from contact with deceased individuals. They found that increasing the fraction of cases reported and increasing the fraction of reported contacts that were traced could bring 0 below 1. They also provided weekly point estimates for the effective reproduction number for Guinea and Sierra Leone. In [18], they had a system of ODEs and a corresponding stochastic model implementation, which included a compartment for improperly buried bodies of infectious individuals, but did not include a hospitalized compartment and did not include the workload of tracing persons who do not become infected. In this work, we will use a similar, but more mechanistic approach of counting persons being traced and accounting for the workload of the contact tracers. Our model will include compartments for hospitalized individuals and for dead bodies from improper burials.
Rivers et al. [19] built an SEIR model of the epidemic in Sierra Leone and Liberia while it was ongoing and before it had reached a peak. They concluded that improved contact tracing could have a large impact on number of cases but that even when combined with two other interventions contact tracing was insufficient to bring the epidemic to an end. They identified the duration of a traditional funeral in Sierra Leone as 4.5 days and the length of the incubation period as 10 days—values which we use in our model. Their work represented improved contact tracing implicitly by increasing the proportion of infected cases that are diagnosed and hospitalized and decreasing the time it takes for an infected individual to be hospitalized (from a baseline scenario), but our model will illustrate contact tracing more explicitly by counting the number of persons being traced. This counting will indicate the people resources needed for the tracing process, not just the effects of the tracing.
In Sierra Leone, contact tracing was hampered by practical difficulties [20]. Olu et al. analyzed contact tracing interview data in the western area districts of Sierra Leone [21], and noted that contact tracing was hindered by under-reporting of exposure, political difficulties in hiring tracers, and an incomplete database for use of tracers. Contacts being traced were supposed to be provided with basic needs, such as food and water, but this often did not occur. Some contacts were difficult to trace because of the stigma of being listed as a contact, and the average number of contacts per case was only 8.5 which was lower than in comparable situations. Olu et al. found that some people gave false information to tracers, withheld information from tracers, and communities tended not to trust tracers. This resulted in missed contacts. According to field staff (personal communication, Centers for Disease Control and Prevention) [22], there were difficulties in procuring additional people to perform contact tracing. In an urban area, a tracer could trace about 15 individuals per day, while in a rural area, a tracer could trace 10 individuals per day. In January 2015, there were 1200 contact tracers in Western Area, Sierra Leone. In neighboring Liberia, tracers faced difficulty finding contacts, difficulty with completing all 21 days of tracing, and resistance of symptomatic contacts to report to an Ebola Treatment Unit (ETU) [23]. Other challenges faced by contact tracers in Liberia included contacts hiding from tracers, people failing to identify all contacts or lying about their own exposure, resistance to in-home isolation, and difficulties in finding contact tracers. Many of the same problems were encountered in Sierra Leone. A study by Swanson et al. found that contact tracing in Liberia was performed for 26.7 % of cases and only identified 3.6 % of new cases [24], suggesting room for improvement. Chowell and Nishiura [25] illustrated the insights for disease management that can come from modeling connected with Ebola epidemiological data and discussed the need for understanding the effectiveness of contact tracing.
Our goal was to carefully and mechanistically represent the contact tracing process to illustrate potential areas of improvement in managing contact tracing efforts. We explored the role contact tracing played in eventually ending the outbreak. Our model uses a novel feature, which is explicitly counting the people being traced and connecting the total persons traced with the workload of contact tracer workers. We will focus our model on Sierra Leone, for which we have data from the Sierra Leone Ministry of Health [26,27]. These data include cumulative confirmed cases and cumulative confirmed deaths as reported online during the outbreak in the daily situation report. We will design a system of Ordinary Differential Equations (ODEs) explicitly incorporating contact tracing, fit this model to our data, and see what insights we might gain from this mechanistic approach.

2. Model

Our model is a compartmental model made up of a system of ODEs and follows an SEIR approach, similar to [17,18,19,21,28]. In addition to the Susceptible, Exposed, Infected, and Recovered classes, we also include a class (D) to account for the persons who have died from Ebola in the community (i.e., having not been effectively isolated in a hospital or by other means), because they are a significant source of infection due to traditional funeral practices, such as hugging and kissing the body of a deceased loved one. We also include a Hospitalized (H) class, in which individuals are assumed to be isolated and not contribute to infection, and if they die their bodies are assumed to be disposed of safely. We place no upper limit on the size of class H, which does not reflect the situation during the outbreak where insufficient beds and staffing were a major limiting factor in controlling the outbreak [29], but allows us to examine the operation of a contact tracing system assuming hospital resources are readily available.
Our investigation of contact tracing begins with adding two new classes of individuals being traced. Since exposure is a hidden trait, individuals being traced are either susceptible or exposed. We created a class called F of susceptible individuals who are being traced but will not become ill and a second class, E F , for individuals being traced who are exposed and will become infectious. Two events can lead to initiation of contact tracing: either an individual enters the D class or an individual enters the H class. The contacts connected to the individuals involved in either of these two events will be contacted each day for 21 days by a contact tracer. We assume that individuals in the F class being traced will follow isolation guidelines to prevent them from becoming exposed. Individuals in E F are moved to the hospital when they present symptoms. The function f alters the completion rate of key contact tracing steps based on the amount of work to be done along with the number of available contact tracing staff. There is a limited number of contact tracers, and each contact tracer is able to trace a limited number of individuals at a time. To account for this, we place a threshold on the total number of contacts that can be traced at a time. Part of the work carried out by contact tracers is moving individuals to the hospital, and the remaining effort is dedicated to visiting contacts who have not (yet) displayed any symptoms of Ebola. In our flow diagram in Figure 1, one can see the terms with coefficient f representing the effects of contact tracing. Our model with eight compartments is below:
S = β 1 S I β 2 S D f S N + θ F
F = f S N θ F
E = β 1 S I + β 2 S D q f E N α E
E F = q f E N r E F
I = α E f I N γ I ϕ 1 I ν I
H = r E F + f I N + γ I ϕ 2 H μ H
R = ϕ 1 I + ϕ 2 H
D = ν I ω D
where N = S + E + I . The function f depends on F, E F , and I and gives the rate of finding new contacts
f = κ 1 γ I + κ 2 ν I if F + E F < ( 15 ) ( 1200 ) p 0 else .
Here, 1 p is the proportion of the total available contact tracing effort dedicated to hospitalizing individuals identified as symptomatic. Note that the two events (movement into H and D) can be seen in the function f with the rates γ I and ν I . In the cutoff for f, the number 15 is how many contacts on average one contact tracer can trace, and the number 1200 is the maximum number of contact tracers that were employed in the Western Area, Sierra Leone (containing the capital city of Freetown), during the 2014–2016 epidemic [22]. Although the total number of contact tracers varied throughout the outbreak, we decided to assume the maximum of 1200 was available throughout the outbreak. The units of f are persons per day. The units of each compartment are individuals. The units and interpretation of each parameter are listed in Table 1. Note that we do not account for births or for deaths from any other cause than Ebola.
People can move from Susceptible to Exposed by coming into contact with a member of the Infectious class (term β 1 S I ) or by coming into contact with an infectious dead body (term β 2 D S ). People who are being traced move from Susceptible to F or from Exposed to E F by coming into contact with a person who has just been hospitalized or attending a funeral for somebody who has just died of Ebola (term f S N = ( κ 1 γ I + κ 2 ν I ) S N ). This term is scaled by N because the persons moving in tracing are moved proportionally to the ratio of persons in their current class. For example, a person being traced from S moves to F at a rate proportional to S N = S S + E + I . A person is more likely to be in E F while being traced than to be in F because of the contact they had with either an infected person or a dead body. To account for this, we multiply the term f E N by a number q > 1 , a scaling factor to increase the likelihood of E F s being traced relative to that of F s being traced. People who have completed their time being traced and have not developed symptoms move back into S (term θ F ). Once a person has been in the Exposed class for an average of 10 days, they move to the Infectious class (term α E ). A person in the class E F is moved to the hospital once they develop symptoms (term r E F ). If an individual being traced shows symptoms the first time they are contacted, they are immediately moved to the hospital (term f I N ). Some Infectious people decide to go to the hospital on their own (term γ I ). Some Infectious people manage to survive Ebola and move to R (term ϕ 1 I ) but others die of the disease and we assume they are not safely buried and contribute to the class D (term ν I ). This is a simplifying assumption, because, as the epidemic drew on, many people who died in the community were safely buried. Some Hospitalized individuals will recover (term ϕ 2 H ) but others will die and be safely buried (term μ H ). After some time has passed, an unsafely buried dead body is no longer able to infect people (with decay term ω D ).

3. Reproductive Number

We will derive the basic reproductive number 0 using the standard method of the Next Generation Matrix [30,31,32,33]. We expect that near the disease-free equilibrium (DFE), the number of infections will be small but non-zero. The population affected by the outbreak consisted entirely of susceptibles at the beginning of the outbreak. Therefore, for this analysis, we assume that f = κ 1 γ I + κ 2 ν I .
Now, Equation (7) implies
ϕ 1 I = ϕ 2 H .
Giving I = H = 0 . From Equation (8), we get D = 0 . Since I = 0 , Equation (2) gives F = 0 and Equation (5) gives E = 0 . Since I = H = 0 , Equation (6) gives E F = 0 . Since E = I = 0 , we conclude that S = N ( 0 ) . We have the DFE: ( S , 0 , 0 , 0 , 0 , 0 , R , 0 ) . However, we take R = 0 for computation of the Next Generation Matrix. The diseased classes here are: E, E F , I, H, and D, with corresponding vectors F V forming the right hand side of the system with only diseased classes,
F = β 1 S I + β 2 S D 0 0 0 0 , V = α E + q ( κ 1 γ I + κ 2 ν I ) E S + I + E r E F q ( κ 1 γ I + κ 2 ν I ) E S + I + E ( ϕ 1 + ν + γ ) I + ( κ 1 γ I + κ 2 ν I ) I S + I + E α E ( ϕ 2 + μ ) H r E F γ I ( κ 1 γ + κ 2 ν ) I 2 S + I + E ω D ν I .
It is easy to show that our model satisfies the assumptions required for use of the Next Generation Method. Note that our DFE is not unique and this is not required. We get the Jacobian matrices D F ( E , E F , I , H , D ) and D V ( E , E F , I , H , D ) at the DFE, ( 0 , 0 , 0 , 0 , 0 ) with S = S ,
D F ( 0 , 0 , 0 , 0 , 0 ) = 0 0 β 1 S 0 β 2 S 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 , D V ( 0 , 0 , 0 , 0 , 0 ) = α 0 0 0 0 0 r 0 0 0 α 0 ϕ 1 + ν + γ 0 0 0 r γ ϕ 2 + μ 0 0 0 ν 0 ω .
Thus, the basic reproductive number we obtain as the spectral radius of the matrix D F ( 0 , 0 , 0 , 0 , 0 ) ( D V ) 1 ( 0 , 0 , 0 , 0 , 0 ) is
0 = β 1 S ϕ 1 + ν + γ + ν β 2 S ω ( ϕ 1 + ν + γ ) .
The first term describes the number of new infections that we expect per individual from the I class, and the second term describes the number of new infections that we expect per body in the D class.

4. Parameter Estimation

Our data are taken from the Sierra Leone Ministry of Health daily situation reports, published on their website during the epidemic. We accessed these old web sites via the Wayback Machine at https://web.archive.org/web/20150314233800/http:/health.gov.sl/?page_id=583 (accessed on 28 February 2020). Data are listed in Appendix A. Situation reports were available beginning at Day 77 with the final day being Day 504, but not every intermediate day had a report. There were 343 total reports available for us to use. From these reports, we used confirmed cases and deaths. There was one report we chose to exclude because it listed more confirmed deaths than subsequent reports, making our total number of data points 342.
We chose some parameters from the literature and estimated others using our data. The parameters
α = 0.1 , 1 ω = 4.5 , 1 θ = 21
were taken from the literature [16,19,21,28]. Our data indicated that the initial condition for the H class was H ( 0 ) = 94 individuals. We assumed the initial condition for the recovered class was R ( 0 ) = 0 individuals, and that the initial condition for S was roughly equivalent to the population of Sierra Leone at the time, S ( 0 ) = 6,348,350 people. We estimated the following parameters:
β 1 , β 2 , γ , κ 1 , κ 2 , r , p , ν , μ , ϕ 1 , ϕ 2 .
We estimated the following initial conditions:
F ( 0 ) , E ( 0 ) , E F ( 0 ) , I ( 0 ) , D ( 0 ) .
See Table 1 for parameter interpretations and units.
We estimated the above parameters in MATLAB using multistart to generate many vectors of starting parameter estimates. Each vector was used to initialize a search in fmincon, which is a local minimizer. In MATLAB ode45 served as our ODE solver. Parameter upper and lower bounds were based on ranges of parameters from the literature [19,21] and from our data. We used papers [19,21] for some ranges because they rely on data from Sierra Leone. For example, parameters comparable to our β 1 , β 2 , ϕ 1 , and ϕ 2 were found in Rivers [19]. Our lower limit for r was based on both papers [19,21]. There is also a parameter in Olu comparable to our parameter κ 1 [21]. For example, the upper bound for F 0 was taken as 2500 because our data indicated that in early days this was roughly the number of contacts being traced. To estimate our cumulative simulated cases, we summed over the entries into the H class, assuming that cases for people in the community were unconfirmed. To estimate our cumulative simulated deaths, we summed over the deaths from H and I together. The data to be compared with simulation results are cumulative confirmed cases and cumulative confirmed deaths. We minimized the following:
J = i = 77 504 ( Cases E s t i m a t e d ( i ) Cases D a t a ( i ) ) 2 ( Cases D a t a ( i ) ) 2 + ( Deaths E s t i m a t e d ( i ) Deaths D a t a ( i ) ) 2 ( Deaths D a t a ( i ) ) 2 ,
which is a type of sum of least squares for our model. Our data began at day 77 and ended at day 504, with 342 total data points each for cases and deaths. Note that this does not include every day between day 77 and day 504. The missing data are for days when the Ministry of Health situation report was unavailable. The data from one day, when cumulative deaths were higher than for following days, were excluded. You can see that some days do not have data by the gaps in the red dots in Figure 2.
We had two primary goals during the process of parameter estimation:
  • Fit the data with a low J value;
  • In each class, we wanted reasonable dynamics, meaning approximately the correct magnitude in the size of each compartment.
We tried several ways of fitting the data. First, we estimated all the parameters listed above, holding them all constant throughout the epidemic. This resulted in simulated epidemic curves that did not flatten at the end, indicating the epidemic would have kept going (see Appendix B). Then, we chose five parameters that seemed to vary during the epidemic according to the literature and allowed those five parameters to switch from one value to a second value in the middle of the epidemic with a smoothed transition between the two values. In order to achieve a good simulation of the data with reasonable compartments, we modified the model by inserting the parameter q. Then, we reestimated the parameters using the varying approach for five of the parameters. This resulted in good simulations of the data with reasonable compartments.
In order to achieve a simulated fit of the data, which would include a flattening of the cumulative cases and cumulative deaths curves rather than simulations that indicated the epidemic would not have ended, we decided to allow some parameters (specifically β 1 , β 2 , γ , κ 1 , and κ 2 ) to vary over the course of the epidemic. We chose these parameters because we knew that people’s behavior changed during the epidemic. We smoothed the transition from the first value of each of these parameters to the second value using piecewise functions such as the one below for each of the parameters
β 1 ( t ) = 6.68 × 10 8 t < 160 6.68 × 10 8 ( 1 t 160 30 ) + 3.94 × 10 8 t 160 30 160 t 190 3.94 × 10 8 t > 190 .
Chowell et al. [34] built a system of ODEs representing Ebola outbreaks in Congo and Uganda and used a smooth transition between two transmission rates due to control interventions (such as education and contact tracing followed by quarantine).
The literature supports our decision to allow β 1 , β 2 , γ , κ 1 and κ 2 to change over the course of the epidemic. Senga et al. [28] analyzed data on probable and confirmed cases of EVD and their contacts in Kenema district, Sierra Leone, taken from the national database. They found that the number of contacts per case increased over time. The low number of contacts per case reported early in the epidemic was much lower than those reported in other countries, which they concluded meant that the contact listings were incomplete. Olu et al. found that during the months of June 2014 to November 2014 the average number of contacts per case was nine, and that during the months of December 2014 to May 2015, the average number of contacts per case increased to 16 [21]. Lokuge et al. reported that, later in the epidemic, people were more likely to come to the hospital of their own volition, less likely to report funeral contact, and that contact tracing increased in efficacy [29]. These findings from the literature indicate it is reasonable to conclude that values for β 1 , β 2 , γ , κ 1 and κ 2 changed during the course of the epidemic due to changes in behavior and the level of education in the population about EVD.
However, we were unable to generate reasonable sizes for compartment E F . Our simulations were showing very few people passing through E F , which is not reflective of the success that contact tracing achieved in locating exposed individuals during the outbreak. We decided to modify the model by adding a multiplier, q, in front of the f E N term. We tried several values and found that a value of q = 100 generated reasonable sizes for compartment E F . This multiplier indicates that people who were being traced had had contact with an individual who was infectious or with a dead body, so they were more likely to have been exposed to Ebola than a member of the population who had not had such contact. These changes resulted in the simulations shown in Figure 2, Figure 3 and Figure 4 which were generated using the parameters found in Table 2.
Figure 3 shows how many cases total were identified as part of the contact tracing effort. Near the end of the outbreak, this number reaches about 1100, which represents more than a tenth of all confirmed cases. This demonstrates the importance of successful contact tracing. The peak of contact tracing numbers corresponds to the slowing of the increase in cumulative cases, around day 200. This indicates that contact tracing efforts contributed to ending the epidemic.
Note that in Figure 4, the increase later in the epidemic of S results from people returning to S from F after being traced for 21 days and showing no symptoms. In Figure 4, the peak in E occurs at day 164, the peak in H about two weeks later on day 176, the peak in I about two weeks after that on day 192, and then the peak in D on day 197. It is not surprising that the peak in E precedes the other peaks, but it is surprising that the peak in D is the last peak to occur. This indicates that there may have been unsafely buried bodies later in the epidemic, but that fewer people were catching Ebola from funeral interactions despite this increase in funerals.
In Table 2, there is no difference between β 1 early and β 1 late. However, β 2 changes from an early value of 1.00 10 6 to a much lower later value of 1.00 10 7 . These parameter values indicate that while the rate of transmission from interactions between S and I remained about the same throughout the epidemic, the rate of transmission from D to S decreased dramatically as people became more educated about Ebola. Oddly, γ = 0.41 decreases to a later value of γ = 0.062 , which does not agree with accounts from the literature that people were more likely to come to the hospital once they developed symptoms later in the epidemic than they were earlier in the epidemic. The value of κ 1 = 29.74 early increases to κ 1 = 44.93 late, corresponding to reports from the literature that people were more likely to report more complete lists of contacts later in the epidemic. However, κ 2 = 44.62 early decreased to κ 2 = 16.61 late, adding to the conclusion that people were less likely to attend traditional funerals later in the epidemic. The changes in these parameters during the outbreak might be caused by a combination of factors, including educating the public about Ebola [35], increases in available beds at Ebola Treatment Centers, and more effective implementation of contact tracing.
The value of r = 0.056 means that contacts who were infected took an average of 18 days to show symptoms. This value for r is probably unrealistically small, as it should likely be closer to α = 0.1 . The parameter ν was slightly larger than μ , since those who were treated had slightly lower chance of dying from Ebola. Similarly, ϕ 2 was larger than ϕ 1 because those who were treated were more likely to recover from the disease.

5. Importance of Contact Tracing

Figure 5 shows potential trajectories for epidemics with different numbers of contact tracer workers available, either more or fewer than were actually available during the epidemic. We varied the number of these workers from 0 to 2000, and note that 1200 is the corresponding number in our model. Without contact tracing at all, the highest blue curve, there would have been thousands more cases and deaths. Even a much smaller workforce than existed would have made a dramatic improvement on the trajectory of the epidemic from what would have happened without contact tracing. Once the number of contact tracers reaches about 1000, each increase in the number of workers has much less dramatic effects. More tracers still would have been better, but the difference in trajectories is much less dramatic than the difference between 0 tracers and 200 tracers.
The number of persons traced from each hospitalization ( κ 1 ) and the number from each funeral ( κ 2 ) were estimated as κ 1 = 29.7 early, κ 1 = 44.9 late, κ 2 = 44.6 early, and κ 2 = 16.6 late in our model. We vary those numbers from 5 to 50 to see the effect on the epidemic. If we hold each of the contact tracing parameters κ 1 and κ 2 constant at the values in Figure 6, the heat map shows the total number of deaths by day 504 of the outbreak. Increasing each of the two parameters reduces the total number of deaths, but κ 1 has a much more dramatic effect than κ 2 . This seems to indicate that more deaths resulted from people having contact with infected individuals than resulted from people having contact with dead bodies.

6. Discussion and Conclusions

Better understanding of the mechanisms of contact tracing is important for disease management. Our model is novel in its inclusion of explicit contact tracing of both Susceptible and Exposed individuals, as well as including the limitation on the number of total contact tracers available for the work. We counted the total number of people being traced and tracked the length of time they were being traced. Li et al. analyzed 37 compartmental models of Ebola [9], and they identified models that explicitly included classes of hospitalized individuals and of funerals as more useful to management decisions, because they explicitly included targeted interventions. For this reason, we explicitly included contact tracing in our model, including the logistical limitations resulting from limited numbers of contact tracers, because contact tracing is another targeted intervention.
We found that better matching of the simulations with the data was possible when we allowed five parameters to change over the course of the epidemic: β 1 , β 2 , γ , κ 1 and κ 2 . These parameters are the per capita rate of transmission from the Infectious compartment to the Susceptible compartment, the per capita rate of transmission from the Dead Body compartment to the Susceptible compartment, the rate of transition from the Infectious compartment to the Hospital compartment, the number of contacts per person generated from a hospitalized case, and the number of contacts per person generated from a funeral. These parameters changed during the outbreak because more hospitals were available as the outbreak went on, people became more educated about the disease, and contact tracing became more effective. This work illustrates the value of changing parameters due to known behavior changes.
Early on in the epidemic, people were less likely to report as many contacts as they did later in the epidemic, as demonstrated by the increase from κ 1 = 29.74 early to κ 1 = 44.93 late. Later in the epidemic, people were less likely to attend traditional funerals, as seen in the decrease from κ 2 = 44.62 early to κ 2 = 16.61 late. The transmission parameter β 1 remained unchanged, while β 2 decreased from 1.00 × 10 6 early to 1.00 × 10 7 late.
There was a period when the contact tracing infrastructure was overwhelmed by cases, as seen in the plateaus in Figure 4. More contact tracers available to work would have prevented this plateau, but the number of contact tracers available was sufficient to prevent many more cases and deaths from occurring. Increasing either κ 1 or κ 2 would have decreased the number of deaths that occurred, but κ 1 had a stronger effect than κ 2 . Overall this work makes a strong contribution to understanding the effects of contact tracing and changes in behavior on disease management.
The results of this paper might be improved if we had more details about the number of contact tracers employed and about the number of individuals being traced through time. More knowledge about the change of behavior during this outbreak would have been useful. One limitation of this model is that we assumed there was no within-hospital transmission, while we know this occurred sometimes.
The practical utility of this model is its use to disease management. One conclusion of our model is that behavior change over the course of an outbreak significantly impacts dynamics and should be considered when formulating models and management responses. It could be interesting to retrospectively analyze other past outbreaks allowing for time-dependent parameters. One could try to connect behavior change with specific information campaigns. Figure 5 shows clearly how a linear decrease in the amount of adequate contact tracing during an outbreak can result in a nonlinear increase in the number of cases and deaths. As a result, our time-dependent modeling approach can be used in future outbreaks to assess the amount of contact tracing that should be conducted in order to limit the total number of cases and deaths.
In the future, we plan to further explore the role of contact tracing in epidemics. To add international spread features, one could consider mobility data [36]. We plan to build a model with a more realistic form to the function f which represents how contact tracing capacity grows in response to an epidemic. We will also explore the role contact tracing plays in outbreaks of other diseases, including diseases with a latent period such as COVID-19. The mechanisms of contact tracing procedures for other diseases might be quite different and require the development of disease-specific models. Optimization techniques (such as optimal control) could be used to design management strategies for contact tracing.

Author Contributions

Conceptualization, all authors; Methodology, all authors; Software, D.B., S.L., C.J.E. and B.L.; Validation, D.B., S.L., C.J.E., B.L., M.L.W. and B.R.G.J.; Formal Analysis, all authors; Data Curation, D.B. and B.R.G.J.; Writing—Original Draft Preparation, D.B.; Writing—Review & Editing, all authors; Visualization, D.B.; Supervision, S.L.; Project Administration, S.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded in part by the National Science Foundation, grant number 134651, to the MASAMU Advanced Study Institute, and in part by the National Institute for Mathematical and Biological Synthesis, which is funded by the National Science Foundation, grant number DBI-1300426, with additional support from the University of Tennessee, Knoxville. K.A.J.W. was supported in part by an LMS Scheme 5 grant (51710).

Institutional Review Board Statement

Ethical review and approval were waived for this study, due to the fact that we performed secondary analysis of existing, freely available data that were already deidentified and aggregated.

Informed Consent Statement

Patient consent was waived due to the fact that we performed secondary analysis of existing, freely available data that were already deidentified and aggregated.

Data Availability Statement

The data used in this study are found in Appendix A.

Conflicts of Interest

The authors declare no conflict of interest. The findings and conclusions in this paper are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention or the Agency for Toxic Substances and Disease Registry.

Appendix A. Data

Mathematics 09 00608 i001Mathematics 09 00608 i002Mathematics 09 00608 i003Mathematics 09 00608 i004Mathematics 09 00608 i005

Appendix B. Initial Fitting Results

Note that in Figure A1, the curves are still increasing at day 500, indicating that the epidemic would have continued.
Figure A1. First attempt match to the data of cumulative cases and cumulative deaths with all parameters constant. The value of J is 0.1963 .
Figure A1. First attempt match to the data of cumulative cases and cumulative deaths with all parameters constant. The value of J is 0.1963 .
Mathematics 09 00608 g0a1

References

  1. Frieden, T.R.; Damon, I.; Bell, B.P.; Kenyon, T.; Nichol, S. Ebola 2014–New Challenges, New Global Response and Responsibility. N. Engl. J. Med. 2014, 371, 1177–1180. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Aylward, B.; Barboza, P.; Bawo, L.; Bertherat, E.; Bilivogui, P.; Blake, I.; Brennan, R.; Briand, S.; Chakauya, J.M.; Chitala, K.; et al. Ebola virus disease in West Africa-The first 9 months of the epidemic and forward projections. N. Engl. J. Med. 2014, 371, 1481–1495. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Bogoch, I.I.; Creatore, M.I.; Cetron, M.S.; Brownstein, J.S.; Pesik, N.; Miniota, J.; Tam, T.; Hu, W.; Nicolucci, A.; Ahmed, S.; et al. Assessment of the potential for international dissemination of Ebola virus via commercial air travel during the 2014 west African outbreak. Lancet 2015, 385, 29–35. [Google Scholar] [CrossRef] [Green Version]
  4. Brainard, J.; Hooper, L.; Pond, K.; Edmunds, K.; Hunter, P.R. Risk factors for transmission of Ebola or Marburg virus disease: A systematic review and meta-analysis. Int. J. Epidemiol. 2016, 45, 102–116. [Google Scholar] [CrossRef] [Green Version]
  5. Dietz, P.M.; Jambai, A.; Paweska, J.T.; Yoti, Z.; Ksaizek, T.G. Epidemiology and risk factors for ebola virus disease in Sierra Leone-23 May 2014 to 31 January 2015. Clin. Infect. Dis. 2015, 61, 1648–1654. [Google Scholar] [CrossRef] [Green Version]
  6. Drake, J.M.; Bakach, I.; Just, M.R.; O’Regan, S.M.; Gambhir, M.; Chun-Hai, I.F. Transmission models of historical ebola outbreaks. Emerg. Infect. Dis. 2015, 21, 1447–1450. [Google Scholar] [CrossRef] [Green Version]
  7. Gomes, M.F.; Piontti, A.P.Y.; Rossi, L.; Chao, D.; Longini, I.; Halloran, M.E.; Vespignani, A. Assessing the international spreading risk associated with the 2014 West African Ebola outbreak. PLoS Curr. 2014, 6. [Google Scholar] [CrossRef] [PubMed]
  8. Skrip, L.A.; Fallah, M.P.; Gaffney, S.G.; Yaari, R.; Yamin, D.; Huppert, A.; Bawo, L.; Nyenswah, T.; Galvani, A.P. Characterizing risk of Ebola transmission based on frequency and type of case–contact exposures. Philos. Trans. R. Soc. B Biol. Sci. 2017, 372, 20160301. [Google Scholar] [CrossRef] [Green Version]
  9. Li, S.L.; Bjørnstad, O.N.; Ferrari, M.J.; Mummah, R.; Runge, M.C.; Fonnesbeck, C.J.; Tildesley, M.J.; Probert, W.J.; Shea, K. Essential information: Uncertainty and optimal control of Ebola outbreaks. Proc. Natl. Acad. Sci. USA 2017, 114, 5659–5664. [Google Scholar] [CrossRef] [Green Version]
  10. Cook, B.W.M.; Cutts, T.A.; Nikiforuk, A.M.; Poliquin, P.G.; Court, D.A.; Strong, J.E.; Theriault, S.S. Evaluating environmental persistence and disinfection of the Ebola virus Makona variant. Viruses 2015, 7, 1975–1986. [Google Scholar] [CrossRef]
  11. Senga, M.; Pringle, K.; Ramsay, A.; Brett-Major, D.M.; Fowler, R.A.; French, I.; Vandi, M.; Sellu, J.; Pratt, C.; Saidu, J.; et al. Factors underlying Ebola virus infection among health workers, Kenema, Sierra Leone, 2014–2015. Clin. Infect. Dis. 2016, 63, 454–459. [Google Scholar] [CrossRef] [Green Version]
  12. Garske, T.; Cori, A.; Ariyarajah, A.; Blake, I.M.; Dorigatti, I.; Eckmanns, T.; Fraser, C.; Hinsley, W.; Jombart, T.; Mills, H.L.; et al. Heterogeneities in the case fatality ratio in the west African Ebola outbreak 2013–2016. Philos. Trans. R. Soc. B Biol. Sci. 2017, 372, 20160308. [Google Scholar] [CrossRef] [Green Version]
  13. De Arazoza, H.; Lounes, R. A non-linear model for a sexually transmitted disease with contact tracing. IMA J. Math. Appl. Med. Biol. 2002, 19, 221–234. [Google Scholar] [CrossRef] [PubMed]
  14. Hsieh, Y.H.; Wang, Y.S.; de Arazoza, H.; Lounes, R. Modeling secondary level of HIV contact tracing: Its impact on HIV intervention in Cuba. BMC Infect. Dis. 2010, 10, 1–9. [Google Scholar] [CrossRef] [Green Version]
  15. Hyman, J.M.; Li, J.; Stanley, E.A. Modeling the impact of random screening and contact tracing in reducing the spread of HIV. Math. Biosci. 2003, 181, 17–54. [Google Scholar] [CrossRef]
  16. CDC. Increases in Heroin Overdose Deaths—28 States, 2010 to 2012. MMWR Morb. Mortal. Wkly. Rep. 2014, 63, 849–854. [Google Scholar]
  17. Browne, C.; Gulbudak, H.; Webb, G. Modeling contact tracing in outbreaks with application to Ebola. J. Theor. Biol. 2015, 384, 33–49. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Webb, G.; Browne, C.; Huo, X.; Seydi, O.; Seydi, M.; Magal, P. A model of the 2014 ebola epidemic in West Africa with contact tracing. PLoS Curr. 2015, 7, 1–8. [Google Scholar] [CrossRef]
  19. Rivers, C.M.; Lofgren, E.T.; Marathe, M.; Eubank, S.; Lewis, B.L. Modeling the impact of interventions on an epidemic of Ebola in Sierra Leone and Liberia. PLoS Curr. 2014, 6, 1–12. [Google Scholar] [CrossRef]
  20. Stehling-Ariza, T.; Rosewell, A.; Moiba, S.A.; Yorpie, B.B.; Ndomaina, K.D.; Jimissa, K.S.; Leidman, E.; Rijken, D.J.; Basler, C.; Wood, J.; et al. The impact of active surveillance and health education on an Ebola virus disease cluster - Kono District, Sierra Leone, 2014–2015. BMC Infect. Dis. 2016, 16, 1–7. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Olu, O.O.; Lamunu, M.; Nanyunja, M.; Dafae, F.; Samba, T.; Sempiira, N.; Kuti-George, F.; Abebe, F.Z.; Sensasi, B.; Chimbaru, A.; et al. Contact Tracing during an Outbreak of Ebola Virus Disease in the Western Area Districts of Sierra Leone: Lessons for Future Ebola Outbreak Response. Front. Public Health 2016, 4, 1–9. [Google Scholar] [CrossRef] [Green Version]
  22. Washington, M.; (Centers for Disease Control and Prevention, Atlanta, GA, USA). Personal communication, 2017.
  23. Wolfe, C.M.; Hamblion, E.L.; Schulte, J.; Williams, P.; Koryon, A.; Enders, J.; Sanor, V.; Wapoe, Y.; Kwayon, D.; Blackley, D.J.; et al. Ebola virus disease contact tracing activities, lessons learned and best practices during the Duport Road outbreak in Monrovia, Liberia, November 2015. PLoS Negl. Trop. Dis. 2017, 11, 1–16. [Google Scholar] [CrossRef] [Green Version]
  24. Swanson, K.C.; Altare, C.; Wesseh, C.S.; Nyenswah, T.; Ahmed, T.; Eyal, N.; Hamblion, E.L.; Lessler, J.; Peters, D.H.; Altmann, M. Contact tracing performance during the Ebola epidemic in Liberia, 2014–2015. PLoS Negl. Trop. Dis. 2018, 12, 2014–2015. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Chowell, G.; Nishiura, H. Transmission dynamics and control of Ebola virus disease (EVD): A review. BMC Med. 2014, 12, 1–17. [Google Scholar] [CrossRef] [Green Version]
  26. Ebola Situation Report, Ministry of Health and Sanitation, Sierra Leone. 2015. Available online: https://web.archive.org/web/20150314233800/http:/health.gov.sl/?page_id=583 (accessed on 28 February 2020).
  27. Ebola Situation Report, Ministry of Health and Sanitation, Sierra Leone. 2016. Available online: https://web.archive.org/web/20160509014636/http:/health.gov.sl/?page_id=583 (accessed on 28 February 2020).
  28. Senga, M.; Koi, A.; Moses, L.; Wauquier, N.; Barboza, P.; Fernandez-Garcia, M.D.; Engedashet, E.; Kuti-George, F.; Mitiku, A.D.; Vandi, M.; et al. Contact tracing performance during the ebola virus disease outbreak in kenema district, Sierra Leone. Philos. Trans. R. Soc. B Biol. Sci. 2017, 372, 20160300. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Lokuge, K.; Caleo, G.; Greig, J.; Duncombe, J.; McWilliam, N.; Squire, J.; Lamin, M.; Veltus, E.; Wolz, A.; Kobinger, G.; et al. Successful Control of Ebola Virus Disease: Analysis of Service Based Data from Rural Sierra Leone. PLoS Negl. Trop. Dis. 2016, 10, 1–12. [Google Scholar] [CrossRef] [Green Version]
  30. Diekmann, O. Mathematical Epidemiology of Infectious Diseases: Model Building, Analysis, and Interpretation; Wiley series in mathematical and computational biology; John Wiley: Chichester, UK, 2000. [Google Scholar]
  31. Diekmann, O.; Heesterbeek, H.; Britton, T. Mathematical Tools for Understanding Infectious Disease Dynamics; Princeton University Press: Princeton, NJ, USA, 2012; Volume 7. [Google Scholar]
  32. Van den Driessche, P.; Watmough, J. Reproduction numbers and sub-threshold endemic equilibria for compartmental models of disease transmission. Math. Biosci. 2002, 180, 29–48. [Google Scholar] [CrossRef]
  33. Van den Driessche, P.; Watmough, J. Further notes on the basic reproduction number. Lect. Notes Math. 2008, 1945, 159–178. [Google Scholar] [CrossRef] [Green Version]
  34. Chowell, G.; Hengartner, N.W.; Castillo-Chavez, C.; Fenimore, P.W.; Hyman, J.M. The basic reproductive number of Ebola and the effects of public health measures: The cases of Congo and Uganda. J. Theor. Biol. 2004, 229, 119–126. [Google Scholar] [CrossRef] [Green Version]
  35. Levy, B.; Edholm, C.; Gaoue, O.; Kaondera-Shava, R.; Kgosimore, M.; Lenhart, S.; Lephodisa, B.; Lungu, E.; Marijani, T.; Nyabadza, F. Modeling the role of public health education in Ebola virus disease outbreaks in Sudan. Infect. Dis. Model. 2017, 2, 323–340. [Google Scholar] [CrossRef]
  36. Halloran, M.E.; Vespignani, A.; Bharti, N.; Feldstein, L.R.; Alexander, K.; Ferrari, M.; Shaman, J.; Drake, J.M.; Porco, T.; Eisenberg, J.N.; et al. Ebola: Mobility data. Science 2014, 346, 433. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. Flow diagram with Susceptible–Exposed–Infectious–Recovered ( S E I R ) standard disease compartments, F , E F compartments due to contact tracing, and H , D for Hospitalized and Dead bodies as appropriate for Ebola. The coefficient f represents transitions due to contact tracing. The parameters and compartments are defined in Table 1.
Figure 1. Flow diagram with Susceptible–Exposed–Infectious–Recovered ( S E I R ) standard disease compartments, F , E F compartments due to contact tracing, and H , D for Hospitalized and Dead bodies as appropriate for Ebola. The coefficient f represents transitions due to contact tracing. The parameters and compartments are defined in Table 1.
Mathematics 09 00608 g001
Figure 2. The value of the objective for this simulation was J = 0.0423 .
Figure 2. The value of the objective for this simulation was J = 0.0423 .
Mathematics 09 00608 g002
Figure 3. Dynamics of class F in the upper left, class E F in the upper right, their sum on the bottom left, and the integral of those leaving E F to be hospitalized on the bottom right. These classes correspond to the parameters from Table 2 and the data simulations from Figure 2.
Figure 3. Dynamics of class F in the upper left, class E F in the upper right, their sum on the bottom left, and the integral of those leaving E F to be hospitalized on the bottom right. These classes correspond to the parameters from Table 2 and the data simulations from Figure 2.
Mathematics 09 00608 g003
Figure 4. The graphs above correspond to the parameters from Table 2 and the data simulations from Figure 2. Note that the scales are all different.
Figure 4. The graphs above correspond to the parameters from Table 2 and the data simulations from Figure 2. Note that the scales are all different.
Mathematics 09 00608 g004
Figure 5. Effect of varying the number of contact tracers available from 0 to 2000, with 1200 as the corresponding number in our model.
Figure 5. Effect of varying the number of contact tracers available from 0 to 2000, with 1200 as the corresponding number in our model.
Mathematics 09 00608 g005
Figure 6. Effect of varying contact tracing parameters κ 1 and κ 2 on the total number of deaths by day 504 of the epidemic.
Figure 6. Effect of varying contact tracing parameters κ 1 and κ 2 on the total number of deaths by day 504 of the epidemic.
Mathematics 09 00608 g006
Table 1. The parameters and compartment names in our model with their interpretations and units.
Table 1. The parameters and compartment names in our model with their interpretations and units.
SymbolInterpretationUnits
β 1 transmission from interactions between I and Sper person per time
β 2 transmission from interactions between D and Sper person per time
1 / θ number of days a person is tracedtime
1 / α length of the exposed periodtime
rrate of hospitalization for traced individualsper time
γ rate of hospitalization for untraced individualsper time
ϕ 1 recovery rate for untreatedper time
ϕ 2 recovery rate for treatedper time
ν death rate for untreatedper time
μ death rate for treatedper time
ω rate at which dead bodies become non-infectiousper time
κ 1 contacts recruited from hospitalization of one personunitless
κ 2 contacts recruited from funeral of one personunitless
qscaling factor for exposed contactsunitless
Ssusceptiblesindividuals
Fsusceptibles being tracedindividuals
Eexposedindividuals
E F exposed being tracedindividuals
Iinfectiousindividuals
Hhospitalizedindividuals
Ddead bodiesindividuals
Rrecoveredindividuals
Table 2. Values for parameters, with five parameters having early and late values. Parameters with * were taken from the data or the literature. Others were estimated.
Table 2. Values for parameters, with five parameters having early and late values. Parameters with * were taken from the data or the literature. Others were estimated.
ParameterValueParameterValue
β 1 early 1.00 × 10 9 r 0.056
β 1 late 1.00 × 10 9 p 0.90
β 2 early 1.00 × 10 6 ν 0.024
β 2 late 1.00 × 10 7 μ 0.010
γ early 0.41 ϕ 1 0.020
γ late 0.062 ϕ 2 0.028
κ 1 early 29.74 F ( 0 ) 2451.10
κ 1 late 44.93 E ( 0 ) 32.04
κ 2 early 44.62 E F ( 0 ) 124.88
κ 2 late 16.61 I ( 0 ) 71.76
D ( 0 ) 6.09
α 0.1 1 / ω 4.5
H ( 0 ) 94 S ( 0 ) 6 , 348 , 350
R ( 0 ) 0 1 / θ 21
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Burton, D.; Lenhart, S.; Edholm, C.J.; Levy, B.; Washington, M.L.; Greening, B.R., Jr.; White, K.A.J.; Lungu, E.; Chimbola, O.; Kgosimore, M.; et al. A Mathematical Model of Contact Tracing during the 2014–2016 West African Ebola Outbreak. Mathematics 2021, 9, 608. https://doi.org/10.3390/math9060608

AMA Style

Burton D, Lenhart S, Edholm CJ, Levy B, Washington ML, Greening BR Jr., White KAJ, Lungu E, Chimbola O, Kgosimore M, et al. A Mathematical Model of Contact Tracing during the 2014–2016 West African Ebola Outbreak. Mathematics. 2021; 9(6):608. https://doi.org/10.3390/math9060608

Chicago/Turabian Style

Burton, Danielle, Suzanne Lenhart, Christina J. Edholm, Benjamin Levy, Michael L. Washington, Bradford R. Greening, Jr., K. A. Jane White, Edward Lungu, Obias Chimbola, Moatlhodi Kgosimore, and et al. 2021. "A Mathematical Model of Contact Tracing during the 2014–2016 West African Ebola Outbreak" Mathematics 9, no. 6: 608. https://doi.org/10.3390/math9060608

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop