Next Article in Journal
Data Analysis of the Risks of Type 2 Diabetes Mellitus Complications before Death Using a Data-Driven Modelling Approach: Methodologies and Challenges in Prolonged Diseases
Next Article in Special Issue
DEVS-Based Building Blocks and Architectural Patterns for Intelligent Hybrid Cyberphysical System Design
Previous Article in Journal
An Empirical Study on the Impact of E-Commerce Live Features on Consumers’ Purchase Intention: From the Perspective of Flow Experience and Social Presence
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Agent-Based Simulation Framework for Epidemic Forecasting during Hajj Seasons in Saudi Arabia

by
Sultanah Mohammed Alshammari
1,*,
Mohammed Hassan Ba-Aoum
2,3,
Nofe Ateq Alganmi
1 and
Arwa AbdulAziz Allinjawi
1
1
Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
2
Industrial and Systems Engineering Department, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA
3
Systems Engineering Department, King Fahd University of Petroleum & Minerals, Dhahran 31261, Saudi Arabia
*
Author to whom correspondence should be addressed.
Information 2021, 12(8), 325; https://doi.org/10.3390/info12080325
Submission received: 1 July 2021 / Revised: 30 July 2021 / Accepted: 10 August 2021 / Published: 12 August 2021
(This article belongs to the Special Issue Discrete-Event Simulation Modeling)

Abstract

:
The religious pilgrimage of Hajj is one of the largest annual gatherings in the world. Every year approximately three million pilgrims travel from all over the world to perform Hajj in Mecca in Saudi Arabia. The high population density of pilgrims in confined settings throughout the Hajj rituals can facilitate infectious disease transmission among the pilgrims and their contacts. Infected pilgrims may enter Mecca without being detected and potentially transmit the disease to other pilgrims. Upon returning home, infected international pilgrims may introduce the disease into their home countries, causing a further spread of the disease. Computational modeling and simulation of social mixing and disease transmission between pilgrims can enhance the prevention of potential epidemics. Computational epidemic models can help public health authorities predict the risk of disease outbreaks and implement necessary intervention measures before or during the Hajj season. In this study, we proposed a conceptual agent-based simulation framework that integrates agent-based modeling to simulate disease transmission during the Hajj season from the arrival of the international pilgrims to their departure. The epidemic forecasting system provides a simulation of the phases and rituals of Hajj following their actual sequence to capture and assess the impact of each stage in the Hajj on the disease dynamics. The proposed framework can also be used to evaluate the effectiveness of the different public health interventions that can be implemented during the Hajj, including size restriction and screening at entry points.

1. Introduction

Emerging and re-emerging infectious disease outbreaks pose a serious threat to public health. According to the World Health Organization (WHO), more than 30 new infectious diseases have emerged in the last three decades [1,2] with the recent acute respiratory disease (COVID-19) [3] caused by a novel coronavirus SARS-CoV-2. Infectious diseases contribute to millions of deaths every year [4]. The risk of infectious diseases will continue to cause major concern to public health authorities and governments worldwide. Therefore, advanced disease surveillance is essential for early detection and preparedness for potential disease outbreaks [5,6].
Preventive and mitigation measures, such as vaccination, isolation of infected individuals, public closures, and travel restrictions [7,8], are applied at different scales to control and slow down the disease transmission in the affected regions and populations. The decision of enforcing these various pharmaceutical and non-pharmaceutical interventions depends on the severity of the epidemics and the extent of their health implications.
Due to their large scale and dynamic nature, mass gatherings (MGs) pose a significant threat to public health [9]. These gatherings, such as sports events, religious pilgrimages, music concerts, and other festivals, are challenging settings to estimate, control, and respond to potential disease outbreaks. The mixing of a large number of people attending these events creates an ideal environment for the rapid transmission of infections. Furthermore, international MGs have the potential of spreading diseases across multiple regions of the world more rapidly due to the associated travel patterns. Participants and attendees in MGs arrive from various locations, have diverse demographics, and might be exposed to different diseases. The social mixing of such heterogeneous crowds in these events facilities the spread of various pathogens and diseases.
Several public health measures and recommendations can be applied to detect and monitor disease outbreaks during MGs [10]. In times of pandemics, postponing or canceling these gatherings are recommended [11,12]. However, the planning process for most of these events, especially the international gatherings, cost the host countries billions of dollars. Therefore, canceling or postponing these major events may cause massive economic losses. Moreover, religious pilgrimages must be held on specific dates and sites, and they cannot be subject to changes in their timing or location. Consequently, these events can be held with a limited number of attendees or a restriction to participants attending from specific regions. Furthermore, screening at entry points and vaccination requirements are recommended for known diseases. In addition to all these alternative preventative measures, it is significant to find effective ways to predict the effectiveness of the proposed prevention measures before implementing them.
Advance infectious disease surveillance at local, regional, and global levels provides needed data to detect natural or deliberate infection threats for better epidemic response planning. Epidemic models and disease spread simulations can play an effective role in studying and predicting disease outbreaks in several settings. These models can be used for an efficient assessment of the control and prevention measures before the emergence of a disease outbreak or during ongoing epidemics. Epidemic modeling and computer simulations of the transmission of infectious diseases are increasingly used to estimate the risk, size, and duration of potential epidemics to develop appropriate response planning. These models assist the What-if analysis to determine the impact of the different disease parameters and support the decision-making process in evaluating the different control measures and applied strategies to contain the disease spread.
Epidemic modeling and simulation have a long successful history in studying and estimating the spread of different diseases in different settings and populations, such as influenza [13], measles [14], zika [15,16], and SARS [17]. Recently, mathematical and individual-based epidemic models have been the subject of extensive research due to the ongoing COVID-19 pandemic. Several epidemic models were explored to simulate the spread of the COVID-19 disease [18], estimate the size of the ongoing COVID-19 outbreak locally and globally [19], and to assess the effectiveness of the applied preventative measures in different settings [20,21]. Furthermore, novel mathematical models were presented to model the novel coronavirus [22,23,24]. However, in the context of the global mass gatherings, there is very limited research in the field of developing epidemic models to assess the risk of epidemics and the impact of these gatherings on global health. The available research about epidemic modeling and simulation at global MGs either discusses the main aspects and challenges in this research area [25,26] or presents epidemic models targeting small-scale international gatherings [27], local events [28], or virtual gatherings [29].
The application of epidemic modeling and simulation in mass gatherings to study the spread of diseases at these unique settings, and to assess the risk of global epidemics poses several challenges [25,26]. Effective epidemic models are best to be constructed based on real data. However, collecting the required data to develop an epidemic model is a challenging phase for any setting, and it is even more challenging to identify and gather data requirements in the context of global MGs. A variety of data, including epidemiological, population, contact, and mobility data, needs to be identified and obtained from available sources.
Required data about the population expected to attend a mass gathering include size, demographics, disease history, contact patterns, and behaviors of the participants and attendees at the event. It is infeasible to collect these data before the start of the MG. However, previous occurrences of the same MG or similar gatherings can be used to provide estimates of these data about future participants. Furthermore, different aspects about the MG itself, such as timing, setting, schedule of activities, restrictions, and associated travel patterns, need to be incorporated for better epidemic modeling and simulation.
While there is massive and ongoing research in the field of forecasting disease epidemics in different settings, only a few studies have focused on studying and modeling epidemics in the context of mass gatherings in general and in global gatherings in particular. Thus, this research contributes to addressing the limitations in this significant research area. This paper is part of ongoing research to develop an agent-based simulation epidemic forecasting framework that can be used to assess the risk of epidemics during the global religious gathering of Hajj in Saudi Arabia. The contributions of this study are explained as follows:
  • A data-driven framework was proposed for assessing disease spread at the annual global religious gathering of Hajj using epidemic modeling and agent-based simulation.
  • In the proposed framework, the pre-event and post-event stages of Hajj were included. To the best of our knowledge, existing works have not considered simulating the disease spread during the period before a pre-planned MG, the MG itself, and after the event (MG) is concluded.
  • Simulation of disease spread at the different phases and rituals of Hajj integrating the spatial and temporal features of each phase.
The remainder of the paper is organized as follows. In the following section, we present a background of global mass gatherings and infectious diseases, with a focus on the religious event of Hajj. Then, Section 3 presents the proposed framework for epidemics forecasting in Hajj using disease spread modeling and agent-based simulation. Section 4 presents results of the epidemic simulations in the different stages of Hajj obtained using initial implementation of the proposed framework. Finally, Section 5 concludes the paper.

2. Background

2.1. Global Mass Gatherings and Disease Epidemics

The World Health Organization defines mass gathering as “any occasion, either organized or spontaneous, that attracts sufficient numbers of people to strain the planning and response resources of the community, city or nation hosting the event” [10]. Mass gatherings can be local, regional, or global, and can last for a few hours, several days, or months. Table 1 lists the main features of selected global pre-planned mass gatherings. Disease epidemics at global Mass gatherings are uncommon, but have been reported in several events throughout the past years. Figure 1 maps some of the major reported epidemics in different global gatherings.

2.2. Religious Global Gathering of Hajj

Hajj is the Muslim pilgrimage to Makkah in Saudi Arabia, Makkah is the official spelling approved by the Saudi government for the holy city of Mecca. Hajj is a pillar of Islam religion, and attendance is required for all Muslims once in a lifetime. Around two to three million pilgrims are expected to arrive in the Hajj each year prior to the 8th and stay up to the 13th day of the 12th month of the Islamic Lunar calendar (Dhul-Hijjah). The Islamic calendar varies with respect to the Gregorian calendar shifting about 11 days earlier each year. International pilgrims come from about 186 different countries around the world. While domestic pilgrims from inside Saudi Arabia arrived at Mecca on the actual dates of the Hajj rituals, the international pilgrims may arrive one month before the starting date of Hajj and stay one month after the completion of Hajj.
Several rituals make up the Hajj, such as Tawaf and Sa’ay, staying at holy cities of Mina, Arafat, and Muzdalifah. Completing Tawaf and Sa’ay is one of the rituals that pilgrims perform multiple times during Hajj. Tawaf involves walking seven times around the kaaba (a 627 square feet building located in the grand mosque in Mecca), followed by Sa’ay which is completed by walking seven times between the hills of Safa and Marwah, located also in the grand mosque near the Kaaba. On the 8th of Dhu al-Hijjah, the pilgrims head to Mina (second ritual) where accommodation arrangements are made to stay there while completing the rest of Hajj rituals. On the next day, pilgrims arrive at Arafat, which is 12 miles east of Mecca. Before sunset, all pilgrims must leave Arafat for Muzdalifah (an area between Arafat and Mina) where they spend the whole night. The next three days (10th, 11th, and 12th), pilgrims will go back and forth from Mina to Jamarat to complete the stoning ritual, throw seven stones at walls that represent the devil. The final step is to perform the farewell Tawaf around the kaaba in the grand mosque in Mecca. The pre stage, the actual rituals, and the post stage of Hajj are illustrated in Figure 2.

2.3. Public Health Response during Hajj Seasons

Saudi Arabia prepares for the potential health hazards before, during and post each Hajj season. A risk assessment is done every year with identification of the health requirements for pilgrims. The guidelines for pilgrims vaccination, emergency plans, and infection control are updated every year. Health care facilities are prepared for the pilgrims, this includes 16 hospitals and 34 health care centers in Mecca and Madina. Furthermore, health care providers and clinicians are available close to the pilgrims at the point of entries as well as during Hajj movement to Mina and Araffat to ensure that the public health activities are taking place. During Hajj, curative and preventive measures are considered for pilgrims. On arrival to Saudi Arabia, each pilgrim vaccination card is checked and a medical screening is performed to check any sign of infectious diseases. The healthcare services in the entry point will provide the vaccination if needed and isolate any positive sign of infection. For curative purposes, Saudi Arabia provides more than 5000 beds and 500 ICU beds available for pilgrims. After the Hajj period is done, the briefing of what happened during the current Hajj season is discussed among the ministry of health and other Hajj stakeholders to improve the Hajj in the next year. However, Saudi Arabia overcame many pandemic challenges.
In 2009, the H1N1 influenza pandemic was declared [30]. The World Health Organization stated that H1N1 influenza had been confirmed in more than 100 countries and there are at least 2185 confirmed deaths. Saudi Arabia authorities continued to host the Hajj pilgrimage during the H1N1 pandemic. The number of pilgrims decreased in 2009 compared to previous years reaching 2.3 million due to the timing of Hajj 2009 in the month of November, in the middle of influenza H1N1 pandemic. However, a low prevalence of pandemic influenza A (H1N1) was reported during the Hajj season. This is due to the preparation procedures made by Saudi Arabia authorities, such as the vaccination program for all healthcare workers and other people participating in the Hajj of 2009. Furthermore, thermal screening was implemented to detect febrile passengers at the points of entry. A 24-h ENT specialist was available to handle any possible positive H1N1 cases.
Another viral respiratory illness, MERS-CoV (Middle East Respiratory Syndrome Coronavirus) emerged in 2012, raising more concerns during Hajj as the first cases of MERS-CoV were reported in Saudi Arabia a few months before Hajj 2012 [31]. While the initial cases of the virus did not affect the number of pilgrims in 2012, the size of Hajj 2013 decreased by 37% due to concerns about MERS-CoV transmission with the highest number of cases reported in Saudi Arabia. The Saudi Ministry of Health issued recommendations to pilgrims at high risk, including pregnant women, children (under 12), elderly (above 65), and those with chronic and acute diseases [32].
The most recently emerging highly contiguous disease COVID-19 caused by the novel coronavirus (SARS-CoV-2) presents an ongoing global health threat affecting more than 178 million people across the world by the mid of June 2021. Saudi Arabia Hajj authorities were aware of the risks associated with the Hajj pilgrimage during the COVID-19 pandemic. Therefore, they used Jeddah Tool [33,34], a risk assessment tool, to estimate the risk of COVID-19 by modeling the expected burden considering the previous Hajj data and other factors such as attack rate, Hajj quota per country, and WHO’s global disease severity index. The result of the analysis showed that healthcare facilities would not be sufficient to treat all expected COVID-19 cases and other patients during Hajj. Thus, the Saudi government restricted the Hajj in 2020 to a limited number of domestic pilgrims to ensure the well-being and safety of pilgrims and local residents of Mecca.
Table 2 summarizes the most recent epidemics of novel viruses and the dates of the immediate affected Hajj season. The table also includes the basic epidemiology of the listed epidemics: the basic reproduction number ( R 0 ) and incubation and infectious periods. R 0 represents the average number of secondary cases caused by an infected individual. The incubation period of an infectious disease is the time interval from infection or the exposure to an infectious agent to the onset of the clinical symptoms of the disease. The period between the initial exposure and infectiousness is called the latent period. During the infectious period, the infected individual can transmit the disease to other susceptible individuals.

3. Epidemic Forecasting Framework

As stated previously, the Saudi government responded to the rapid spread of the COVID-19 pandemic by restricting the immediate affected Hajj season of 2020 to a limited number of domestic pilgrims. This decision was taken according to the current situation worldwide and based on the results of the Jeddah tool. The development and deployment of the Jeddah tool is a significant step towards incorporating data-driven approaches in risk assessment and decision-making in Hajj. Jeddah tool provides an efficient method to identifying and prioritizing potential hazards based on historical data and critical risk criteria like vulnerability assessment, capacity assessment, reputational risk, and health risk estimation.
However, this tool is not capable of assessing risks and forecast infectious diseases in future events. The static nature of the Jeddah tool prevents decision-makers from testing various interventions and evaluating the recommended preparedness plans in advance. The framework proposed in this study allows for the incorporating of synthetic data, which may improve the risk assessment process in the future Hajj seasons. The dynamic computer simulation feature included in this framework would allow the testing of various policies. Thus, the framework would aid in assessing the effectiveness of Hajj Season preparedness and prevention plans prior to the starting dates of the Hajj season. We believe that the Jeddah tool and framework complement each other in empowering the decision-making process in utilizing advanced data-driven approaches for better risk assessment and preparedness.
The proposed epidemic simulation framework as shown in Figure 3, consists of a Data Sources layer, Underlying Models layer, Epidemic Simulation layer, and an Output layer. The data requirements to develop the epidemic simulation were collected in the Data layer from various official sources, such as the Saudi General Authority for Statistics (GASTAT), Saudi General Authority of Civil Aviation (GACA), and the Ministry of Hajj and Umrah. In the Model layer, the population module is used to generate the international and domestic pilgrims based on demographic and travel data. The related data about the different stages and rituals of Hajj are stored in the Event module. The underlying epidemic model is defined in this layer. Different disease spread scenarios will be executed in the Epidemic Simulation layer according to changes in the model parameters and assumptions. Simulation results will be summarized and accessed in the Output layer.
The agent-based simulation epidemic forecasting framework integrates these layers using a user interface layer which provides potential users access to the simulation results and allows them to set up and update the model parameters and simulation settings. Via this layer, the computational models to simulate the disease transmission in Hajj will be accessible to decision-makers providing a tool for epidemic response.

3.1. Underlying Models

Several modules are developed to generate and store the data about the participating population, the mass gathering settings, and the epidemic model. The data in the Underlying Models layer will be used in the simulation layer. In the population model, the data collected about domestic and international pilgrims and the travel data will be used to create the participants’ database storing all features about the pilgrims. In the global event of Hajj, there are different groups of individuals, including residents of the two holy cities of Mecca and Madinah, official workers who organize the Hajj, and domestic and international pilgrims. Depending on the available data about these sub-populations, the Population model will generate the individuals. The Population model maintains a generic individual class that can be extended to generate sub-classes of the different types of individuals composing the whole population of Hajj.
Figure 4 shows the classification of the different groups in the Hajj population, where the international pilgrims are assigned to six different establishments based on their country of origin. For example, all pilgrims from South Asia belong to the first establishment. This classification of international pilgrims is used to organize accommodation arrangements in campuses in the holy site of Mina. Furthermore, the flow of pilgrims while completing the different rituals of Hajj will be guided based on this classification.
In the Event model, the data related to the different stages of the global event of Hajj are generated. Unlike most global MGs, the travel patterns associated with the pre- and post-stages of the Hajj are monitored and documented. These data are used in the Event model to generate the arrival and departure stages of international pilgrims. As shown previously in Figure 2, these stages are composed of entry points and staying in Mecca and Madinah phases.
The sequence of Hajj rituals is also generated in the Event model using the data collected about previous Hajj seasons and the main features of each rite, such as the location where this ritual is completed, the maximum capacity, and the average duration to complete the ritual.

3.2. Disease Dynamics

The most prominent compartmental SEIR epidemic model is used to model the disease dynamics in the population of pilgrims, where each individual in the population can have one of the disease-related states: Susceptible (S), Exposed (E), Infectious (I), and Recovered/Removed (R). In the susceptible state, an individual is assumed to be healthy but can acquire the disease once being exposed to an infection. In the second compartment (Exposed), a susceptible person is infected but not infectious yet. Individuals remain in this state throughout the latent period of the disease. Then, an exposed individual becomes infectious and transmits the disease to other susceptible individuals in the population. Infected individuals either recover at the end of the infectious period or die from the complication of the disease. Figure 5 illustrates the transition from one state to another in the basic SEIR model. The length of the latent and infectious periods and the possibility of gaining post-infection immunity are disease-dependent parameters. The disease parameters are assigned in the Epidemic Simulation layer via the Model Parameters module.
Based on the disease epidemiology and the implemented measures during a disease spread, additional compartments and states can be integrated into the basic SEIR model, such as Infectious Asymptomatic, Infectious Symptomatic, Immunized, Quarantined or Isolated, and Died. While in the proposed framework, we used the basic SEIR epidemic model, extensions of the model can be included in the layer of the Underlying Models. Figure 6 illustrates an extended epidemic model that includes more states where a newly infected individual (exposed) can be either symptomatic or asymptomatic. In the figure, the preventative measure of isolating infected individuals is shown by assuming that the infected individuals in the symptomatic state can be either quarantined or not quarantined. The selection of the epidemic model that will be implemented in the simulation layer can be made either by the end-users or based on the available data.

3.3. Agent-Based Simulation

Agent-based modeling (ABM) can simulate human behaviors in real-world complex systems in different settings [44], such as emergency evacuation [45], crowd dynamics [46], and disease outbreak modeling [47,48,49,50]. In ABM, the population is represented as a group of autonomous entities [44]. Each entity or agent can represent an individual or a subgroup in a population with specific features. The interactions and contact patterns among the individual agents, and between agents and their environment, are modeled following specific behavioral rules. The decisions and actions of agents determine the model’s outcome. Agents in these models can be heterogeneous with different features and change their behaviors according to the surrounding environment. In the context of epidemic modeling and forecasting, agent-based models provide a better understanding of the disease transmission process at an individual level.
In the developed agent-based simulation, each pilgrim in the generated population of pilgrims is represented as an agent and maintains several features, such as age, gender, health status, arrival date, and departure date. These features are grouped into four categories as shown in Figure 7, for instance, the fields related to Hajj include pilgrim type (international or domestic) and dates to complete specific rituals. These assigned fields for each agent either remain constant throughout the entire simulation or changed according to the simulation behavior. For instance, the disease state and related timers will be updated based on the disease spread simulation.
The contact patterns define the social mixing between individuals or agents in an environment. In mass gatherings, several elements influence the social mixing between the participants in these events. These factors include the type of the gathering, the capacity and settings where attendees gather, and the characteristics of the expected population attending the event. The interactions between the pilgrims during a stage or a ritual of the Hajj will be defined according to the constraints of that specific stage. For example, in the pre-Hajj phase, the social mixing and interactions between pilgrims are assumed to be within groups arriving on the same flights as they will be at the same locations while completing the arrival procedures at the entry points.
Based on the nature of the population and rituals of the Hajj, we defined a set of mixing and behavioral rules:
  • Agents who arrive on the same flight will complete all rituals as a group.
  • Agents will mix randomly with a higher rate towards agents belong to the same group and a lower rate across other groups.
  • Agents interactions will be restricted to agents from the same gender at their tents at the campus in Mina.
  • At Mina campus, agents will remain in their accommodations (tents) throughout the night time.
  • Agents will follow an hourly schedule per establishment on the first day of the Jamarat ritual.
While the contact patterns between individuals in a specific setting contribute to the disease progression, the number of contacts per individual can determine how fast the disease will spread in the population. A contact estimation function [51] that estimates the number of contacts based on the population density and the maximum capacity of the location for each stage of the Hajj is contained in the Epidemic Simulation layer. Population densities and interactions differ in each simulation of the same ritual due to whether all pilgrims complete it at the same time or there is a wider window of time to complete this ritual. Without time constraints or restrictions, lower rates of pilgrims are expected with more diverse crowd combined from all establishments or countries.
The settings of the simulation, such as the duration of each simulation, the time step, and the order of the simulations, will be determined by the Simulation settings function. By default, these two functions will be activated to provide inputs for the epidemic simulation unless the user assigns values for some of the input parameters. The simulation of the interactions and the disease spreads at an individual level for such mass gathering as the Hajj is expensive in terms of computational capabilities. Pilgrims or agents in a specific stage can be aggregated in another stage either by country, gender, or other demographic features. Thus, aggregated agents will form sub-population agents in stages where contacts between individual agents between sub-population are unlikely to occur. Figure 8 shows an example of international pilgrims aggregation.

4. Results and Discussion

An initial implementation of the proposed framework for epidemic forecasting during Hajj seasons was developed using Python Programming Language (Python 3.6). The prototype was built using a simulated synthetic population of pilgrims based on historical data of previous Hajj seasons before 2015 to predict the spread of H1N1 influenza among international pilgrims [52]. In the simulations executed using the agent-based model, we restricted the Hajj population to almost one million international pilgrims, with 0.2% initially infected pilgrims. The obtained results, summarized in Table 3, provide insights into the effectiveness of the different components of the framework in tracking the disease transmission among pilgrims throughout the sequence of the Hajj rituals, including the arrival and departure stages.
The duration of mixing at the entry points (Jeddah and Madinah international airports) in the arrival and departure stages lasts for a few hours. Thus, there will be no changes in the infectious status or timer of the initially infected pilgrims. The only transition that can be seen after executing the epidemic simulation at both entry points is in the number of newly infected pilgrims and the number of individuals in the exposed status. By the end of the departure phase of Hajj, the agent-based epidemic simulation predicted almost 22% of the arriving international pilgrims acquired the disease, with 6% of the returning pilgrims were either in the exposed or infectious state. By tracking infections in international pilgrims departing back to their home countries, the model can estimate the potential risk of disease outbreaks among returning pilgrims and their contacts to identify the likelihood of disease spread at regional and global levels.
The prototype presented previously in [52] shared the main architecture with the epidemic forecasting framework proposed in this study. However, components and modules in the middle-end layer in the initial prototype were modified and combined for better design and implementation. For instance, the travel model in the first implementation was included in the event model in the current framework as the arrival and departure stages are part of the whole event of Hajj. In addition, a user interface layer included in the current version of the framework allows users to set up, execute various simulations, and retrieve and download the obtained results. The FastAPI Python web framework and ReactJS [53] will be used to integrate the underlying system and create the user interface layer. When fully implemented, this system will improve public health responses by providing critical information for planning and responding to actions during the Hajj gathering.
Although many epidemic forecasting models have been proposed in different settings, especially with the emergence of the recent coronavirus, this study provides increased enhancements for epidemic forecasting during global mass gatherings in general and specifically in the Hajj season. For public health authorities, the proposed simulation provides insights to improve the quality of care, provide medical services to the holy sites of Hajj across the different stages, and plan the needed number of clinicians and equipment depending on the expected severity of risk and duration predicted by the proposed framework. Thus, overcoming the lack of trained medical expertise for potential epidemics and decreases morbidity and mortality to achieve a successful Hajj season. Economically, the proposed system will allow decision-makers to find alternative strategies to avoid canceling, postponing, or restricting the mass gathering. The proposed framework can be used to explore the effectiveness of pharmaceutical and non-pharmaceutical interventions to ensure the safety of the attendees while avoiding massive costs for canceling these events. Finally, the suggested framework includes (i) several discrete-event simulations that simulate all the stages of Hajj starting from arrival, actual Hajj rituals, and departure; (ii) an intelligent multi-agent system that allows the aggregation of agents at multiple levels based on the simulated stage; and (iii) user interface layer allowing users to execute simulations and retrieve results.

5. Conclusions

Computational epidemic modeling and simulation provide public health authorities with advanced and powerful tools to control and prevent disease epidemics in different settings. Mass gatherings are ideal settings for the rapid spread of infectious diseases. Therefore, epidemic forecasting before pre-planned gatherings can contribute to better emergency preparedness and response during these events. In this paper, we proposed an agent-based simulation framework to assess the risk of disease outbreaks in the global religious gathering of Hajj.
The proposed framework in this work allows researchers to conceptualize epidemic forecasting in the context of mass gatherings and recognize the big picture, which will help bridge the gap between epidemic molders, health researchers, and the policy-makers and organizers of the pre-planned events. The background about mass gatherings and infectious diseases, the detailed description of the Muslim pilgrimage of Hajj, and the representation of the conceptual framework, presented in this study, could provide researchers with insights into the simulation of epidemics at mass gatherings of different scales.
This study deepens the knowledge of forecasting disease epidemics in mass gatherings and global gatherings context. It helps policy-makers identifying potential epidemic spread risks and develop high-leverage policies that protect the health and safety of attendees and participants in these gatherings. However, the general application of the proposed framework may be limited as the characteristics of mass gatherings vary across sites. In the future, other data-driven simulation and modeling approaches, such as system dynamics, could be used to compare results and insights. Moreover, once being fully implemented, this agent-based simulation framework can be integrated with existing planning tools for the religious gathering of Hajj.

Author Contributions

Conceptualization, S.M.A.; methodology, S.M.A.; writing—original draft preparation, S.M.A., M.H.B.-A. and N.A.A.; writing—review and editing, S.M.A., M.H.B.-A., N.A.A. and A.A.A.; supervision, S.M.A.; funding acquisition, S.M.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research work was funded by the Institutional Fund Projects under Grant No. (IFPHI-020-612-2020). Therefore, the authors gratefully acknowledge technical and financial support from the Ministry of Education and King Abdulaziz University, DSR, Jeddah, Saudi Arabia.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author.

Acknowledgments

The authors wish to thank King Abdulaziz University Jeddah, Saudi Arabia for providing the necessary facilities for conducting their research.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Nii-Trebi, N.I. Emerging and neglected infectious diseases: Insights, advances, and challenges. BioMed Res. Int. 2017, 2017, 5245021. [Google Scholar] [CrossRef] [Green Version]
  2. Mukherjee, S. Emerging infectious diseases: Epidemiological perspective. Indian J. Dermatol. 2017, 62, 459–467. [Google Scholar]
  3. World Health Organization. Coronavirus Disease 2019 (COVID-19): Situation Report, 51. Available online: https://www.who.int/emergencies/diseases/novel-coronavirus-2019?adgroupsurvey={adgroupsurvey}&gclid=EAIaIQobChMIkuzvya6o8gIVB7qWCh2QHQk9EAAYASAAEgJddvD_BwE (accessed on 1 July 2021).
  4. World Health Organization. WHO Report on Global Surveillance of Epidemic-Prone Infectious Diseases; Technical Report; World Health Organization: Geneva, Switzerland, 2000. [Google Scholar]
  5. Heymann, D.L.; Rodier, G.R. Hot spots in a wired world: WHO surveillance of emerging and re-emerging infectious diseases. Lancet Infect. Dis. 2001, 1, 345–353. [Google Scholar] [CrossRef]
  6. World Health Organization. Communicable Disease Surveillance and Response Systems: Guide to Monitoring and Evaluating; Technical Report; World Health Organization: Lyon, France, 2006. [Google Scholar]
  7. Anderson, R.M.; Heesterbeek, H.; Klinkenberg, D.; Hollingsworth, T.D. How will country-based mitigation measures influence the course of the COVID-19 epidemic? Lancet 2020, 395, 931–934. [Google Scholar] [CrossRef]
  8. Bedford, J.; Enria, D.; Giesecke, J.; Heymann, D.L.; Ihekweazu, C.; Kobinger, G.; Lane, H.C.; Memish, Z.; Oh, M.D.; Schuchat, A.; et al. COVID-19: Towards controlling of a pandemic. Lancet 2020, 395, 1015–1018. [Google Scholar] [CrossRef]
  9. Hoang, V.T.; Gautret, P. Infectious diseases and mass gatherings. Curr. Infect. Dis. Rep. 2018, 20, 44. [Google Scholar] [CrossRef]
  10. World Health Organization. Public Health for Mass Gatherings: Key Considerations. Available online: https://www.who.int/publications/i/item/public-health-for-mass-gatherings-key-considerations (accessed on 1 July 2021).
  11. Ahmed, Q.A.; Memish, Z.A. The cancellation of mass gatherings (MGs)? Decision making in the time of COVID-19. Travel Med. Infect. Dis. 2020, 34, 101631. [Google Scholar] [CrossRef]
  12. World Health Organization. How to Use WHO Risk Assessment and Mitigation Checklist for Mass Gatherings in the Context of COVID-19: Interim Guidance, 20 March 2020; Technical Report; World Health Organization: Geneva, Switzerland, 2020. [Google Scholar]
  13. Chao, D.L.; Halloran, M.E.; Obenchain, V.J.; Longini, I.M., Jr. FluTE, a publicly available stochastic influenza epidemic simulation model. PLoS Comput. Biol. 2010, 6, e1000656. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Liu, F.; Enanoria, W.T.; Zipprich, J.; Blumberg, S.; Harriman, K.; Ackley, S.F.; Wheaton, W.D.; Allpress, J.L.; Porco, T.C. The role of vaccination coverage, individual behaviors, and the public health response in the control of measles epidemics: An agent-based simulation for California. BMC Public Health 2015, 15, 447. [Google Scholar] [CrossRef] [Green Version]
  15. Gwalani, H.; Hawamdeh, F.; Mikler, A.R.; Xiong, K. Modeling the 2013 zika outbreak in French Polynesia: Intervention strategies. Appl. Syst. Innov. 2018, 1, 31. [Google Scholar] [CrossRef] [Green Version]
  16. Wiratsudakul, A.; Suparit, P.; Modchang, C. Dynamics of Zika virus outbreaks: An overview of mathematical modeling approaches. PeerJ 2018, 6, e4526. [Google Scholar] [CrossRef] [Green Version]
  17. Zhou, Y.; Ma, Z.; Brauer, F. A discrete epidemic model for SARS transmission and control in China. Math. Comput. Model. 2004, 40, 1491–1506. [Google Scholar] [CrossRef]
  18. Rocklöv, J.; Sjödin, H.; Wilder-Smith, A. COVID-19 outbreak on the Diamond Princess cruise ship: Estimating the epidemic potential and effectiveness of public health countermeasures. J. Travel Med. 2020, 27, taaa030. [Google Scholar] [CrossRef] [Green Version]
  19. Hamzah, F.B.; Lau, C.; Nazri, H.; Ligot, D.; Lee, G.; Tan, C.; Shaib, M.; Zaidon, U.; Abdullah, A.; Chung, M.; et al. CoronaTracker: Worldwide COVID-19 outbreak data analysis and prediction. Bull. World Health Organ. 2020, 1, 32. [Google Scholar] [CrossRef]
  20. Goscé, L.; Phillips, A.; Spinola, P.; Gupta, R.K.; Abubakar, I. Modelling SARS-COV2 Spread in London: Approaches to Lift the Lockdown. J. Infect. 2020, 81, 260–265. [Google Scholar] [CrossRef]
  21. Prem, K.; Liu, Y.; Russell, T.W.; Kucharski, A.J.; Eggo, R.M.; Davies, N.; Flasche, S.; Clifford, S.; Pearson, C.A.; Munday, J.D.; et al. The effect of control strategies to reduce social mixing on outcomes of the COVID-19 epidemic in Wuhan, China: A modelling study. Lancet Public Health 2020, 5, e261–e270. [Google Scholar] [CrossRef] [Green Version]
  22. Alkahtani, B.S.T.; Alzaid, S.S. A novel mathematics model of covid-19 with fractional derivative. Stability and numerical analysis. Chaos Solitons Fractals 2020, 138, 110006. [Google Scholar] [CrossRef]
  23. Hedayati, M.; Ezzati, R.; Noeiaghdam, S. New Procedures of a Fractional Order Model of Novel Coronavirus (COVID-19) Outbreak via Wavelets Method. Axioms 2021, 10, 122. [Google Scholar] [CrossRef]
  24. Noeiaghdam, S.; Micula, S.; Nieto, J.J. A Novel Technique to Control the Accuracy of a Nonlinear Fractional Order Model of COVID-19: Application of the CESTAC Method and the CADNA Library. Mathematics 2021, 9, 1321. [Google Scholar] [CrossRef]
  25. Alshammari, S.M.; Mikler, A.R. Modeling Disease Spread at Global Mass Gatherings: Data Requirements and Challenges. In Recent Advances in Information and Communication Technology 2016; Springer: Berlin/Heidelberg, Germany, 2016; pp. 17–26. [Google Scholar]
  26. Chowell, G.; Nishiura, H.; Viboud, C. Modeling Rapidly Disseminating Infectious Disease during Mass Gatherings. BMC Med. 2012, 10, 159. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Stehlé, J.; Voirin, N.; Barrat, A.; Cattuto, C.; Colizza, V.; Isella, L.; Régis, C.; Pinton, J.F.; Khanafer, N.; Van den Broeck, W.; et al. Simulation of an SEIR Infectious Disease mMdel on the Dynamic Contact Network of Conference Attendees. BMC Med. 2011, 9, 87. [Google Scholar] [CrossRef]
  28. Buldú, J.M.; Antequera, D.R.; Aguirre, J. The resumption of sports competitions after COVID-19 lockdown: The case of the Spanish football league. Chaos Solitons Fractals 2020, 138, 109964. [Google Scholar] [CrossRef]
  29. Shi, P.; Keskinocak, P.; Swann, J.L.; Lee, B.Y. The Impact of Mass Gatherings and Holiday Traveling on the Course of an Influenza Pandemic: A Computational Model. BMC Public Health 2010, 10, 778. [Google Scholar] [CrossRef] [Green Version]
  30. Khan, K.; Arino, J.; Hu, W.; Raposo, P.; Sears, J.; Calderon, F.; Heidebrecht, C.; Macdonald, M.; Liauw, J.; Chan, A.; et al. Spread of a novel influenza A (H1N1) virus via global airline transportation. N. Engl. J. Med. 2009, 361, 212–214. [Google Scholar] [CrossRef]
  31. Al Tawfiq, J.; Smallwood, C.; Arbuthnott, K.; Malik, M.; Barbeschi, M.; Memish, Z. Emerging respiratory and novel coronavirus 2012 infections and mass gatherings. EMHJ-East. Mediterr. Health J. 2013, 19 (Suppl. 1), S48–S54. [Google Scholar] [CrossRef]
  32. Memish, Z.A.; Al-Tawfiq, J.A.; Al-Rabeeah, A.A. Hajj: Preparations underway. Lancet Glob. Health 2013, 1, e331. [Google Scholar] [CrossRef] [Green Version]
  33. Bieh, K.L.; Khan, A.; El-Ganainy, A.; Alotaibi, B.; Ghallab, S.; Abdulmalek, N.; Mukhtar, N.; Jokhdar, H. Guidance for health risk assessment at recurrent mass gatherings: The Jeddah tool framework. Prehospital Disaster Med. 2021, 36, 348–353. [Google Scholar] [CrossRef] [PubMed]
  34. Jokhdar, H.; Khan, A.; Asiri, S.; Motair, W.; Assiri, A.; Alabdulaali, M. COVID-19 mitigation plans during Hajj 2020: A success story of zero cases. Health Secur. 2021, 19, 133–139. [Google Scholar] [CrossRef] [PubMed]
  35. Coburn, B.J.; Wagner, B.G.; Blower, S. Modeling influenza epidemics and pandemics: Insights into the future of swine flu (H1N1). BMC Med. 2009, 7, 30. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Liu, J.; Xie, W.; Wang, Y.; Xiong, Y.; Chen, S.; Han, J.; Wu, Q. A comparative overview of COVID-19, MERS and SARS. Int. J. Surg. 2020, 81, 1–8. [Google Scholar] [CrossRef]
  37. Liu, Y.; Gayle, A.A.; Wilder-Smith, A.; Rocklöv, J. The reproductive number of COVID-19 is higher compared to SARS coronavirus. J. Travel Med. 2020. [Google Scholar] [CrossRef] [Green Version]
  38. Zhai, P.; Ding, Y.; Wu, X.; Long, J.; Zhong, Y.; Li, Y. The epidemiology, diagnosis and treatment of COVID-19. Int. J. Antimicrob. Agents 2020, 55, 105955. [Google Scholar] [CrossRef]
  39. Jhung, M.A.; Swerdlow, D.; Olsen, S.J.; Jernigan, D.; Biggerstaff, M.; Kamimoto, L.; Kniss, K.; Reed, C.; Fry, A.; Brammer, L.; et al. Epidemiology of 2009 Pandemic Influenza A (H1N1) in the United States. Clin. Infect. Dis. 2011, 52, S13–S26. [Google Scholar] [CrossRef]
  40. Tuite, A.R.; Greer, A.L.; Whelan, M.; Winter, A.L.; Lee, B.; Yan, P.; Wu, J.; Moghadas, S.; Buckeridge, D.; Pourbohloul, B.; et al. Estimated Epidemiologic Parameters and Morbidity Associated with Pandemic H1N1 Influenza. Can. Med. Assoc. J. 2010, 182, 131–136. [Google Scholar] [CrossRef] [Green Version]
  41. Assiri, A.; Al-Tawfiq, J.A.; Al-Rabeeah, A.A.; Al-Rabiah, F.A.; Al-Hajjar, S.; Al-Barrak, A.; Flemban, H.; Al-Nassir, W.N.; Balkhy, H.H.; Al-Hakeem, R.F.; et al. Epidemiological, demographic, and clinical characteristics of 47 cases of Middle East respiratory syndrome coronavirus disease from Saudi Arabia: A descriptive study. Lancet Infect. Dis. 2013, 13, 752–761. [Google Scholar] [CrossRef] [Green Version]
  42. Song, R.; Han, B.; Song, M.; Wang, L.; Conlon, C.P.; Dong, T.; Tian, D.; Zhang, W.; Chen, Z.; Zhang, F.; et al. Clinical and epidemiological features of COVID-19 family clusters in Beijing, China. J. Infect. 2020, 81, e26–e30. [Google Scholar] [CrossRef] [PubMed]
  43. Bulut, C.; Kato, Y. Epidemiology of COVID-19. Turk. J. Med. Sci. 2020, 50, 563–570. [Google Scholar] [CrossRef] [PubMed]
  44. Macal, C.M.; North, M.J. Tutorial on agent-based modeling and simulation. In Proceedings of the Winter Simulation Conference, Orlando, FL, USA, 4 December 2005. [Google Scholar]
  45. Ren, C.; Yang, C.; Jin, S. Agent-based modeling and simulation on emergency evacuation. In International Conference on Complex Sciences; Springer: Berlin/Heidelberg, Germany, 2009; pp. 1451–1461. [Google Scholar]
  46. Luo, L.; Zhou, S.; Cai, W.; Low, M.Y.H.; Tian, F.; Wang, Y.; Xiao, X.; Chen, D. Agent-based human behavior modeling for crowd simulation. Comput. Animat. Virtual Worlds 2008, 19, 271–281. [Google Scholar] [CrossRef]
  47. Carley, K.M.; Fridsma, D.B.; Casman, E.; Yahja, A.; Altman, N.; Chen, L.C.; Kaminsky, B.; Nave, D. BioWar: Scalable agent-based model of bioattacks. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2006, 36, 252–265. [Google Scholar] [CrossRef]
  48. Perez, L.; Dragicevic, S. An agent-based approach for modeling dynamics of contagious disease spread. Int. J. Health Geogr. 2009, 8, 1–17. [Google Scholar] [CrossRef] [Green Version]
  49. Cuevas, E. An agent-based model to evaluate the COVID-19 transmission risks in facilities. Comput. Biol. Med. 2020, 121, 103827. [Google Scholar] [CrossRef] [PubMed]
  50. Marini, M.; Brunner, C.; Chokani, N.; Abhari, R.S. Enhancing response preparedness to influenza epidemics: Agent-based study of 2050 influenza season in Switzerland. Simul. Model. Pract. Theory 2020, 103, 102091. [Google Scholar] [CrossRef]
  51. Hu, H.; Nigmatulina, K.; Eckhoff, P. The Scaling of Contact Rates with Population Density for the Infectious Disease Models. Math. Biosci. 2013, 244, 125–134. [Google Scholar] [CrossRef] [PubMed]
  52. Alshammari, S.M.; Gwalani, H.; Helsing, J.E.; Mikler, A.R. Disease spread simulation to assess the risk of epidemics during the global mass gathering of Hajj pilgrimage. In Proceedings of the 2019 Winter Simulation Conference (WSC), National Harbor, MD, USA, 8–11 December 2019; pp. 215–226. [Google Scholar]
  53. Fedosejev, A. React. js essentials; Packt Publishing Ltd.: Birmingham, UK, 2015. [Google Scholar]
Figure 1. Some of the major reported disease outbreaks in different global gatherings.
Figure 1. Some of the major reported disease outbreaks in different global gatherings.
Information 12 00325 g001
Figure 2. The main rituals and phases of the global event of Hajj.
Figure 2. The main rituals and phases of the global event of Hajj.
Information 12 00325 g002
Figure 3. The proposed framework for epidemic forecasting in Hajj seasons.
Figure 3. The proposed framework for epidemic forecasting in Hajj seasons.
Information 12 00325 g003
Figure 4. The classification of the sub-populations participate and interact during Hajj seasons.
Figure 4. The classification of the sub-populations participate and interact during Hajj seasons.
Information 12 00325 g004
Figure 5. States (compartments) of the basic SEIR model, Susceptible (S), Exposed (E), Infectious (I), and Removed (R), corresponding to the disease development stages over time.
Figure 5. States (compartments) of the basic SEIR model, Susceptible (S), Exposed (E), Infectious (I), and Removed (R), corresponding to the disease development stages over time.
Information 12 00325 g005
Figure 6. Extended SEIR epidemic model including additional states of Infectious Asymptomatic, Infectious Symptomatic, Quarantine, and Non-Quarantine.
Figure 6. Extended SEIR epidemic model including additional states of Infectious Asymptomatic, Infectious Symptomatic, Quarantine, and Non-Quarantine.
Information 12 00325 g006
Figure 7. The main features assigned to each pilgrim in the agent-based simulation model.
Figure 7. The main features assigned to each pilgrim in the agent-based simulation model.
Information 12 00325 g007
Figure 8. Hierarchy of agents representing international pilgrims. The top-level aggregate represents one of the six establishments where international pilgrims are grouped based on country of origin. The next level consists in country aggregates. Agents on the lowest level are aggregated based on arriving in the same flight.
Figure 8. Hierarchy of agents representing international pilgrims. The top-level aggregate represents one of the six establishments where international pilgrims are grouped based on country of origin. The next level consists in country aggregates. Agents on the lowest level are aggregated based on arriving in the same flight.
Information 12 00325 g008
Table 1. List of various types of global mass gatherings.
Table 1. List of various types of global mass gatherings.
EventTypeLocationFrequencyDurationSize (in Millions)
World ExpoFairMultiple locationsEvery 5 Years6 monthsup to 73
Summer/Winter OlympicsSportsMultiple locationsEvery 2 Years16 daysup to 10
FIFA World CupSportsMultiple locationsEvery 4 Years32 days1.5–3.5
World Youth DayReligiousMultiple locationsEvery 3 Years6 days0.5–2
HajjReligiousMecca, Saudi ArabiaAnnual5–8 days2–3
Table 2. Major disease outbreaks of novel viruses and the affected Hajj seasons.
Table 2. Major disease outbreaks of novel viruses and the affected Hajj seasons.
PandemicH1N1MERS-CoVCOVID-19
IdentificationApril 2009September 2012December 2019
Hajj Season200920122020
Hajj Dates25–30 November23–28 October28 July–2 August
Vaccine ApprovalSeptember 2009No vaccineDecember 2020
Estimated Value of R 0 1.4–1.6 [35]0.8–1.3 [36]2.4–3.58 [37,38]
Incubation Period2.7 days [39,40]5.2–6.0 [41]2–14 days [38,42]
Infectious Period3.8 days [40]7.6 days [41]3–6 days [42,43]
Confirmed Cases482,00013224,854,140
By November 2009By September 2013By August 2020
Deaths600058838,924
By November 2009By September 2013By August 2020
Table 3. Summary of the results of disease spread simulations in the whole population of international pilgrims at the end of the main stages of Hajj: the arrival stage, the actual Hajj rituals, and the departure stage.
Table 3. Summary of the results of disease spread simulations in the whole population of international pilgrims at the end of the main stages of Hajj: the arrival stage, the actual Hajj rituals, and the departure stage.
StageDurationSusceptibleExposedInfectiousRecovered
Arrival35 days
Jeddah airport500,476 ( 99.7 % )373 ( 0.1 % )851 ( 0.2 % )0
Madinah airport409,675 ( 99.7 % )310 ( 0.1 % )705 ( 0.2 % )0
Pre-Hajj35 days
Madinah407,539 ( 99 % )418 ( 0.1 % )1107 ( 0.3 % )1625 ( 0.4 % )
Mecca888,633( 97.4 % )3805 ( 0.42 % )4470 ( 0.48 % )15,482 ( 1.7 % )
Hajj rituals6 days831,485 ( 91 % )18,579 ( 2 % )25,475 ( 3 % )36,851 ( 4 % )
Post-Hajj30 days
Mecca395,435 ( 78.5 % )7540 ( 1.5 % )19,097 ( 4 % )79,628 ( 16 % )
Madinah313,203 ( 76 % )6316 ( 1.5 % )15,947 ( 4 % )75,224 ( 18 % )
Departure30 days
Jeddah airport392,535 ( 78 % )10,440 ( 2 % )19,097 ( 4 % )79,628 ( 16 % )
Madinah airport310,781 ( 76 % )8738 ( 2 % )15,947 ( 4 % )75,224 ( 18 % )
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Alshammari, S.M.; Ba-Aoum, M.H.; Alganmi, N.A.; Allinjawi, A.A. Agent-Based Simulation Framework for Epidemic Forecasting during Hajj Seasons in Saudi Arabia. Information 2021, 12, 325. https://doi.org/10.3390/info12080325

AMA Style

Alshammari SM, Ba-Aoum MH, Alganmi NA, Allinjawi AA. Agent-Based Simulation Framework for Epidemic Forecasting during Hajj Seasons in Saudi Arabia. Information. 2021; 12(8):325. https://doi.org/10.3390/info12080325

Chicago/Turabian Style

Alshammari, Sultanah Mohammed, Mohammed Hassan Ba-Aoum, Nofe Ateq Alganmi, and Arwa AbdulAziz Allinjawi. 2021. "Agent-Based Simulation Framework for Epidemic Forecasting during Hajj Seasons in Saudi Arabia" Information 12, no. 8: 325. https://doi.org/10.3390/info12080325

APA Style

Alshammari, S. M., Ba-Aoum, M. H., Alganmi, N. A., & Allinjawi, A. A. (2021). Agent-Based Simulation Framework for Epidemic Forecasting during Hajj Seasons in Saudi Arabia. Information, 12(8), 325. https://doi.org/10.3390/info12080325

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop