Evaluating Interventions in Response to COVID-19 Outbreak by Multiple-Criteria Decision-Making Models

: The COVID-19 outbreak has currently led to serious social and economic consequences. In poor and developing countries, there are more challenges and barriers to tackling the pandemic. The study’s aim is to propose a hybrid approach to multiple-criteria decision-making (MCDM) models for determining the most efﬁcient intervention strategies. The methodology is a combination between the Best-Worst Model (BWM) and Group Best-Worst Model (GBWM) to estimate the efﬁciency score of intervention. Based on the background of knowledge, ﬁve groups of stakeholders including Academicians, Entrepreneurs, Commons Residents, Social Workers, Health Workers are considered decision-makers (DMs). A set of nine potential strategies was evaluated and prioritized by all DMs. The ﬁndings have shown that different groups of stakeholders prioritized differently the importance of criteria due to their interests. In the context of Vietnam, however, the Availability of Health Systems is prioritized as the most important intervention. The results and proposed model of this paper contributed to MCDM literature as well as a good reference to apply practically in many different countries.


Introduction
At the end of 2019, an infectious disease which is caused by a novel form of coronavirus appeared in Wuhan (China). Afterward, the disease spread rapidly and became a global pandemic. The World Health Organization (WHO) labeled the form of pneumonia disease as COVID-19 which is caused by the Severe Acute Respiratory Syndrome Corona Virus 2 (SARS-CoV-2) [1]. The number of infected cases soared to the point of outbreaks in many countries. By 10th May 2022, according to the statistics on worldometers.info, it accounted for about 518 million SARS-CoV-2 infected cases around the world, with about 6.3 million deaths [2]. The coronavirus COVID-19 is affecting 220 countries and territories of which the USA, India, and Brazil are the three leading nations in the number of infected cases. The COVID-19 pandemic has become the largest crisis since World War II [3]. Vietnam is not an exception with the alarmingly increasing number of newly infected people. By 18 July 2021, the Vietnamese Government imposed the second complete lockdown throughout the country since the figures reported daily climbed to 3000 cases on average. The rapid spread of COVID-19 can collapse any healthcare system and that is what happened in the USA, Italy, and India. Although the vaccine types were invented in developed countries, variants of the coronavirus have occurred and the number of infections has risen rapidly. In addition, developing or low-income countries have encountered many challenges in approaching the COVID-19 vaccine. The situation could be worsened if governments delay in planning to face the challenges and threats. Nations like Vietnam, therefore, must design other strategies to tackle the spread of the pandemic. Many different intervention strategies have been adopted such as lockdowns, digital surveillance, isolation of infected patients, and widespread testing [3]. Nevertheless, governments face many difficulties in implementing such interventions. For example, issues related to ethics and privacy arise with the use of digital surveillance to track and monitor mechanisms; or a complete lockdown of the country significantly influences economics or food security for poor people. Greater attention is merited to scientific research studying the relationship between intervention strategies and public health decision-making, particularly during the COVID-19 pandemic.
In terms of decision-making studies, the Multi-Criteria Decision-Making method (MCDM) and its applications have become the most common technique, especially in the industrial sector or supply chain management [4,5]. For example, Wang et al. (2021) applied Data Envelopment Analysis (DEA) to evaluate the environmental efficiency across ASEAN nations [6] and Wang et al. (2021) used a two-stage hybrid approach of DEA and AHP to select the most optimal Wind Plants in Vietnam [4]. In terms of tourism, Wang and Nguyen (2022) evaluated the sustainability of hotels using MCDM models [7]. However, these studies are based on subjective individuals or experts to evaluate the performance efficiency of units (i.e., location, firms, countries, etc.). The authors, hence, found a research gap in the literature on applying MCDM models for group decision-makers. One of the most recent MCDM models is the Best-Worst Method (BWM) introduced by Rezaei (2015) [8].
The BWM method has some advantages compared to other MCDM models in analytical evaluation and interpretation of human perception with fewer comparisons. It could be developed for group decision-makers. Moreover, the literature on the application of the MCDM models in the public sector is attracted many scholars' attention. For example, Ozgur (2013) used the AHP model to evaluate and prioritize the pandemic mitigation strategies [9]; or Lopez and Gumasekaran used the fuzzy logic-based VIKOR to evaluate the H1N1 vaccine in Vellore city of India in 2015 [10]. Nevertheless, limited research has been conducted to evaluate public health strategies related to the COVID-19 pandemic [3].
Therefore, the study aimed to propose a novel model to evaluate and prioritize the identified factors in intervention strategies to handle COVID-19 outbreaks in developing countries. Vietnam was selected as a case study. The research model is a combination of two extended MCDM approaches, namely the Best-Worst Model (BWM) and Group Best-Worst Model (GBWM). In addition, the proposed research framework in this study was expected to fill a research gap in the literature related to the application of MCDM in public health sectors. The findings in the study can be seen as a good reference for decision-makers or policymakers to have a good evaluation of interventions and have better strategies.

Multi-Criteria Decision-Making Model
Decision-making is primarily a complex process that involves various aspects concerned and the existence of differences in perception. It is necessary to have tools and techniques supporting decision-makers to take appropriate strategies in complex contexts. Beginning in the 1950s, Neumann and Morgenstern [11] introduced the foundation of modern multi-criteria decision-making methods (MCDM) to minimize or maximize a utility function in the presence of constraints. The MCDM methods have been developed in various models. For example, Charnes (1978) introduced Data Envelopment Analysis (DEA) [12]; Saaty (1980) proposed the Analytic Hierarchy Process (AHP) [13]; Hwang and Lin (1981) published a study on Group Decision-Making under multi-criteria [14]; in 2015, one of the most recent MCDM models invented by Rezaei is the Best-Worst multi-criteria decision-making method [8]. The procedure for running MCDM models includes two basic steps: estimating criteria and ranking of alternatives [15]. Many different MCDM methods and their applications are implemented to support decision-makers in various fields. (See more in Table 1). In reviewing previous literature, the lack of studies applying the Best-Worst Method in health public decision sectors is one of the reasons the authors decided to conduct the present study.

Best-Worst Method
In general, the procedure for MCDM methods involves two steps in which data are collected to estimate pairwise comparisons in the first step, and then the weights and ranking of criteria are calculated in the second step. Researchers can base procedures on information requirements, modeling efforts, or outcomes to choose a suitable MCDM method [15]. To evaluate and rank identified factors, models that might be considered include the Measuring Attractiveness by a Categorical-Based Evaluation Technique (MACBE), Multi-Attribute Utility Theory (MAUT), Analytic Hierarchy Process (AHP), Analytic Network Process (ANP), and Best-Worst Method (BWM).
In this paper, the authors decided to choose the BWM due to its advantages over other MCDM applications. First, the BWM model considers fewer comparisons than matrix-based methods such as AHP, ANP, or TOPSIS. For example, the BWM model conducts 2n − 3 comparisons, as opposed to (n(n − 1))/2 comparisons in the AHP model. Second, in the BWM model, there are only two vectors for Best to others and Other to worst comparisons. It probably saves time needed for collecting data, calculating, and analyzing [19]. Third, the results provided by the BWM are highly reliable as the redundant comparisons are eliminated. The integer values (i.e., 1-9) scale is used to make pairwise comparisons in BWM, whereas AHP uses a fractional values scale (i.e., 1/9-9). This leads to easy analytical evaluation and interpretation of human perception for applying BWM Fourth, if the Consistency Ratio (CR) is greater than 0.1, AHP requires revision of all the comparisons in the matrix. Instead, it is more data-efficient when the BWM needs to revise only two vectors. In 2015, Rezaei introduced the Best-Worst Method to solve MCDM problems [8]. In BWM, the best and worst criteria are determined initially by Decision-Makers (DMs). Afterward, the DMs use the numeric scale as Table 2 to covert linguistic terms (i.e., from 1 = "Equally important" to 9 = "Extremely Important") and make pairwise comparisons between the best-worst criterion and others. These comparisons become foundations for a mathematical model presented in the next section. The BWM has been applied to a wide variety of sectors such as energy industries, manufacturing, and operation management.

Methodology
The study conducted a model of a two-stage approach combining the Best-Worst Method and Group Best-Worst Method (GBWM) to consider combining all DMs. In the first stage, the BWM is used to determine the local weight for each group of stakeholders, then in the second stage, the GBWM is applied to figure out the most efficient strategy prioritized by all DMs from five groups. In general, the procedure of BWM and GBWM models has the following basic steps (as in Figure 1).

Data Collection
The COVID-19 outbreak has caused many challenges and impacts on the countries that have encountered it. These consequences have been more serious in poor or developing countries which have limited resources. The pandemic has had major effects on Asia-Pacific nations, from territories' economies to health systems. For instance, Vietnam has Step 1: Setting goals and criteria determination The relevant literature is reviewed and the decision criteria are identified.
Step 2: Data collection The authors conduct interviews with experts or decision-makers (i.e., DMs) to collect data on the set of criteria. Afterward, the DMs use the numeric scale in Table 2 to covert linguistic terms (i.e., from 1 = "Equally important" to 9 = "Extremely Important") and make pairwise comparisons between the best-worst criterion and others.
Step 3: Modeling and Estimation The authors estimate the pairwise comparison of preferences between the best-worst criterion and all other criteria. The Best to Others (O B ) and Others to Worst (Ow) preferences are shown in Equations (1) and (2) below.
In which, O Bj and O jW represents the pairwise preference value between best to other criteria j and j to worst criterion, ∀j = 1, 2, . . . , n.
To find the optimal weights (W 1 *,W 2 *, . . . , W n *) for ranking of criteria, the following mathematical program model of GBWM is utilized as equations below.
Min ξ Subject to In which, W B and W W : the weights of the most (best) and the least (worst) important criteria, respectively. D: the set of Decision-maker; j index for criteria; λ k : the weight of decision-maker k W j : the weight of criterion j ξ k : the inconsistency in pairwise comparisons obtained by kth decision-maker Step 4: The optimal values ξ k are used to calculate the consistency ratio for each decisionmaker (CR k ) and group decision making (CR G ) as below In which, ξ * k is the optimal value of inconsistency for the kth decision-maker λ k : the weight given to the kth DM based on expertise level CI: Consistency Index values are given in Table 3, corresponding to the value of O BW as CR G decreases, the consistency increases and if CR G = 0, it implies that the solution is fully consistent.

Data Collection
The COVID-19 outbreak has caused many challenges and impacts on the countries that have encountered it. These consequences have been more serious in poor or developing countries which have limited resources. The pandemic has had major effects on Asia-Pacific nations, from territories' economies to health systems. For instance, Vietnam has experienced a fourth COVID-19 outbreak. The practice of selecting the most optimal strategy is very challenging for decision-makers and attracts high public attention due to the involvement of various criteria and the interests of different stakeholders. In addition, the replication of the pandemic strategies of developed nations in developing countries faces many barriers due to the different capabilities of each country. The practice of prioritizing strategies is necessary for any government, especially in a developing country like Vietnam.
The study implemented the Delphi method to determine the set of criteria for evaluating the efficiency of the intervention. This means that the authors have reviewed relevant literature such as Thunstrom et al. (2020) for the criterion of social distancing [24]; Ranney et al. (2020) for the criterion of Health system [25]; or Kumar et al. (2020) for the criterion of Digital surveillance [26], etc. The description of each criterion is referenced in Table 4. Afterward, the list of criteria is proposed to experts for review. After several rounds of discussion and revision, the list is finalized with nine criteria that decision-makers consider for tackling COVID-19 strategies, as shown in Figure 2. The "Academician" group includes people who are researchers, teaching assistants, and lecturers at two big national universities. The group of "Health Workers" includes doctors and nurses working at a local hospital. The group of "Social Workers" includes staff working at two social clubs and a non-profit organization. The "Entrepreneur" group includes managers and owners of five different companies. The group of common residents includes people living in Ho Chi Minh City who have experienced serious impacts during the COVID-19 outbreak.
In a study evaluating strategies to tackle COVID-19, Naeem et al. (2021) made a questionnaire to conduct a survey for four groups of decision-makers including health workers, academicians, social workers, and common citizens [3]. These groups of stakeholders are all impacted by the COVID-19 outbreak. However, in the present study, the authors consider one more group of entrepreneurs who have been negatively affected by the pandemic. From different backgrounds, these groups of evaluators are expected to give different viewpoints on the various interventions implemented by the government. Therefore, the authors decided to conduct this study with five groups of decision-makers including academicians, health workers, social workers, entrepreneurs, and common residents. The data collection process includes two parts, in which the former is related to demographic information and the impact of COVID-19, and the latter process is making pairwise comparisons among different criteria. The representatives of groups are ap-

CR3.
Controlling prices and ensuring availability of essential items The price and availability of food, groceries, medicines, and other essential products needs to be controlled, because they are an urgent necessity demand during the pandemic, especially for low-income workers.

CR5.
Providing the left information to the public Fake news is a significant barrier in the effort of tackling the pandemic. For instance, people over-stored a larger number of foods than usual due to wrong information on social media. It is important to inform the public about up-to-date reports and figures to assure them.
Ros and Neuwirth (2020) [29] CR6. Accurately measuring the overall impacts The measurement or estimation of impacts is important to allow for pro-active and timely actions. For instance, governments will save time and money in producing and supplying numbers of quarantine facilities, medical oxygen, and financial packages. CR9.

Support to banks and SMEs
Banks are the backbones of any economy and small and medium enterprises (SMEs) are very vulnerable. These sectors suffer greatly during outbreaks. If there are not timely and suitable supports, the country may encounter a recession or financial crisis post-pandemic.
Utz et al. (2020) [32] The authors used flexible online interviewing to collect data from five groups namely academicians, health workers, social workers, entrepreneurs, and common residents.
The "Academician" group includes people who are researchers, teaching assistants, and lecturers at two big national universities. The group of "Health Workers" includes doctors and nurses working at a local hospital. The group of "Social Workers" includes staff working at two social clubs and a non-profit organization. The "Entrepreneur" group includes managers and owners of five different companies. The group of common residents includes people living in Ho Chi Minh City who have experienced serious impacts during the COVID-19 outbreak.
In a study evaluating strategies to tackle COVID-19, Naeem et al. (2021) made a questionnaire to conduct a survey for four groups of decision-makers including health workers, academicians, social workers, and common citizens [3]. These groups of stakeholders are all impacted by the COVID-19 outbreak. However, in the present study, the authors consider one more group of entrepreneurs who have been negatively affected by the pandemic. From different backgrounds, these groups of evaluators are expected to give different viewpoints on the various interventions implemented by the government. Therefore, the authors decided to conduct this study with five groups of decision-makers including academicians, health workers, social workers, entrepreneurs, and common residents.
The data collection process includes two parts, in which the former is related to demographic information and the impact of COVID-19, and the latter process is making pairwise comparisons among different criteria. The representatives of groups are approached and invited to participate through online channels. An online survey is designed with some questions related to their demography as well as the impacts of COVID-19 on them. In the latter, respondents are asked to choose the most and least important and make the relative level between strategies by choosing the most and the least importance on a scale 1-9 as mentioned previously. Finally, the software BWM solver [8] was used to calculate the weights of the criteria for each group of stakeholders.

Results
A total of 43 responses were collected, but we selected 40 responses (i.e., eight decisionmakers for each group) because of incomplete answers. A total of five models as below with D 1 , D 2 , D 3 , D 4 , D 5 , respectively, represent five groups of eight decision-makers, namely Academicians, Entrepreneurs, Commons Residents, Social Workers, and Health Workers. The models use the common symbols defined as below W B and W W : the weights of the most (best) and the least (worst) important criteria, respectively. D: the set of Decision-maker; j index for criteria; λ k : the weight of decision-maker k W j : the weight of criterion j ξ k : the inconsistency in pairwise comparisons obtained by kth decision-maker.

Model for Health Workers
Min ξ Subject to ξ ≥ ξ k ∀k D 5 = {33, 34 The pairwise comparison by Health Workers are presented in Table 9.

Local Weights of Criteria
The results are obtained and ranked in Table 10. These values are local weights implying the important level of each criterion evaluated by each group.
The values of ξ * k , CR k for each DM is obtained to determine the CR G for each group as Table 11. The values of CR k for each DM and CR G for groups are consistent and acceptable. The least value of consistency is for Academicians with CR G = 0.0567, and the most consistent value is for the Social Workers group with CR G = 0.0194.

Global Weights of Criteria
The weights presented in Table 10 are local weights with respect to pairwise comparisons of each group. To suggest an effective strategy, however, the global weights of all criteria need to be estimated. The authors consider the preference of all groups together with a new mathematical model based on GBWM as follows [3]: Min ξ Subject to ξ ≥ (λ 1 /8)ξ k ∀k D 1 = {{1, 2, 3, 4, 5, 6, 7, 8} (48) In which, λ 1 , λ 2 , λ 3 , λ 4 , and λ 5 are the corresponding weights assigned, respectively, to the group of Academicians, Entrepreneurs, Commons Residents, Social Workers, and Health Workers. The sum of all weights equals 1, i.e., λ 1 + λ 2 + λ 3 + λ 4 + λ 5 = 1. To have a more precise analysis, the authors decided to set up four different scenarios with different weights for each group as Table 12. In the Weight Set 1 (WS1), the authors set all weights equally for all groups. However, the weights are varied for each group because of different expertise levels and knowledge about the COVID-19 outbreak. In these weight sets, the Health Workers group is given the highest weightage (i.e., λ 5 = 0.4-0.5) and the Social Workers group is given the second-highest weightage (i.e., λ 4 = 0.2-0.3) due to their expertise concerning the pandemic. Table 13 presents the global weights of strategies obtained when we combine all preferences of all groups together. Across different scenarios with the weight sets varied, the second criteria (i.e., Availability of Health Systems) is evaluated as the most important strategy by all decision-makers. Then, the "Social Distancing" (CR1) and the "Providing the right information to the public" (CR5) are second-most important. The three least important strategies are namely determining a suitable duration for complete lockdown (CR7.), Digital Surveillance (CR8.), and support to banks and SMEs (CR9.).   Table 11. Group Consistency ratio (CR G ).

Discussion and Conclusions
The practice of determining the most prioritized strategies to tackle the COVID-19 outbreak is attracting a great deal of concern among government officials. This is a complex decision with multiple challenges and criteria that directly or indirectly influence the interests of stakeholders. One of the efficiency techniques is the Multiple-Criteria Decision Making model and the Best-Worst Method is chosen to apply for this case due to its advantages in data collection and calculations. The study shows differences in selecting the prioritized actions between developed countries and developing or poor countries. Decision-making has become more challenging for governments of developing nations and Vietnam is an example, especially when the Delta Variant of coronavirus is fast-spreading and threatening all aspects of Vietnam.
In the proposed research model, the authors considered a total of nine strategies applied in many countries around the world. These strategies are entitled Social Distancing (CR1.); Availability of Health Systems (CR2.); Controlling prices and ensuring availability of essential items (CR3.); Financial Support to low-income residents (CR4.); Providing the right information to the public (CR5.); Accurately measuring the overall impacts (CR6.); Determining a suitable duration for complete lockdown (CR7.); Digital surveillance (CR8.); Support to banks and SMEs (CR9.). These strategies are set as nine criteria in the BWM model. The authors have conducted online interviews with five different groups of stakeholders (i.e., Academicians; Entrepreneurs; Common Residents; Social Workers, and Health Workers) to evaluate and rank the most important levels of all criteria. Then, the extended BWM model known as the GBWM model is applied to identify the global weightage and suggest the most efficient strategy.
The results of local weights have shown the most and least important criteria for each group of stakeholders. For the group of Academicians and Health Workers, the Availability of Health Systems (CR2.) is evaluated as the most important criterion; health workers have a better understanding of the impacts of the COVID-19 pandemic on the pressure of the health system than anyone. It is important to ensure the quantity of equipment for treatment and protection during pandemic outbreaks. The group of Entrepreneurs and Social Workers, meanwhile, evaluated Providing the right information to the public (CR5.) as the most important criteria because fake news is also a barrier to their plans and makes people feel puzzled. The criterion Controlling prices and ensuring availability of essential items (CR3.), however, can be seen as the most important to the Common Residents' group. This may be explained by its direct impact on the daily life of residents. If the prices of necessary categories are not controlled, it may cause trouble in the market with supply and demand. Poor people would be significantly influenced. On the other hand, the group of Academicians and Entrepreneurs evaluated Determining a suitable duration for complete lockdown (CR7.) as the least important strategy. A complete lockdown seriously impacts economic activities, and for this reason, Entrepreneurs do not favor this intervention. The Support to banks and SMEs (CR9.) strategy is the lowest priority for the group of Social Workers and Health Workers. For these groups, the health of people and the stability of society are the most important priorities. The group of Common Residents ranked Digital Surveillance (CR8.) as the least important strategy due to its privacy issues.
In the next stage, the authors implemented the GBWM to determine the most efficient and prioritized strategy by combining all decision-makers together. Taking the advantage of considering fewer pairwise comparisons and providing more reliable outcomes from the GBWM model, the global weights of criteria are estimated and ranked across different weightage sets. The results of global weights have shown that the availability of Health Systems (CR2.) is the most prioritized criterion for all decision-makers. This finding is also consistent with the local weight results mentioned above and many previous studies by Nardo et al. (2020) [33]; Naeem (2021) [3] and so on. This is also aligned with the directorgeneral of the World Health Organization's (WHO) argument in the journal Nature: "the provision of universal health care remains a priority . . . Governments, scientists, and the public should support this goal because it's in everyone's best interests." [34]. In addition, the practical utility of results could be useful for governments, particularly in developing nations like Vietnam. Due to the limited resources in such nations, it is important to have optimization strategies in healthcare settings. Such practices increase the availability of Health Systems, the criterion that was shown as the most prioritized intervention in the findings of the present study.
The limitation of this study is its sample of only 40 decision-makers from five different groups that are not representative of the whole of the country. In addition, the categories of decision-maker groups should be more varied such as government representatives; public health scientists, who play key roles in issuing the interventions. This leads to challenges in determining the generalizability of the findings. However, the present study contributes not only to the literature on decision-making with complex combined interests of different stakeholders, but the results also provide useful references to support decisionmakers such as policymakers in prioritizing the strategies for tackling COVID-19 outbreaks. This research framework could be duplicated in any context or any country because the respondents do a simple task by giving comparisons through a numerical scale. Moreover, the proposed model is a good example of BWM and GBWM applications in dealing with a complex problem and minimizing the conflicting interests of different stakeholders. For future studies, this work can be extended by combining other MCDM methods such as AHP, DEA, TOPSIS, and others to provide additional insight into decision-making problems not only in health public sectors but in various fields.