Cross-Entropy Method in Application to the SIRC Model

Abstract

The study considers the usage of a probabilistic optimization method called Cross-Entropy (CE). This is the version of the Monte Carlo method created by Reuven Rubinstein (1997). It was developed in the context of determining rare events. Here we will present the way in which the CE method can be used for problems of optimization of epidemiological models, and more specifically the optimization of the Susceptible–Infectious–Recovered–Cross-immune (SIRC) model based on the functions supervising the care of specific groups in the model. With the help of weighted sampling, an attempt was made to find the fastest and most accurate version of the algorithm.

1. Introduction

In this study our aim is to develop the possibilities of numerically solving variational problems that appear in epidemic dynamics models. The algorithm that we will use for this purpose will be a modified Monte Carlo method. In a few sentences, we recall those stages of the Monte Carlo method development and its improvement, which constitute the essence of the applied approach (cf. Section 1.1 of Martino et al. [1]).

Before constructing the computer, the numerical analysis of deterministic models used approximations that aimed, among other things, at reducing and simplifying the calculations. The ability to perform a significantly larger number of operations in a short time led Ulam (cf. Metropolis [2]) to the idea of transforming a deterministic task to an appropriate stochastic task and applying the numerical analysis using the simulation of random quantities of the equivalent model (cf. Metropolis and Ulam [3]). The problem Ulam was working on was part of a project headed by von Neumann, who accepted the idea, and Metropolis gave it the name of the Monte Carlo Method [1] (in Section 1.1).

The resulting idea allowed one to free oneself from difficult calculations, replacing them with a large number of easier calculations. The problem was the errors that could not be solved by increasing the number of iterations. Work on improving the method consisted of reducing the variance of simulated samples, which led to the development of the Importance Sampling (IS) method in 1956 (cf. Marshall [4]).

Computational algorithms in this direction are a rapidly growing field. Currently, the computing power of computers is already large enough to provide an opportunity to solve problems that cannot be solved analytically. However, some problems and methods still require a lot of power. A lot of research is devoted to finding or improving such methods. The development of this field is currently very important. There are many models describing current problems. Not all offer the possibility of an analytical solution. That is why various studies are appearing to find the best methods for accurate results. the cross-entropy method helps to realize IS and it may, among other things, be an alternative to current methods in problems in epidemiological models.

1.1. Sequential Monte Carlo Methods

The CE method was proposed as an algorithm for the estimation of probabilities of rare events. Modifications of cross-entropy to minimize variance were used for this by Rubinstein [5] in his seminal paper [5] in 1997. This is done by translating the “deterministic” optimization problem in the related “stochastic” optimization and then simulating a rare event. However, to determine this low probability well, the system would need a large sample and a long time to simulate.

1.2. Importance Sampling

Therefore, it was decided to apply the importance sampling (IS), which is a technique used for reducing the variance. The system is simulated using other parameter sets (more precisely, a different probability distribution) that helps increase the likelihood of this occurrence. Optimal parameters, however, are very often difficult to obtain. This is where the important advantage of the CE method comes in handy, which is a simple procedure for estimating these optimal parameters using an advanced simulation theory. Sani and Kroese [6] have used the cross-entropy method as the support of the importance sampling to solve the problem of the optimal epidemic intervention of HIV spread. The idea of his paper will be adopted to treat the Susceptible–Infectious–Recovered–Cross-immune (SIRC) models, which has a wide application to modeling the division of population to four groups of society members based on the resistance to infections (v. modeling of bacterial Meningitis transmission dynamics analyzed by Asamoah et al. [7], Vereen [8]). The result will be compared with an analytical solution of Casagrandi et al. [9].

1.3. Cross-Entropy Method

This part will contain information on the methods used in the paper. The cross-entropy method developed in [5] is one of the versions of the Monte Carlo method developed for problems requiring the estimation of events with low probabilities. It is an alternative approach to combinatorial and continuous, multi-step, optimization tasks. In the field of rare event simulation, this method is used in conjunction with weighted sampling, a known technique of variance reduction, in which the system is simulated as part of another set of parameters, called reference parameters, to increase the likelihood of a rare event. The advantage of the relative entropy method is that it provides a simple and fast procedure for estimating optimal reference parameters in IS.

Cross-entropy is the term from the information theory (v. [10,11] (Chapter 2)). It is a consequence of measuring the distance of the random variables based on the information provided by their observation. The relative entropy, a close idea to the cross-entropy, was first defined by Kullback and Leibler [12] as a measure of the distance $\mathcal{D}\left(p\right|\left|q\right)={\mathbf{E}}_{p}log\frac{p\left(X\right)}{q\left(X\right)}$ (v. [11] ((1.6) on p. 9; Definition on Section 2.3 p. 19), [13]) for two distributions p, q and is known also as Kullback–Leibler distance. This idea was studied in detail by Csiszár [14] and Amari [15]. The application of this distance measure in the Monte Carlo refinements is related to a realization of the importance sampling technique.

The CE algorithm can be seen as a self-learning code covering the following two iterative phases:

• Generating random data samples (trajectories, vectors, etc.) according to a specific random mechanism.
• Updating parameters of the random mechanism based on data to obtain a “better” sample in the next iteration.

Now the main general algorithm behind the cross-entropy method will be presented on the examples.

1.4. Application to Optimization Problems

This section is intended to show how extensive the CE method is. The examples will show the transformation of the optimization problems to the Monte Carlo task of estimation for some expected values. The equivalent MC task uses the CE method. Other very good examples can be found in [16].

The first variational problem presented in this context is the specific multiple stopping model. Consider the so-called secretary problem (v. Ferguson [17] or the second author’s paper [18]) for multiple choice, i.e., the issue of selecting the best proposals from a finite set with at most k attempts. There is a set with N objects numbered from 1 to N. By convention, the object with the number 1 is classified as the best and the object with the number N as the worst. Objects arrive one at a time in random order. We can accept this object or reject it. However, if we reject an object, we cannot return. The task is to find such an optimal stopping rule that the sum of the ranks of all selected objects is the lowest (their value is then the highest). So we strive for a designation

$$\underset{\tau }{inf}\mathbf{E}{X}_{\tau }=\mathbf{E}{X}_{{\tau }^{*}}.$$

Let $a_i$ be the rank of the selected object. The goal is to find a routine for which the value of $\mathbf{E}(a_{{\tau }_1}+\dots +a_{{\tau }_k})$ for $k⩾2$, is the smallest. More details are moved to Appendix A.1.

Next, the problem was transformed into the minimization of the mean of the sum of ranks. The details are presented in Appendix A.2. This problem and its solution by CE was described by Polushina [19]. Its correctness was checked and programmed in a different environment by Stachowiak [20].

The second example is presented in detail in Appendix A.3. It is a formulation of the vehicle routing problem (v. [21,22]). This is an example of stochastic optimization where the cross-entropy is used.

1.5. Goal and Organization of This Paper

In the following parts of the paper, we will focus on models of the dynamics of the spread of infection over time when the population consists of individuals susceptible to infection, sick, immune to vaccination or past infection, and partially susceptible i.e., the population is assigned to four compartments of individuals. Actions taken and their impact on the dynamics of the population are important. It is precisely the analysis of the impact of the preventive diagnosis and treatment that is particularly interesting in the model. How the mathematical model covers such actions is presented in Section 2.1. The model created in this way is then adapted for Monte Carlo analysis in Section 2.2 and the results obtained on this basis are found in Section 2.3. The analysis of computational experiments concludes this discussion in Section 3.

2. Optimization of Control for SIRC Model

Let us focus the attention on the introduction of SIRC model and presentation of the logic behind using the CE method to determine functions that optimize the spread of epidemics. The ideology behind the creation of the SIRC model and its interpretation will be presented in order to match the right parameters for calculating the cost functions.

The CE method here is used to solve the variational deterministic problem. Solving such problems has long been undertaken with the help of numerical methods with a positive result. In the position [23] proposals of the route for variational problems for optimization problems in time are presented. The main focus was on the non-convex problems, which often occur with optimal control problems. In the book by Glowinski [24] a review of the methods for solving variational problems was made. Then [25] presented a complicated method of approximating functions for the problem optimization.

2.1. SIRC Model

The subject of the work is to solve the problem of optimizing the spread of the disease, which can be modeled with the SIRC model. This model was proposed by Casagrandi et al. [9]. Its creation was intended to create a better model that would describe the course of influenza type A. Contrary to appearances, this is an important topic because the disease, although it is widely regarded as weak, is a huge problem for healthcare. In the US alone, the cost associated with the influenza epidemic in the season is estimated to exceed USD 10 billion (v. [26]) and the number of deaths is over 21,000 per season (v. [27]).

Various articles have previously proposed many different mathematical models to describe a pandemic of influenza type A. An overview of such models was made by Earn [28]. In general, it came down to combining SIR models with the use of cross-resistance parameters (cf. details in the papers by Andreasen et al. [29] and Lin et al. [30]). The authors of the article write that the main disadvantage of this approach is the difficulty of the analysis and calculations with a large number of strains. The results of their research showed that the classic SIR model cannot be used to model and study influenza epidemics. Instead, they proposed the extension of the SIR model to a new class C that will simulate a state between a susceptible state and a fully protected state. This extension aims to cover situations where vulnerable carriers are exposed to similar stresses as they had before. As a result, their immune system is stronger when fighting this disease.

The SIRC model divides the community into four groups:

▪

S—persons susceptible to infection, who have not previously had contact with this disease and have no immune defense against this strain

▪

I—people infected with the current disease

▪

R—people who have had this disease and are completely immune to this strain

▪

C—people partially resistant to the current strain (e.g., vaccinated or those who have had a different strain)

Figure 1 contains a general scheme of this model. The four rhombi represent the four compartments of individuals, the movement between the compartments is indicated by the continuous arrows. The main advantage of this model over other SIR users is the fact that after recovering from a given strain, in addition to being completely resistant to this strain, they are also partly immune to a new virus that will appear later. This allows you to model the resistance and response of people in the group to different types of diseases.

Parameters $\alpha ,\delta$ i $\gamma$ can be interpreted as the reciprocal of the average time spent by a person in order of ranges $I,R,C$. Parameters $\mu$ with indices $S,I,R,C$ represent the natural mortality in each group, respectively. In some versions of the model, additional mortality rates are considered for the group of infected people. Here we assume that this factor is not affected by the disease and we will denote it as $\mu$ in later formulas (${\mu }_S={\mu }_I={\mu }_R={\mu }_C=\mu$). The next one $\sigma$ is the likelihood of reinfection of a person who has cross-resistance while the parameter $\beta$ describes the contact indicator.

The SIRC model is represented as a set of four ordinary differential equations. Let $S\left(t\right)$, $I\left(t\right)$ be the number of people in adequate compartments. We have (v. [9])

$$\begin{array}{cc}\hfill \frac{dS}{dt}& =\mu (1-S(t\left)\right)-\beta S\left(t\right)I\left(t\right)+\gamma C\left(t\right),\hfill \\ \hfill \frac{dI}{dt}& =\beta S\left(t\right)I\left(t\right)+\sigma \beta C\left(t\right)I\left(t\right)-(\mu +\alpha )I\left(t\right),\hfill \\ \hfill \frac{dR}{dt}& =(1-\sigma )\beta C\left(t\right)I\left(t\right)+\alpha I\left(t\right)-(\mu +\delta )R\left(t\right),\hfill \\ \hfill \frac{dC}{dt}& =\delta R\left(t\right)-\beta C\left(t\right)I\left(t\right)-(\mu +\gamma )C\left(t\right),\hfill \end{array}$$

(1)

with initial conditions

$$\begin{array}{c}\hfill S\left(0\right)=S_0,I\left(0\right)=I_0,R\left(0\right)=R_0,C\left(0\right)=C_0.\end{array}$$

(2)

2.2. Derivation of Optimization Functions

An approach to optimize this model was proposed by Iacoviello and Stasio [31]. In this article one can find the suggestion concerning performing the calculations as outlined by Zaman et al. [32]. Two functions were proposed in the approach as the parameter or controls. One relates to the susceptible people and the other describes the number of sick people. The method described by Kamien and Schwartz [33] was used to determine the optimal solution. In order to apply it, the set of Equation (1) should be updated accordingly. After these modifications, it has the following form:

$$\begin{array}{cc}\hfill \frac{dS}{dt}& =\mu (1-S)-\beta SI+\gamma C-g\left(S\right(t),u(t\left)\right),\hfill \\ \hfill \frac{dI}{dt}& =\beta S\left(t\right)I\left(t\right)+\sigma \beta C\left(t\right)I\left(t\right)-(\mu +\alpha )I\left(t\right)-h\left(I\right(t),v(t\left)\right),\hfill \\ \hfill \frac{dR}{dt}& =(1-\sigma )\beta C\left(t\right)I\left(t\right)+\alpha I\left(t\right)-(\mu +\delta )R\left(t\right)+g\left(S\right(t),u(t\left)\right)+h\left(I\right(t),v(t\left)\right),\hfill \\ \hfill \frac{dC}{dt}& =\delta R\left(t\right)-\beta C\left(t\right)I\left(t\right)-(\mu +\gamma )C\left(t\right),\hfill \end{array}$$

(3)

where

$$\begin{array}{cc}\hfill g\left(S\right(t),u(t\left)\right)& ={\rho }_{1}S\left(t\right)u\left(t\right),\hfill \\ \hfill h\left(I\right(t),v(t\left)\right)& ={\rho }_{2}I\left(t\right)v\left(t\right),\hfill \end{array}$$

(4)

where $u\left(t\right)$ represent the percentage of those susceptible who have been taken care of thanks to using control on population and $v\left(t\right)$ represent the percentage of infected people with the same description, respectively. ${\rho }_1$ and ${\rho }_2$ are weights that optimize the proportion of given control options. Both functions $g\left(S\right(t),u(t\left)\right)$$h\left(I\right(t),v(t\left)\right)$ and in order they can be interpreted as actions performed on people susceptible to the disease and infected. In addition, two new conditions related to optimization functions are added to the initial conditions.

$$\begin{array}{c}\hfill u_{min}⩽u\left(t\right)⩽u_{max},v_{min}⩽v\left(t\right)⩽v_{max}.\end{array}$$

(5)

They describe the limits on what part of the population care can be given to. The smallest values $u_{min}$ and $v_{min}$ is zero. The existence of the solution is shown by Iacoviello et al. [31] and they are functions that minimize the following cost index:

$$J(u,v)={\int }_{t_1}^{t_2}\phi (S\left(t\right),I\left(t\right),u\left(t\right),v\left(t\right))dt={\int }_{t_1}^{t_2}\left[{\alpha }_{1}S\left(t\right)+{\alpha }_{2}I\left(t\right)+\frac{1}{2}{\tau }_1{u}^2\left(t\right)+\frac{1}{2}{\tau }_2{v}^2\left(t\right)\right]dt,$$

(6)

${\alpha }_1,{\alpha }_2$ are used to maintain the balance in the susceptible and infected group. ${\tau }_1,{\tau }_2$ is interpreted as weighting in the cost index. $t_1,t_2$ determine the time interval. The square next to the functions $u\left(t\right)$ and $v\left(t\right)$ indicates increasing intensity of functions [32]. Thus the function $J(u,v)$ reflects the human value of susceptible and infected persons, taking into account the growing value of funds used over a specified period of time.

The objective function can be changed as long as there is a minimum. The CE method does not impose any restrictions here. However, to compare the results with [31], the function proposed by them is used also here. There, the objective function must be square in order for the quadratic programming techniques to be used. Usually, the quadratic function is good in mathematics, but not in practice. Changing the objective function and impact of it can be considered further.

2.3. Optimization of the Epistemological Model by the CE Method

Sani and Kroese [6] present the way in which the cross-entropy method can be used to solve the optimization of the epistemological model consisting of ordinary differential equations. The main task was to minimize the objective function $J\left(u\right)$ depending on one optimization function $u\left(t\right)$ over a certain set U consisting of continuous functions u. The minimum can be saved as:

$${\gamma }^{*}=J\left(u^{*}\right)=\underset{u\in U}{min}J\left(u\right)$$

(7)

Parameterizing the minimum problem looks as follows:

$$\underset{c\in C}{min}J\left(u_c\right),$$

(8)

where $u_c$ is a function from U, which is parameterized by a certain control vector $c\in {\mathbb{R}}^m$. Collection C is a set of vectors c. It should be noted that the selected set C should be big enough to get the enough precise solution. Then in such a set there are such $c^{*}$—which will be the optimal control vector and ${\gamma }^{*}$—optimal value, respectively.

One way to parameterize a problem is to divide the time interval $[t_1,t_2]$ at small intervals and using these intervals together with control vector points c to define a function $u_c$. This function can be created by interpolating between points. Such interpolation can be done using e.g., the finite element method, finite difference method, finite volume method or cubic B-Spline. Fang et al. [34] and Caglar et al. [35] show that all of this methods can be used to the two-point boundary value problems. Here the FEM method was used (v. [36]). The results for the other methods have not been checked. The choice of the approximation method is a difficult and interesting issue, but this is not what is consider here. With this function $u_c$, it takes form

$$u_c\left(t\right)=\sum _{t=0}^n{c}_i{k}_i\left(t\right)$$

(9)

where

$$k_i\left(t\right)=\left\{\begin{array}{cc}\frac{t-t_{i-1}}{t_i-t_{i-1}}\hfill & \mathrm{for}t\in [t_{i-1},t_i]\hfill \\ \frac{t_{i+1}-t}{t_{i+1}-t_i}\hfill & \mathrm{for}t\in [t_i,t_{i+1}]\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(10)

Now when $u_c$ is represented as a parameterized function, the value can be counted $J\left(u_c\right)$ by solving the system of ordinary differential Equation (3) e.g., by the Runge–Kutta method. The idea of using the CE method in this problem is to use a multi-level algorithm with generated checkpoints $c_i$, here, from a normal distribution. The CE method does not impose the distribution. Usually it is chosen from the family of density which is expected. Here, it is expected that there will be one high peak at the peak of the epidemic. Hence the normal distribution. Then, update the distribution parameters using the CE procedure using the target indicator as a condition for selecting the best samples. The calculations are carried out until the empirical variance of optimal control function is smaller than the given $ϵ$. This means that the function values are close to the optimal expected value The idea of applying the CE method to the SIRC model is similar. The only difficulty is the fact that when optimizing the SIRC model there are two functions instead of one as in the case of the algorithm described here. In the form of an Algorithm 1, it looks as follows:

Algorithm 1: (Modification of the Sani and Kroese’s algorthm (v. [6]) to two optimal functions case):

1.  Initialize: Choose ${\mu }_0^{\left(u\right)}=\{{\mu }_{i0},i=0,\dots ,n\}$, ${\sigma }_0^{\left(u\right)}=\{{\sigma }_{i0},i=0,\dots ,n\}$, ${\mu }_0^{\left(v\right)}=\{{\mu }_{i0},i=0,\dots ,n\}$ and ${\sigma }_0^{\left(v\right)}=\{{\sigma }_{i0},i=0,\dots ,n\}$, set $k=1$  2.  Draw: Generate a random samples $C_1,\dots ,C_N\sim N({\mu }_{k-1}^{\left(u\right)},{{\sigma }_{k-1}^{\left(u\right)}}^2)$ and $D_1,\dots ,D_N\sim N({\mu }_{k-1}^{\left(v\right)},{{\sigma }_{k-1}^{\left(v\right)}}^2)$ with $C_m=\{C_{mi},i=0,\dots ,n\}$ and $D_l=\{D_{li},i=0,\dots ,n\}$.  3.  Evaluate: For each control vector $C_m$ and $D_m$ evaluate the objective function $J(u_{C_m},u_{D_l})$ by solving the ODE system  4.  Find the p best performing samples, based on the values $\left\{J\left(u_{C_m,D_l}\right)\right\}$. Let I be the corresponding set of indices.  5.  Update: for all $i=0,\dots ,n$ let $$\begin{array}{c}\hfill {\widehat{{\mu }_{ki}}}^{\left(u\right)}=\frac{1}{p}\sum _{m\in I}^{}{C}_{mi};{\left({\widehat{{\sigma }_{ki}}}^{\left(u\right)}\right)}^2=\frac{1}{p}\sum _{m\in I}^{}(C_{mi}-{\mu }_{ki}^{\left(u\right)}),\\ \hfill {\widehat{{\mu }_{ki}}}^{\left(v\right)}=\frac{1}{p}\sum _{l\in I}^{}{D}_{li};{\left({\widehat{{\sigma }_{ki}}}^{\left(v\right)}\right)}^2=\frac{1}{p}\sum _{l\in I}^{}(D_{li}-{\mu }_{ki}^{\left(v\right)}).\end{array}$$  6.  Smooth: For a fixed smoothing parameter $0<\alpha ⩽1$ let $$\begin{array}{ccc}\hfill {\widehat{{\mu }_k}}^{\left(u\right)}& =\alpha {\mu }_k^{\left(u\right)}+(1-\alpha ){\widehat{{\mu }_{k-1}}}^{\left(u\right)};{\widehat{{\sigma }_k}}^{\left(u\right)}\hfill & \hfill =\alpha {\sigma }_k^{\left(u\right)}+(1-\alpha ){\widehat{{\sigma }_{k-1}}}^{\left(u\right)}\\ \hfill {\widehat{{\mu }_k}}^{\left(v\right)}& =\alpha {\mu }_k^{\left(v\right)}+(1-\alpha ){\widehat{{\mu }_{k-1}}}^{\left(v\right)};{\widehat{{\sigma }_k}}^{\left(v\right)}\hfill & \hfill =\alpha {\sigma }_k^{\left(v\right)}+(1-\alpha ){\widehat{{\sigma }_{k-1}}}^{\left(v\right)}\end{array}$$  7.  Repeat 2–6 until $ma{x}_i{\sigma }_{ki}^{\left(u\right)}<ϵ$ and $ma{x}_i{\sigma }_{ki}^{\left(v\right)}<ϵ$ with $ϵ={10}^{-5}$ . Let L be the final iteration number. Return ${\mu }_L$ as an estimate of the optimal control parameter $c^{*}$

The value of the criterion $J(·,·)$ is calculated in steps 3 and 4. The selection of the function $\phi$ allows us to fit the cost function to the modeled case. Here, the criterion given by (6) is adopted.

3. Description of the Numerical Results

This section will present the results obtained using the cross-entropy method. For the correct comparison the same parameters for the SIRC model were used. These values are:

$$\begin{array}{cccccc}\hfill \mu & =\frac{1}{75}\left[{\mathrm{year}}^{-1}\right],\hfill & \hfill \gamma & =\frac{1}{2}\left[{\mathrm{year}}^{-1}\right],\hfill & \hfill \alpha & =\frac{365}{5}\left[{\mathrm{year}}^{-1}\right],\hfill \\ \hfill \delta & =1\left[{\mathrm{year}}^{-1}\right],\hfill & \hfill \beta & \approx 146\left[{\mathrm{year}}^{-1}\right],\hfill & \hfill \sigma & \approx 0.078.\hfill \end{array}$$

For optimization functions $u\left(t\right)$ and $v\left(t\right)$ the following restrictions have been applied:

$$0⩽u\left(t\right)⩽0.9,0⩽v\left(t\right)⩽0.9.$$

Restrictions were proposed by Lenhart and Workman [37]). Their value was explained by the fact that the whole group cannot be controlled. In addition, the weights used for functions have values ${\rho }_1=2$ and ${\rho }_2=2$

Now, all that’s left is to propose variable values for the objective function:

$${\alpha }_1={10}^{-3},{\alpha }_2=0.997,{\tau }_1={10}^{-3},{\tau }_2={10}^{-3}.$$

The parameter values are adjusted using historical data like parameter mortality rate $\mu$, which is counted as average lifetime of a host or parameters $\alpha ,\delta ,\gamma$ mean the inverted time of belonging to the group $I,R$ and C. A detailed description of determining parameters along with their limitations is presented in the article by Casagrandi et al. [9].

3.1. A Remark about Adjusting the Parameters of the Control Determination Procedure

Here these parameters are given and describe the influenza A epidemics well, because of long historical data sets. However, this may not be the case, and then these parameters are subject to adjustment in the control determination procedure. An overview of applications and accuracy of calibration methods, which can be used, is presented in the article by Hazelbag et al. [38], which provides an overview of the model calibration methods. All parameters in the model are constant and independent of time, so methods that try to optimize a goodness-of-fit (GOF) can be used for this example. GOF is a measure that assesses consistency between model the output and goals. As a result, it gives the best combination of parameters. Examples of such methods are grid search, Nelder–Mead method, the iterative, descent-guided optimization algorithm, sampling from a tolerable range. After finding the appropriate parameters, other algorithms like the profile likelihood method or Fisher information can be used to calculate the confidence intervals for these coefficients. If the epidemic described by the model consists of transition probabilities that cannot be estimated from currently available data, calibrations can be performed to many end points. Then GOF is measured as the mean percentage deviation of the values obtained at the endpoints (v. [39]).

3.2. Proposed Optimization Methods in the Model Analysis

The results will be presented for two moments in the model: at the beginning and in the middle of the epidemic. This is initialized with other initial parameters. For the beginning of the epidemic, they are as follows:

$$S_0=1-I_0,I_0={10}^{-6},R_0=0,C_0=0.$$

For the widespread epidemic:

$$S_0=0.99,I_0=5\ast {10}^{-3},R_0=3\ast {10}^{-3},C_0=2\ast {10}^{-3}.$$

The values in the model have been normalized for the entire population, and they add up to 1. The results for both beginning (v. Figure 2a) and during (v. Figure 2b) the epidemic were obtained using the Runge-Kutty method and shown here. The cost index in this case was in order $0.00799$ and $0.00789$ for the model of the beginning and the development of an epidemic.

In Section 3.3 and Section 3.4 two ways of calculating optimal functions $u\left(t\right)$ and $v\left(t\right)$ for the SIRC model will be considered. In the first version, the functions will be calculated separately. In the following, both $u\left(t\right)$ and $v\left(t\right)$ will be counted at the same time. All versions will be presented at two moments of the epidemic: when it began and when it has already spread. The results will be compared with those obtained in the article by Iacoviello et al. [31], where the problem was solved using the sequential quadratic programming method using a tool from Matlab. Unfortunately, the article does not specify the cost index value for solving the obtained sequential quadratic programming method. Therefore, in the paper, there were attempts to recreate the form of the function $u\left(t\right)$ and $v\left(t\right)$ and the cost index obtained for them ($0.003308$ for the first situation and $0.006489$ for the second one) and the model solution.

As can be seen in Figure 3 in the first situation, the control functions helped reduce peak infection, which was previously (without control Figure 2a) occurred between 2 and 4 months. The largest peak was around $20\%$ occurred between 2 and 4 months. Previously, the largest peak was around $10\%$. In the case of the second situation, the peak also decreased, but not very clearly (it is also visible in the value of the cost index, which for the first situation is $0.003308$, and for the other one $0.006489$). It can be seen that timely control is also very important to reducing the harmful effects of an epidemic. For more conclusions on how to apply control properly in this model, see [31].

3.3. CE Method Version 1

In the first method, the algorithm is shown at the end of Section 2.3 has been applied twice. Once for the calculation of the optimal function $u\left(t\right)$ with a fixed $v\left(t\right)$, and then $v\left(t\right)$ with a fixed $u\left(t\right)$. In the next step, the results were combined and the result for the SIRC model was calculated using these two functions. This approach is possible due to the lack of a combined restriction on $u\left(t\right)$ and $v\left(t\right)$. Features are not dependent on each other with resources.

The cost index was $0.003086$ for the start of the epidemic and $0.006443$ for a developed epidemic. The Figure 4 show the graphs of functions $u\left(t\right)$ and $v\left(t\right)$ along with a graph of solutions for both moments during the epidemic.

For functions $u\left(t\right)$ the results are similar, only the decrease here is more linear. However, the solution of the SIRC model gives the same results. Function $v\left(t\right)$ is much more interesting. The CE method showed that it should have very low values, even close to 0 throughout the entire period in both cases. However, this does not affect the cost index (it is even smaller than in the previous case) and the SIRC model. More on this subject will be included in the summary.

3.4. CE Method Version 2

In the second case, it was necessary to change slightly the algorithm from Section 2.3 to be able to use it for two control functions. In point 1 of the algorithm a variable is generated $C_m\sim N({\mu }_k,{\sigma }_k^2)$, which serves to designate $u_{C_m}$, where ${\mu }_k,{\sigma }_k$ are requested by the user. Here, 2 $\mu$ will need to be initialized ${\mu }_k^{\left(u\right)},{\mu }_k^{\left(v\right)}$ and 2 $\sigma$: ${\sigma }_k^{\left(u\right)},{\sigma }_k^{\left(v\right)}$. Then designate $C_1^{\left(u\right)},\dots ,C_N^{\left(u\right)}\sim N({\mu }_k^{\left(u\right)},{\sigma }_k^{\left(u\right)2})$ and similarly $C_1^{\left(v\right)},\dots ,C_N^{\left(v\right)}\sim N({\mu }_k^{\left(v\right)},{\sigma }_k^{\left(v\right)2})$. Now $J(u,v)$ can be count without problems. Further the algorithm goes the same way, and in point 6, all four parameters ${\mu }_k^{\left(u\right)},{\mu }_k^{\left(v\right)},{\sigma }_k^{\left(u\right)},{\sigma }_k^{\left(v\right)}$ are updated. The Figure 5 presents these functions.

In this case, we have very similar results as in the previous section. This can be seen in both the charts and the cost index values ($0.003044$ and $0.006451$). The only problem here is the need for a large sample to bring the results closer.

3.5. Comparison of Results

This section gathers the results from each version and compares them with each other. At first, the behavior of each group of the SIRC model was looked at and compared with the solution of the uncontrolled model. This is shown in Figure 6, Figure 7, Figure 8 and Figure 9. The results are presented only for the first situation, where the epidemic begins to spread, because there you can see the results of using control much better.

The most interesting is Figure 7 comparing infected people. You can see here very well how functions $u\left(t\right)$ and $v\left(t\right)$ help in the fight against the epidemic. Let two versions of the CE method overlap. The functions obtained by the sequential quadratic programming method give slightly better results. However, they have a worse cost index (the comparison is presented in

Table 1. Summary of cost indexes and times for all situations.

Method	Cost Index	Time
Results without control functions	0.00799	-
	0.00789
Sequential quadratic programming method	0.003308	-
	0.006489
Cross-entropy method version 1	0.003086	30 s
	0.006443
Cross-entropy method version 2	0.003044	4 min 54 s
	0.006451

). This is probably due to the high value of the function $v\left(t\right)$, which increases the cost index. The question remains why there is such a discrepancy between the obtained function values $v\left(t\right)$ with both methods. In this article [31] weight and impact of functions $u\left(t\right)$ and $v\left(t\right)$ were compared. The chart in the article compares the results obtained for the application of both functions, one at a time and none. It turned out that the main influence on the number of infected has control over susceptible persons, i.e., the function $u\left(t\right)$. Choosing the optimal values for $v\left(t\right)$ is not very important for the solution of the model. Therefore, such discrepancies are possible when choosing optimal values.

Table 1 contains a comparison of the cost index values and the duration of the algorithms for each method. The first value is when the epidemic started, and the second is when it has already developed. The article does not specify the algorithm time of the method used.

It can be seen that for each version the cost index values came out very similar. The differences can be seen in time. This is because in the first version of the CE method a smaller sample was simulated because both functions were considered separately. Fortunately, the functions were not closely related and it was possible to consider them separately.

4. Summary

The study examined the possibility of using the CE method to determine the optimal control in the SIRC model. Similar models are being developed to model the propagation of malicious software in computer networks (v. Taynitskiy et al. [40]). However, the number of works with this approach is still relatively small, although its universality should encourage checking its effectiveness. There are two control functions in the model, which, when properly selected, optimize the cost function. The CE method was used in two versions: considering the two functions separately and together. The results were then compared with those obtained by Iacoviello and Stasio [31] using routing of Matlab (Matlab is a paid environment with a number of functions. Unlike other environments and programming languages, additional features are created by developers in a closed environment and can only be obtained through purchase.). The Cross-Entropy method had little problems when considering two functions at the same time. Due to the large number of possibilities, it was necessary to simulate a large sample, which significantly extended the algorithm’s time. The second way came out much better. However, it only worked because the functions were not dependent on each other. In the opposite situation, there is also a possibility to get the result via CE results however, it will be necessary to change the algorithm and the dependence of how the functions depend on each other and add this to the algorithm when initiating optimization functions in the code. The same situation applies to changing the objective function and the density used for approximation. Changing the objective function gives the possibility to use other methods. If it stays quadratic and the constraints are linear, quadratic programming techniques like the sequential quadratic programming method can still be used. If the objective function and the constraints stay convex, we use general methods from convex optimization. The CE method is based on well-known and simply classical rules and is therefore quite problem-independent. There are no rigid formulas, and therefore it requires consideration for each problem separately. So it is very possible to reconsider it in another way, which can be interesting for further work. The CE method may also facilitate the analysis of modifications of epidemiological models related to virus mutation or delayed expression (v. Gubar et al. [41], Kochańczyk et al. [42]).