Parameter Learning of Bayesian Network with Multiplicative Synergistic Constraints

Abstract

Learning the conditional probability table (CPT) parameters of Bayesian networks (BNs) is a key challenge in real-world decision support applications, especially when there are limited data available. The traditional approach to this challenge is introducing domain knowledge/expert judgments that are encoded as qualitative parameter constraints. In this paper, we focus on multiplicative synergistic constraints. The negative multiplicative synergy constraint and positive multiplicative synergy constraint in this paper are symmetric. In order to integrate multiplicative synergistic constraints into the learning process of Bayesian Network parameters, we propose four methods to deal with the multiplicative synergistic constraints based on the idea of classical isotonic regression algorithm. The four methods are simulated by using the lawn moist model and Asia network, and we compared them with the maximum likelihood estimation (MLE) algorithm. Simulation results show that the proposed methods are superior to the MLE algorithm in the accuracy of parameter learning, which can improve the results of the MLE algorithm to obtain more accurate estimators of the parameters. The proposed methods can reduce the dependence of parameter learning on expert experiences. Combining these constraint methods with Bayesian estimation can improve the accuracy of parameter learning under small sample conditions.

1. Introduction

Bayesian networks (BNs) can model probabilistic dependent relationships among variables in many real world problems. Therefore, they have become very popular in the artificial intelligence (AI) field over the last two decades. BNs have become a powerful tool with many applications such as medical diagnosis, financial analysis, bioinformatics, medical diagnosis, financial analysis, bioinformatics and industrial applications [1], target tracking [2], robot control [3], gene analysis [4], ecosystem modeling [5], signal processing [6], and educational measurement [7]. A BN model consists of a network structure and a set of conditional probability tables (CPTs). This paper focuses on the parameter learning of discrete BNs when the structure is known.

In practice, when performing parameter learning, we need sufficient samples. If we have sufficient data, BNs can be easily constructed using traditional methods such as the maximum likelihood (ML) method [8]. However, the ML method is difficult to obtain accurate parameters when the data is insufficient, and thus it is difficult to give the right decision [9]. It is very difficult to collect abundant data under certain circumstances, such as, in the cases of earthquake prediction [10], parole assessment [11], and rare disease diagnosis [12]. Thus, the data are not sufficient in many cases, which may lead to inaccurate structure and parameters of a BN. Therefore, many scholars began to pay attention to the parameter learning problems of BNs under the condition of small data sets, and proposed some algorithms to solve these problems.

Altendorf et al. [13] converted the qualitative influence constraints to penalty function and gave the objective optimization function combined with the ML function, and then applied the gradient method to solve it. Feelders and Linda [14] converted qualitative influence constraints to order constraints and applied isotonic regression algorithm to adjust the size of parameters so that the parameters satisfy order constraints. Cassio and Ji [15] converted monotonic constraints to penalty function and applied a convex optimization algorithm to solve the objective optimization function. Ren et al. [16] transformed interval constraints into beta distribution to constrain priori parameters, and combined Bayesian estimation to obtain the parameters. Niculescu [17] studied some equality constraints such as normative constraint and proportional constraint by introducing Lagrange multiplier. Rui [18] used Monte Carlo sampling method to extract virtual data from non-monotonic constraints space, and then used virtual data to construct prior distribution. Finally, Bayesian estimation was used to combine prior distribution with real data to obtain the parameters of BNs. Kobra [19] proposed a multi-experts parameter learning framework to fuse multiple experts’ knowledge. In addition to the above methods based on constraints, some scholars proposed the parameter learning methods of BN based on the minimum free energy (see [20]) and Noisy- or -Gates (see [21]). Zhou et al. [22] studied a class of constraints that is naturally encoded in the edges of BNs with monotonic influences. Gao et al. [23] developed “MiniMax Fitness” algorithm to address the problem that imposing prior distributions can reduce the fitness between parameters and data. For more studies on BNs, one can refer to [24,25,26,27].

It is found that the main idea of the algorithms is to introduce expert experiences or domain knowledge into the parameter learning process of BNs with some constraints in existing literature. However, the constraints involved in the existing literature are mainly network parameters under the condition of single parent node, and the constraints under the synergistic condition of multiple parent nodes are rarely studied, and the methods involved are relatively complex. There are many constraints under the condition of multiple parents nodes such as additive synergy, multiplicative synergy (see [28]), etc. In this paper, isotonic regression is used to study the parameter learning of BNs under multiplicative synergistic constrains. The proposed methods can reduce the dependence of parameter learning on expert experiences. Combining these constraint methods with Bayesian estimation can improve the accuracy of parameter learning under small sample conditions.

The reminder of this paper is organized as follows. In Section 2, we briefly review some basic theories of BNs and parameter learning. In Section 3, the classical isotonic regression algorithm is introduced. In Section 4, we provide the parameter learning algorithms of this paper under multiplicative synergistic constrains by referring to the idea of pool adjacent violators (PAV) algorithm. In Section 5, the effectiveness and performance of four mentioned algorithms are verified by simulations. In Section 6, some conclusions are given.

2. Preliminaries

In this section, some concepts of Bayesian networks will be briefly reviewed, so that one can understand the paper well.

2.1. Bayesian Networks

Bayesian networks are represented as a directed acyclic graph that contains some nodes and edges. Nodes represent random variables, while edges represent the probabilistic relationship between random variables. For each variable node $X_i$, a conditional probability table is specified as $P\left(X_i\left|\pi \left(X_i\right)\right.\right)$, which describes the probability over the possible values of $X_i$ and possible configurations of parent variables $\pi \left(X_i\right)$. In a BN, the joint probability can be written as follows:

$$P\left(X_1,\cdots ,X_n\right)={\displaystyle \prod _{i=1}^{n}P\left(X_i\left|\pi \left(X_i\right)\right.\right)}.$$

To illustrate the BNs more clearly, the lawn moist BN is employed, which is depicted as Figure 1, where “1” stands for “Cloudy”, “2” stands for “Rain”, “3” stands for “Sprinkle”, and “4” stands for “Wet”. The model will be used as the experimental model later. The variables in the network are binary, the value is 0 or 1, that is, if the event occurs or is present, it has the state 1. Our purpose is to learn the estimators of parameters in the following BN.

Figure 1. Lawn moist model.

2.2. Maximum Likelihood Estimation

Maximum likelihood estimation is an important method for parameters learning of BN. The MLE of the parameters is

$${\theta }_{ijk}^{*}=\left\{\begin{array}{ll}\frac{m_{ijl}}{m_{ijk}}& {\displaystyle \sum _{k=1}^{r_i}{m}_{ijk}>0}\\ \frac{1}{r_i}& {\displaystyle \sum _{k=1}^{r_i}{m}_{ijk}=0}\end{array}\right.,$$

where $i$ is the index of node $X_i$, $j$ is the index of parent nodes’ configuration, $k$ is $X_i^{\prime }s$ states, $r_i$ is the number of the states of $X_i$, and $m_{ijk}$ is the number of cases that satisfy $X_i=k$ and $\pi \left(X_i\right)=j$ in the data set (see [1]).

3. Isotonic Regression

We know that the variables satisfy order relation, but the order obtained by observation or counting does not satisfy the known order relation, then the data will be adjusted by the weighted average method. This is the problem to be solved by the isotonic regression. Let $u_1,u_2\dots ,u_k$ be a set of variables, $u_1\le u_2\le \dots \le u_k$, and ${u^{\prime }}_1,{u^{\prime }}_2\dots ,{u^{\prime }}_k$ are estimators. The process of using pool adjacent violators (PAV) algorithm to isotonic regression is as follows:

• Step 1: Start with ${u^{\prime }}_1$, and compare ${u^{\prime }}_1,{u^{\prime }}_2\dots ,{u^{\prime }}_k$ in pairs, if $u_i\le u_{i+1}$, there is no adjusment, if $u_i<u_{i+1}<\dots <u_{i+j}$, $\left(1\le j\le k-i\right)$, let

  $$u_{i+1}=u_{i+2}=\dots =u_{i+j}=\frac{1}{j+1}{\displaystyle \sum _{m=0}^j{u^{\prime }}_{i+m}},$$

  and so on.
• Step 2: If ${u^{\prime }}_1,{u^{\prime }}_2\dots ,{u^{\prime }}_k$ do not satisfy the size relationship of variables in step 1, repeat the process in step 1 from ${u^{\prime }}_1$. Thus, the values of ${u^{\prime }}_1,{u^{\prime }}_2\dots ,{u^{\prime }}_k$ adjusted by PAV algorithm can be obtained.

The uniqueness of the solution of isotonic regression has been verified (see [29]) and the solutions obtained by the above process that satisfying the order relation of $u_1\le u_2\le \dots \le u_k$ are unique.

4. Parameter Learning of BNs under Multiplicative Synergistic Constraints

4.1. The Model of Multiplicative Synergistic

Multiplicative synergy constraints describe the synergy size relationship of parameters among three node variables. Let $A$, $B$ and $C$ be the three variables in the BNs, and all of them are discrete binary variables. Suppose that $A$ and $B$ are the parent nodes of $C$, then the negative multiplicative synergy constraint of $A$ and $B$ to $C$ can be expressed as:

$$P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)\le P\left(c\left|\overline{a},b\right.\right)\cdot P\left(c\left|a,\overline{b}\right.\right).$$

Similarly, positive multiplicative synergy constraint can be expressed as:

$$P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)\ge P\left(c\left|\overline{a},b\right.\right)\cdot P\left(c\left|a,\overline{b}\right.\right).$$

4.2. Description of Algorithm

Under the condition of small data sets, if the variables $A$, $B$ and $C$ in the network conform to the above multiplicative synergy constraint, how to apply this constraint and combine the small data sets to limit the Bayesian network parameter learning and get more accurate results is very important.

The algorithm steps adopted in this paper are as follows:

• Step 1: Using the existing small data sets and by the maximum likelihood estimation algorithm to get relevant parameters.
• Step 2: For the structural part of multiple parent nodes, judge whether the parameters of this part meet the multiplicative synergy constraint. If they meet, conform to step 5, otherwise conform to step 3.
• Step 3: Take the left and right sides of multiplicative synergy constraint as a whole, and use the idea of “averaging” of PAV algorithm to modify them, respectively.
• Step 4: Adjust the whole (product of parameters) in step 3, and then modify each parameter.
• Step 5: Obtain the final parameter learning result.

Steps 3 and 4 are the focus of the algorithm, which are described in detail below. By referring to the idea of PAV algorithm, this paper proposes four different methods to complete the order preserving of parameters and the contents mentioned in steps 3 and 4. Taking the negative multiplicative synergy constraint mentioned in Section 4.1 as an example, the following specific calculation methods are given successively. Parameters order preserving under the condition of positive multiplicative synergy can be obtained in a similar way. The proposed four methods are as follows:

Method 1

$$mean=\left(P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)+P\left(c\left|\overline{a},b\right.\right)\cdot P\left(c\left|a,\overline{b}\right.\right)\right)/2$$

$$d=\left|mean-P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)\right|$$

$$\begin{array}{l}P{\left(c\left|a,b\right.\right)}^{\prime }=P\left(c\left|a,b\right.\right)-d/2\\ P{\left(c\left|\overline{a},\overline{b}\right.\right)}^{\prime }=P\left(c\left|\overline{a},\overline{b}\right.\right)-d/2\\ P{\left(c\left|\overline{a},b\right.\right)}^{\prime }=P\left(c\left|\overline{a},b\right.\right)+d/2\\ P{\left(c\left|a,\overline{b}\right.\right)}^{\prime }=P\left(c\left|a,\overline{b}\right.\right)+d/2\end{array}$$

Method 2

$$mean=\frac{\left(\left(N_1\cdot N_4\right)\cdot \left(P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)\right)+\left(N_2\cdot N_3\right)\cdot \left(P\left(c\left|\overline{a},b\right.\right)\cdot P\left(c\left|a,\overline{b}\right.\right)\right)\right)}{N_1\cdot N_2\cdot N_3\cdot N_4}$$

$$d=\left|mean-P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)\right|$$

$$P{\left(c\left|a,b\right.\right)}^{\prime }=P\left(c\left|a,b\right.\right)-d\cdot N_1/\left(N_1\cdot N_4\right)$$

$$P{\left(c\left|\overline{a},\overline{b}\right.\right)}^{\prime }=P\left(c\left|\overline{a},\overline{b}\right.\right)-d\cdot N_4/\left(N_1\cdot N_4\right)$$

$$P{\left(c\left|\overline{a},b\right.\right)}^{\prime }=P\left(c\left|\overline{a},b\right.\right)+d\cdot N_2/\left(N_2\cdot N_3\right)$$

$$P{\left(c\left|a,\overline{b}\right.\right)}^{\prime }=P\left(c\left|a,\overline{b}\right.\right)+d\cdot N_3/\left(N_2\cdot N_3\right)$$

Method 3

$$mean=\left(P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)+P\left(c\left|\overline{a},b\right.\right)\cdot P\left(c\left|a,\overline{b}\right.\right)\right)/2$$

$$d=\left|mean-P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)\right|$$

$$P{\left(c\left|a,b\right.\right)}^{\prime }=P\left(c\left|a,b\right.\right)-d\cdot N_1/\left(N_1\cdot N_4\right)$$

$$P{\left(c\left|\overline{a},\overline{b}\right.\right)}^{\prime }=P\left(c\left|\overline{a},\overline{b}\right.\right)-d\cdot N_4/\left(N_1\cdot N_4\right)$$

$$P{\left(c\left|\overline{a},b\right.\right)}^{\prime }=P\left(c\left|\overline{a},b\right.\right)+d\cdot N_2/\left(N_2\cdot N_3\right)$$

$$P{\left(c\left|a,\overline{b}\right.\right)}^{\prime }=P\left(c\left|a,\overline{b}\right.\right)+d\cdot N_3/\left(N_2\cdot N_3\right)$$

Method 4

$$mean=\frac{\left(\left(N_1\cdot N_4\right)\cdot \left(P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)\right)+\left(N_2\cdot N_3\right)\cdot \left(P\left(c\left|\overline{a},b\right.\right)\cdot P\left(c\left|a,\overline{b}\right.\right)\right)\right)}{N_1\cdot N_2\cdot N_3\cdot N_4}$$

$$d=\left|mean-P\left(c\left|a,b\right.\right)\cdot P\left(c\left|\overline{a},\overline{b}\right.\right)\right|$$

$$\begin{array}{l}P{\left(c\left|a,b\right.\right)}^{\prime }=P\left(c\left|a,b\right.\right)-d/2\\ P{\left(c\left|\overline{a},\overline{b}\right.\right)}^{\prime }=P\left(c\left|\overline{a},\overline{b}\right.\right)-d/2\\ P{\left(c\left|\overline{a},b\right.\right)}^{\prime }=P\left(c\left|\overline{a},b\right.\right)+d/2\\ P{\left(c\left|a,\overline{b}\right.\right)}^{\prime }=P\left(c\left|a,\overline{b}\right.\right)+d/2\end{array}$$

where $N_1$ stands for the sample size when the state of parent nodes is $\left(a,b\right)$, $N_2$ stands for the sample size when the state of parent nodes is $\left(\overline{a},b\right)$, $N_3$ stands for the sample size when the state of parents node is $\left(a,\overline{b}\right)$, and $N_4$ stands for the sample size when the state of parent nodes is $\left(\overline{a},\overline{b}\right)$.

The algorithms obtained from Methods 1 to 4 are convergent, and we can obtain the theorem as follows:

Theorem 1.

Let$p_{01}\cdot p_{04}>p_{02}\cdot p_{03}$, then there exists$k$such that.

$$p_{k1}\cdot p_{k4}\le p_{k2}\cdot p_{k3}$$

after$k$ steps adjustment as Method 1, where $p_1=P\left(c\left|a,b\right.\right)$, $p_2=P\left(c\left|\overline{a},b\right.\right)$, $p_3=P\left(c\left|a,\overline{b}\right.\right)$, $p_4=P\left(c\left|\overline{a},\overline{b}\right.\right)$, $p_{ki}$ represents the value of$p_i\left(i=1,2,3,4\right)$ after the adjustment of $k$ steps, $m_k=\left|p_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}\right|/2$, and$d_k=\left|m_k-p_{k1}\cdot p_{k4}\right|$.

Proof of Theorem 1.

Without loss of generality, we only prove the case of Method 1, as the proof of the other three cases is analogous.

Proving $p_{k1}\cdot p_{k4}\le p_{k2}\cdot p_{k3}$ is to show that

$$p_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}\le 0.$$

$$p_{k1}=p_{k-1,1}-d_{k-1}/2,p_{k4}=p_{k-1,4}-d_{k-1}/2,p_{k2}=p_{k-1,2}+d_{k-1}/2,p_{k3}=p_{k-1,3}+d_{k-1}/2,$$

then

$$\begin{array}{l}{p}_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}\\ =\left(p_{k-1,1}-d_{k-1}/2\right)\left(p_{k-1,4}-d_{k-1}/2\right)-\left(p_{k-1,2}+d_{k-1}/2\right)\left(p_{k-1,3}+d_{k-1}/2\right)\\ =p_{k-1,1}\cdot p_{k-1,4}-p_{k-1,2}\cdot p_{k-1,3}-\frac{d_{k-1}}{2}\left(p_{k-1,1}+p_{k-1,4}+p_{k-1,2}+p_{k-1,3}\right)\\ =\left(p_{k-2,1}-d_{k-2}/2\right)\left(p_{k-2,4}-d_{k-2}/2\right)-\left(p_{k-2,2}+d_{k-2}/2\right)\left(p_{k-2,3}+d_{k-2}/2\right)\\ -\frac{d_{k-1}}{2}\left(p_{k-2,1}-\frac{d_{k-2}}{2}+p_{k-2,4}-\frac{d_{k-2}}{2}+p_{k-2,2}+\frac{d_{k-2}}{2}+p_{k-2,3}+\frac{d_{k-2}}{2}\right)\\ =p_{k-2,1}\cdot p_{k-2,4}-p_{k-2,2}\cdot p_{k-2,3}-\frac{d_{k-2}}{2}\left(p_{k-2,1}+p_{k-2,4}+p_{k-2,2}+p_{k-2,3}\right)\\ -\frac{d_{k-1}}{2}\left(p_{k-2,1}+p_{k-2,4}+p_{k-2,2}+p_{k-2,3}\right)\\ =p_{k-2,1}\cdot p_{k-2,4}-p_{k-2,2}\cdot p_{k-2,3}-\frac{d_{k-2}+d_{k-1}}{2}\left(p_{k-2,1}+p_{k-2,4}+p_{k-2,2}+p_{k-2,3}\right)\\ =p_{k-3,1}\cdot p_{k-3,4}-p_{k-3,2}\cdot p_{k-3,3}-\frac{d_{k-3}+d_{k-2}+d_{k-1}}{2}\left(p_{k-3,1}+p_{k-3,4}+p_{k-3,2}+p_{k-3,3}\right)\\ =\cdots \cdots \\ =p_{11}\cdot p_{14}-p_{12}\cdot p_{13}-\frac{d_1+d_2+\cdots +d_{k-1}}{2}\left(p_{11}+p_{14}+p_{12}+p_{13}\right)\\ =p_{01}\cdot p_{04}-p_{02}\cdot p_{03}-\frac{d_0+d_1+\cdots +d_{k-1}}{2}\left(p_{01}+p_{04}+p_{02}+p_{03}\right)\\ =p_{01}\cdot p_{04}-p_{02}\cdot p_{03}-\frac{d_0+d_1+\cdots +d_{k-1}}{2}\cdot M\end{array}$$

where $p_{01}+p_{04}+p_{02}+p_{03}\triangleq M$ ($M$ is a positive constant).

Therefore, we can find that

$$\begin{array}{l}{d}_k=\left|m_k-p_{k1}\cdot p_{k4}\right|\\ =\left|\frac{p_{k1}\cdot p_{k4}+p_{k2}\cdot p_{k3}}{2}-p_{k1}\cdot p_{k4}\right|\\ =\frac{\left|p_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}\right|}{2}\\ =\frac{\left|\left(p_{k-1,1}-\frac{d_{k-1}}{2}\right)\left(p_{k-1,4}-\frac{d_{k-1}}{2}\right)-\left(p_{k-1,2}-\frac{d_{k-1}}{2}\right)\left(p_{k-1,3}-\frac{d_{k-1}}{2}\right)\right|}{2}\\ =\frac{\left|p_{k-1,1}\cdot p_{k-1,4}-p_{k-1,2}\cdot p_{k-1,3}-\frac{d_{k-1}}{2}\cdot M\right|}{2}\end{array}$$

Since $p_{k-1,1}\cdot p_{k-1,4}-p_{k-1,2}\cdot p_{k-1,3}>0$, we have

$$\begin{array}{l}{d}_{k-1}=\frac{\left|p_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}\right|}{2}\\ =\frac{p_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}}{2}\end{array}$$

Hence,

$$p_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}=2d_{k-1}.$$

Therefore,

$$\begin{array}{l}{d}_k=\frac{\left|p_{k-1,1}\cdot p_{k-1,4}-p_{k-1,2}\cdot p_{k-1,3}-\frac{d_{k-1}}{2}\cdot M\right|}{2}\\ =\frac{\left|2d_{k-1}-\frac{d_{k-1}}{2}\cdot M\right|}{2}=\frac{\left|2d_{k-1}\left(1-\frac{M}{4}\right)\right|}{2}\\ =d_{k-1}\left|1-\frac{M}{4}\right|\end{array}$$

By $0<M\le 4$, we have

$$d_k=d_{k-1}\left|1-\frac{M}{4}\right|=d_{k-1}\left(1-\frac{M}{4}\right).$$

When $M=4$,$d_k=0\left(k\ge 1\right)$, then

$$\begin{array}{l}{p}_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}\\ \hspace{1em}\hspace{1em}\hspace{1em}=p_{01}\cdot p_{04}-p_{02}\cdot p_{03}-\frac{d_0+d_1+\cdots +d_{k-1}}{2}\cdot M\\ \hspace{1em}\hspace{1em}\hspace{1em}=p_{01}\cdot p_{04}-p_{02}\cdot p_{03}-\frac{d_0}{2}\cdot 4\\ =2d_0-2d_0=0\end{array}$$

Therefore, Theorem 1 holds in this case.

When $0<M<4$,

$$d_k=d_{k-1}\left(1-\frac{M}{4}\right)=d_{k-2}{\left(1-\frac{M}{4}\right)}^2=\cdots =d_0{\left(1-\frac{M}{4}\right)}^k,$$

thus

$$\begin{array}{l}{p}_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}\\ =p_{01}\cdot p_{04}-p_{02}\cdot p_{03}-\frac{d_0+d_1+\cdots +d_{k-1}}{2}\cdot M\\ =2d_0-8d_0\left[1-{\left(1-M/4\right)}^k\right]\end{array}$$

Let $p_{k1}\cdot p_{k4}-p_{k2}\cdot p_{k3}$$=2d_0-8d_0\left[1-{\left(1-M/4\right)}^k\right]\le 0$, we can find that

$${\left(1-M/4\right)}^k\le \frac{3}{4}.$$

Since $0<M<4$, we have

$$0<1-M/4<1.$$

Hence, there exists $k$ such that

$${\left(1-M/4\right)}^k\le \frac{3}{4}.$$

From the above, we known that there exists $k$ such that

$$p_{k1}\cdot p_{k4}\le p_{k2}\cdot p_{k3}.$$

This completes the proof of Theorem 1. □

5. Experiments

In this section, we verify the effectiveness and performance of four algorithms mentioned in this paper by two simulations.

5.1. Experiment 1

5.1.1. Simulation Model

This simulation adopts the lawn moist model, as shown in Figure 1.

In the model, $R$, $S$ and $W$ meet the multiplicative synergy constraint, which can be expressed as follows:

$$P\left(W=1\left|R=1,S=1\right.\right)\cdot P\left(W=1\left|R=0,S=0\right.\right)\le P\left(W=1\left|R=1,S=0\right.\right)\cdot P\left(W=1\left|R=0,S=1\right.\right).$$

In the above inequality, when the value of the variable is 1, it means that the event occurs, and 0 means that it does not occur. Table 1 shows the real parameters in the network. In order to quantitatively analyze the performance of several methods in this paper, KL divergence from real parameters is introduced as an index to measure the accuracy of the algorithm. The expression of KL divergence (see [30]) is as follows:

$$KL\left(\widehat{\theta },\theta \right)={\displaystyle \sum _X{p}_{\widehat{\theta }}\left(X\right)}\ln \frac{p_{\widehat{\theta }}\left(X\right)}{p_{\theta }\left(X\right)}.$$

Table 1. Real parameters of the simulation network.

	$X_5=0,X_7=0$	$X_5=0,X_7=1$	$X_5=1,X_7=0$	$X_5=1,X_7=1$
$W=0$	0.6	0.4	0.4	0.15
$W=1$	0.4	0.6	0.6	0.85

5.1.2. Simulation Analysis

Take the sample size as 20, the simulation results obtained by several algorithms are shown from Table 2, Table 3, Table 4, Table 5 and Table 6, and the KL divergences between the learning results and the real parameters are shown in Table 7.

Table 2. Learning parameters of MLE.

	$R=0,S=0$	$R=0,S=1$	$R=1,S=0$	$R=1,S=1$
$W=0$	0.5814	0.2500	0.5250	0.0545
$W=1$	0.4186	0.7500	0.4750	0.9455

Table 3. Learning parameters of Method 1.

	$R=0,S=0$	$R=0,S=1$	$R=1,S=0$	$R=1,S=1$
$W=0$	0.6012	0.2311	0.5052	0.0743
$W=1$	0.3988	0.7689	0.4948	0.9257

Table 4. Learning parameters of Method 2.

	$R=0,S=0$	$R=0,S=1$	$R=1,S=0$	$R=1,S=1$
$W=0$	0.6333	0.1774	0.4524	0.1755
$W=1$	0.3667	0.8226	0.5476	0.8245

Table 5. Learning parameters of Method 3.

	$R=0,S=0$	$R=0,S=1$	$R=1,S=0$	$R=1,S=1$
$W=0$	0.6012	0.2444	0.5171	0.0611
$W=1$	0.3988	0.7556	0.4829	0.9389

Table 6. Learning parameters of Method 4.

	$R=0,S=0$	$R=0,S=1$	$R=1,S=0$	$R=1,S=1$
$W=0$	0.6390	0.1924	0.3684	0.2121
$W=0$	0.3610	0.8076	0.6316	0.7879

Table 7. KL divergences between the learning results and the real parameter.

Algorithm	MLE	Method 1	Method 2	Method 3	Method 4
KL divergence	0.1279	0.1133	0.1258	0.1203	0.0724

Table 2, Table 3, Table 4, Table 5 and Table 6 show the network parameters learned by MLE algorithm and the four algorithms proposed in this paper when the sample size is 20. Table 7 shows the KL divergences between the learning results and the real parameters. The experimental results show that the KL divergence between the learning parameters of the four methods proposed in this paper and real parameters is smaller than that between the learning parameters of MLE and the real parameters when the sample size is small. It shows that each method proposed in this paper is superior to the MLE algorithm in the accuracy of parameter learning. In addition, it can be seen from Table 7 that among the four algorithms proposed in this paper, the learning accuracy of Method 2 is the highest, while that of Method 1 is the lowest.

5.2. Experiment 2

5.2.1. Simulation Model

This simulation adopts Asia Network, as shown in Figure 2, where ‘1’ stands for $X_1$, ‘2’ stands for $X_2$, ‘3’ stands for $X_3$, ‘4’ stands for $X_4$, ‘5’ stands for $X_5$, ‘6’ stands for $X_6$, ‘7’ stands for $X_7$, and ‘8’ stands for $X_8$.

Figure 2. Asia network.

In the model, $X_7$ and, $X_8$ meet the multiplicative synergy constraint, which can be expressed as follows:

$$\begin{array}{l}P\left({\mathrm{X}}_8=1\left|{\mathrm{X}}_5=1,{\mathrm{X}}_7=1\right.\right)\cdot P\left({\mathrm{X}}_8=1\left|{\mathrm{X}}_5=0,{\mathrm{X}}_7=0\right.\right)\\ \le P\left({\mathrm{X}}_8=1\left|{\mathrm{X}}_5=1,{\mathrm{X}}_7=0\right.\right)\cdot P\left({\mathrm{X}}_8=1\left|{\mathrm{X}}_5=0,{\mathrm{X}}_7=1\right.\right)\end{array}$$

In the above inequality, if the value of the variable is 1, it means that the event occurs, and 0 means that it does not occur. Table 8 shows the real parameters in the network.

Table 8. Real parameters of the simulation network.

	$X_5=0,X_7=0$	$X_5=0,X_7=1$	$X_5=1,X_7=0$	$X_5=1,X_7=1$
$X_8=0$	0.65	0.4	0.3	0.05
$X_8=1$	0.35	0.6	0.7	0.95

The explanations of $X_i$ are as follows:

$X_1$—Visit To Asia (2): Visit, No_Visit;

$X_2$—Tuberculosis (2): Present, Absent;

$X_3$—Smoking (2): Smoker, Nonsmoker;

$X_4$—Lung Cancer (2): Present, Absent;

$X_5$—Tuberculosis or Lung Cancer (2): True, False;

$X_6$—Xray Result (2): Abnormal, Normal;

$X_7$—Bronchitis (2): Present, Absent;

$X_8$—Dyspnoea (2): Present, Absent.

5.2.2. Simulation Analysis

Take the sample size as 20, the simulation results obtained by several algorithms are shown from Table 9, Table 10, Table 11, Table 12 and Table 13, and the KL divergences between the learning results and the real parameters are shown in Table 14.

Table 9. Learning parameters of MLE.

	$X_5=0,X_7=0$	$X_5=0,X_7=1$	$X_5=1,X_7=0$	$X_5=1,X_7=1$
$X_8=0$	0.6299	0.3333	0.4976	0.0015
$X_8=1$	0.3701	0.6667	0.5024	0.9985

Table 10. Learning parameters of Method 1.

	$X_5=0,X_7=0$	$X_5=0,X_7=1$	$X_5=1,X_7=0$	$X_5=1,X_7=1$
$X_8=0$	0.6437	0.3155	0.4798	0.0193
$X_8=1$	0.3563	0.6845	0.5202	0.9807

Table 11. Learning parameters of Method 2.

	$X_5=0,X_7=0$	$X_5=0,X_7=1$	$X_5=1,X_7=0$	$X_5=1,X_7=1$
$X_8=0$	0.6774	0.2653	0.4286	0.1148
$X_8=1$	0.3226	0.7347	0.5714	0.8852

Table 12. Learning parameters of Method 3.

	$X_5=0,X_7=0$	$X_5=0,X_7=1$	$X_5=1,X_7=0$	$X_5=1,X_7=1$
$X_8=0$	0.6413	0.3035	0.5082	0.1011
$X_8=1$	0.3587	0.6965	0.4918	0.8989

Table 13. Learning parameters of Method 4.

	$X_5=0,X_7=0$	$X_5=0,X_7=1$	$X_5=1,X_7=0$	$X_5=1,X_7=1$
$X_8=0$	0.6988	0.3634	0.3277	0.1714
$X_8=1$	0.3012	0.6366	0.6723	0.8286

Table 14. KL divergences between the learning results and the real parameter.

Algorithm	MLE	Method 1	Method 2	Method 3	Method 4
KL divergence	0.1410	0.0961	0.1113	0.1356	0.1078

Table 9, Table 10, Table 11, Table 12 and Table 13 show the network parameters learned by MLE algorithm and the four algorithms proposed in this paper when the sample size is 20. Table 14 shows the KL divergences between the learning results and the real parameters. The experimental results show that the KL divergence between the learning parameters of the four methods proposed in this paper and real parameters is smaller than that between the learning parameters of MLE and the real parameters when the sample size is small. It shows that each method proposed in this paper is superior to the MLE algorithm in the accuracy of parameter learning. In addition, it can be seen from Table 14 that among the four algorithms proposed in this paper, the learning accuracy of Method 1 is the highest, while that of Method 3 is the lowest.

6. Conclusions

By referring to the idea of PAV algorithm, this paper proposes four methods to deal with multiplicative synergy constraints. We analyze and compare the algorithms from the algorithm accuracy. The simulations results show that the four algorithms mentioned in this paper are superior to the MLE algorithm in the accuracy of parameter learning, which can improve the results of the MLE algorithm to obtain more accurate estimators of the parameters.

The methods proposed in this paper can reduce the dependence of parameter learning on expert experiences. Combining these constraint methods with Bayesian estimation can improve the accuracy of parameter learning under small sample conditions. However, the algorithms in this paper also have limitations. When there are many parent nodes, it is difficult to give the parameter size relationship of the network. In the future research, the constraints presented in this paper can be combined with other existing constraints to reduce the dependence of constraints on expert experiences and improve the accuracy of parameter learning.