Efficient Modeling of Deterministic Decision Trees for Recognition of Realizable Decision Rules: Bounds on Weighted Depth

Abstract

In this paper, an efficient algorithm for modeling the operation of a DDT (Deterministic Decision Tree) solving the problem of realizability of DRs (Decision Rules) is proposed and analyzed. For this problem, it is assumed that a DRS (Decision Rule System) is given; for an arbitrary tuple of feature values, it is required to recognize whether there is a DR realizable on this tuple, i.e., a DR for which the left-hand side is true on the tuple. It is shown that the weighted depth of the modeled DDT does not exceed the square of the minimum weighted depth of the NDT (Nondeterministic Decision Tree) solving the realizability problem.

1. Introduction

DTs (Decision Trees) [1,2,3,4] and DRSs (Decision Rule Systems) [5,6,7,8,9] are common tools for structuring and expressing knowledge. They act as classifiers, providing predictions for unseen cases, and are also employed as algorithms in diverse domains such as fault diagnosis, combinatorial optimization, and beyond. Among classification and knowledge representation models, DTs and DRSs stand out for their high level of interpretability [10,11]. Exploring the connections and transformations between DTs and DRSs has become an important focus of research within computer science. In this work, we continue to develop the syntactic approach to study of this task proposed in [12,13]. This approach is based on the assumption that only the DRS is available to us, and not the input data. The results of previous studies in this area are summarized mainly in the book [14] and in the conference papers [15,16,17]. Earlier research has focused on the depth of DTs; in contrast, this paper investigates weighted depth as a measure of the time complexity of DTs.

In this work, we study the following realizability problem. Let a DRS be given. For a tuple of feature values, it is required to recognize whether this DRS contains a DR (Decision Rule) that is realizable for this tuple, i.e., a DR for which the left-hand side is true for this tuple. The tuples under consideration can contain values that do not appear in the DRS. This allows us to model the natural situation when the input tuple for the classifier may contain values that are not present in the training data. In addition, we do not allow the presence in the DRS of two DRs such that the set of conditions from the left-hand side of the first DR is a proper subset of the set of conditions from the left-hand side of the second DR; removing the second DR will not change the solution of the realizability problem.

When addressing the considered problem, it is important to note that in real-life situations we do not have direct access to the tuple of feature values. To determine the value of a feature, it is necessary to compute it for the given input, which can be an expensive procedure.

The complexity of finding feature values is determined by the weight function w, which assigns a weight $w\left(f_i\right)$ that is a positive integer to each feature $f_i$, $i=1,\dots ,n$ present in the DRs from the DRS. The weight $w\left(f_i\right)$ is interpreted as the complexity of finding the value of the feature $f_i$. To minimize the total weight of computed feature values, we consider DDTs (Deterministic Decision Trees) and NDTs (Nondeterministic Decision Trees) solving the considered problem and study their weighted depth.

To clarify the possibilities and limitations of using DTs, we modify examples considered in the papers [15,17] and in Lemma 13.10 of the book [14]. First, we discuss a sequence of DRSs for which the minimum depth of deterministic decision DDTs is growing as a logarithm on the number of different features in the DRs from the DRS. In such a situation, the use of DTs seems appropriate. A sequence of DRSs is also considered for which the minimum number of vertices in DDTs grows exponentially with the sum of length of DRs from the DRS. This means that in the general case, instead of constructing the entire DT, its operation on a given tuple of feature values should be modeled using a sufficiently efficient algorithm.

This paper is devoted to the consideration of a new efficient algorithm for modeling the operation of a DDT solving the realizability problem. The weighted depth of this DT is bounded from below by the minimum weighted depth of an NDT and from above by the square of the minimum weighted depth of an NDT.

Similar upper bounds for the depth were obtained in [18,19,20] for Boolean functions (see [21] for details); in [14] for functions of k-valued logic, $k\ge 2$; and in the paper [15] for the problem of finding all realizable DRs. To derive such upper bounds for each of the mentioned cases, an NDT with the minimum depth was considered. Based on this DT, for a given tuple of values of variables or features, the operation of a DDT satisfying the upper bound was described. It is important to note that this description of a DDT cannot be considered an efficient algorithm, since the NDT under consideration can have a huge number of vertices.

Note that this paper is a generalization of a previous conference paper [17]. In [17], we proposed an efficient algorithm for modeling the operation of DDTs that solves the realizability problem. The main contribution was showing that the depth of such DTs is bounded from below by the minimum depth of an NDT and from above by the square of the minimum depth of an NDT. In this paper, we extend that approach by considering the notion of weighted depth, which takes into account the complexity of finding feature values. This generalization better reflects real-world applications, where the complexity of finding feature values might vary.

In the present paper, when creating an algorithm for modeling the operation of a DDT for the realizability problem, it was possible to do without directly using an optimal NDT. As a result, the designed modeling algorithm has a polynomial complexity depending on the length of the description of the DRS.

The structure of the paper is as follows: in Section 2, we provide the key definitions and notation; Section 3 clarifies possibilities and limitations of using DTs; Section 4 considers the minimum weighted depth of NDTs; Section 5 is devoted to the analysis of an efficient algorithm for modeling the operation of a DDT; finally, Section 6 presents brief conclusions.

2. Definitions

This section introduces the notation and key definitions for DRSs and DTs.

2.1. DRSs—Decision Rule Systems

Let $N_0=\{0,1,2,\dots \}$, $k\in N_0\setminus \{0,1\}$, and $n\in N_0\setminus \left\{0\right\}$. Denote $E_k=\{0,1,\dots ,k-1\}$ and $F_n=\{f_1,\dots ,f_n\}$. Elements of the set $F_n$ will be called features.

A $(n,k)$-DR is an expression of the form

$$(f_{i_1}={\sigma }_1)\wedge \cdots \wedge (f_{i_t}={\sigma }_t)\to \gamma ,$$

where $t\in N_0\setminus \left\{0\right\}$, $f_{i_1},\dots ,f_{i_t}$ are pairwise different features from $F_n$, ${\sigma }_1,\dots ,{\sigma }_t$$\in E_k$, and $\gamma \in N_0$. We denote this DR by r. The number $\gamma$ is interpreted as the decision of the DR r. The number t will be called the length of the DR r, denoted $l\left(r\right)$. We denote $Ft\left(r\right)=\{f_{i_1},\dots ,f_{i_t}\}$ and $K\left(r\right)=\{f_{i_1}={\sigma }_1,\dots ,f_{i_t}={\sigma }_t\}$.

A restricted $(n,k)$-DRSS is a finite nonempty set of $(n,k)$-DRs such that there are no DRs $r,r^{\prime }\in S$ for which $K\left(r\right)\subset K\left(r^{\prime }\right)$. We denote $Ft\left(S\right)={\bigcup }_{r\in S}F\left(r\right)$. We can describe the DRS S by a word in the alphabet $\{0,1,(,),\wedge ,f,\to ,;\}$ such that the indexes of features, their values, and the decisions of DRs are in binary representations. The sign “;” is used to separate DRs. This word will be called the description of the DRS S.

A weight function for the DRS S is a map $w:Ft\left(S\right)\to N_0\setminus \left\{0\right\}$. The total weight of features in the DR r will be called the weight of the DR r and is denoted $w\left(r\right)$.

Let us fix a restricted $(n,k)$-DRS S, with which we will work later. We assume for definiteness that $S=\{r_1,\dots ,r_m\}$ and $Ft\left(S\right)=F_n$.

For $\overline{v}=(v_1,\dots ,v_n)\in E_{k+1}^n$, we denote $K(S,\overline{v})=\{f_1=v_1,\dots ,f_n=v_n\}$. We will say that a DR $r_i$ from S is realizable for a tuple $\overline{v}\in E_{k+1}^n$ if $K\left(r_i\right)\subseteq K(S,\overline{v})$.

For each tuple $\overline{v}\in E_{k+1}^n$, we define the value $dec\left(\overline{v}\right)$ in the following way: if there is a DR from S that is realizable for $\overline{v}$, then $dec\left(\overline{v}\right)=1$; otherwise, $dec\left(\overline{v}\right)=0$. The problem of realizability$\mathcal{R}\left(S\right)$ is defined as follows: for a given tuple $\overline{v}\in E_{k+1}^n$, it is required to find the value $dec\left(\overline{v}\right)$.

We denote by $ES\left(S\right)$ the set of systems of equations of the form

$$\alpha =\{f_{i_1}={\sigma }_1,\dots ,f_{i_t}={\sigma }_t\},$$

where $t\in N_0$, $f_{i_1},\dots ,$$f_{i_t}\in F_n$, and ${\sigma }_1,\dots ,{\sigma }_t\in E_{k+1}$. This system will be called inconsistent if there exist $p,q\in \{1,\dots ,t\}$ such that $p\ne q$, $i_p=i_q$, and ${\sigma }_p\ne {\sigma }_q$. If the system of equations $\alpha$ is not inconsistent, then it will be called consistent. The total weight of features in the system of equations will be called the weight of the system $\alpha$, denoted $w\left(\alpha \right)$. We will say that a tuple $\overline{v}=(v_1,\dots ,v_n)\in E_{k+1}^n$ is a solution of the equation system $\alpha$ if $\alpha \subseteq K(S,\overline{v})$.

We will say that the equation system $\alpha$ supports the decision $d\in \{0,1\}$ if $\alpha$ has no solutions from $E_{k+1}^n$ or if, for any solution $\overline{v}\in E_{k+1}^n$ of $\alpha$, $dec\left(\overline{v}\right)=d$.

2.2. DTs—Decision Trees

A finite rooted directed tree is a finite directed tree with exactly one vertex that has no incoming edges, referred to as the root. Vertices with no outgoing edges are called leaves, while those that are neither root nor leaves are termed internal vertices. A complete path is a sequence $\xi =u_1,e_1,\dots ,u_m,e_m,u_{m+1}$, where $u_1$ is the root, $u_{m+1}$ is a leaf, and each $e_i$ connects $u_i$ to $u_{i+1}$ for $i=1,\dots ,m$.

A DT over the problem $\mathcal{R}\left(S\right)$ is a finite rooted labeled directed tree $\mathcal{G}$ with at least two vertices, such that:

• The root and its outgoing edges are not labeled.
• Every internal vertex of $\mathcal{G}$ is labeled by a feature from $Ft\left(S\right)$, and the edges leaving it are labeled by elements of $E_k$.
• Each leaf of $\mathcal{G}$ is labeled with a decision from the set $\{0,1\}$.

A DT over the problem $\mathcal{R}\left(S\right)$ is termed a deterministic if exactly one edge leaves from the root; at every internal vertex, the outgoing edges are labeled with pairwise distinct labels.

Let $\mathrm{\Gamma }$ be a DT over the problem $\mathcal{R}\left(S\right)$. We denote by $CP\left(\mathrm{\Gamma }\right)$ the set of complete paths in the DT $\mathrm{\Gamma }$. Let $\xi =u_1,e_1,\dots ,u_t,e_t,u_{t+1}$ be a complete path in $\mathrm{\Gamma }$. We correspond to this path an equation system $K\left(\xi \right)\in ES\left(S\right)$. If $t=1$ and $\xi =u_1,e_1,u_2$, then $K\left(\xi \right)=\varnothing$. Let $t\ge 2$ and (for $j=2,\dots t$) the vertex $u_j$ be labeled with the feature $f_{i_j}$. Let the edge $e_j$ be labeled with the number ${\sigma }_j\in E_{k+1}$. Then, $K\left(\xi \right)=\{f_{i_2}={\sigma }_2,\dots ,f_{i_t}={\sigma }_t\}$. We denote by $d\left(\xi \right)$ the decision attached to the vertex $u_{t+1}$. We will say that the complete path $\xi$ accepts the tuple $\overline{v}\in E_{k+1}^n$ if $K\left(\xi \right)\subseteq K(S,\overline{v})$.

We will say that $\mathcal{G}$ solves the problem $\mathcal{R}\left(S\right)$ nondeterministically if, for any tuple $\overline{v}\in E_{k+1}^n$, there exists a path $\xi \in CP\left(\mathcal{G}\right)$ which accepts the tuple $\overline{v}$ and if for any path $\xi \in CP\left(\mathcal{G}\right)$, the system of equations $K\left(\xi \right)$ supports the decision $d\left(\xi \right)$. In this case, we will also say that $\mathcal{G}$ is an NDT solving the problem $\mathcal{R}\left(S\right)$.

We will say that $\mathcal{G}$ solves the problem $\mathcal{R}\left(S\right)$ deterministically if $\mathcal{G}$ is a DDT, which solves the problem $\mathcal{R}\left(S\right)$ nondeterministically. In this case, we will also say that $\mathcal{G}$ is a DDT solving the problem $\mathcal{R}\left(S\right)$.

For any complete path $\xi \in CP\left(\mathcal{G}\right)$, we denote by $h\left(\xi \right)$ the number of internal vertices in $\xi$ and denote by $h_w\left(\xi \right)$ the total weight of features attached to internal vertices of $\xi$. The value $h\left(\mathcal{G}\right)=max\left\{h\right(\xi ):\xi \in CP(\mathcal{G}\left)\right\}$ is called the depth of the DT $\mathcal{G}$. The value $h_w\left(\mathcal{G}\right)=max\{h_w\left(\xi \right):\xi \in CP\left(\mathcal{G}\right)\}$ is called the weighted depth of the DT $\mathcal{G}$.

We denote by $h_w^a\left(S\right)$ the minimum weighted depth of an NDT over the problem $\mathcal{R}\left(S\right)$ which solves this problem. We denote by $h_w^d\left(S\right)$ the minimum weighted depth of a DDT over the problem $\mathcal{R}\left(S\right)$ which solves this problem. It is clear that $h_w^a\left(S\right)\le h_w^d\left(S\right)$.

3. Two Sequences of DRSs (Decision Rule Systems)

In this section, we consider two sequences of DRSs. For each DRS in the first sequence, the minimum depth of a DDT is significantly less than the number of features in the DRS. This example shows that using DTs is reasonable. For each DRS in the second sequence, the number of vertices in any DDT is exponential related to the sum of length of the DRs in the DRS. This example shows that rather than constructing the entire DT, it is more reasonable to model its operation for a given tuple of feature values.

Let us begin with the first sequence of DRSs. Let $t\in N_0\setminus \{0,1\}$. A complete binary tree of depth t is a finite directed tree with root in which each non-leaf vertex has exactly two outgoing edges and the length of each complete path is equal to t. The vertex set of this tree is naturally divided into $t+1$ levels: for $i=0,\dots ,t$, the ith level contains all vertices that are located at distance i from the root. It is clear that each level i includes $2^i$ vertices. Consequently, the number of non-leaf vertices equals $2^0+\dots +2^{t-1}=2^t-1$, while the number of leaf vertices is $2^t$.

Let $H_t$ denote a labeled complete binary tree of depth t, where non-leaf nodes are assigned features $f_1,\dots ,f_{2^t-1}$ and leaf vertices are labeled with integers $1,\dots ,2^t$. For every non-leaf vertex, its outgoing edges are labeled by 0 and 1, respectively. For $j=1,\dots ,2^t$, we define a DR $r_j$ as follows. Consider a full path $u_0,e_0,\dots ,u_{t-1},e_{t-1},u_t$ in $H_t$ ending at the leaf node labeled j, where for each $i=0,\dots ,t-1$, the vertex $u_i$ is labeled with feature $f_{l_i}$ and the connecting edge $e_i$ is labeled with a number $v_i\in E_2$. Then, the corresponding DR $r_j$ is provided by

$$(f_{l_0}=v_0)\wedge \cdots \wedge (f_{l_{t-1}}=v_{t-1})\to j.$$

We denote the set of all such DRs by $R_t=\{r_1,\dots ,r_{2^t}\}$. It is straightforward that $|Ft\left(R_t\right)|=2^t-1$. For the DRS $R_t$, we will consider tuples of values of features from the set $E_3^{2^t-1}$.

Next, let us analyze the problem $\mathcal{R}\left(R_t\right)$ and show that $h^d\left(R_t\right)\le t$. We transform the tree $H_t$ into a DDT $G_t$ over the problem $\mathcal{R}\left(R_t\right)$. For each $j=1,\dots ,2^t$, we replace the label j attached to a leaf vertex of $H_t$ with the label 1. For each non-leaf vertex w of $H_t$, we add to the tree $H_t$ a vertex $u_w$ and an edge $e_w$ that leaves the vertex w and enters the vertex $u_w$. The edge $e_w$ is labeled with the number 2 and the vertex $u_w$ is labeled with the number 0. We add to the tree $H_t$ a vertex v and an edge d that leaves the vertex v and enters the root of $H_t$. Both v and d are unlabeled. It can be shown that the obtained DDT $G_t$ solves the problem $\mathcal{R}\left(R_t\right)$ and has depth t.

Hence, for every $t\in N_0\setminus \{0,1\}$, we obtain a DRS $R_t$ satisfying $|Ft\left(R_t\right)|=2^t-1$ and there exists a DDT $G_t$ solving $\mathcal{R}\left(R_t\right)$ for which the depth equals t.

Now, let us move to the second sequence of DRSs. For any $q\in N_0\setminus \left\{0\right\}$, we define $U_q$ the DRS $\{(f_{2i-1}=0)\wedge (f_{2i}=0)\to 0:i=1,\dots ,q\}$. Clearly, $\left|Ft\right(U_q\left)\right|=2q$. For $U_q$, we consider tuples of values of features from the set $E_3^{2q}$.

We denote by $\Delta$ the set of tuples $(v_1,\dots ,v_{2q})\in {\{0,2\}}^{2q}$ such that $\{v_{2i-1},v_{2i}\}=\{0,2\}$ for $i=1,\dots ,q$. It is clear that $\left|\Delta \right|=2^q$ and that for any $\overline{v}\in \Delta$ there is no a DR from $U_q$ that is realizable for the tuple $\overline{v}$. Let $\mathcal{G}$ be a DDT over the problem $\mathcal{R}\left(U_t\right)$ which solves this problem. Let $\overline{v},\overline{\sigma }\in \Delta$ and $\overline{v}\ne \overline{\sigma }$. It is clear that there exist complete paths $\xi$ and $\tau$ in $\mathcal{G}$ such that $\xi$ accepts $\overline{v}$ and $\tau$ accepts $\overline{\sigma }$. Let us now show that $\xi \ne \tau$. Let us first assume the contrary, $\xi =\tau$. It is clear that the equation system $K\left(\xi \right)$ supports the decision 0. Because $\overline{v}\ne \overline{\sigma }$, there exists $i\in \{1,\dots ,q\}$ such that the $(2i-1)$th and $\left(2i\right)$th digits of the tuples $\overline{v}$ and $\overline{\sigma }$ are different. Therefore, the features $f_{2i-1}$ and $f_{2i}$ are not attached to any vertex of the path $\xi$. Using this fact, it is easy to show that there exists a tuple $\overline{\gamma }\in E_3^{2q}$ such that $\overline{\gamma }$ is a solution of the equation system $K\left(\xi \right)$ and that the DR $(f_{2i-1}=0)\wedge (f_{2i}=0)\to 0$ from $U_t$ is realizable for the tuple $\overline{\gamma }$; however, this is impossible. Therefore, $\xi \ne \tau$. Thus, there are at least $2^q$ pairwise different complete paths in the tree $\mathcal{G}$.

As a result, for each $q\in N_0\setminus \left\{0\right\}$ we obtain an example of a DRS $U_q$ which consists of q DRs of length 2 and for which any DDT solving the problem $\mathcal{R}\left(U_q\right)$ has at least $2^q$ vertices.

4. On the Minimum Weighted Depth of NDTs (Nondeterministic Decision Trees)

In this section, we return to the study of the restricted $(n,k)$-DRS $S=\{r_1,\dots ,r_m\}$ with $Ft\left(S\right)=\{f_1,\dots ,f_n\}$ and prove three lemmas related to the minimum weighted depth of an NDT solving the problem $\mathcal{R}\left(S\right)$.

Let $\overline{v}\in E_{k+1}^n$. A system of equations $\alpha \in ES\left(S\right)$ will be called a certificate for the tuple $\overline{v}$ if $\alpha$ supports the decision $dec\left(\overline{v}\right)$ and if $\overline{v}$ is a solution of $\alpha$.

Let $dec\left(\overline{v}\right)=1$. We denote $cer\left(\overline{v}\right)=K\left(r_i\right)$, where $r_i$ is a DR from S with the minimum weight that is realizable for $\overline{v}$. Let $dec\left(\overline{v}\right)=0$. We denote by $cer\left(\overline{v}\right)$ a subsystem with the minimum weight of the system $K(S,\overline{v})$ such that $cer\left(\overline{v}\right)\cup K\left(r_i\right)$ is inconsistent for each DR $r_i\in S$.

Lemma 1. 

For any tuple $\overline{v}\in E_{k+1}^n$, the equation system $cer\left(\overline{v}\right)$ is a certificate for $\overline{v}$ with the minimum weight.

Proof. 

Let $\overline{v}\in E_{k+1}^n$, $dec\left(\overline{v}\right)=1$, $\alpha \in ES\left(S\right)$, and $\overline{v}$ be a solution of $\alpha$. We now show that $\alpha$ is a certificate for $\overline{v}$ if and only if $K\left(r_i\right)\subseteq \alpha$ for some DR $r_i\in S$ that is realizable for $\overline{v}$. If such a DR exists, then evidently $\alpha$ is a certificate for $\overline{v}$. Let there be no such DR. We change the value of each feature in $\overline{v}$ that does not belong to $\alpha$ to k. As a result, we obtain a tuple ${\overline{v}}^{\prime }\in E_{k+1}^n$ such that $dec\left({\overline{v}}^{\prime }\right)=0$ and $\overline{v}$ is a solution of $\alpha$. Therefore, $\alpha$ is not a certificate for $\overline{v}$. From here, it follows that the equation system $K\left(r_i\right)$, where $r_i$ is a DR from S with the minimum weight that is realizable for $\overline{v}$, is a certificate for $\overline{v}$ with the minimum weight.

Let $\overline{v}\in E_{k+1}^n$, $dec\left(\overline{v}\right)=0$, and $\alpha \in ES\left(S\right)$; additionally, let $\overline{v}$ be a solution of $\alpha$. We now show that $\alpha$ is a certificate for $\overline{v}$ if and only if the system of equations $\alpha \cup K\left(r_i\right)$ is inconsistent for each DR $r_i\in S$. If $\alpha \cup K\left(r_i\right)$ is inconsistent for each DR $r_i\in S$, then evidently $\alpha$ is a certificate for $\overline{v}$. Let the system $\alpha \cup K\left(r_i\right)$ be not inconsistent for some DR $r_i\in S$. In $\overline{v}$, change the values of all features belonging to $r_i$ to values from $K\left(r_i\right)$. As a result, we obtain a tuple ${\overline{v}}^{\prime }$ from $E_{k+1}^n$ which is a solution of $\alpha$ and for which $dec\left({\overline{v}}^{\prime }\right)=1$. Therefore, $\alpha$ is not a certificate for $\overline{v}$. From here, it follows that a subsystem $\alpha$ with the minimum weight of the system $K(S,\overline{v})$ such that $\alpha \cup K\left(r_i\right)$ is inconsistent for each DR $r_i\in S$ is a certificate for $\overline{v}$ with the minimum weight.    □

Lemma 2. 

$h_w^a\left(S\right)=max\{w\left(cer\left(\overline{v}\right)\right):\overline{v}\in E_{k+1}^n\}$.

Proof. 

Let $\mathcal{G}$ be an NDT over the problem $\mathcal{R}\left(S\right)$ which solves this problem. Let $\overline{v}\in E_{k+1}^n$. Then, $\mathcal{G}$ has a complete path $\xi$ accepting $\overline{v}$. This means that $\overline{v}$ is a solution of the equation system $K\left(\xi \right)$. Therefore, the system of equations $K\left(\xi \right)$ supports the decision $dec\left(\overline{v}\right)$. Thus, $K\left(\xi \right)$ is a certificate for $\overline{v}$, and by Lemma 1 we have $h_w\left(\xi \right)\ge w\left(cer\left(\overline{v}\right)\right)$. As a result, we obtain $h_w^a\left(S\right)\ge max\{w\left(cer\left(\overline{v}\right)\right):\overline{v}\in E_{k+1}^n\}$.

Let us now describe an NDT ${\mathcal{G}}_0$ over the problem $\mathcal{R}\left(S\right)$. The set of complete paths of ${\mathcal{G}}_0$ is equal to $\{{\xi }_{\overline{v}}:\overline{v}\in E_{k+1}^n\}$, where $K\left({\xi }_{\overline{v}}\right)=cer\left(\overline{v}\right)$ and the leaf vertex of ${\xi }_{\overline{v}}$ is labeled with $dec\left(\overline{v}\right)$. From Lemma 1, it follows that the complete path ${\xi }_{\overline{v}}$ accepts the tuple $\overline{v}$ and that the equation system $K\left({\xi }_{\overline{v}}\right)$ supports the decision $dec\left(\overline{v}\right)$. Therefore, ${\mathcal{G}}_0$ solves the problem $\mathcal{R}\left(S\right)$. Thus, $h_w^a\left(S\right)\le max\{w\left(cer\left(\overline{v}\right)\right):\overline{v}\in E_{k+1}^n\}$.    □

For $b\in \{0,1\}$, we denote $h^b\left(S\right)=max\left\{\right|cer\left(\overline{v}\right)|:\overline{v}\in E_{k+1}^n,dec\left(\overline{v}\right)=b\}$ and $h_w^b\left(S\right)=max\{w\left(cer\left(\overline{v}\right)\right):\overline{v}\in E_{k+1}^n,dec\left(\overline{v}\right)=b\}$.

Corollary 1. 

$h_w^a\left(S\right)=max\{h_w^0\left(S\right),h_w^1\left(S\right)\}$.

Lemma 3. 

$h_w^1\left(S\right)=max\{w\left(r_i\right):r_i\in S\}$.

Proof. 

Let us show that $\{cer\left(\overline{v}\right):\overline{v}\in E_{k+1}^n,dec\left(\overline{v}\right)=1\}=\{K\left(r_1\right),\dots ,K\left(r_m\right)\}$. It is clear that for each $\overline{v}\in E_{k+1}^n$ such that $dec\left(\overline{v}\right)=1$, we have $cer\left(\overline{v}\right)\in \{K\left(r_1\right),\dots ,$$K\left(r_m\right)\}$. Let $r_i\in S$. We consider a tuple $\overline{v}\in E_{k+1}^n$ such that values of features from $r_i$ are equal to corresponding values from $r_i$ and values of all other features are equal to k. It is clear that $r_i$ is the only DR from S that is realizable for $\overline{v}$. Therefore, $cer\left(\overline{v}\right)=K\left(r_i\right)$. As a result, we obtain $\{cer\left(\overline{v}\right):\overline{v}\in E_{k+1}^n,dec\left(\overline{v}\right)=1\}=\{K\left(r_1\right),\dots ,K\left(r_m\right)\}$. From here, it follows that $h_w^1\left(S\right)=max\{w\left(r_i\right):r_i\in S\}$.    □

5. Algorithm Simulating the Operation of a DDT (Deterministic Decision Tree)

In this section, we continue to study the restricted $(n,k)$-DRS $S=\{r_1,\dots ,r_m\}$ with $Ft\left(S\right)=\{f_1,\dots ,f_n\}$. Let $\overline{v}=(v_1,\dots ,v_n)\in E_{k+1}^n$. We now consider an algorithm that describes the operation on the tuple $\overline{v}$ of a DDT $\mathcal{G}\left(S\right)$ over the problem $\mathcal{R}\left(S\right)$ which solves this problem. As a result, we obtain the description of a complete path $\tau \left(\overline{v}\right)$ in the DT $\mathcal{G}\left(S\right)$ that accepts the tuple $\overline{v}$. The set of complete paths of the DT $\mathcal{G}\left(S\right)$ coincides with the set $\{\tau \left(\overline{v}\right):\overline{v}\in E_{k+1}^n\}$. The text in square brackets below is not a description of the algorithm’s and DT’s actions, and will only be used to prove a statement about this Algorithm 1.

Algorithm 1 Simulation of DDT operation

Step 1. For $i=1,\dots ,m$, set $Q_i:=K\left(r_i\right)$. Set $W:=\{Q_1,\dots ,Q_m\}$. [For any $\overline{v}\in E_{k+1}^n$ with $dec\left(\overline{v}\right)=0$, set $P\left(\overline{v}\right)=cer\left(\overline{v}\right)$. Set $V:=\{P\left(\overline{v}\right):\overline{v}\in E_{k+1}^n,dec\left(\overline{v}\right)=0\}$.] Step 2. If $W=\varnothing$, then $\mathcal{G}\left(S\right)$ returns the decision 0 and stops. If W contains an empty system of equations, then $\mathcal{G}\left(S\right)$ returns the decision 1 and stops. Let $W\ne \varnothing$ and W contain no empty systems of equations. Choose a system $Q_j$ from W with the minimum number j. Let $f_{i_1},\dots ,f_{i_p}$ be all features from $Q_j$ and $i_1<\cdots <i_p$. The DT $\mathcal{G}\left(S\right)$ computes values of features $f_{i_1},\dots ,f_{i_p}$ and obtains the system of equations $\alpha =\{f_{i_1}=v_{i_1},\dots ,f_{i_p}=v_{i_p}\}$. For each $Q_l\in W$, we remove $Q_l$ from W if the system $Q_l\cup \alpha$ is inconsistent. Otherwise, we set $Q_l:=Q_l\setminus \alpha$. [For each $P\left(\overline{v}\right)\in V$, we remove $P\left(\overline{v}\right)$ from V if the system $P\left(\overline{v}\right)\cup \alpha$ is inconsistent. Otherwise, we set $P\left(\overline{v}\right):=P\left(\overline{v}\right)\setminus \alpha$.] Return to Step 2.

Remark 1. 

It is possible to show that the time complexity of Algorithm 1 is polynomial depending on the length of the description of DRS S.

Theorem 1. 

The DDT $\mathcal{G}\left(S\right)$ solves the problem $\mathcal{R}\left(S\right)$ and satisfies the inequality $h_w\left(\mathcal{G}\left(S\right)\right)\le h^0\left(S\right)h_w^1\left(S\right)$.

Proof. 

First, we show that $\mathcal{G}\left(S\right)$ solves the problem $\mathcal{R}\left(S\right)$. Let $\overline{v}\in E_{k+1}^n$. It is clear that the complete path $\tau \left(\overline{v}\right)$ in the DT $\mathcal{G}\left(S\right)$ accepts $\overline{v}$. Let $dec\left(\overline{v}\right)=0$. We can show that the equation system $K\left(\tau \left(\overline{v}\right)\right)\cup K\left(r_i\right)$ is inconsistent for each DR $r_i\in S$. Then, $d\left(\tau \left(\overline{v}\right)\right)=0$. It is clear that for each solution $\overline{\sigma }\in E_{k+1}^n$ of the system $K\left(\tau \left(\overline{v}\right)\right)$ we have $dec\left(\overline{\sigma }\right)=0$. Therefore, $K\left(\tau \left(\overline{v}\right)\right)$ supports the decision 0. Let $dec\left(\overline{v}\right)=1$. We can show that $K\left(r_i\right)\subseteq K\left(\tau \left(\overline{v}\right)\right)$ for some DR $r_i\in S$. Then, $d\left(\tau \left(\overline{v}\right)\right)=1$. It is clear that for each solution $\overline{\sigma }\in E_{k+1}^n$ of the system $K\left(\tau \left(\overline{v}\right)\right)$, we have $dec\left(\overline{\sigma }\right)=1$. Therefore, $K\left(\tau \left(\overline{v}\right)\right)$ supports the decision 1. Thus, the DT $\mathcal{G}\left(S\right)$ solves the problem $\mathcal{R}\left(S\right)$.

Let us consider the work of the DT $\mathcal{G}\left(S\right)$ on a tuple $\overline{v}\in E_{k+1}^n$. By Lemma 3, during each complete repetition of Step 2, the DT $\mathcal{G}\left(S\right)$ computes the values of features for which the total weight is at most $h_w^1\left(S\right)$. We now show that the number of complete repetitions of Step 2 is at most $h^0\left(S\right)$.

Let $r_i\in S$, $\overline{\sigma }\in E_{k+1}^n$, and $dec\left(\overline{\sigma }\right)=0$. From the definition of $cer\left(\overline{\sigma }\right)$, it follows that the system of equations $K\left(r_i\right)\cup cer\left(\overline{\sigma }\right)$ is inconsistent. From here, it follows that before each repetition of Step 2 of Algorithm 1, for each nonempty system $Q_l\in W$ and each nonempty system $P\left(\overline{\sigma }\right)\in V$, the system of equations $Q_l\cup P\left(\overline{\sigma }\right)$ is inconsistent. Using this fact, it can be shown that after each complete repetition of Step 2, each system of equations from V will either be removed from V or its cardinality will decrease by at least 1. Evidently, the cardinality of each system from V before the first repetition of Step 2 is at most $h^0\left(S\right)$. Therefore, after $h^0\left(S\right)$ complete repetitions of Step 2, the set V will be empty or will contain only empty systems of equations.

We denote by $\beta$ the system of equations consisting of all features computed by $\mathcal{G}\left(S\right)$ during $h^0\left(S\right)$ complete repetitions of Step 2 with their values from $\overline{v}$.

Let V contain an empty system. Then, $cer\left(\overline{\rho }\right)\subseteq \beta$ for some tuple $\overline{\rho }\in E_{k+1}^n$ such that $dec\left(\overline{\rho }\right)=0$. Thus, the set W is empty and $\mathcal{G}\left(S\right)$ stops. Let V be empty. Then, the system $\beta \cup cer\left(\overline{\rho }\right)$ is inconsistent for any $\overline{\rho }\in E_{k+1}^n$ such that $dec\left(\overline{\rho }\right)=0$. Let us show that $K\left(r_i\right)\subseteq \beta$ for some DR $r_i\in S$. First, assume the contrary, that for each DR $r_i\in S$, $K\left(r_i\right)$ is not a subset of $\beta$. In the tuple $\overline{v}$, we change the values of all features that do not belong to $\beta$ to k. We denote the obtained tuple ${\overline{v}}^{\prime }$. It is clear that ${\overline{v}}^{\prime }$ is a solution of the equation system $\beta$ and $dec\left({\overline{v}}^{\prime }\right)=0$; however, this is impossible, as the system $\beta \cup cer\left({\overline{v}}^{\prime }\right)$ is inconsistent. As a result, we have that $\mathcal{G}\left(S\right)$ stops after at most $h^0\left(S\right)$ complete repetitions of Step 2. Thus, $h_w\left(\mathcal{G}\left(S\right)\right)\le h^0\left(S\right)h_w^1\left(S\right)$. □

Corollary 2. 

$h_w\left(\mathcal{G}\left(S\right)\right)\le h_w^a{\left(S\right)}^2$.

Proof. 

This statement follows from the fact that $h^0\left(S\right)\le h_w^0\left(S\right)$, Theorem 1 and Corollary 1. □

We denote by $w^{max}\left(S\right)$ the maximum weight of a DR from S.

Corollary 3. 

$h_w\left(\mathcal{G}\left(S\right)\right)\le w^{max}\left(S\right)h_w^a\left(S\right)$.

Proof. 

This statement follows from Theorem 1, Lemma 3, and Corollary 1. □

From Theorem 1, it follows that $h_w^d\left(S\right)\le h_w^0\left(S\right)h_w^1\left(S\right)$. We now show that this bound is unimprovable even for depth, i.e., when $h_w=h$.

Proposition 1. 

For any $p,q\in N_0\setminus \left\{0\right\}$, there exists a restricted $(pq,2)$-DRS $S_{pq}$ such that $h_1\left(S_{pq}\right)=p$, $h_0\left(S_{pq}\right)=q$, and $h^d\left(S_{pq}\right)=pq$.

Proof. 

Let $p,q\in N_0\setminus \left\{0\right\}$. We now consider a restricted $(pq,2)$-DRS $S_{pq}=\{r_1,\dots ,r_q\}$, where for $i=1,\dots ,q$ the DR $r_i$ is equal to

$$(f_{p(i-1)+1}=0)\wedge \cdots \wedge (f_{p(i-1)+p}=0)\to i.$$

It is clear that $Ft\left(S_{pq}\right)=\{f_1,\dots ,f_{pq}\}$ and that the values of all features from the DRs $r_1,\dots ,r_q$ belong to the set $E_2$. For the DRS $S_{pq}$, we will consider tuples of values of features from the set $E_3^{pq}$.

The length of each DR from $S_{pq}$ is equal to p. Using Lemma 3, we obtain that $h_1\left(S_{pq}\right)=p$. Let $\overline{v}\in E_3^{pq}$ and $dec\left(\overline{v}\right)=0$, i.e., let all DRs from $S_{pq}$ be non-realizable for the tuple $\overline{v}$. It is clear that $Ft\left(r_i\right)\cap$$Ft\left(r_j\right)=\varnothing$ for any two DRs $r_i,r_j\in S_{pq}$ such that $i\ne j$. Using this fact, it is easy to show that the minimum cardinality of a certificate for $\overline{v}$ is equal to q. Therefore, $h_0\left(S_{pq}\right)=q$.

It is clear that $h^d\left(S_{pq}\right)\le \left|Ft\left(S_{pq}\right)\right|=pq$. Let us show that $h^d\left(S_{pq}\right)\ge pq$. To this end, we consider a DDT $\mathcal{G}$ over the problem $\mathcal{R}\left(S_{pq}\right)$ which solves this problem and for which $h\left(\mathcal{G}\right)=h^d\left(S_{pq}\right)$. Now, let us analyze the work of $\mathcal{G}$, which is described as follows. Let $\mathcal{G}$ compute the value of feature $f_t$ from DR $r_s$. If the values of all other features from $r_s$ have already been computed, then $f_t=2$; otherwise, $f_t=0$. If $\mathcal{G}$ does not compute the values of all features, then we will not know the solution to the realizability problem. Therefore, $h\left(\mathcal{G}\right)\ge pq$ and $h^d\left(S_{pq}\right)\ge pq$. Thus, $h^d\left(S_{pq}\right)=pq$. □

It follows from Proposition 1 that we cannot improve the accuracy bound for Algorithm 1 provided by Theorem 1 and that we cannot find algorithms with better accuracy bounds based on parameters $h_w^0\left(S\right)$ and $h_w^1\left(S\right)$.

6. Conclusions

In this paper, an algorithm for modeling the operation of a DDT solving the problem of realizability is proposed. The running time of this algorithm is polynomial depending on the length of the description of the DRS under consideration. The weighted depth of the modeled DDT does not exceed the square of the minimum weighted depth of an NDT solving the realizability problem. In the future, we plan to design similar algorithms for some other problems related to DRSs. In the more distant future, we plan to conduct an experimental study of such algorithms.