EntropyBased Greedy Algorithm for Decision Trees Using Hypotheses
Abstract
:1. Introduction
 Decision trees that use only attributes.
 Decision trees that use only hypotheses.
 Decision trees that use both attributes and hypotheses.
 Decision trees that use only proper hypotheses.
 Decision trees that use both attributes and proper hypotheses.
2. Decision Tables
3. Decision Trees
 Each terminal node is labeled with a number from the set $D\left(T\right)\cup \left\{0\right\}$.
 Each node, which is not terminal (such nodes are called working), is labeled with an attribute from the set $F\left(T\right)$ or with a hypothesis over T.
 If a working node is labeled with an attribute ${f}_{i}$ from $F\left(T\right)$, then for each answer from the set $A\left({f}_{i}\right)$, there is exactly one edge labeled with this answer, which leaves this node and there are no any other edges that leave this node.
 If a working node is labeled with a hypothesis $H=\{{f}_{1}={\delta}_{1},\dots ,{f}_{n}={\delta}_{n}\}$ over T, then for each answer from the set $A\left(H\right)$, there is exactly one edge labeled with this answer, which leaves this node and there are no any other edges that leave this node.
 The node v is terminal if and only if the subtable $TS(\Gamma ,v)$ is degenerate.
 If v is a terminal node and the subtable $TS(\Gamma ,v)$ is empty, then the node v is labeled with the decision 0.
 If v is a terminal node and the subtable $TS(\Gamma ,v)$ is nonempty, then the node v is labeled with the decision attached to all rows of $TS(\Gamma ,v)$.
 Decision trees that use only attributes.
 Decision trees that use only hypotheses.
 Decision trees that use both attributes and hypotheses.
 Decision trees that use only proper hypotheses.
 Decision trees that use both attributes and proper hypotheses.
4. Greedy Algorithm Based on Entropy
Algorithm 1$\mathcal{E}$. 
Input: A nonempty decision table T and a number $k\in \{1,\dots ,5\}$. Output: A decision tree of the type k for the table T.

5. Results of Experiments
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
 Breiman, L.; Friedman, J.H.; Olshen, R.A.; Stone, C.J. Classification and Regression Trees; Chapman and Hall/CRC: Boca Raton, FL, USA, 1984. [Google Scholar]
 Moshkov, M. Time complexity of decision trees. In Transactions on Rough Sets III; Lecture Notes in Computer Science; Peters, J.F., Skowron, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2005; Volume 3400, pp. 244–459. [Google Scholar]
 Rokach, L.; Maimon, O. Data Mining with Decision Trees—Theory and Applications; Series in Machine Perception and Artificial Intelligence; World Scientific: Singapore, 2007; Volume 69. [Google Scholar]
 Pawlak, Z. Rough sets. Int. J. Parallel Program. 1982, 11, 341–356. [Google Scholar] [CrossRef]
 Pawlak, Z. Rough Sets—Theoretical Aspects of Reasoning about Data; Theory and Decision Library: Series D; Kluwer: Alfen am Rhein, The Netherlands, 1991; Volume 9. [Google Scholar]
 Pawlak, Z.; Skowron, A. Rudiments of rough sets. Inf. Sci. 2007, 177, 3–27. [Google Scholar] [CrossRef]
 Chegis, I.A.; Yablonskii, S.V. Logical methods of control of work of electric schemes. Trudy Mat. Inst. Steklov 1958, 51, 270–360. (In Russian) [Google Scholar]
 Angluin, D. Queries and concept learning. Mach. Learn. 1988, 2, 319–342. [Google Scholar] [CrossRef][Green Version]
 Angluin, D. Queries revisited. Theor. Comput. Sci. 2004, 313, 175–194. [Google Scholar] [CrossRef][Green Version]
 Valiant, L.G. A theory of the learnable. Commun. ACM 1984, 27, 1134–1142. [Google Scholar] [CrossRef][Green Version]
 AbouEisha, H.; Amin, T.; Chikalov, I.; Hussain, S.; Moshkov, M. Extensions of Dynamic Programming for Combinatorial Optimization and Data Mining; Intelligent Systems Reference Library; Springer: Berlin/Heidelberg, Germany, 2019; Volume 146. [Google Scholar]
 Aglin, G.; Nijssen, S.; Schaus, P. Learning optimal decision trees using caching branchandbound search. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, AAAI 2020, New York, NY, USA, 7–12 February 2020; pp. 3146–3153. [Google Scholar]
 Alsolami, F.; Azad, M.; Chikalov, I.; Moshkov, M. Decision and Inhibitory Trees and Rules for Decision Tables with ManyValued Decisions; Intelligent Systems Reference Library; Springer: Berlin/Heidelberg, Germany, 2020; Volume 156. [Google Scholar]
 Narodytska, N.; Ignatiev, A.; Pereira, F.; MarquesSilva, J. Learning optimal decision trees with SAT. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018, Stockholm, Sweden, 13–19 July 2018; pp. 1362–1368. [Google Scholar]
 Verwer, S.; Zhang, Y. Learning optimal classification trees using a binary linear program formulation. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, Honolulu, HI, USA, 27 January–1 February 2019; pp. 1625–1632. [Google Scholar]
 Azad, M.; Chikalov, I.; Hussain, S.; Moshkov, M. Minimizing depth of decision trees with hypotheses (to appear). In Proceedings of the International Joint Conference on Rough Sets (IJCRS 2021), Bratislava, Slovakia, 19–24 September 2021. [Google Scholar]
 Azad, M.; Chikalov, I.; Hussain, S.; Moshkov, M. Minimizing number of nodes in decision trees with hypotheses (to appear). In Proceedings of the 25th International Conference on KnowledgeBased and Intelligent Information & Engineering Systems (KES 2021), Szczecin, Poland, 8–10 September 2021. [Google Scholar]
 Quinlan, J.R. C4.5: Programs for Machine Learning; Morgan Kaufmann: Burlington, MA, USA, 1993. [Google Scholar]
 Dua, D.; Graff, C. UCI Machine Learning Repository. University of California, Irvine, School of Information and Computer Sciences. 2017. Available online: http://archive.ics.uci.edu/ml (accessed on 12 April 2017).
 Quinlan, J.R. Induction of decision trees. Mach. Learn. 1986, 1, 81–106. [Google Scholar] [CrossRef][Green Version]
 Rokach, L.; Maimon, O. Topdown induction of decision trees classifiers—A survey. IEEE Trans. Syst. Man Cybern. Part C 2005, 35, 476–487. [Google Scholar] [CrossRef][Green Version]
Decision  Number of  Number of 

Table  Rows  Attributes 
balancescale  625  5 
breastcancer  266  10 
cars  1728  7 
hayesrothdata  69  5 
lymphography  148  18 
nursery  12,960  9 
soybeansmall  47  36 
specttest  169  22 
tictactoe  958  10 
zoodata  59  17 
Decision Table T  ${\mathit{h}}_{\mathcal{E}}^{\left(1\right)}\left(\mathit{T}\right)$  ${\mathit{h}}_{\mathcal{E}}^{\left(2\right)}\left(\mathit{T}\right)$  ${\mathit{h}}_{\mathcal{E}}^{\left(3\right)}\left(\mathit{T}\right)$  ${\mathit{h}}_{\mathcal{E}}^{\left(4\right)}\left(\mathit{T}\right)$  ${\mathit{h}}_{\mathcal{E}}^{\left(5\right)}\left(\mathit{T}\right)$ 

balancescale  4  4  4  4  4 
breastcancer  9  9  9  8  9 
cars  6  6  6  6  6 
hayesrothdata  4  4  4  4  4 
lymphography  11  11  9  13  10 
nursery  8  8  8  8  8 
soybeansmall  2  6  2  8  2 
specttest  20  5  5  14  11 
tictactoe  7  8  7  8  7 
zoodata  8  6  5  8  5 
Average  7.9  6.7  5.9  8.1  6.6 
Decision Table T  ${\mathit{L}}_{\mathcal{E}}^{\left(1\right)}\left(\mathit{T}\right)$  ${\mathit{L}}_{\mathcal{E}}^{\left(2\right)}\left(\mathit{T}\right)$  ${\mathit{L}}_{\mathcal{E}}^{\left(3\right)}\left(\mathit{T}\right)$  ${\mathit{L}}_{\mathcal{E}}^{\left(4\right)}\left(\mathit{T}\right)$  ${\mathit{L}}_{\mathcal{E}}^{\left(5\right)}\left(\mathit{T}\right)$ 

balancescale  556  5234  4102  5234  4102 
breastcancer  255  446,170  304  10,3642  266 
cars  1136  65,624  3944  65,624  3944 
hayesrothdata  73  72  72  367  72 
lymphography  123  6,653,366  162  8,515,841  153 
nursery  4460  12,790,306  14,422  12,790,306  14,422 
soybeansmall  7  10,029  7  157,640  7 
specttest  123  6983  5495  398,926  1116 
tictactoe  648  864,578  200,847  946,858  940 
zoodata  33  2134  35  13,310  35 
Average  741.4  2,084,449.6  22,939  2,299,774.8  2505.7 
Number of Variables n  ${\mathit{h}}_{\mathcal{E}}^{\left(1\right)}$  ${\mathit{h}}_{\mathcal{E}}^{\left(2\right)}$  ${\mathit{h}}_{\mathcal{E}}^{\left(3\right)}$  ${\mathit{h}}_{\mathcal{E}}^{\left(4\right)}$  ${\mathit{h}}_{\mathcal{E}}^{\left(5\right)}$ 

3  ${}_{0}{2.94}_{3}$  ${}_{0}{2.02}_{3}$  ${}_{0}{1.86}_{3}$  ${}_{0}{2.02}_{3}$  ${}_{0}{1.86}_{3}$ 
4  ${}_{4}{4.00}_{4}$  ${}_{2}{3.05}_{4}$  ${}_{2}{2.97}_{3}$  ${}_{2}{3.05}_{4}$  ${}_{2}{2.97}_{3}$ 
5  ${}_{5}{5.00}_{5}$  ${}_{4}{4.11}_{5}$  ${}_{3}{3.99}_{4}$  ${}_{4}{4.11}_{5}$  ${}_{3}{3.99}_{4}$ 
6  ${}_{6}{6.00}_{6}$  ${}_{5}{5.09}_{6}$  ${}_{5}{5.00}_{5}$  ${}_{5}{5.09}_{6}$  ${}_{5}{5.00}_{5}$ 
Number of Variables n  ${\mathit{L}}_{\mathcal{E}}^{\left(1\right)}$  ${\mathit{L}}_{\mathcal{E}}^{\left(2\right)}$  ${\mathit{L}}_{\mathcal{E}}^{\left(3\right)}$  ${\mathit{L}}_{\mathcal{E}}^{\left(4\right)}$  ${\mathit{L}}_{\mathcal{E}}^{\left(5\right)}$ 

3  ${}_{1}{9.60}_{15}$  ${}_{1}{12.33}_{22}$  ${}_{1}{9.61}_{15}$  ${}_{1}{12.33}_{22}$  ${}_{1}{9.61}_{15}$ 
4  ${}_{15}{21.02}_{29}$  ${}_{14}{44.75}_{70}$  ${}_{11}{27.69}_{58}$  ${}_{14}{44.75}_{70}$  ${}_{11}{27.69}_{58}$ 
5  ${}_{31}{42.54}_{51}$  ${}_{125}{218.05}_{292}$  ${}_{25}{70.19}_{176}$  ${}_{125}{218.05}_{292}$  ${}_{25}{70.19}_{176}$ 
6  ${}_{69}{86.30}_{101}$  ${}_{649}{1171.03}_{1538}$  ${}_{75}{292.99}_{807}$  ${}_{649}{1171.03}_{1538}$  ${}_{75}{292.99}_{807}$ 
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. 
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Azad, M.; Chikalov, I.; Hussain, S.; Moshkov, M. EntropyBased Greedy Algorithm for Decision Trees Using Hypotheses. Entropy 2021, 23, 808. https://doi.org/10.3390/e23070808
Azad M, Chikalov I, Hussain S, Moshkov M. EntropyBased Greedy Algorithm for Decision Trees Using Hypotheses. Entropy. 2021; 23(7):808. https://doi.org/10.3390/e23070808
Chicago/Turabian StyleAzad, Mohammad, Igor Chikalov, Shahid Hussain, and Mikhail Moshkov. 2021. "EntropyBased Greedy Algorithm for Decision Trees Using Hypotheses" Entropy 23, no. 7: 808. https://doi.org/10.3390/e23070808