Next Article in Journal
Sir Isaac Newton Stranger in a Strange Land
Next Article in Special Issue
How to Utilize My App Reviews? A Novel Topics Extraction Machine Learning Schema for Strategic Business Purposes
Previous Article in Journal
Dynamic Topology Reconfiguration of Boltzmann Machines on Quantum Annealers
Previous Article in Special Issue
Deep-Learning-Based Power Generation Forecasting of Thermal Energy Conversion
Article

TNT: An Interpretable Tree-Network-Tree Learning Framework using Knowledge Distillation

1
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
2
PCL Research Center of Networks and Communications, Peng Cheng Laboratory, Shenzhen 518055, China
3
Ping An Life Insurance Company of China, Ltd., Shenzhen 518046, China
*
Authors to whom correspondence should be addressed.
Entropy 2020, 22(11), 1203; https://doi.org/10.3390/e22111203
Received: 24 September 2020 / Revised: 18 October 2020 / Accepted: 20 October 2020 / Published: 24 October 2020
(This article belongs to the Special Issue Statistical Machine Learning for Multimodal Data Analysis)
Deep Neural Networks (DNNs) usually work in an end-to-end manner. This makes the trained DNNs easy to use, but they remain an ambiguous decision process for every test case. Unfortunately, the interpretability of decisions is crucial in some scenarios, such as medical or financial data mining and decision-making. In this paper, we propose a Tree-Network-Tree (TNT) learning framework for explainable decision-making, where the knowledge is alternately transferred between the tree model and DNNs. Specifically, the proposed TNT learning framework exerts the advantages of different models at different stages: (1) a novel James–Stein Decision Tree (JSDT) is proposed to generate better knowledge representations for DNNs, especially when the input data are in low-frequency or low-quality; (2) the DNNs output high-performing prediction result from the knowledge embedding inputs and behave as a teacher model for the following tree model; and (3) a novel distillable Gradient Boosted Decision Tree (dGBDT) is proposed to learn interpretable trees from the soft labels and make a comparable prediction as DNNs do. Extensive experiments on various machine learning tasks demonstrated the effectiveness of the proposed method. View Full-Text
Keywords: deep neural networks; James–Stein Decision Trees; distillable gradient boosted decision tree; interpretable machine learning; knowledge distillation deep neural networks; James–Stein Decision Trees; distillable gradient boosted decision tree; interpretable machine learning; knowledge distillation
Show Figures

Figure 1

MDPI and ACS Style

Li, J.; Li, Y.; Xiang, X.; Xia, S.-T.; Dong, S.; Cai, Y. TNT: An Interpretable Tree-Network-Tree Learning Framework using Knowledge Distillation. Entropy 2020, 22, 1203. https://doi.org/10.3390/e22111203

AMA Style

Li J, Li Y, Xiang X, Xia S-T, Dong S, Cai Y. TNT: An Interpretable Tree-Network-Tree Learning Framework using Knowledge Distillation. Entropy. 2020; 22(11):1203. https://doi.org/10.3390/e22111203

Chicago/Turabian Style

Li, Jiawei; Li, Yiming; Xiang, Xingchun; Xia, Shu-Tao; Dong, Siyi; Cai, Yun. 2020. "TNT: An Interpretable Tree-Network-Tree Learning Framework using Knowledge Distillation" Entropy 22, no. 11: 1203. https://doi.org/10.3390/e22111203

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop