Next Article in Journal
Thermodynamics in the Universe Described by the Emergence of Space and the Energy Balance Relation
Previous Article in Journal
Quality-Oriented Perceptual HEVC Based on the Spatiotemporal Saliency Detection Model
Previous Article in Special Issue
A Novel Belief Entropy for Measuring Uncertainty in Dempster-Shafer Evidence Theory Framework Based on Plausibility Transformation and Weighted Hartley Entropy
Article Menu

Article Versions

Export Article

Open AccessConcept Paper
Entropy 2019, 21(2), 166;

Learning Entropy as a Learning-Based Information Concept

Department of Mechanics, Biomechanics, and Mechatronics, Research Centre for Low-Carbon Energy Technologies, Faculty of Mechanical Engineering, Czech Technical University, Prague, Czech Republic
Department of Electrical and Computer Engineering, University of Manitoba, Canada
Dpt. of Radiological Imaging and Informatics, Tohoku Univ. Grad. School of Medicine, Intelligent Biomed. Sys. Eng. Lab., Grad. School of Biomed. Eng., Tohoku University, Sendai 980-8575, Japan
Author to whom correspondence should be addressed.
Received: 30 December 2018 / Revised: 28 January 2019 / Accepted: 5 February 2019 / Published: 11 February 2019
PDF [1662 KB, uploaded 11 February 2019]


Recently, a novel concept of a non-probabilistic novelty detection measure, based on a multi-scale quantification of unusually large learning efforts of machine learning systems, was introduced as learning entropy (LE). The key finding with LE is that the learning effort of learning systems is quantifiable as a novelty measure for each individually observed data point of otherwise complex dynamic systems, while the model accuracy is not a necessary requirement for novelty detection. This brief paper extends the explanation of LE from the point of an informatics approach towards a cognitive (learning-based) information measure emphasizing the distinction from Shannon’s concept of probabilistic information. Fundamental derivations of learning entropy and of its practical estimations are recalled and further extended. The potentials, limitations, and, thus, the current challenges of LE are discussed.
Keywords: learning; information; novelty detection; non-probabilistic entropy; learning systems learning; information; novelty detection; non-probabilistic entropy; learning systems
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

Share & Cite This Article

MDPI and ACS Style

Bukovsky, I.; Kinsner, W.; Homma, N. Learning Entropy as a Learning-Based Information Concept. Entropy 2019, 21, 166.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top