Next Article in Journal / Special Issue
The Fractional Differential Polynomial Neural Network for Approximation of Functions
Previous Article in Journal
Spectral Entropy, Empirical Entropy and Empirical Exergy for Deterministic Boundary-Layer Structures
Previous Article in Special Issue
Fractional Heat Conduction in an Infinite Medium with a Spherical Inclusion
Due to scheduled maintaince work on one of the switches in our server center, our websites www.mdpi.com and susy.mdpi.com may experience short service dirsuptions of up to 5 minutes on 30 January 2015 00:00 PM EST time.
Entropy 2013, 15(10), 4159-4187; doi:10.3390/e15104159
Article

Learning Entropy: Multiscale Measure for Incremental Learning

Received: 26 July 2013 / Revised: 17 September 2013 / Accepted: 22 September 2013 / Published: 27 September 2013
(This article belongs to the Special Issue Dynamical Systems)
Download PDF [729 KB, uploaded 27 September 2013]
Abstract: First, this paper recalls a recently introduced method of adaptive monitoring of dynamical systems and presents the most recent extension with a multiscale-enhanced approach. Then, it is shown that this concept of real-time data monitoring establishes a novel non-Shannon and non-probabilistic concept of novelty quantification, i.e., Entropy of Learning, or in short the Learning Entropy. This novel cognitive measure can be used for evaluation of each newly measured sample of data, or even of whole intervals. The Learning Entropy is quantified in respect to the inconsistency of data to the temporary governing law of system behavior that is incrementally learned by adaptive models such as linear or polynomial adaptive filters or neural networks. The paper presents this novel concept on the example of gradient descent learning technique with normalized learning rate.
Keywords: incremental learning; adaptation plot; multiscale; learning entropy; individual sample learning entropy; approximate learning entropy; order of learning entropy; learning entropy of a model; non-Shannon entropy; novelty detection; chaos; time series; HRV; ECG incremental learning; adaptation plot; multiscale; learning entropy; individual sample learning entropy; approximate learning entropy; order of learning entropy; learning entropy of a model; non-Shannon entropy; novelty detection; chaos; time series; HRV; ECG
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Export to BibTeX |
EndNote


MDPI and ACS Style

Bukovsky, I. Learning Entropy: Multiscale Measure for Incremental Learning. Entropy 2013, 15, 4159-4187.

AMA Style

Bukovsky I. Learning Entropy: Multiscale Measure for Incremental Learning. Entropy. 2013; 15(10):4159-4187.

Chicago/Turabian Style

Bukovsky, Ivo. 2013. "Learning Entropy: Multiscale Measure for Incremental Learning." Entropy 15, no. 10: 4159-4187.


Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert