Next Article in Journal
The Quantum-Classical Transition as an Information Flow
Next Article in Special Issue
Entropy Variation in the Two-dimensional Phase Transition of Anthracene Adsorbed at the Hg Electrode/Ethylene Glycol Solution Interface
Previous Article in Journal
Arguments for the Integration of the Non-Zero-Sum Logic of Complex Animal Communication with Information Theory
Entropy 2010, 12(1), 136-147; doi:10.3390/e12010136

On the Entropy Based Associative Memory Model with Higher-Order Correlations

Department of Electrical Engineering, Faculty of Engineering, Nagaoka University of Technology, Kamitomioka 1603-1, Nagaoka, Niigata 940-21, Japan
Received: 2 January 2010 / Accepted: 18 January 2010 / Published: 22 January 2010
(This article belongs to the Special Issue Entropy in Model Reduction)
View Full-Text   |   Download PDF [488 KB, uploaded 24 February 2015]   |   Browse Figures


In this paper, an entropy based associative memory model will be proposed and applied to memory retrievals with an orthogonal learning model so as to compare with the conventional model based on the quadratic Lyapunov functional to be minimized during the retrieval process. In the present approach, the updating dynamics will be constructed on the basis of the entropy minimization strategy which may be reduced asymptotically to the above-mentioned conventional dynamics as a special case ignoring the higher-order correlations. According to the introduction of the entropy functional, one may involve higer-order correlation effects between neurons in a self-contained manner without any heuristic coupling coefficients as in the conventional manner. In fact we shall show such higher order coupling tensors are to be uniquely determined in the framework of the entropy based approach. From numerical results, it will be found that the presently proposed novel approach realizes much larger memory capacity than that of the quadratic Lyapunov functional approach, e.g., associatron.
Keywords: association model; entropy; memory capacity; lyapunov functional association model; entropy; memory capacity; lyapunov functional
This is an open access article distributed under the Creative Commons Attribution License (CC BY) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Share & Cite This Article

Further Mendeley | CiteULike
Export to BibTeX |
EndNote |
MDPI and ACS Style

Nakagawa, M. On the Entropy Based Associative Memory Model with Higher-Order Correlations. Entropy 2010, 12, 136-147.

View more citation formats

Related Articles

Article Metrics

For more information on the journal, click here


[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert