# From Learning to Consciousness: An Example Using Expected Float Entropy Minimisation

## Abstract

**:**

## 1. Introduction

#### 1.1. Definitions and Theory

**Definition**

**1.**

**Remark**

**1.**

**Definition**

**2**(Weighted relations)

**.**

- 1.
- reflexive if $R(a,a)=1$ for all $a\in S$;
- 2.
- symmetric if $R(a,b)=R(b,a)$ for all $a,b\in S$.

**Remark**

**2.**

**Definition**

**3.**

**Definition**

**4.**

**Definition**

**5**(Float entropy)

**.**

Algorithm 1: Partition algorithm for when using conditional entropy similarly to $\mathrm{efe}$. |

Step 1: Let ${S}_{i}\in {\mathsf{\Omega}}_{S,V}$ be such that: - (a)
- ${S}_{i}$ has yet to be allocated to one of the subsets $A\subseteq {\mathsf{\Omega}}_{S,V}$ partitioning ${\mathsf{\Omega}}_{S,V}$;
- (b)
- of all the elements of ${\mathsf{\Omega}}_{S,V}$ satisfying (a), ${S}_{i}$ has the greatest probability;
- (c)
- for ${A}_{{S}_{i}}:=\{{S}_{j}\in {\mathsf{\Omega}}_{S,V}:\mathrm{d}(R,R\{U,{S}_{j}\})\le \mathrm{d}(R,R\{U,{S}_{i}\})\}$, if there is more than one element of ${\mathsf{\Omega}}_{S,V}$ satisfying (a) and (b) then ${S}_{i}$ is such that ${A}_{{S}_{i}}$ contains the fewest elements. If ${S}_{i}$ is still not unique then choose any such ${S}_{i}$ that satisfies (a), (b) and (c). It will turn out that $\widehat{\mathsf{\Omega}}(R,U)$ is well defined because if $\#{A}_{{S}_{i}}=\#{A}_{{S}_{j}}$ then ${A}_{{S}_{i}}={A}_{{S}_{j}}$.
Step 2: Define the next subset of ${\mathsf{\Omega}}_{S,V}$ contributing to the partition of ${\mathsf{\Omega}}_{S,V}$ to be
$$\begin{array}{c}\{{S}_{j}\in {A}_{{S}_{i}}:{S}_{j}\mathrm{has}\mathrm{yet}\mathrm{to}\mathrm{be}\mathrm{allocated}\mathrm{to}\mathrm{one}\mathrm{of}\mathrm{the}\mathrm{subsets}A\mathrm{partitioning}{\mathsf{\Omega}}_{S,V}\}.\end{array}$$
Step 3: If all the elements of ${\mathsf{\Omega}}_{S,V}$ have been allocated then define the partition $\widehat{\mathsf{\Omega}}(R,U)$ of ${\mathsf{\Omega}}_{S,V}$ as the set of the subsets defined during each occurrence of Step 2. Otherwise, if there are still elements of ${\mathsf{\Omega}}_{S,V}$ to be allocated then return to Step 1. |

## 2. Methods and Materials

#### 2.1. Typical Data from Digital Photographs

#### 2.2. Using Efe-Histograms Obtained from Monte-Carlo Methods

## 3. Results

## 4. Discussion

**Definition**

**6**(Multi-relational float entropy)

**.**

#### 4.1. The Inherent Probability Distribution a System Defines

#### 4.2. Experimental Testing on the Brain

#### 4.3. Observable Phenomenon and Unobservable Effects of Relationship Isomorphism

## 5. Conclusions

## Supplementary Materials

## Funding

## Acknowledgments

## Conflicts of Interest

## Abbreviations

FE | Float Entropy |

EFE | Expected Float Entropy |

## Appendix A.

**Table A1.**Notation (most of the formal definitions can be found in Section 1.1).

Symbol | Description |
---|---|

$\alpha $ | a relationship isomorphism $\alpha :X\to Y$. |

$a,b,c,\dots $ | elements of S but also used to denote elements of other sets. |

A | an element of ${2}^{{\mathsf{\Omega}}_{S,V}}$. For clarification, A is just a subset of ${\mathsf{\Omega}}_{S,V}$. |

$B(n,p)$ | the Binomial distribution. |

${C}_{0},{C}_{1},{C}_{2},\dots $ | conditions, involving weighted relations, in the definition of |

multi-relational float entropy. | |

$\mathrm{d}$ | a metric on the set of all weighted relations on S or, in places, a metric on |

${[0,1]}^{n}$. | |

${\mathrm{d}}_{n}$ | for $n\in \mathbb{N}\cup \{\infty \}$, a metric (on the set of all weighted relations on S) |

obtained from the corresponding p-norm, for $p=n$, on a finite dimensional | |

vector space. | |

$\mathrm{efe}(R,U,P)$ | the expected float entropy, relative to U and R, of the given system. |

$\mathrm{efe}(R,U,T)$ | the mean approximation of $\mathrm{efe}(R,U,P)$. |

$\mathrm{fe}(R,U,{S}_{i})$ | the float entropy, relative to U and R, of the data element ${S}_{i}$. |

$\mathrm{fe}(R,U,{R}_{1},{U}_{1},{R}_{2},{U}_{2},\dots ,{S}_{i})$ | the multi-relational float entropy, relative to $U,{U}_{1},{U}_{2},\dots $ and $R,{R}_{1},{R}_{2},\dots $, |

of the data element ${S}_{i}$. | |

${f}_{i}$ | the map ${f}_{i}:S\to V$ corresponding to the data element ${S}_{i}$. |

H | the Shannon entropy of the system. |

node 1,node 2,node 3,… | elements of S. |

$N(np,np(1-p\left)\right)$ | the Normal approximation of the Binomial distribution $B(n,p)$. |

P | the probability distribution $P:{\mathsf{\Omega}}_{S,V}\to [0,1]$ of the random variable defined |

by the bias of the given system. P extends to a probability measure on ${2}^{{\mathsf{\Omega}}_{S,V}}$. | |

R | an element of ${\mathsf{\Psi}}_{S}$. |

$R\{U,{S}_{i}\}$ | the element of ${\mathsf{\Psi}}_{S}$ given by the canonical definition |

$R\{U,{S}_{i}\}(a,b):=U({f}_{i}\left(a\right),{f}_{i}\left(b\right))$ for all $a,b\in S$. | |

S | a nonempty finite set; in most places S denotes the set of nodes of a system. |

${S}_{i}$ | a data element for S, i.e., a system state given by the aggregate of the node |

states. | |

T | the typical data for the given system, i.e., T is a finite set of numbered |

observations of the given system. | |

$\tau $ | the map $\tau :\{1,\dots ,\#T\}\to \{i:{S}_{i}\in {\mathsf{\Omega}}_{S,V}\}$ for which ${S}_{\tau \left(k\right)}$ is the data |

element representation of observation number k in T. $\tau $ need not be | |

injective. | |

U | an element of ${\mathsf{\Psi}}_{V}$. |

${v}_{1},{v}_{2},{v}_{3},\dots $ | elements of V. |

V | the node repertoire, i.e., the set of node states for a given system. |

W | a finite test set of numbered observations of the given system. |

${W}_{\mathrm{mfe}}$ | the set of numbered minimum float entropy completions of the obfuscated |

elements of W. | |

$\omega $ | the map $\omega :\{1,\dots ,\#W\}\to \{i:{S}_{i}\in {\mathsf{\Omega}}_{S,V}\}$ for which ${S}_{\omega \left(k\right)}$ is the data |

element representation of observation number k in W. $\omega $ need not be | |

injective. | |

${\mathsf{\Psi}}_{S}$ | the set of all reflexive, symmetric weighted-relations on S. |

${\mathsf{\Psi}}_{V}$ | the set of all reflexive, symmetric weighted-relations on V. |

${\mathsf{\Omega}}_{S,V}$ | the set of all data elements ${S}_{i}$, given S and V. |

$\widehat{\mathsf{\Omega}}(R,U)$ | a partition of ${\mathsf{\Omega}}_{S,V}$ determined by R, U and, implicitly, P. |

${2}^{{\mathsf{\Omega}}_{S,V}}$ | the power set of ${\mathsf{\Omega}}_{S,V}$. |

## References

- Mason, J.W.D. Quasi-conscious multivariate systems. Complexity
**2016**, 21, 125–147. [Google Scholar] [CrossRef] - Mason, J.W.D. Consciousness and the structuring property of typical data. Complexity
**2013**, 18, 28–37. [Google Scholar] [CrossRef] - Zhang, Q.; Sepulveda, F. A model study of the neural interaction via mutual coupling factor identification. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Seogwipo, Korea, 11–15 July 2017; pp. 3329–3332. [Google Scholar] [CrossRef]
- Aru, J.; Rutiku, R.; Wibral, M.; Singer, W.; Melloni, L. Early effects of previous experience on conscious perception. Neurosci. Conscious.
**2016**, 2016, niw004. [Google Scholar] [CrossRef] [PubMed] - Audiffren, J.; Kadri, H. Equivalence of Learning Algorithms. arXiv, 2014; arXiv:1406.2622. [Google Scholar]
- Bienenstock, E.L.; Cooper, L.N.; Munro, P.W. Theory for the development of neuron selectivity—Orientation specificity and binocular interaction in visual-cortex. J. Neurosci.
**1982**, 2, 32–48. [Google Scholar] [CrossRef] [PubMed] - Kirkwood, A.; Rioult, M.G.; Bear, M.F. Experience-dependent modification of synaptic plasticity in visual cortex. Nature
**1996**, 381, 526–528. [Google Scholar] [CrossRef] [PubMed] - Dudek, S.M.; Bear, M.F. Homosynaptic Long-Term Depression in Area CA1 of Hippocampus and Effects of N-Methyl-D-Aspartate Receptor Blockade. Proc. Natl. Acad. Sci. USA
**1992**, 89, 4363–4367. [Google Scholar] [CrossRef] [PubMed] - Lamme, V.A.; Supèr, H.; Spekreijse, H. Feedforward, horizontal, and feedback processing in the visual cortex. Curr. Opin. Neurobiol.
**1998**, 8, 529–535. [Google Scholar] [CrossRef] - Friston, K. A Free Energy Principle for Biological Systems. Entropy
**2012**, 14, 2100–2121. [Google Scholar] [CrossRef] [Green Version] - Haken, H.; Portugali, J. Information and Self-Organization. Entropy
**2017**, 19, 18. [Google Scholar] [CrossRef] - Cover, T.; Thomas, J. Elements of Information Theory, 2nd ed.; Wiley: Hoboken, NJ, USA, 2006; p. 776. [Google Scholar]
- Krejic, N.; Jerinkic, N.K. Stochastic Gradient Methods for Unconstrained Optimization. Pesqui. Oper.
**2014**, 34, 373–393. [Google Scholar] [CrossRef] - Nocedal, J.; Wright, S. Numerical Optimization, 2nd ed.; Springer Series in Operations Research and Financial Engineering; Springer: New York, NY, USA, 2006; p. 664. [Google Scholar]
- Conn, A.R.; Scheinberg, K.; Vicente, L.N. Introduction to Derivative-Free Optimization; MPS-SIAM Series on Optimization; SIAM: Philadelphia, PA, USA, 2009; p. 295. [Google Scholar]
- Oizumi, M.; Albantakis, L.; Tononi, G. From the Phenomenology to the Mechanisms of Consciousness: Integrated Information Theory 3.0. PLoS Comput. Biol.
**2014**, 10, e1003588. [Google Scholar] [CrossRef] [PubMed] - Balasubramanian, M.; Polimeni, J.; Schwartz, E.L. The V1-V2-V3 complex: Quasiconformal dipole maps in primate striate and extra-striate cortex. Neural Netw.
**2002**, 15, 1157–1163. [Google Scholar] [CrossRef] - Schwartz, E.L. Spatial mapping in primate sensory projection—Analytic structure and relevance to perception. Biol. Cybern.
**1977**, 25, 181–194. [Google Scholar] [CrossRef] [PubMed] - Grindrod, P. On human consciousness: A mathematical perspective. Netw. Neurosci.
**2017**, 2, 23–40. [Google Scholar] [CrossRef] - Edelman, G.M.; Tononi, G. A Universe of Consciousness: How Matter Becomes Imagination; Basic Books: New York, NY, USA, 2000; p. 274. [Google Scholar]
- Ascoli, G.A. The complex link between neuroanatomy and consciousness. Complexity
**2000**, 6, 20–26. [Google Scholar] [CrossRef] - Sporns, O. Network analysis, complexity, and brain function. Complexity
**2002**, 8, 56–60. [Google Scholar] [CrossRef] [Green Version] - Miyawaki, Y.; Uchida, H.; Yamashita, O.; Sato, M.A.; Morito, Y.; Tanabe, H.C.; Sadato, N.; Kamitani, Y. Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders. Neuron
**2008**, 60, 915–929. [Google Scholar] [CrossRef] [PubMed] [Green Version]

**Figure 2.**Graph illustration of the weighted relations in Table 3, showing strongest relationships (solid lines) and intermediate relationships (dash lines).

**Figure 3.**An $\mathrm{efe}$-histogram for the training data using 2000 $\mathrm{efe}$ observations and a bin interval of 0.05. The $\mathrm{efe}$ value of the approximate solution is shown (triangular marker).

**Figure 4.**A histogram showing the proportion of the 200 test data elements that have n out of four nodes correctly completed, for $n\in \{0,1,2,3,4\}$, when using minimum float entropy completion (solid line) and when completing each node independently of the others by selecting for each node the most commonly observed state for that node in the training set (light dash line). For further comparison, the binomial distribution B(4,1/4) which gives the probability of correctly completing n out of four nodes when guessing node states uniformly at random for $n\in \{0,1,2,3,4\}$ (heavy dash line).

**Figure 5.**A histogram showing the proportion of the elements of the obfuscated version of $T\cup W$ that have n out of nine nodes correctly completed, when using nftool, for $n\in \{0,1,\cdots ,9\}$ (dotted line). For comparison, the results shown in Figure 4 are included. The results when using minimum float entropy completion are shown (solid line), the results are shown for when completing each node independently of the others by selecting for each node the most commonly observed state for that node in the training set (light dash line), and the distribution for guessing node states uniformly at random is shown (heavy dash line).

**Figure 6.**An $\mathrm{efe}$-histogram for the test data as sampled using 20,000 $\mathrm{efe}$ observations and a bin interval of 0.025. The $\mathrm{efe}$ value of the approximate solution is shown (triangular marker).

**Figure 7.**An $\mathrm{efe}$-histogram for the minimum float entropy completed test data using 20,000 $\mathrm{efe}$ observations and a bin interval of 0.025. The $\mathrm{efe}$ value of the approximate solution is shown (triangular marker).

**Figure 8.**An $\mathrm{efe}$-histogram for the uniform randomly completed test data using 20,000 $\mathrm{efe}$ observations and a bin interval of 0.025. The $\mathrm{efe}$ value of the approximate solution is shown (triangular marker).

**Figure 9.**Eight tuples each containing three of the initial nodes. The geometric layout of the initial node is that of the sampling locations in Figure 1.

U | ${\mathit{v}}_{1}$ | ${\mathit{v}}_{2}$ | ${\mathit{v}}_{3}$ | ⋯ | R | Node 1 | Node 2 | Node 3 | ⋯ |
---|---|---|---|---|---|---|---|---|---|

${v}_{1}$ | 1 | ${u}_{1,2}$ | ${u}_{1,3}$ | ⋯ | node 1 | 1 | ${r}_{1,2}$ | ${r}_{1,3}$ | ⋯ |

${v}_{2}$ | ${u}_{2,1}$ | 1 | ${u}_{2,3}$ | ⋯ | node 2 | ${r}_{2,1}$ | 1 | ${r}_{2,3}$ | ⋯ |

${v}_{3}$ | ${u}_{3,1}$ | ${u}_{3,2}$ | 1 | ⋯ | node 3 | ${r}_{3,1}$ | ${r}_{3,2}$ | 1 | ⋯ |

⋮ | ⋮ | ⋮ | ⋮ | ⋱ | ⋮ | ⋮ | ⋮ | ⋮ | ⋱ |

**Table 2.**Node states of the typical data element obtained from the sampling in Figure 1.

Node 1 | Node 2 | Node 3 | Node 4 | Node 5 | Node 6 | Node 7 | Node 8 | Node 9 | |
---|---|---|---|---|---|---|---|---|---|

${S}_{\tau \left(1\right)}$ | 0.000 | 294.449 | 294.449 | 0.000 | 147.224 | 294.449 | 147.224 | 0.000 | 147.224 |

U | 0 | 147.224 | 294.449 | 441.673 | |||||

0 | 1 | 0.29688 | 0.04688 | 0.01563 | |||||

147.224 | 0.29688 | 1 | 0.42188 | 0.10938 | |||||

294.449 | 0.04688 | 0.42188 | 1 | 0.32813 | |||||

441.673 | 0.01563 | 0.10938 | 0.32813 | 1 | |||||

R | node 1 | node 2 | node 3 | node 4 | node 5 | node 6 | node 7 | node 8 | node 9 |

node 1 | 1 | 0.95313 | 0.73438 | 0.95313 | 0.79688 | 0.60938 | 0.73438 | 0.60938 | 0.60938 |

node 2 | 0.95313 | 1 | 0.95313 | 0.79688 | 0.95313 | 0.79688 | 0.60938 | 0.73438 | 0.60938 |

node 3 | 0.73438 | 0.95313 | 1 | 0.60938 | 0.79688 | 0.95313 | 0.60938 | 0.60938 | 0.73438 |

node 4 | 0.95313 | 0.79688 | 0.60938 | 1 | 0.95313 | 0.73438 | 0.95313 | 0.79688 | 0.60938 |

node 5 | 0.79688 | 0.95313 | 0.79688 | 0.95313 | 1 | 0.95313 | 0.79688 | 0.95313 | 0.79688 |

node 6 | 0.60938 | 0.79688 | 0.95313 | 0.73438 | 0.95313 | 1 | 0.60938 | 0.79688 | 0.95313 |

node 7 | 0.73438 | 0.60938 | 0.60938 | 0.95313 | 0.79688 | 0.60938 | 1 | 0.95313 | 0.73438 |

node 8 | 0.60938 | 0.73438 | 0.60938 | 0.79688 | 0.95313 | 0.79688 | 0.95313 | 1 | 0.95313 |

node 9 | 0.60938 | 0.60938 | 0.73438 | 0.60938 | 0.79688 | 0.95313 | 0.73438 | 0.95313 | 1 |

© 2019 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Mason, J.W.D.
From Learning to Consciousness: An Example Using Expected Float Entropy Minimisation. *Entropy* **2019**, *21*, 60.
https://doi.org/10.3390/e21010060

**AMA Style**

Mason JWD.
From Learning to Consciousness: An Example Using Expected Float Entropy Minimisation. *Entropy*. 2019; 21(1):60.
https://doi.org/10.3390/e21010060

**Chicago/Turabian Style**

Mason, Jonathan W. D.
2019. "From Learning to Consciousness: An Example Using Expected Float Entropy Minimisation" *Entropy* 21, no. 1: 60.
https://doi.org/10.3390/e21010060