Symmetry, Entropy and Computer Science ^†

Abstract

We analyze Symmetry, Entropy, and therefore, Uncertainty Measures, to obtain new ways to model adequate conditions, constructed from vague pieces of information. Over time, it was adapted by Shannon, thus creating the Information Theory. However, the Hungarian mathematician Rényi shows that there exist different and valid entropy measures, according to the need of applications. So, it will be very necessary to clarify the different types of measures, and their mutual relationships.

The contributions of Kolmogorov to this mathematical theory provide great advances to the Shannon formulations, proposing a new complexity theory, now translated to Computation. According to such theory, the complexity of a message is given by the size of the program necessary to make possible the reception of such a message. From these ideas, he analyzes the entropy of literary texts. Such entropy appears as a function of the semantic capacity of the texts, depending on factors as their extension and also the flexibility of the corresponding language.

Wiener, founder of Cybernetics, also proposed a similar vision of such a problem in 1948. However, the approach used by Shannon differs from that of Wiener in the nature of the transmitted signal and in the type of decision made at the receiver. In the Shannon model, messages are firstly encoded, and then transmitted, whereas in the Wiener model the signal is communicated directly through the channel without it needing to be encoded.