Methodology of Information Study ^†

Abstract

A general study of information is necessary to set firm foundations for the studies of secondary concepts of intelligence, consciousness, and their relation to computation. The lack of such foundations can be blamed for the global-scale promotion of pseudo-scientific myths about the future of humanity. Information study is not ready for this overarching role due to the lack of attention to its methodology. In particular, methods of structurally analyzing information independent of probability and the long overdue development of information semantics and ontology are missing or underdeveloped. This paper has a limited scope, but some directions for development are suggested in particular for the changes in the quest for information semantics.

1. Introduction

The term “information” and its dizzying career in all domains of contemporary life on the global scale brings one more example of technology outpacing our understanding. Claude Shannon, who is frequently considered the father of information study, was aware of the dangers of jumping on the bandwagon of a fashionable and successful theory of communication when writing about it in 1956 [1]. His solution was to separate the study of information into multiple studies of diverse concepts of information relevant to the particular contexts and regulated by the empirical methodologies of these contexts.

This solution has two fundamental flaws. The history of science shows that progress in the understanding of reality has usually been achieved through crossing artificial disciplinary divisions. Moreover, the focus on the narrow field of inquiry is frequently achieved by sweeping the inconvenient and difficult-to-conceptualize ideas under the carpet of the neighboring disciplines. Everyone agrees that the concepts of intelligence, consciousness, and computing are related to information and that it is information that has a primary ontological status to the other concepts. As a consequence, those working on these secondary concepts do not take responsibility for inquiring about the definition, ontological status, structure, or properties of information. Whatever is published about information that fits the preferences of researchers in these related domains is imported by them uncritically in a rather random manner.

Thus, as long as information lacks firm foundations, the secondary studies of intelligence, consciousness, or computing are built on thin ice. It would be naive to claim that the empirical methods of these domains would weed out these incorrect theoretical claims, as purely empirical methods promoted in the past by logical empiricism are well-known illusions. There are no purely empirical methods, and every experiment involves some theoretical assumptions. Hidden theoretical assumptions about information imported to empirical research of associated disciplines may influence the results which are supposed to support the inquiry of information.

The outcomes of the lack of conceptual clarity in the study of information are projected on the understanding (or rather, on the lack) of intelligence, consciousness, and especially on their relationship to computing. This lack of understanding gives an opportunity to “experts” whose only qualification is big wealth or political influence to promote their outlandish visions, such as the future dominance of hostile non-human intelligence leading to the extinction of humanity. The actual already present danger is not the hostile high-level artificial intelligence, but the loss of control over not necessarily intelligent technology. The more widespread the use of technology, the more dependent humans become and the less in control they are of their life, work, and environment.

This is not a criticism of the limitations of technological knowledge, but rather of the limitations of the overarching study of information that integrates specific domains of inquiry such as computer science, physics, genetics, neurophysiology, psychology, biology, etc. The main issue is whether the status of the information study is adequate to the level of development in technology, or in the more specific disciplines where information is recognized as an important concept. The answer is definitely negative.

Although I opposed Shannon’s suggestion to distribute the study of information in multiple empirically-oriented domains without attempting a unified, general approach, I fully agree with his other recommendation: “The subject of information theory has certainly been sold, if not oversold. We should now turn our attention to the business of research and development at the highest scientific plane we can maintain” [1]. In my view, in which a general, unified study of information is necessary and indispensable, this “business of research and development at the highest plane” requires a highly disciplined and comprehensive methodology. Thus far, most of the research on the general concept of information was conducted and guided mainly by common sense rather than a specific methodology. As a result, what was supposed to be the subject of a theory of information lost its association with meaning and became a structureless, indefinite object that is devoid of clear ontological status.

The fact that information has many different definitions and that these differences stimulate never-ending discussions should not trouble us. This just shows that this concept is non-trivial and difficult to conceptualize. More troublesome is the fact that the diverse definitions of information are not followed by the development of comprehensive tools for inquiry or its consistent models. Definitions cannot be true or false. If correctly formulated, they can serve to develop theories of the defined concepts. There are many definitions of information, but the concepts that they provide are rarely followed by theoretical studies of their qualitative (structural) characteristics, and their quantitative descriptions are usually limited to the attempts to reconstruct some elements of Shannon’s communication theory without much concern that this theory programmatically eliminates from its focus the key aspect of information (its meaning), assumes without any justification or recognition the conduit metaphor of information (information is subjugated to the process of communication), and is based on the probabilistic conceptual framework.

2. Putting Information (the Horse) Ahead of Probability (the Cart)

Probability is a very useful tool for inquiries as we can see in physics, chemistry, sociology, or any other disciplines. However, in all these applications, probability enters after structural analysis justifies its use. Shannon writes in his famous paper “[E]ntropy of the set of probabilities” [2]. In the finite case, we can find entropy for every probability distribution, and the concept of information becomes spurious if it does not precede the concept of probability.

That information should be setting the foundations for probability, not the other way around is not a new idea. It stimulated Andrey Kolmogorov in his inquiries in computational complexity almost a half-century ago [3]. More recently, Rota, in his 1998 Turin Lectures, proposed that the direction of the study of information should be based on the logic of partitions with the same objectives of setting the foundations for probability [4]. In both cases, priority was given to the structural analysis of information independent of the concept of probability to serve as its conceptual foundation. Kolmogorov’s proposal was restricted to the context of computing and cannot serve as a solution for the general study of information. Rota presented only an outline of the research direction. Thus, putting the horse ahead of the cart has not yet been accomplished fully. Some steps in this direction are reported elsewhere [5,6].

It is an irony of history that the first study of information of this type was published by Ralph Hartley twenty years before Shannon’s famous paper. Unfortunately, although this paper was cited by Shannon, it was also misinterpreted as being about a special case of Shannon’s probabilistic approach for uniform distribution. In Hartley’s paper [7], the concept of probability does not appear at all. Instead, he refers to the invariance of information with respect to its encoding. This idea of the invariance of information with respect to the changes in encoding is the most important contribution of Hartley that was overlooked by Shannon (and for a long time by everyone else) who identified information with its encoding. Any change of encoding was a change of information. This brings us to the second deficiency in the present state of the study of information: its separation from the meaning.

3. The Meaning of Information Meaning

Hartley postulated the elimination of the psychological aspects of information, such as the language used in its transmission, from the analysis of information. He assumed that the operator of the transmission can use different numbers of primary symbols grouped into arbitrary selections representing the secondary symbols without a change of information. Based on this assumption, he derived his formula for the amount of information. He did not give the name to the invariant of variable encoding and, most likely, he would not use the name “meaning”, but even without an explicit name, his analysis involves a semantical aspect of information. Significantly, in several ways, Hartley’s analysis anticipates the ideas of Kolmogorov that were mentioned above. For instance, Kolmogorov’s minimal program that produces a given configuration of characters has some similarity to optimal encoding that was considered by Hartley. Also, the representation of secondary characters as groupings of primary signs is an example of a partition in Rota’s partition logic of information. Shannon focused on the amount of information carried by the characters and calculated the amount of information carried by the message by adding the amount carried by its characters. Since the characters rarely have any meaning, it was natural for him to dismiss the meaning from the consideration.

A strong criticism of the negligence of meaning, or more generally, the semantic aspect of information in Shannon’s approach that disqualifies it as a theory of information, was made by Yehoshua Bar-Hillel and Rudolf Carnap, and it opened an ever-lasting discussion of this topic [8]. Bar-Hillel and Carnap did not succeed in promoting their approach, and at present, there are no fully developed general semantics of information, despite several attempts. The failures were typically caused by a lack of understanding of the depth of the issue. For instance, there were some attempts to import elements of the logical (linguistic) semantic concepts. The attempts involving the use of the concept of truth (the distinction of true information) were doomed by Alfred Tarski’s Theorem on the Undefinability of the Truth [9]. Tarski’s theorem is for a specific type of information system of predicate logic of the first order, but what does not work in this specific case cannot work for a more general approach. Moreover, the undefinability of the truth is only a tip of the iceberg of logico-philosophical issues involving the concept of the truth within a theory, which can use itself as a subject of inquiry. Obviously, the discourse on information is an informational process.

Due to the limited volume of this paper, it is impossible to review the complicated history of linguistic semantics that casts a long shadow on the attempts to develop semantics of information patterned on the examples from logic or the methodology of science. Some developments in linguistic semantics, such as the Peircean tripartite division into the significant, the signifier, and the interpretant stimulated the development of biosemiotics; however, they cannot be extended to many other contexts of information. My methodological recommendation is to consider “reversed semantics” [10,11].

In linguistic semantics, we start with an information system consisting of a pre-defined language with established logical/grammatical rules. Then, we search for the meaning of a term/sentence/theory among the entities or their relations within an entirely different (“external”) reality.

In reversed semantics, the point of departure is at the informational structures present in this “external reality”. In this, I fully agree with Ralph Landauer’s motto “Information Is Physical” [12], except that in my view, the physical reality has a multistory hierarchic architecture in which there are interactions between different levels, but there is no possibility to reduce the phenomena or their inquiry of the higher levels to those at the lower (reductionism or physicalism) or the other way around. My favorite description of the architecture of reality is in Philip Anderson’s famous article “More is Different” [13]. Thus, my preference would be to say that “Reality is informational and it has a multistory architecture.” The task for semantics is not to search for its entities corresponding to the items or structures of the linguistic or other symbolic information systems, but rather the other way around: to ask about how the informational structures of reality can be encoded symbolically.

4. Conclusions

To make information the fundamental concept of the inquiry of reality in its complex hierarchy, from the level of elementary particles; through molecular structures, living organisms, and their populations embedded in the environment; and to conscious human beings and their cultural organizations, it is necessary to develop an adequate complex of methodologies for the information inquiry. The most urgent tasks are to develop a structural methodology for the study of information independent of the concept of probability and to develop a sufficiently general semantics of information which may have a form of the reversed relationship between natural and symbolic informational structures defined by encoding as it was outlined above.