Mark Burgin’s Theory of Information

Abstract

A review of a major, definitive source book on the basis of information theory is presented.

Mark Burgin’s Theory of Information [1], published in 2010, is a landmark in the theory and the history of the theory of information. It is an expert, totally convincing combination of three major functional parts: a complete resume of the “state-of-the-art” of the theoretical and practical knowledge of information; a new parametric type of comprehensive definition of information that makes it possible to build a unified general theory of information (GTI) that explains and determines what information is; and a clear outline of the major problems and challenges for further research.

The result is that the Theory of Information provides both an accessible, focused examination of the contemporary state of information theory and its ramifications for the computer industry, networks, artificial intelligence, embedded systems, and the Internet. The book demonstrates how the GTI can explain relations between a huge diversity of meanings of the word information, making it possible to unify these understandings and to explicate the essence of such a critical phenomenon. A new practically oriented perspective on information processes, including education, entertainment, and networking is developed, and problems of information dynamics and pragmatics are also addressed.

The originality of this book lies in the author’s ability to integrate perspectives on information from logic, epistemology and systems theory in a rigorous mathematical framework. For example, for a long time, parametric systems have been frequently used in mathematics and its applications but the parameters are, as a rule, only numerical and are considered as quantities that define certain characteristics of systems. In the case of the general theory of information (GTI), the parameter is very general. The parametric definition of information utilizes a system parameter called an infological system that plays the role of a parameter that discerns different kinds of information, e.g., social, personal, chemical, biological, genetic, or cognitive, and combines all existing kinds and types of information in one general concept of “information”.

This parametric approach provides a tool for building the GTI as a synthetic approach, which organizes and encompasses all main directions in information theory. In Burgin’s next step, on the meta-axiomatic level, it is formulated as a system of principles, explaining what information is (by means of Ontological Principles) and how to measure information (by means of Axiological Principles). On the level of science, mathematical models of information are constructed. One type of model bases the mathematical stratum of the general theory of information on category theory. Abstract categories allow one to develop flexible models for information and its flow, as well as for computers, networks and computation. Another type of model establishes functional representation of infological systems representing information as an operator in functional spaces.

The book is organized as a three-component system. At first (in Chapter 1), contemporary interpretations and explications of the term information are analyzed (Section 1.1) and the role of information in contemporary society (Section 1.2), nature (Section 1.3) and technology (Section 1.4) is discussed. In Chapter 2, the general theory of information is presented. The third component of the book (Chapters 3–7) contains an exposition of popular and not so popular information theories. For these theories, only some basics that allow one to better comprehend a more detailed exposition are given. It provides an opportunity for the reader to understand how the general theory of information unifies existing knowledge in information studies.

In more detail, Chapter 2 contains an informal exposition of the GTI and its applications to the theory of knowledge and psychology. Section 2.1 develops theoretical and methodological foundations for the GTI. These foundations include the general structures of the world, mathematical definition of the concept structure, and elements of semiotics, the discipline that studies signs, symbols and their systems.

Sections 2.2 and 2.3 contain the basics of the general theory of information, giving its conceptual and methodological foundations. In order to develop the general theory of information, three levels of formalization are used: principles, postulates, and axioms. Principles represent informal assumptions related to studied phenomena, which, in our case, are information, information processes and systems. Postulates describe connections between the theory domain and theoretical constructions. Axioms characterize properties of theoretical constructions (e.g., mathematical structures) used in this theory. Thus, the base of the general theory of information is a system of principles and there are two groups of such principles: ontological and axiological (See also [2]).

Ontological principles studied in Section 2.2 reflect major features of information essence and behavior. They give an answer to the question "What is information?" In particular, the Ontological Principle O2 and its versions define information as a phenomenon that exists in nature, society, mentality of people, virtual reality, and in the artificial world of machines and mechanisms created by people.

Axiological principles studied in Section 2.3 explain how to evaluate, estimate and measure information and what measures of information are necessary. Section 2.4 demonstrates what kinds of information exist and how to discern them. The relationships between information, data, and knowledge are studied in Section 2.5 and the relationships between emotions and information in Section 2.6.

Chapter 3 contains a brief exposition of several directions and applications of statistical information theory. The relationships between information and communication are studied in Section 3.1. The relationships between information, uncertainty and entropy are considered in Section 3.2. The problem of the difference between information in conventional systems and quantum information is treated in Section 3.3. Section 3.4 demonstrates what relations exist between information and problem solving. Section 3.5 describes axiomatic foundations of the statistical theory of information. How the theory of information is used in physics is explained in Section 3.6.

In contrast to statistical theories of information, the semantic theories of information, which are described in Chapter 4, study the meaning of information. The meaning of information is generally understood as the assumption that every piece of information has the characteristic that it makes a positive assertion and at the same time clearly denies the opposite of that assertion. However, meaning is a more complicated phenomenon and to understand it in the context of information theory, Chapter 4 starts with a study of three communicational aspects, or dimensions, of information: syntax, semantics and pragmatics (cf. Section 4.1). Then, semantic information theories are described, that is, theories that make emphasis on the semantics of information. Section 4.2 gives an exposition of the first semantic information theory developed by Bar-Hillel and Carnap [3], as well as its later developments and improvements. Section 4.3 reflects on knowledge-oriented information theories.

Chapter 5 provides a broad perspective on algorithmic information theory with its main peculiarities and constructions. In comparison with the majority of sources on algorithmic information theory, the exposition of this theory is given on several levels. At first, the conventional level of recursive algorithms is considered in Sections 5.1 and 5.2. Burgin is able to upgrade the usual recursive algorithmic approach to information to a next, super-recursive level of the algorithmic universe in Section 5.3. This is a necessary step as super-recursive algorithms are essentially more efficient in processing information than recursive algorithms and give more adequate models for the majority of information systems. Section 5.4 contains a relativized version of algorithmic information theory. Finally, the highest level is presented in the form of an axiomatic algorithmic information theory considered in Section 5.5.

Chapter 6 contains an exposition of pragmatic theories of information. Section 6.1 considers the economics of information developed by Marschak, Arrow, and others, the role of information in economic activity and measures for estimation of the economic value and cost of information and information sources. Section 6.2 further studies such important characteristics as value, cost, and quality of information. Section 6.3, which was one of those of the most interest to this reviewer, contains elements of the qualitative information theory developed by Mazur [4].

Dynamic theories of information are presented in Chapter 7 starting with, in Section 7.1, the theories of information flow developed by Dretske, Barwise and Seligman. Section 7.2 contains an exposition of the operator theory of information developed by Chechkin [5].

Section 7.3 exhibits information algebra and geometry. Elements of the mathematical component of the GTI are expounded in Subsection 7.3.1. This mathematical theory is based on functional models where information dynamics is represented by operators in function spaces. Other publications by Burgin show that it is also possible to develop a mathematical component of the GTI based on category theory.

Abstract information algebra in the sense of Kohlas, which captures a variety of formalisms in computer science, is presented in Subsection 7.3.2. For instance, abstract information algebras can model operations and processes with information carriers, such as data, texts, and documents. Information geometry is discussed in Subsection 7.3.3, demonstrating the application of geometrical methods and structures to information studies.

The last Chapter, Chapter 8 contains conclusions and directions for future research, and a series of valuable Appendices on the Mathematical Foundations of Information Theory, set theoretical, logical, algorithmic, number theoretical etc., is also provided.

The exposition of the material is such that different groups of readers will each find something of interest and value. Those who want to know more about the history of information studies and get a general perspective of the current situation in this area can skip proofs and even many results that are given in the strict mathematical form. Those who have a sufficient mathematical training and are interested in formalized information theories can skip the preliminary discussions and go directly to the sections that contain the mathematical expositions.

If there is any criticism to be made of this Grundlage in the best sense of the German term, it is in the author’s tendency to err on the side of completeness, listing all the ways a particular subject may be viewed, or giving the lists of definitions that different authors provide without always indicating which is to be preferred. In view of Burgin’s own prior contributions over a period of 40 years (some 80 references in this book alone), one would welcome, from time to time, a more personal statement. But this is only a quibble about style, not about the scientific content of the book.

In conclusion, I feel that this book masterfully presents and organizes the bulk of the major existing directions in information theory, giving a broader picture than any other published book in this area. Its new approach drastically changes how information should be viewed and studied, showing, for example, that what is commonly termed “information” is, as a rule, only a container or carrier of information but not information itself.

Thus, this detailed foundational exposition of a General Theory of Information, supported by numerous examples and illustrations, can serve as the basis for the development of a new paradigm for information science and information practice including information retrieval, processing and transmission. Researchers and advanced students of information theory and practice will find the book an essential resource for a large variety of methods, techniques and theories in information studies. In my view, the Theory of Information is suitable as a textbook in information theory for students of technical, scientific, and mathematical subjects, as well as a supplementary textbook in traditional courses on information theory at all levels.

Conclusions

In conclusion, Burgin’s GTI reveals fascinating relations between matter, knowledge, energy, and information and makes it possible to discuss new types of information such as affective information and effective information in an organized fashion. Both the expert and the general reader will gain an essential set of new ideas and tools with which to understand, use and further develop the concept of information.

Replies to Reviews

One reviewer has asked for some additional information in very pertinent areas that I am pleased to include in this review:

• Infological systems: Burgin proposes the concept of “infological systems” as part of the characterization of informational systems and processes that emphasizes their ontological aspects (the term is a contraction of informational-ontological). The information involved in an infological system is, roughly, defined as both constituted by and acting upon structural subsystems which one designates as its infological system. For example, systems of knowledge are infological systems.
• Energy: With regard to energy, suggests that the phenomena of physical, mental, and emotional energy, and information itself as structural energy, are all particular cases of information in broad sense. However, Burgin does not undertake, in this book, a detailed analysis of information processes in biological systems in which information and energy problems must be looked at in tandem. The suggestion that he could do so elsewhere is noted.
• Observer dependence of information: contra the reviewer, I feel that Burgin is concerned mainly with observer-independent information and the complex rules that it follows. In any event, he states clearly that “we do not need any implication that the observer, or the observer system in quantum mechanics, is human nor need have any peculiar property other than that of interacting with the ‘observed’ system.”

Finally, another reviewer criticized my book review, and thus by implication the Burgin book, for not referring to the work of some authors, say A and B. My reply to him is that Burgin’s Theory of Information is not intended to be a complete review of the literature. I noted the absence, for example, of such well-known authors in the field as C, D and E. My feeling is that one may agree or disagree with Burgin’s theory, more or less, as the case may be, but any judgment should be made on its own merits.