1. Introduction
As with any natural phenomenon, there are two main problems related to information. The first one is to define what information is and to find what basic properties it has. The second problem is how to measure and evaluate information. In this paper, we consider the second problem.
From the beginning of the development of information theory, it was known more how to measure information than what information is. Hartley and Shannon gave effective formulas for measuring the quantity of information. However, without understanding the phenomenon of information, these formulas bring misleading results when applied to irrelevant domains.
At the same time, a variety of information definitions have been introduced. Being mostly vague and limited, these definitions have brought confusion into information studies (cf., for example, [
34,
4,
19,
39]).
The existing confusion with the term information is increased when researchers call by the name “information” a measure of information or even a value of such a measure. For example, many call by the name “information” Shannon’s quantity of information
I = -Σ
i=1n pi log
pi or Renyi’s measure of information
Hα(
X) = (1 - α)
-1log Σ
x∈X Pα(
x) ([
29,
15]) or pragmatic measure of information
IM(
p, q) = Σ
i,m pi/m ϕ
m log
2 (
pi/m/
qi ) ([
37]). At the same time, some researchers (cf., for example, [
30] or [
21]) never did this. That is why it is so important to explain and understand distinctions between some phenomena and their measures. This completely refers to information.
Even if we have an answer to the question what is information, it is not sufficient for practical purposes of information processing. The main problem in this perspective is how to measure or, at least, to evaluate information. The results of [
8], which describe information as a general phenomenon and its basic properties by means of ontological principles, provide a base for developing a unified theory for information evaluation and measurement. This is done in the second section of the paper, which goes after Introduction. It contains the axiological component of the general theory of information. This component is developed on the base of axiomatic methodology, providing basic axiological principles for information evaluation and measurement. Basic axiological principles explain what are basic properties of measures and estimates for information. These principles systematize and unify different approaches, existing as well as possible, to construction and utilization of information measures. This, axiological aspect of the theory is not less important than the ontological one because methods of modern science emphasize importance of measurement and evaluation that are technical tools for observation and experiment in scince, as well as for engineering. In the third section of this paper, it demonstrated how the main directions of information theory as a whole are unified and systemized in the context of the general theory of information.
2. Basic Axiological Principles of the General Theory of Information
Basic axiological principles explain how to evaluate information and what measures of information are necessary.
According to the ontological principles [
8,
11,
10], information causes changes either in the whole system
R that receives information or in an inforlogical subsystem IF(
R) of this system. Consequently, it is natural to assume that measure of information is determined by the results that are caused by reception of the corresponding portion of information. It is formulated in the first principle.
Axiological Principle A1. A measure of information I for a system R is some measure of changes caused by I in R (for information in the strict sense, in the infological system IF(R) ) .
Next principles describe what information measures reflect. This implies several classifications for information measures.
The first criterion for measure classification is the time of changes.
Axiological Principle A2. According to time orientation, there are three temporal types of measures of information: 1) potential or perspective; 2) existential or synchronic; 3) actual or retrospective.
Definition 4.1. Potential or perspective measures of information I determine (reflects) what changes (namely, their extent) may be caused by I in R.
Definition 4.2. Existential or synchronic measures of information I determine (reflects) what changes (namely, their extent) are going in R during some fixed interval of time after receiving I. This interval of time may be considered as the present time.
Definition 4.3. Actual or retrospective measures of information I determine (reflects) what changes (namely, their extent) were actually caused by I in the system R.
Let us consider the following example. Some student R studies a textbook C. After two semesters she acquires no new knowledge from C and finishes to use it. At this time an existential measure of information contained in C for R is equal to zero. The actual measure of information in C for R is very big if R is a good student and C is a good textbook. But the potential measure of information in C for R may be also bigger than zero if in future R returns to C and finds in C such things that she did not understand in her youth.
Different types of information measures can estimate information in separate infological systems. For example, synchronic measures reflect the changes of the short-term memory, while retrospective measures represent transformations in the long-term memory of a human being.
The second criterion for measure classification is derived from the system separation triad:
Here, R is a system, W is the environment of this system and l represent different links between R and W.
Axiological Principle A3. There are three structural types of measures of information: external, intermediate, and internal.
Definition 4.4. An internal information measure reflects the extent of inner changes caused by I . Examples are given by the change of the length (the extent) of a thesaurus.
Definition 4.5. An intermediate information measure reflects the extent of changes caused by I in the links between R and W.
Examples are given by the change of the probability p(
R,
g) of achievement of a particular goal
g by the system
R. This information measure was suggested by [
20] and is called the quality of information.
Definition 4.6. An external information measure reflects the extent of outer changes caused by I, i.e., the extent of changes in W.
Examples are given by the change of the dynamics (functioning, behavior) of the system R or by the complexity of changing R.
Axiological Principle A4. There are three constructive types of measures of information: abstract, realistic, and experiential.
Definition 4.7. An abstract information measure is determined theoretically under general assumptions.
Examples are given by the change of the length (the extent) of a thesaurus.
Definition 4.8. A realistic information measure is determined theoretically subject to realistic conditions.
Quality of information [
20] is an example of such measure.
Those people who worked with information technology and dealt with problems of information security and reliability have discovered the difference between abstract and realistic measures of information. They found that if you have an encrypted message, you know that information contained in this message is available. Those who know the cipher can get it. However, if you do not possess this cipher and do not have working algorithms for deciphering, then this information is inaccessible to you. To reflect this situation, exact concepts of available and acceptable information have been introduced. Available information is measured by abstract information measures, while acceptable information is measured by realistic information measures.
The third type of measures from the Principle A4 is defined as follows.
Definition 4.9. An experiential information measure is obtained through experimentation.
Remark 4.1. In some cases, one information measure may belong to different types. In other words, classes of information measures overlap.
As an example of such a measure, we may take the measure that is used to estimate the computer memory content as well as the extent of a free memory in computer or on the disk. Namely, information in computers is represented as strings of binary symbols and the measure of such a string is the number n of these symbols. The length of the string is taken as the value of its information measure. The unit of such a measure is called a bit. Computer memory is measured in bits, bytes, kilobytes, which contain 1,000 bytes, megabytes, which contain 1,000,000 bytes and so on. This reflects the length of the strings of symbols can be stored in a memory. This is the simplest measure of symbolic information. However, this measure is necessary because storage devices (such as computer disks) have to be relevant to needs in information storage. For example, if you have a file containing five megabytes and a floppy disk of 1.4 megabytes, then cannot store this file on this floppy disk.
Moreover, some authors consider information in such a simplistic way. For example, in one article, it is assumed that information is a one-dimensional string comprising a sequence of atomic symbols. Each such symbol is a token drawn, with replacement, from a set of available symbol types. Sets of symbol types may be the binary digits (0 and 1) or the alphanumeric characters or some other convenient set.
It is necessary to remark that different information measures may give different values for the same string. For example, according to the measures used in the algorithmic information theory, the algorithmic measure of this string having length
n may be much less that
n (cf., [
21,
13]).
Let us look how this measure relates to the axiological principles of the general theory of information. When such a string is written into the computer memory, it means that some information is stored in the memory. Changes in the memory content might be measured in a different way. The simplest is to measure the work that has been performed when the string has been written. The simplest way to do this is to count how many elementary actions of writing unit symbols have been performed. However, this number is just the number of bits in this string. So, conventional measure of the size of a memory and its information content correlates with the axiological principles of the general theory of information.
Let us take classifications of measures that are presented in the axiological principles A2-A4 and apply it to the conventional measure of the size of a memory. We see that it is an internal measure (cf. Principle A3), both abstract and realistic measure (cf. Principle A4), and belong to all three classes of potential, existential and actual measures (cf. Principle A2).
The axiological principles A2-A4 have the following consequences.
A unique measure of information exists only for oversimplified system. Any complex system R with a developed infological subsystem IF(R) has many parameters that may be changed. So, such systems demand many different measures of information in order to reflect the full variety of these systems properties as well as of conditions in which these systems function. Thus, the problem of finding one universal measure for information is unrealistic.
Uncertainty elimination (which is measured by the Shannon’s quantity of information, cf.
Section 3) is only one of the possible changes, which are useful to measure for information. Another important property is a possibility to obtain a better solution of a problem (which is more complete, more adequate, demands less resources, for example, time, for achievement a goal). Changes of this possibility reflect the utility of information. Different kinds of such measures of information are introduced in the theory of information utility [
20] and in the algorithmic approach in the theory of information [
12,
13,
21].
Axiological Principle A5. Measure of information I, which is transmitted from C to a system R, depends on interaction between C and R.
Stone [
33] gives an interesting example of this property. Distortions of human voice, on one hand, are tolerable in an extremely wide spectrum, but on the other hand, even small amounts of distortion create changes in interactive styles.
The next principle is a clarification of Principle A4.
Axiological Principle A6. Measure of information transmission reflects a relation (like ratio, difference etc.) between measures of information that is accepted by the system R in the process of transmission and information that is presented by C in the same process.
It is known that the receiver accepts not all information that is transmitted by a sender. Besides, there are different distortions of transmitted information. For example, there is a myth that the intended understanding may be transmitted whole from a sender to a receiver. In almost every process of information transmission the characteristic attitudes of the receiver “interfere” in the process of comprehension. People make things meaningful for themselves by fitting them into their preconceptions. Ideas come to us raw, and we dress and cook them. The standard term for this process is
selective perception. We see what we wish to see, and we twist messages around to suit ourselves. All this is demonstrated explicitly in the well-known ‘Mr. Biggott’ studies [
35]. An audience was shown a series of anti-prejudice cartoons featuring the highly prejudiced Mr. Biggott. Then people from the audience were subjected to detailed interviews. The main result was that about two thirds of the sample clearly misunderstood the anti-prejudice intention of the cartoons. The major factors accounting for this selective perception, according to the researchers, were the predispositions of the audience. Those who were already prejudiced saw the cartoons as supporting their position. Even those from them who understood the intentions of the cartoons found ways of evading the anti-prejudice 'effect.' Only those with a predisposition toward the message interpreted the films in line with the intended meanings of the communicators.
3. Systematizing Theoretical Approaches in Information Science
Other developed theoretical approaches in information science are particular cases of the general theory of information because they explicitly or implicitly consider treat information from the functional point of view as a kind of transformations in a system. To prove this, we find in each case a relevant infological system IF(R) and demonstrate that in each of these approaches information is what changes this system.
It is necessary to remark that there are some approaches, which consider information as some kind of knowledge. More exactly, in modern information theory, according to [
17], a distinction is made between structural-attributive and functional-cybernetic types of theories. While representatives of the former approach conceive information as structure, like knowledge or data, variety, order, and so on; members of the latter understand information as functionality, functional meaning or as a property of organized systems. The general theory of information treats information from the functional and, more exactly, dynamic perspective. As it is demonstrated in [
10,
11], structural-attributive interpretation does not represent information itself but relates to information carriers. Consequently, structural-attributive types of information theories are also included in the scope of the general theory of information because structures and attributes are represented in this theory by infological elements and their properties and systems.
3.1. Shannon's Information Theory
The
statistical approach is now the most popular direction in the information sciences. It is traditionally called Shannon's information theory or, as it was at first named by Shannon, the theory of communication [
30]. It is a mathematical theory formulated principally by the American scientist Claude E. Shannon to explain aspects and problems of information and communication.
The basic problem that needed to be solved, according to Shannon, was the reproduction at one point of a message produced at another point. He deliberately excluded from his investigation the question of the meaning of a message, i.e., the reference of the message to the things of the real world. He wrote [
30]:
“Frequently the messages have meaning; that is they refer to or are correlated according to some system with physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages. The system must be designed to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design.”
While the statistical theory of information is not specific in many respects, it proves the existence of optimum coding schemes without showing how to find them. For example, it succeeds remarkably in outlining the engineering requirements of communication systems and the limitations of such systems.
In the statistical theory of information, the term information is used in a special sense: it is a measure of the freedom of choice with which a message is selected from the set of all possible messages. Information is thus distinct from meaning, since it is entirely possible for a string of nonsense words and a meaningful sentence to be equivalent with respect to information content.
In the statistical theory of information, information is measured in bits (short for binary digit). One bit is equivalent to the choice between two equally likely choices. For example, if we know that a coin is to be tossed but are unable to see it as it falls, a message telling whether the coin came up heads or tails gives us one bit of information. A special measure that is called quantity of information of a message about some situation is defined by the formula I = -Σi=1n pi log pi where n is the number of possible cases of the situation in question and pi is the probability of the case i . So, information is considered as elimination of uncertainty, i.e., as a definite change in the knowledge system that is the infological system IF(R) of the receptor of information. Consequently, we have the following result.
Proposition 3.1. The statistical information theory is a subtheory the general theory of information.
Interestingly, the mathematical expression for information content closely resembles the expression for entropy in thermodynamics. The greater the information in a message, the lower its randomness, or “noisiness,” and hence the smaller its entropy. Since the information content is, in general, associated with a source that generates messages, it is often called the entropy of the source.
However, many saw limitations of Shannon's information theory, especially when it was applied outside technical areas. As a result, other directions have been suggested in information science. However, to this day, there is no measure in information theory that is as well-supported and as generally accepted as Shannon's quantity of information. On the other hand, Shannon's work is rightly seen as lacking indications for a conceptual clarification of information.