Hierarchy and the Nature of Information

Abstract

We address the nature of information from a systemic structural point of view. Starting from the Natural hierarchy of living systems, we elucidate its decomposition into two partial hierarchies associated with its extant levels and inter-level regions, respectively. External observation of a hierarchical system involves the generation of approximate hyperscalar representations of these two partials, which then reintegrate to give a singular metascalar result. We relate Havel’s categories of reality and Peirce’s categories of experience to this result, and indicate that the ultimate result of the reintegration of hyperscalar data and context is a sign which is information.

1. Introduction

Nature tends towards hierarchical relationships. Unfortunately, conventional views of hierarchy do not provide an all-inclusive representation of the properties of a hierarchical system. Either we are restricted to a representation in which differently sized entities are nested within each other (e.g., atoms, molecules, cells… called a scalar or compositional hierarchy by Salthe [1]), or one in which there is no simple extant link between the different descriptive levels of the system (e.g., physics, chemistry, biology… called a specification or subsumption hierarchy by Salthe [1]). The situation is complicated by the pragmatic tendency of evolution to scavenge prior existing features in creating new ones, and consequently biology does not uniquely exhibit hierarchical configurations. For example, “What serves for thermoregulation is re-adapted for gliding; what was part of the jaw becomes a sound receiver; guts are used as lungs and fins turn into shovels. Whatever happens to be at hand is made use of” [2]. Consequently, although we believe that the Nature tends towards the hierarchical relationships we describe, this may not always be apparent in the results of evolution. Gilaie-Dotan et al. [3], for example, have demonstrated non-hierarchical functioning in the human cortex. A second aspect is that massive information processing, as in the brain, is faster in paralleled structures. This leads to the heterarchical character of neural networks, although the compression of parallel processing to the serial output required for muscle action takes place through hierarchical structures. Dodig-Crnkovic and Giovagnoli [4] have described Nature as a hierarchically organized network of networks, which corresponds to our own viewpoint.

Salthe has published extensively on the concept of hierarchy (see, for example, [1]). While he has before now expressed to the authors that hierarchy is a human mental construct, devoid of any other reality, we note that in [5] with Eldredge, for example, he takes a more ontological position (in reference to Salthe’s 1975 book, among other references, they state that “The several concrete proposals of this kind…have for the most part been constructed only from an epistemological perspective”). Hierarchy permits the generation of simple representations of complex informational domains, thus supporting faster survival-promoting reactions to environmental stimuli, and as such it is a primary cognitive mechanism used by all living systems, not just humans. This, of course, raises the question of the validity of representation in information theory. Our own position is clearly that representation is a necessary “computational” device for survival and therefore for evolution itself. Any other position would negate the importance of hierarchy in Nature. As Dodig-Crnkovic and Giovagnoli [4] comment in their discussion of connectionist approaches, “…it is correct that there is ‘no computation without representation’”.

We present a radically different representation of hierarchy from Salthe’s two variants—a model hierarchy—which is on a higher organizational plane when compared to the two preceding representations, and which is in fact the parent of both of them [6]. It is worth noting that this formulation of a model hierarchy automatically includes heterarchy in its purview, as individual model layers can be formulated as neural networks, for example.

Current approaches to the definition of information lack attention to this particular model-hierarchical character of Nature. Information is most often defined on the basis of order or entropy, but it is questionable whether this approach is valid for living systems, where a duality of order or entropy is prevalent [6]. We find it parsimonious to first define information with respect to the Natural (living) hierarchy—where the greatest complications and complexity are found—and then to reduce this formulation where necessary to correspond to a simpler specific context: we will address this aspect later in the paper.

We will first look at the general properties of a model hierarchy in its context of representing a living system, and then tackle the thorny problem of external access to hierarchical scales, leading to the definition of hyperscale and the reintegrative character of metascale. Next we will look at the relationships between this formulation and the work of Ivan Havel [7] and Charles Peirce [8], which will lead us to a derivation of information in Natural hierarchies.

2. Hierarchical Systems

Figure 1 illustrates our general representational form of a model hierarchy. It is vital to note that this kind of representation is not consistent with, most particularly, a composition hierarchy. A model hierarchy is an entirely new kind of representation, which is generic to other hierarchical schemes. We cannot emphasize this enough: understanding of the way a model hierarchy “operates” requires complete disconnection from other more conventional and, at first sight, similar arrangements. We make the bold move of proposing that this representation is primarily ontological in character, as we will describe later in the paper. The different levels of such a hierarchy are NOT nested within each other, as are those of a composition hierarchy, while individual levels refer to different specifications of the described system, in a manner similar to those of a subsumption hierarchy (Salthe himself has referred to a model hierarchy as “a subsumption hierarchy constructed in terms of scale”!).

Figure 1. The simple representation of a model hierarchy. Vertical line length indicates how much information is necessary to set up the local model.

Figure 1. The simple representation of a model hierarchy. Vertical line length indicates how much information is necessary to set up the local model.

Other hierarchical relationships for information have been previously published (e.g., recently [9,10]), but none of these are related to the present exposition, in that the published proposals do not have the generality of character of a model hierarchy. Most particularly, the work of Floridi (e.g., [11]) on Levels of Abstraction is not directly relatable to the hierarchical scheme described in this paper, as he is solely referring to views of a system from outside—his Levels of Abstraction are a guide to epistemological experience rather than references to ontology, unlike the levels in a model hierarchy. A second point of disagreement, although Floridi would describe a model hierarchy as a gradient of abstractions, is that in our representation all of the levels of the scheme are coupled together to create a self-consistent whole, and local events or structures (i.e., generally, models and transits between them) are specifically controlled by global properties (and, of course, vice versa). It is worth noting here Havel’s [7] insistence that things and events are only distinguished by differences in time scale: “In the world of all scales there is no essential difference: things are just long-lasting events and events are just short-lived things, where -long- and -short- are relative with respect to our temporal scale perspective”.

Figure 2 illustrates the basic nature of a model hierarchy. It is important to note that complexity is engendered as soon as modeling is attempted; consequently, a modeled system always exists within a more-or-less complex phase space, as indicated in the figure. We do not address here the way in which a model hierarchy is instantiated or evolves, merely the form which it ultimately exhibits (the startup problem is one which besets analysis and synthesis of all kinds of scientific endeavors). We have portrayed the system in terms of a graph in Figure 3; the most difficult aspect of this is that the graph axes are uncommon in that they are automatically nonlinear and possibly discontinuous—thus the simplification which we make in our habitual representations of a model hierarchy (e.g., as in Figure 1). The vertical axis describes the degree of Rosennean complexity [12] at particular locations in the phase space, while the horizontal axis is more problematic in terms of its definition (Rosen’s definition of complexity is in terms of representational incompleteness: he defines complexity as a situation which is not simulable except by application of an infinite number of different representations). Havel [7] has characterized investigative endeavors in terms of three descriptions: he refers to absolute, subjective and objective representations. We will meet all of these three later in the paper, but for the moment we will retain solely the absolute, which, for Havel, is a purely ontological description. A further significant aspect of Havel’s work [7] is the development of his idea of scale. This is a purely relative concept, which refers to the “distance” in terms of the size of a feature or event from the location in terms of “size” which is the focus of interest. We believe that the best characterization of the horizontal axis in Figure 1 is as a combination of these two parts of Havel’s work: it is an absolute (ontological) formulation of Havel’s scale. It is, therefore, easy to see how a model hierarchy could be unfortunately mistaken for a composition hierarchy. Figure 2 indicates how a survivalist tendency to promote “computational ease” creates islands of simplicity within the co-generated complex phase space of a system. These islands appear as Newtonian potential wells, and the evolution of their depth results in local complexity being “dumped” out of them and into the local phase space. The net result of evolution, therefore, is of a pair of identifiable local features: one which is a comparatively easy-to-compute simplified model, and the other which is the consequent “ecosystem” from which it has apparently “emerged”. It is clear in this scheme that the resulting extant “scales” are not nested within each other—they are local individual emergences from the global systemic phase space. Habitually in hierarchical representation, there is a clear distinction between the hierarchy of control and the hierarchy of organization. This, however, is not the case for a model hierarchy, such as is discussed here: different hierarchical levels consist of systemic models, which can imply either control, or organization (information systems or matter-and-energy exchange), or any one of these. We believe that this model-hierarchical representation of Nature is more ontologically correct than that implied by other lower-level representations, such as a composition hierarchy, which implies the necessity for a completely new look at our presuppositions about the emergence of Nature from the big bang, and of its consequent evolution.

Figure 2. The representation of a system in terms of a complex phase space with localized extant levels or scales which appear as Newtonian potential wells that present a simplified route to effective survivalist “computation”.

Figure 2. The representation of a system in terms of a complex phase space with localized extant levels or scales which appear as Newtonian potential wells that present a simplified route to effective survivalist “computation”.

Figure 3. A model hierarchy, indicating the different model levels A, B, C, D and the inter-scalar complex regions P, Q, R.

Figure 3. A model hierarchy, indicating the different model levels A, B, C, D and the inter-scalar complex regions P, Q, R.

We will always portray a hierarchy lying on its side, as in Figure 1, because we do not believe that in a Natural hierarchy the conventionally indicated “top” of the hierarchy has precedence or overall control. The length of the vertical lines indicates the amount of information required to circumscribe the associated model: more complicated models require more information, and in the figure the simplest, or highest, level of the hierarchy is on the right-hand side (labeled “A”) as indicated by the line lengths. The word “information” appears somewhat awkwardly here, as we will later define information as appearing at a completely different location in a hierarchy; it is tempting to use “data” instead, but this would give a wrong impression—we will later address this question of “information” or “data” appearing in different guises in different functional locations. With the aim of consolidating this approach, it is easy to see how this representation could describe a business organization, for example, where A would be the chief executive, B the vice-executives, C the middle management and D the workforce. We must again insist that we are here talking about a model hierarchy and not a composition hierarchy!

Regions in between the extant models (e.g., between B and C) are complex in character. This mirrors the unexpected difficulty with an “equation” such as 1 + 1 = 2. There is no way that we can independently and logically derive the 2 from 1 + 1—fortunately, it is a result defined in number theory so we never have to! Such an “equation” is, in fact, both hierarchical and partially irreversible: given that we have moved from left to right, we then lack sufficient information to say what was initially on the left-hand side (it could well have been 1 + 1, but equally well 4 − 2 or 8/4!). The complex regions between levels of a model hierarchy are similar in character: not only is it impossible to purely logically carry out the transit between, for example, C and B in the figure, but having done so we lack sufficient information to logically return to C. This seems, at first sight, to be a trivial property, but it controls everything that takes place in a Natural hierarchy.

The complex inter-model regions have yet another important characteristic. If we consider, for example, model C in the figure, then it is clear that it is just a reduced representation of the entire system itself. The model, however, invisibly “contains” a large amount of information which is absent from its formulation (in our example of a business structure, the model C of middle management misses out most of the characteristics of the work force D which it represents). This missing (or “contained”) information is itself represented in the complex region leading up to C (by way of the globality of communications linking the work force D to and from middle management C). Figure 3 illustrates the complicated character of a Natural hierarchy. Transit from model D to model C passes through complex region R where there is an absence of simple logical relationships. This indicates the biggest problem when dealing with a Natural hierarchy: if the inter-scalar regions are so complex (in fact, they are multiply fractal in character [6]), then how on earth can we deal with relations between adjacent models? Possibly the most important aspect of Natural hierarchy is that knowledge of the entire system’s global properties is required to deal with local inter-scalar transit [6].

In Figure 3, model A is very limited, and, consequently, its “contained” information appearing in region P is very extensive. Model B is more complicated, so its “contained” information in region Q is more constrained, and so on. Figure 4 illustrates this situation. The black lines, as before, represent the local system models; the gray regions represent the associated “contained” or “hidden” information (it is tempting to associate this “contained” or “hidden” information with the “hidden variables” found in, for example, David Bohm’s formulation of quantum theory; we believe that this association is justified). Intuitively, this set of gray regions forms a second partial hierarchy in opposition to the first one. This provides us with the general form of a Natural hierarchy [13]. The gray complex regions in Figure 4 (e.g., Q in Figure 3) operate as locally constructed scaled representations of the system’s environment—regions from which their associated models (e.g., B in Figure 3) have emerged.

Figure 4. The relationship between the two contra-oriented hierarchies. Black lines indicate the local models; gray regions indicate their associated “contained” information.

Figure 4. The relationship between the two contra-oriented hierarchies. Black lines indicate the local models; gray regions indicate their associated “contained” information.

3. The Generation of Hyperscale

As we have indicated in Figure 4, the two (partial) hierarchies are interleaved in a single structure. It will be convenient in this context, however, for us to draw them out separately as shown in Figure 5. From here on in we will retain this convenient fictitious separation.

Figure 5. The two partial hierarchies drawn out separately for convenience.

Figure 5. The two partial hierarchies drawn out separately for convenience.

We have pointed out already the difficulty of transiting between the adjacent model levels of a Natural hierarchy. The problem, however, is much more serious than that. At some point in our argument we must take into account the location from which we are observing the hierarchy. If it represents ourselves, we must define from which scale we are observing the structure. If, however, it represents a living system different from ourselves, we must be looking in from the outside. In conjunction with the problem of inter-scalar transit, this makes it virtually impossible to know what is happening inside the hierarchy, and, in particular, it is impossible to access the different models themselves to any degree of accuracy. This is surprising, as we are apparently mentally capable of systematically addressing all the different scales of a living entity.

Nature has come up with a neat solution to this problem. We generate, from outside a Natural hierarchy, a high-level approximate representation of all of the internal scales, which we can then make use of as if it were the hierarchy itself [14]. This representation, illustrated in Figure 6, is called hyperscale. Not only is this representation generated in terms of observation from outside a living entity, it is a property of the living entity itself—automatically generated internally as a step towards being able to represent itself to its environment in a chosen manner (e.g., through evolution, biological cells have been shut off from their environment by their lipid layer, which then permits advantageous channels of communication between them and their environments).

Figure 6. The establishment of a hyperscalar representation of the internal scales of a Natural hierarchy.

Figure 6. The establishment of a hyperscalar representation of the internal scales of a Natural hierarchy.

We must not forget that our derivation resulted in two partial hierarchies. Consequently, we will have two hyperscalar representations (Figure 7), one for the entity itself and one for its context or environment.

Figure 7. The generation of a hyperscalar representation for both of the partial hierarchies in a Natural hierarchy.

Figure 7. The generation of a hyperscalar representation for both of the partial hierarchies in a Natural hierarchy.

4. Hyperscale Reintegration and Metascale

As we intimated earlier, the two separate hierarchies are only partially independent—they are both part of the same system description. It turns out to be a major characteristic, most particularly of living systems but also marginally of non-biological systems, that all aspects of a complete system are partially independent, partially coupled. Living systems exhibit this partiality more strongly than non-biological systems for two reasons: firstly because their scalar character is more complex, and secondly because their inter-scalar transits are more difficult as these regions are more complexly fractal. Specifically for living systems, we find that the two hierarchies are partially coupled, the two hyperscales are partially independent, and those hyperscales are ultimately reintegrated to give a singular system unification in the form of metascale (see Figure 8).

Figure 8. Reintegration of the two hyperscales to give a singular unified metascalar representation.

Figure 8. Reintegration of the two hyperscales to give a singular unified metascalar representation.

We have here described these processes as a temporal sequence, but more realistically they are temporally unified, and consequently the overall character of a Natural hierarchy is one of unification, and the dual nature of its internal processes is not necessarily evident from outside. So, hyperscale is itself a dual concept: living entities generate it internally themselves, and it is also constructed in the mind of an observing cognitive agent. For an external cognitive agent, the processes of hyperscale generation and metascalar reintegration are part of an epistemological experience.

5. Havel‘s Categories of Reality

Ivan Havel [7] has circumscribed the ontological character of Nature by three levels of “reality”. He characterizes the underlying ontological character of Nature as absolute: a level of reality which is fundamental but inaccessible to direct investigation. He accepts the conventional “subjective” character as that pertaining to a single cognitive being.

However, his take on “objective” is very different (see Figure 9). Havel points out that what we call “objective” is the common acceptance by a majority of human agents of the results of their epistemological experiences: he re-formulates “objective” as a group subjective. In our present context of hierarchy we can find all of these three levels of description. The two partial hierarchies—whose internal properties are directly inaccessible—correspond to Havel’s absolute. A version of his objective corresponds to the integration of the different systemic scales into an externally accessible hyperscalar representation, and the unified metascalar representation corresponds to his (or conventional) subjective.

Figure 9. The correspondence between parts of the hierarchical system description and Havel’s absolute, objective and subjective descriptions.

Figure 9. The correspondence between parts of the hierarchical system description and Havel’s absolute, objective and subjective descriptions.

6. Peirce’s Categories of Experience

Charles Peirce’s [8] semiotic approach to Nature categorizes the character of experience into three levels—firstness, secondness and thirdness:

• firstness, as a quality of feeling (referenced to an abstraction);
• secondness, as an actuality (referenced to a correlate);
• thirdness, as a forming of habits (referenced to an interpretant).

In his semiotics, everything is a sign. Nothing “exists” except as a sign. “Redness” is a sign:

-

the quality or feeling of redness...where you are aware of it as “something existent” but not conscious of it as distinct and “red”...is firstness;

-

when you are aware that “there’s a distinct and unique experience of ‘redness’ in my vision”...that is secondness. It is distinct; you can define it as unique in itself in that time and space;

-

when you are talking about that experience in the abstract or generality, for example, “when it’s going to be a nice day tomorrow, there is always a redness in the sky at sunset”… that is thirdness (we are indebted to Edwina Taborsky for this succinct comparison of firstness, secondness and thirdness).

If we associate Peirce’s three levels with our description of Natural hierarchy, we get Figure 10. The process leading to the formulation of hyperscale (referenced to an abstraction) corresponds to firstness; hyperscalar representation itself (referenced to a correlate) corresponds to secondness; the reintegration process leading to the unification of metascale (referenced to an interpretant) corresponds to thirdness; the metascalar representation itself is a sign, where the interpretant is the cognitive organism in its subjectivity.

Figure 10. The correspondence of Peirce’s three levels of experience with regions of the dual-to-singular hierarchical representation.

Figure 10. The correspondence of Peirce’s three levels of experience with regions of the dual-to-singular hierarchical representation.

7. The Derivation of Information

It is unfortunate that the English language is relatively poor in words which relate to different levels of completeness of epistemological content, and in the precise characterization of meaning of such words as exist. This aspect relates also to our earlier comment on the common use of the word “information” in different epistemological settings. At the lowest level of definition, in this work we take data to be elements of epistemological observation which are devoid of contextual clarification, for example an abstract number or a word isolated from its sentence. On the other hand, at the highest level of definition we take information to be a complete epistemological experience, so far as we are able to define it. This means that information effectively constitutes data in its most complete available context. However, given the impossibility we have described to unequivocally observe from outside the internal scalar levels of a hierarchical system, the nature of the content of those levels will be less well defined externally than internally.

Consequently, although when viewed from outside the scalar content of a hyperscalar representation may appear to be complete, its approximated origin precludes completeness either of content or context. This suggests that we would need other words to relate to content which is intermediate between data and information. However, far more scientific attention to hierarchy will need to exist before such words are automatically forthcoming. Suffice it to say that when viewed from its own level, the content of a hierarchical scale is correctly described as information, while its appearance from outside the model of partial hierarchy is less correctly described as such.

The dissociation of scalar model levels from the complex inter-scalar regions in a hierarchy, and the recognition that a specific scalar level is derived from its local complex layer, means that this is a (partial) separation of data from context, as indicated in Figure 11. Reintegration of these two (partial) characters then gives a unified metascale, as shown in the figure. Conventionally, cognition is only associated with neural processing, but a wider view is beginning to prevail, where the activity of non-neural organisms is included in the definition. For example, amoebas can hunt their prey and orient themselves with respect to infrared light, although they have no neurons; plant roots are related to neural cells, and in the related domain of study their actions are associated with the concept of cognition (see, for example [15]).

Figure 11. The derivation of information from the two partial hierarchies via the two (partial) hyperscales pertaining to data and context.

Figure 11. The derivation of information from the two partial hierarchies via the two (partial) hyperscales pertaining to data and context.

We hypothesize that the reality of a singular metascale corresponds to information per se: as Schroeder [16] points out, “…information has been formulated as identification of a variety, where identification is understood as that which either selects, distinguishes one out of many, or that which makes the many into one (a whole)”. This high-level metascalar formulation incorporates the properties of all of a living system’s scalar properties in such a unified form that individual selection is also possible: it combines Schroeder’s [17] “two complementary manifestations of information”.

In Peircian terms (Figure 10), information is a sign which exists as the content of an interpretant.

8. Information in Less-than-Hierarchical Systems

It is a moot point whether a non-hierarchical arrangement can be called a system. Systems always include life at some level of their reality [18], and life is always multi-scalar. Be that as it may, we should look at the derivation of information in non-hierarchical situations. Surprisingly, there is no difference from the hierarchical case. Even a single-leveled “system” must be examined from outside (which is an apparent scalar difference), leading to the generation, in our mind, of an approximate hyperscalar representation of the single level. After that, things progress in much the same way as for the hierarchical case to the mental creation of a metascalar representation.

But what about Shannon’s [19] single-level definition of information, for example? Surely this will be different, precise and accurate as it appears to be? Well, no. Order and entropy are external representations of internal properties. The difference here is that if we are in a clearly non-living situation, the “inter-scalar” difference between inside and outside views is negligible, and Shannon’s formula gives good correspondence with other theoretical derivations or measurements. Consequently, the primary criterion for information definition is whether we are addressing a living setting or not. Our own context in doing so is clearly a living one, and this is the basic aspect of the entire scale-to-hyperscale-to-metascale sequence we have described.