An Algebraic Theory of Information: An Introduction and Survey

Abstract

This review examines some particular, but important and basic aspects of information: Information is related to questions and should provide at least partial answers. Information comes in pieces, and it should be possible to aggregate these pieces. Finally, it should be possible to extract that part of a piece of information which relates to a given question. Modeling these concepts leads to an algebraic theory of information. This theory centers around two different but closely related types of information algebras, each containing operations for aggregation or combination of information and for extracting information relevant to a given question. Generic constructions of instances of such algebras are presented. In particular, the close connection of information algebras to logic and domain theory will be exhibited.

1. Introduction: Modeling Information

The approach to the concept of information proposed here is based on the view that information comes in pieces, that information relates to questions and that it provides at least partial answers to questions. Since information comes in pieces, it should be aggregated or combined. It must be possible to extract from any piece of information the part pertaining to a given question, to focus the information on a given question. This gives rise to algebraic structures capturing these aspects of information. The presentation and discussion of these structures is the aim of this survey.

In fact, we propose two different types of algebraic structures, each one emphasizing a different point of view, but shown to be essentially equivalent. So each of these two types may be used at convenience. The first type starts from the idea that each piece of information is associated with a specified question, a question it answers at least partially. The piece of information refers to the associated question by its label. Two or more pieces of information can be combined into a new piece of information whose label is determined by the labels of the pieces to be combined. This represents aggregation of information. Further a piece of information can be transported or projected to another question. This models extraction of information. These ideas are captured by algebraic structures with the two basic operations of combination and transport (or projection). Such structures are called called labeled information algebras. Section 2 is devoted to an introduction and a first discussion of such algebras.

Alternatively, one may also consider pieces of information as abstract entities, unrelated to particular questions. Again such pieces of information can be combined to aggregate them into a new piece of information. Further it must be possible to extract the part relevant to any given question by extraction operations which associate to a piece of information others which represent its extracted parts. Again, we obtain algebraic structures with two basic operations, namely combination and extraction. Such structures are called domain-free information algebras. They are presented in Section 3.

Both types of algebraic structures are intimately related. They represent essentially two different views on the same subject, namely the concept of information. Each point of view has its own merits. The labeled algebras originated as a structure representing a certain computational approach. The domain-free algebras are better suited for theoretical investigations. We shall show that from each of the two types it is possible to pass to the other one and back: They represent two sides of the same coin. According to our purpose, we may thus work with one or the other type at our convenience. The domain-free version of an information algebra is generally more suited for theoretical investigations, while the labeled version is better suited for modeling and computational purposes.

There are many instances or models of information algebras. Some of them will be presented in this paper. Furthermore there are different general, generic ways to construct whole classes of such algebras. In particular, systems of mathematical logic provide important sources of information algebras, illustrating the close relationship between logic and information. Information algebras appear in this respect as objects of algebraic logic, adding some new features to this field. This is the subject of Section 4.

The algebraic structures presented here, especially the labeled ones, are based on a proposal for certain axiomatic algebraic structures in [1,2]. These structures, in turn, were proposed with the goal to generalize a successful computation scheme, presented in [3] for Bayesian networks. The algebraic structures proposed in [2] are now called valuation algebras [2,4,5,6]. So, valuation algebras provide a basis for generic, so-called local computation schemes [7]. Information algebras, in our sense, add however one important concept to valuation algebras, namely idempotency. That is, the combination of a piece of information with itself or part of itself gives nothing new. This allows to introduce an order between pieces of information, reflecting their information content. Order is an essential element of the present theory of information algebras, representing a very important feature of any concept of “information”.

With this order structure, information algebras become related to domain theory. Domain theory essentially studies information order with the goal to capture some aspects of computation and semantics of programming languages. By introducing the concept of finite information pieces, information algebras as ordered structures become in fact domains. They are more rich, however, because of the operations of combination and especially information extraction. It turns out that many results and concepts of domain theory carry over to information algebras. This is discussed in Section 5.

In the concluding Section 6, an outlook on subjects not treated in detail in this survey and on open questions in the theory of information algebras is given. So far, there are not too many publications on information algebras available. Most of this survey is based on Chapter 6 in the book [6] and on [8]. A related paper is [9]. So our hope is that this survey may attract some interest to this new and promising theory of important aspects of information.

2. Labeled Algebras

2.1. Modeling Questions

According to the introductory section, we need to consider two types of elements in a labeled information algebra: pieces of information and questions. We start by addressing the concept of questions. First, there are, in general, many questions of interest in a given situation. Further, questions have a certain granularity. A question may be refined, made more precise, or on the contrary coarsened, made less precise. In addition, a question may be the refinement respectively the coarsening of several other questions. This can be captured by an ordering of questions where x ≤ y means that question x is coarser than y or y finer than x. So, we consider a family Q of elements thought to represent questions, and we assume that (Q, ≤) is a partially ordered set, that is, ≤ satisfies:

(1)

Reflexivity: x ≤ x for all x ∈ Q.

(2)

Antisymmetry: x ≤ y and y ≤ x imply x = y.

(3)

Transitivity: x ≤ y and y ≤ z imply x ≤ z.

The following example is based on the idea that a question may be identified with the set of all its possible answers. Let $\{X_i,i\in \mathbf{N}\}$ be a countable set of variables, and fix, for all i ∈ N, a set of values V_i which X_i may take (the sets V_i need not be distinct). For any I ⊆ N the set ${\prod }_{i\in I}{V}_i$ represents all possible answers to the question “which combinations of values of the variables X_i with i ∈ I occur?” The elements of ${\prod }_{i\in I}{V}_i$, or more precisely, the set of all maps s from I to ${\cup }_{i\in I}{V}_i$, satisfying $s\left(i\right)\in V_i$ for all i ∈ I, are called I-tuples with components in V_i.

Write Q for the collection of all such questions ${\prod }_{i\in I}{V}_i$, I ⊆ N. It is natural to call a question I coarser than a question J, if I ⊆ J. This imposes a partial order ≤ on Q. In this way, (Q, ≤) becomes a distributive lattice, where the lattice operations of meet (∧) and join (∨) are given by

$$\begin{array}{ccc}\hfill {\displaystyle \prod _{i\in I}}{V}_i\wedge {\displaystyle \prod _{i\in J}}{V}_i& =& {\displaystyle \prod _{i\in I\cap J}}{V}_i\hfill \\ \hfill {\displaystyle \prod _{i\in I}}{V}_i\vee {\displaystyle \prod _{i\in J}}{V}_i& =& {\displaystyle \prod _{i\in I\cup J}}{V}_i\hfill \end{array}$$

(1)

So (Q, ≤) is isomorphic to the power set $\mathcal{P}\left(N\right)$, ordered by set inclusion. A variant of this example is obtained by admitting only finite sets I ⊆ N. The result is again a distributive lattice, although without a greatest element. We call this important model for a system of questions a multivariate system for further reference.

It is convenient to assume generally that (Q, ≤) is a lattice, although not necessarily a distributive one. The join of two question x, y ∈ Q, written x ∨ y, is the coarsest question, finer than both x and y. It represents the combined question of x and y. The meet, written as x ∧ y, is the finest question, coarser then both x and y. It represents the common part of the questions x and y. A nondistributive model for a set of questions is given by the partitions of some universe U. The blocks of a partition $\mathcal{P}$ are the possible answers of the question represented by the partition. A partition ${\mathcal{P}}_1$ is finer than a partition ${\mathcal{P}}_2$, if every block of ${\mathcal{P}}_1$ is contained in some block of partition ${\mathcal{P}}_2$. We write ${\mathcal{P}}_2\le {\mathcal{P}}_1$ in this case. Note that this is the reverse of the order usually considered when comparing partitions [10]. However, in our case this order makes sense semantically, since ${\mathcal{P}}_1$ has more possible answers than ${\mathcal{P}}_2$, or any answer to ${\mathcal{P}}_1$ determines also an answer to ${\mathcal{P}}_2$, but not vice versa. The join of two partitions in this order is defined by

$${\mathcal{P}}_1\vee {\mathcal{P}}_2=\{P_1\cap P_2\ne \varnothing :P_1\in {\mathcal{P}}_1,P_2\in {\mathcal{P}}_2\}$$

(2)

The meet is a bit more difficult to define, we abstain from giving a definition here, see [10]. The lattice part(U) of partitions of an universe U is not distributive except for trivial cases. In most cases sublattices of the lattice part(U) are considered, which generally are nondistributive too.

Therefore, in the following, we assume (Q, ≤) to be a lattice. According to the examples above, questions are often represented by the set of their possible answers, representing the domain of the information pertaining to it. Therefore we also speak sometimes of domains instead of questions. Next we consider pieces of information in a labeled view, that is, pieces of information referring to specified questions, hence to elements of Q.

2.2. Labeled Information

Let Ψ be a set of elements, thought to represent labeled pieces of information, generic elements denoted by small Greek letters $\psi ,\varphi ,\dots$. We assume that each of these pieces of information pertains to a unique questions in a lattice (Q, ≤) whose elements will be denoted by lower case letters like $x,y,\dots$. That is, each element of Ψ is labeled with the question it refers to. Pieces of information can be combined to give new aggregated pieces of information. Further any piece of information can be projected to a question coarser than the question it refers to. More formally, three operations are defined in the pair (Ψ, Q):

(1)

Labeling: d: Ψ → Q; ψ ↦ d(ψ).

(2)

Combination: ·: Ψ × Ψ → Ψ; ψ, ϕ ↦ ψ · ϕ.

(3)

Projection: π : Ψ × Q → Ψ; ψ, x ↦ π_x(ψ), defined for x ≤ d(ψ).

The labeling operation retrieves the question $d\left(\psi \right)\in Q$ the information ψ pertains to. The combination operation returns the aggregated piece of information $\psi ·\varphi$ of the two element ψ and ϕ. Projection finally represents the operation of extracting that part of information which pertains to a coarser question than the original one. It is a partial operation, defined only for pairs $\psi ,x$ such that $x\le d\left(\psi \right)$. Thus, we have a two sorted algebra $(\mathrm{\Psi },Q;d,·,\pi )$ with a unary and two binary operations, one of them partial. We collect the elements of Ψ having the same label into sets ${\mathrm{\Psi }}_x=\{\psi \in \mathrm{\Psi }:d\left(\psi \right)=x\}$ for every element $x\in Q$. Thus

$$\mathrm{\Psi }=\bigcup _{x\in Q}{\mathrm{\Psi }}_x$$

(3)

We impose the following axioms:

(1)

Semigroup:$(\mathrm{\Psi };·)$ is a commutative semigroup

(2)

Labeling: For all $\psi ,\varphi \in \mathrm{\Psi }$ and $x\in Q$ such that $x\le d\left(\psi \right)$,

$$d(\psi ·\varphi )=d\left(\psi \right)\vee d\left(\varphi \right),d\left({\pi }_x\left(\psi \right)\right)=x$$

(3)

Unit and Null: For all $x\in Q$ there exist unit and null elements $1_x$ and $0_x$ in ${\mathrm{\Psi }}_x$ such that $1_x·\psi =\psi ·1_x=\psi$ and $0_x·\psi =\psi ·0_x=0_x$ for all ψ with $d\left(\psi \right)=x$. Further, for all $y\le x$, ${\pi }_y\left(1_x\right)=1_y$ and $0_y·1_x=0_x$.

(4)

Projection: For all $\psi \in \mathrm{\Psi }$ and $x,y\in Q$ such that $x\le y\le d\left(\psi \right)$,

$${\pi }_x\left({\pi }_y\left(\psi \right)\right)={\pi }_x\left(\psi \right)$$

(5)

Combination: For all $\psi ,\varphi \in \mathrm{\Psi }$, if $d\left(\psi \right)=x$ and $d\left(\varphi \right)=y$, then

$${\pi }_x(\psi ·\varphi )=\psi ·{\pi }_{x\wedge y}\left(\varphi \right)$$

(6)

Idempotency: For all $\psi \in \mathrm{\Psi }$ and $x\in Q$ such that $x\le d\left(\psi \right)$,

$$\psi ·{\pi }_x\left(\psi \right)=\psi$$

An algebra $(\mathrm{\Psi },Q;d,·,\pi )$ satisfying these axioms is called a labeled information algebra.

The Semigroup axiom states that information can be combined in any order without changing the resulting information. As a result, we do not need to write parentheses when combining several pieces of information. The Labeling axiom says that the combined information refers to the combined question of the questions of the factors to be combined. Similarly, if a piece of information is projected to a coarser question, the result refers to this coarser question. The axiom of Unit and Null means, together with the semigroup and labeling axioms, that all ${\mathrm{\Psi }}_x$ are subsemigroups of Ψ, each having unit and null elements. Unit elements represent vacuous information with respect to the corresponding questions: Combining with whatever other information relating to the same question does not change this later information. Unit elements project to unit elements. Null elements represent contradiction, destroying any other information. According to the Projection axiom a projection may be carried out in two (or several) steps. The most powerful axiom is Combination. It permits to avoid large domains in combination, if a combination is followed by a projection. This is important for computations, but not exclusively. Idempotency finally tells us that to combine a piece of information with a part (a projection) of itself gives nothing new. This important property of information allows to order pieces of information, and permits to develop an order theory for information algebra, as we shall see later.

As an example, let us consider the multivariate model of questions introduced above. The lattice of questions is given by the lattice of subsets of variables. A piece of information R with label $d\left(R\right)=I$ is in this model an arbitrary set of I-tuples, with I finite (see Section 2.1). Note the similarity to relational algebra of relational database theory. A subset of I-tuples is called a relation over I there. So the elements of Ψ are relations over sets of variables I, or more precisely, pairs $\psi =(R,I)$, where R is a relation over I. The label of a relation ψ over I is just I, $d\left(\psi \right)=I$. Combination is essentially relational join. In order to define this operation define the label of an I-tuple s to be $d\left(s\right)=I$ and the projection ${\pi }_J$ of I-tuples to J-tuples with $J\subseteq I$ to be the restriction of the map $s:I\to V$ to J, that is, ${\pi }_J\left(s\right)\left(j\right)=s\left(j\right)$ for $j\in J$. The relational join of two relations R and S over I and J respectively is then defined as

$$R\bowtie S=\{s:d\left(s\right)=I\cup J,{\pi }_I\left(s\right)\in R,{\pi }_J\left(s\right)\in S\}$$

(4)

Then, if $\psi =(R,I)$ and $\varphi =(S,J)$, we define $\psi ·\varphi =(R\bowtie S,I\cup J)$. This clearly defines a commutative semigroup; the null element in domain I is the pair $(\varnothing ,I)$, and the unity element is the pair $(T_I,I)$, where $T_I$ denotes the set of all I-tuples. Projection is based on relational projection. If R is a relation over I and $J\subseteq I$, then relational projection is defined as

$${\pi }_J\left(R\right)=\{{\pi }_J\left(s\right):s\in R\}$$

(5)

Projection in Ψ is then defined for $\psi =(R,I)$ by ${\pi }_J\left(\psi \right)=({\pi }_J\left(R\right),J)$. It can be verified that $(\mathrm{\Psi },Q;d,·,\pi )$ with labeling, combination and projection as just defined satisfies the axioms of a labeled information algebra. If we restrict ourselves to finite subsets of variables, then we still obtain a labeled information algebra, in fact, a subalgebra of the original one.

The pair notation is cumbersome, it is only necessary to distinguish the empty relations of no I-tuples. Once this is understood, we may drop the pair-notation and just work with relations $R,S,\dots$, their domains being implicitly defined. In this case we denote the pair $(\varnothing ,I)$ simply by ${\varnothing }_I$ and call it the empty set on domain I. We refer to this example as a relational information algebra. Many labeled information algebras are subalgebras of a relational information algebra, especially with variables taking real numbers as values, see [6], as for instance the information algebras of convex sets, convex polyhedra or affine subspaces of linear spaces.

As a second example, let us consider the lattice $(Q,\le )$ of partitions of an universe U, discussed in the previous subsection. Pieces of information refer to a partition $\mathcal{P}$ as label there. Similarly as for relational information algebras, we take as elements of Ψ pairs $\psi =(S,\mathcal{P})$, where S is a subset of blocks of partition $\mathcal{P}$. The label $d\left(\psi \right)$ of the piece of information ψ is $\mathcal{P}$. The idea here is that an unknown element of the universe U represents the state of a system or the actual world out of a set of possible worlds. A partition $\mathcal{P}$ represents a question about the unknown state. A set of blocks S is a (partial) answer to the question which block P of the partition $\mathcal{P}$ contains the unknown state. In this sense ψ is a piece of information about the question represented by the partition $\mathcal{P}$. Such situations arise often in expert systems, especially diagnostics systems, where faults for instance may be decomposed or refined into finer descriptions. The partitions of an universe U form a lattice $Part\left(U\right)$ as we have seen in Section 2.1. However we consider only sublattices $\mathcal{S}$ of Part(U) satisfying an additional condition. For two elements $m,n\in U$ we write $m{\equiv }_{\mathcal{P}}n$ if they belong to the same block of the partition $\mathcal{P}$. Then we require, for any pair of partitions $\mathcal{P}$ and $\mathcal{Q}$ in the sublattice $\mathcal{S}$, that

$$m{\equiv }_{\mathcal{P}\wedge \mathcal{Q}}n\Rightarrow \exists l:m{\equiv }_{\mathcal{P}}l,l{\equiv }_{\mathcal{Q}}n$$

(6)

This condition is necessary and sufficient for a subset algebra to become an information algebra (see below). It is clear that a partition lattice in general does not satisfy this condition.

Let us now Φ denote the family of subsets of blocks of partitions of a sublattice $\mathcal{S}$ satisfying Equation (6). Then we define combination and projection similarly as in the case of a relational algebra: If $\mathcal{Q}$ is a partition coarser as $\mathcal{P}$, and P a block of $\mathcal{P}$, then define ${\pi }_{\mathcal{Q}}\left(P\right)=Q$, where Q is the block of $\mathcal{Q}$ which contains P. The label $d\left(P\right)$ of a block of a partition $\mathcal{P}$ is just $\mathcal{P}$. Then combination of two pieces of information $\psi =(S,\mathcal{P})$ and $\varphi =(T,\mathcal{Q})$ is based on a kind of generalized relational join,

$$S\bowtie T=\{P:d\left(P\right)=\mathcal{P}\vee \mathcal{Q},{\pi }_{\mathcal{P}}\left(P\right)\in S,{\pi }_{\mathcal{Q}}\left(P\right)\in T\}$$

(7)

by $\psi ·\varphi =(S\bowtie T,\mathcal{P}\vee \mathcal{Q})$. Projection of a piece of information $\psi =(S,\mathcal{P})$ to a partition $\mathcal{Q}$ coarser than $\mathcal{P}$ is based on a generalization of relational projection:

$${\pi }_{\mathcal{Q}}\left(S\right)=\{{\pi }_{\mathcal{Q}}\left(P\right):P\in S\}$$

(8)

by ${\pi }_{\mathcal{Q}}\left(\psi \right)=({\pi }_{\mathcal{Q}}\left(S\right),\mathcal{Q})$. Condition in Equation (6) is sufficient to guarantee that the structure $(\mathrm{\Phi },\mathcal{S};·,\circ )$ is a labeled information algebra. It is called a subset information algebra. Note that in a general lattice of partitions this is not true. Condition in Equation (6) is sufficient, and in fact also necessary, for $(\mathrm{\Phi },\mathcal{S};·,\circ )$ to satisfy the axioms of a labeled information algebra. We shall also see later that it is no coincidence that this example resembles relational information algebras so much (see Section 4.1).

To complete the picture let’s consider a distributive lattice $(L,{\le }_L)$ with least element $0_L$ and greatest element $1_L$. Meet and join in L are denoted by ${\wedge }_L$ and ${\vee }_L$. Further consider any index set I and an associated indexed family of finite sets ${\Omega }_i$ for $i\in I$. For any finite subset s of I form the Cartesian product ${\Omega }_s={\prod }_{i\in s}{\Omega }_i$. The generic elements of ${\Omega }_s$ are denoted by $x,y,\dots$. These elements can also be considered as maps x from s into ${\Omega }_s$ such that $x\left(i\right)\in {\Omega }_i$, that is, as s-tuples. If $t\subseteq s$, then let ${\pi }_t\left(x\right)$ denote the restriction of x to t. Let $(Q,\le )=\left(\mathcal{P}\right(I),\subseteq )$ be the lattice of all subsets of I ordered under inclusion.

Lattice valuations on s are maps $\psi :{\Omega }_s\to L$. Denote the set of all such lattice valuations on s by ${\mathrm{\Psi }}_s$ and define

$$\mathrm{\Psi }=\bigcup _{s\subseteq I}{\mathrm{\Psi }}_s$$

(9)

Consider the following operations:

(1)

Labeling: For $\psi \in {\mathrm{\Psi }}_s$, let $d\left(\psi \right)=s$.

(2)

Combination: For $\psi ,\varphi \in \mathrm{\Psi }$ with $d\left(\psi \right)=s$, $d\left(\varphi \right)=t$, combination $\psi ·\varphi$ is defined pointwise for all $x\in {\Omega }_{s\cup t}$ as

$$(\psi ·\varphi )\left(x\right)=\psi \left({\pi }_s\left(x\right)\right){\wedge }_L\varphi \left({\pi }_t\left(x\right)\right)$$

(3)

Projection: For $\psi \in \mathrm{\Psi }$ with $d\left(\psi \right)=s$ and $t\subseteq s$, the projection ${\pi }_t\left(\psi \right)$ is defined for all $x\in {\Omega }_t$ as

$${\pi }_t\left(\psi \right)\left(x\right)={\vee }_L\{\psi \left(z\right):z\in {\Omega }_s,{\pi }_t\left(z\right)=x\}$$

This defines a labeled information algebra, see [11]. The vacuous information on domain s is the lattice valuation $1_s\left(x\right)=1_L$, the null element $0_s\left(x\right)=0_L$ for all $x\in {\Omega }_s$. This is called a Lattice Induced Information Algebra. For the choice of L we have many possibilities. For instance we may take the Boolean lattice $\{0,1\}$ with the natural order $0\le 1$. Here lattice valuations represent indicator functions. So, this gives an algebra similar to a subset information algebra. Secondly, we may take for L the real interval $[0,1]$ with its natural order. Meet is then minimum and join maximum. Lattice valuations here can be thought to represent fuzzy sets or possibility distributions. Combination becomes

$$(\psi ·\varphi )\left(x\right)=min\{\psi \left({\pi }_s\left(x\right)\right),\varphi \left({\pi }_t\left(x\right)\right)\}$$

(10)

and projection is defined as

$${\pi }_t\left(\psi \right)\left(x\right)=max\{\psi \left(z\right):z\in {\Omega }_s,{\pi }_t\left(z\right)=x\}$$

(11)

This corresponds to widely used definitions of fuzzy set intersection and fuzzy set projection. These are examples of generic constructions of labeled information algebras. We shall see more such generic constructions later (Section 4).

2.3. Transport of Information

The projection or extraction operator of a labeled information algebra is only defined as a partial map. We shall now extend it to a total map in two steps. So let $(\mathrm{\Psi },Q;d,·,\pi )$ be a labeled information algebra. Consider an element $\psi \in \mathrm{\Psi }$ and a question $x\ge d\left(\psi \right)=y$. Define $e_x\left(\psi \right)=\psi ·1_x$. This is an information element on domain x (according to the Labeling axiom). If we project it back to the original domain y of ψ, we obtain by the Combination and Unit axioms that ${\pi }_y(\psi ·1_x)=\psi ·{\pi }_y\left(1_x\right)=\psi ·1_y=\psi$. So the extension of ψ to domain x by $e_x\left(\psi \right)$ can be inversed without loss of information. Also the combination of ψ with the vacuous information $1_x$ does only change the domain, but does not add information. Therefore the extension operation $e_x$ is called vacuous extension. It still is a partial operation.

Still assume $d\left(\psi \right)=y$, but consider now an arbitrary element $x\in Q$. Then we define a transport operation $t_x$ by

$$t_x\left(\psi \right)={\pi }_x\left(e_{x\vee y}\left(\psi \right)\right)$$

(12)

That is, we extend ψ to the larger domain $x\vee y$ and then project the result back to domain x. Clearly this operation extracts the information pertaining to question x from ψ. The map $t:\psi \times Q\to \mathrm{\Psi }$ defined by $\psi ,x\mapsto t_x\left(\psi \right)$ is called the transport operation, since it transports ψ to domain x. It is a total map. Evidently, if $x\le y=d\left(\psi \right)$, $t_x$ reduces to the projection π. So it is indeed an extension of the partial map π. If $x\ge d\left(\psi \right)=y$, then it reduces to vacuous extension; it is therefore also an extension of vacuous extension.

There are alternative, equivalent ways to define transport. Using the axioms of a labeled information algebra, it follows (see [6]) that for any $z\ge x\vee y$,

$$t_x\left(\psi \right)={\pi }_x\left(e_z\left(\psi \right)\right)$$

(13)

and also

$$t_x\left(\psi \right)=e_x\left({\pi }_{x\wedge y}\left(\psi \right)\right)$$

(14)

For the sake of completeness we mention a few other results on transport, which are easy to derive [6]

(1)

If $d\left(\psi \right)=x$, then $t_x\left(\psi \right)=\psi$,

(2)

If $y\le z$, then $t_y\left(\psi \right)=t_y\left(t_z\left(\psi \right)\right)$,

(3)

$t_y\left(t_x\left(\psi \right)\right)=t_y\left(t_{x\wedge y}\left(\psi \right)\right)$,

(4)

$t_x(t_x\left(\psi \right)·\varphi )=t_x\left(\psi \right)·t_x\left(\varphi \right)$.

A piece of information ψ and its vacuous extension $e_x\left(\psi \right)$ represent in some way the same information, although with reference to different questions. More generally, if ψ and ϕ are two elements of Ψ with domains $d\left(\psi \right)=x$ and $d\left(\varphi \right)=y$ and $t_y\left(\psi \right)=\varphi$ and $t_x\left(\varphi \right)=\psi$, then we may shuffle these elements around between the domains x an d y, without any loss of information. Again, ψ and ϕ represent the same information, although each with reference to a different question. This observation can be captured by defining the following relation ${\equiv }_{\sigma }$:

$$\psi {\equiv }_{\sigma }\varphi ,\mathrm{if}d\left(\psi \right)=x \text{and} d\left(\varphi \right)=y \text{together} \text{imply} t_y\left(\psi \right)=\varphi \text{and} t_x\left(\varphi \right)=\psi .$$

This is an equivalence relation in Ψ. It is even more, namely a congruence with respect to combination and transport (see [6]). That means that if ${\psi }_1{\equiv }_{\sigma }{\varphi }_1$ and ${\psi }_2{\equiv }_{\sigma }{\varphi }_2$, then ${\psi }_1·{\psi }_2{\equiv }_{\sigma }{\varphi }_1·{\varphi }_2$ and if $\psi {\equiv }_{\sigma }\varphi$, then $t_x\left(\psi \right){\equiv }_{\sigma }{t}_x\left(\varphi \right)$ for any $x\in Q$. Furthermore $0_x{\equiv }_{\sigma }{0}_y$ and $1_x{\equiv }_{\sigma }{1}_y$ for all $x,y\in Q$. This allows us to consider the quotient algebra $\mathrm{\Psi }/\sigma$, consisting of the equivalence classes ${\left[\psi \right]}_{\sigma }$ for all $\psi \in \mathrm{\Psi }$. Then, since ${\equiv }_{\sigma }$ is a congruence, the following operations between equivalence classes are all well-defined

$$\begin{array}{ccc}\hfill {\left[\psi \right]}_{\sigma }·{\left[\varphi \right]}_{\sigma }& =& {[\psi ·\varphi ]}_{\sigma }\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill {ϵ}_x\left({\left[\psi \right]}_{\sigma }\right)& =& {\left[t_x\left(\psi \right)\right]}_{\sigma }\hfill \end{array}$$

(16)

So, $\mathrm{\Psi }/\sigma$ becomes a new type of algebra, again with a combination and an extraction operation, but this time applied to equivalence classes and not to individual elements of Ψ. We note that $\mathrm{\Psi }/\sigma$ is a commutative and idempotent semigroup under combination. The ${ϵ}_x$ are mappings of $\mathrm{\Psi }/\sigma$ into itself for every $x\in Q$. Under composition these maps form also a commutative and idempotent semigroup, ${ϵ}_x\circ {ϵ}_y={ϵ}_y\circ {ϵ}_x$ and ${ϵ}_x\circ {ϵ}_x={ϵ}_x$. Further, the maps ${ϵ}_x$ have the interesting properties that (1) ${ϵ}_x\left({\left[1_y\right]}_{\sigma }\right)={\left[1_y\right]}_{\sigma }$, further (2) ${\left[\psi \right]}_{\sigma }·{ϵ}_x\left({\left[\psi \right]}_{\sigma }\right)={\left[\psi \right]}_{\sigma }$ and finally (3) ${ϵ}_x({ϵ}_x\left({\left[\psi \right]}_{\sigma }\right)·{\left[\varphi \right]}_{\sigma })={ϵ}_x\left({\left[\psi \right]}_{\sigma }\right)·{ϵ}_x\left({\left[\varphi \right]}_{\sigma }\right)$. This indicates an algebraic structure which is of interest in itself. It will be formalized and studied in the next section.

3. Domain-free Algebras

3.1. Another View

In this section a quite different view of an information algebra will be presented. It is motivated by the remarks at the end of the previous section. This time we start from a set Φ whose element are thought of as pieces of information, considered as abstract entities independent of any domain or question. That is, the elements of Φ represent domain-free information. The generic elements of Φ are again denoted by lower case Greek letters like $\varphi ,\psi ,\dots$. And again the operation of combination is defined between elements of Φ:

Combination: $·:\mathrm{\Phi }\times \mathrm{\Phi }\to \mathrm{\Phi }$, $\varphi ,\psi \mapsto \varphi ·\psi$.

As before, combination expresses the idea of aggregation of pieces of information, resulting in a new piece of information. Also, there is a unit or neutral element 1 for combination such that $1·\varphi =\varphi ·1=\varphi$ for all $\varphi \in \mathrm{\Phi }$. It represents vacuous information. Further there is a null element 0 such that $0·\varphi =\varphi ·0=0i$. It represents contradiction, which destroys any information. Strictly speaking it does not really represent information as the other elements of Φ do, but it stands for the possibility that pieces of information may be contradictory.

Further we assume the existence of a family E of extraction operators, represented by mappings of Φ into itself:

Extraction: $ϵ:\mathrm{\Phi }\to \mathrm{\Phi }$.

We may think of each element of E as an operation to extract information from the elements of Φ with respect to some question. This point of view will be clarified below. Guided by the algebraic structure derived from a labeled information algebra at the end of the previous section, we impose the following conditions on Φ, E and the interaction of combination and extraction:

• Combination:Φ is a commutative and idempotent semigroup under the combination operation · with a unit 1 and a null element 0.
• Extraction: Each extraction operator $ϵ\in E$ satisfies the following conditions:

  (a)

  $ϵ\left(0\right)=0$,

  (b)

  $\varphi ·ϵ\left(\varphi \right)=\varphi$ for all $\varphi \in \mathrm{\Phi }$,

  (c)

  $ϵ\left(ϵ\right(\varphi )·\psi )=ϵ\left(\varphi \right)·ϵ\left(\psi \right)$ for all $\varphi ,\psi \in \mathrm{\Phi }$.

• Composition:E is a commutative and idempotent semigroup under ordinary composition of mappings.

An algebra $(\mathrm{\Phi },E;·,\circ )$ satisfying these conditions is a called a Domain-Free Information Algebra.

Item 1 states that combination of pieces of information can be done in any order with the same result. As stated above, the unit element 1 represents vacuous information and the null element 0 contradiction. Item 2 (a) says that extraction of information from the vacuous information results simply in vacuous information, there is no information to extract from vacuous information. Further, item 2 (b) requires that combining a piece of information with a part extracted from it gives nothing new, preserving the original piece of information. This corresponds to the Idempotency axiom of labeled algebras. Item 2 (c) corresponds to the Combination axiom of labeled information algebras. Finally, item 3 says that if information is extracted with respect to two questions, the order in which this is done is irrelevant. And once information is extracted with respect to a fixed question, extracting information with respect to the same question gives nothing new.

As discussed at the end of the previous section, any labeled information algebra gives rise to a domain-free information algebra, if equivalent labeled information elements representing the same information are collected into equivalence classes. Combination is inherited from labeled combination and extraction from the transport operation. This hints already at a close connection between labeled and domain-free algebras. We shall in fact show that they are essentially two sides of a same coin and that we may switch between the two types of algebraic structures at our convenience.

If $(\mathrm{\Psi },Q;d,·,\pi )$ is a labeled information algebra and the lattice of questions has a top element ⊤, that is a finest, universal question, then any element $\psi \in \mathrm{\Psi }$ is equivalent to its vacuous extension to the top domain, $\psi {\equiv }_{\sigma }{e}_{\top }\left(\psi \right)$. In the quotient algebra $\mathrm{\Psi }/\sigma$, we may therefore always take $e_{\top }\left(\psi \right)$ as a representative of the class ${\left[\psi \right]}_{\sigma }$. Since we then have

$$\begin{array}{ccc}\hfill {\left[e_{\top }\left(\psi \right)\right]}_{\sigma }·{\left[e_{\top }\left(\varphi \right)\right]}_{\sigma }& =& {[e_{\top }\left(\psi \right)·e_{\top }\left(\varphi \right)]}_{\sigma }\hfill \end{array}$$

(17)

$$\begin{array}{ccc}\hfill {ϵ}_x\left({\left[e_{\top }\left(\psi \right)\right]}_{\sigma }\right)& =& {\left[e_{\top }\left({\pi }_x\left(e_{\top }\left(\psi \right)\right)\right)\right]}_{\sigma }\hfill \end{array}$$

(18)

we see that we may represent the domain-free version of the labeled algebra $(\mathrm{\Psi },Q;d,·,\pi )$ essentially by the elements $e_{\top }\left(\psi \right)$ of the labeled algebra. So, e.g., in the case of a relational information algebra (see previous section), we may adjoin to the lattice Q of the finite subsets of $\mathbf{N}$ the whole index set $\mathbf{N}$ as top element. Let $f:\{1,2,\dots \}\to {\cup }_{i\in \mathbf{N}}{V}_i$ be sequences of elements $f\left(i\right)$ with $f\left(i\right)\in V_i$ and define ${\pi }_I\left(f\right)$ to be the restriction of f to a subset I of variables. Then the vacuous extension of a relation R on I is its so-called cylindrical extension or saturation, defined by

$$e_{\top }\left(R\right)=\{f:{\pi }_I\left(f\right)\in R\}$$

(19)

Thus, the sets $e_{\top }\left(R\right)$ are subsets of ${\prod }_{i\in \mathbf{N}}{V}_i$, the set of all infinite sequences f. Hence, the domain-free version of a relational information algebra is essentially the algebra of subsets $e_{\top }\left(R\right)$ with intersection as combination and ${ϵ}_I\left(e_{\top }\left(R\right)\right)=e_{\top }\left({\pi }_I\left(e_{\top }\left(R\right)\right)\right)$ as extraction.

To complete the picture let us present two further, simple examples of domain-free information algebras. Consider a finite alphabet Σ, the set ${\Sigma }^{*}$ of finite strings over Σ, including the empty string ϵ, and the set ${\Sigma }^{\omega }$ of countably infinite strings over Σ. Let ${\Sigma }^{**}={\Sigma }^{*}\cup {\Sigma }^{\omega }\cup \left\{0\right\}$, where 0 is a symbol not contained in Σ. For two strings $r,s\in {\Sigma }^{**}$, define $r\le s$, if r is a prefix of s or if $s=0$. Then, we define a combination operation in ${\Sigma }^{**}$ as follows:

$$\begin{array}{c}\hfill r·s=\left\{\begin{array}{cc}s,\hfill & \mathrm{if}r\le s,\hfill \\ r\hfill & \mathrm{if}s\le r,\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

(20)

Clearly, this is a commutative, idempotent semigroup. The empty string ϵ is the unit element, and the adjoined element 0 is the null element of combination. For extraction, we define operators ${ϵ}_n$ for any $n\in \mathbf{N}$ and also for $n=\infty$. Let ${ϵ}_n\left(s\right)$ be the prefix of length n of string s, if the length of s is at least n, and let ${ϵ}_n\left(s\right)=s$ otherwise. In particular, define ${ϵ}_{\infty }\left(s\right)=s$ for any string s and ${ϵ}_n\left(0\right)=0$ for any n. So, any ${ϵ}_n$ maps ${\Sigma }^{**}$ into itself. It is easy to verify the each ${ϵ}_n$ satisfies conditions (a) to (c) of item 2 above in the definition of a domain-free information algebra. Further, it can also be easily verified, that $E=\{{ϵ}_n:n\in \mathbf{N}\cup \left\{\infty \right\}\}$ is a commutative and idempotent semigroup under composition of maps. So, the string algebra $({\Sigma }^{**},E,·,\circ \}$ is an instance of a domain-free information algebra.

As another example consider any set X. A partial map on X is a map $\sigma :S\to X$, where S is a subset of X. We denote the domain of σ by $d\left(\sigma \right)$. The set of all partial maps on X, including an adjoined new element 0, is denoted by $X\to \circ \to X$. Partial maps are ordered by $\sigma \le \tau$, if $d\left(\sigma \right)\subseteq d\left(\tau \right)$ and $\sigma \left(x\right)=\tau \left(x\right)$ for all $x\in d\left(\sigma \right)$ or if $\tau =0$. We define combination in $X\to \circ \to X$ by

$$\begin{array}{c}\hfill \sigma ·\tau =\left\{\begin{array}{cc}\sigma \hfill & \mathrm{if}\tau \le \sigma ,\hfill \\ \tau \hfill & \mathrm{if}\sigma \le \tau ,\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(21)

Again, under this operation, $X\to \circ \to X$ is a commutative and idempotent semigroup, with the empty function ϵ, defined on the empty set, as unit, and 0 as null element. For any subset S of X, we define an extraction operation ${ϵ}_S$ by restricting a map σ to S, more precisely,

$${ϵ}_S\left(\sigma \right)=\sigma |d\left(\sigma \right)\cap S$$

(22)

and ${ϵ}_S\left(0\right)=0$. Each of these maps satisfies conditions (a) to (c) in item 2 of the definition of a domain-free information algebra. Further, $E=\{{ϵ}_S:S\subseteq X\}$ is a commutative and idempotent semigroup under composition. Thus the algebra of partial maps $(X\to \circ \to X,E,·,\circ )$ is a domain-free information algebra.

Further examples will be given later, in particular in Section 4. Here we shall first show that idempotency of Φ and E permits to define an order in each of these two sets. Especially important is the order of information in Φ. Many interesting features of an information algebra are in fact order-theoretic, as we shall see subsequently and in particular in Section 5. Order will be introduced in Section 3.2 and exploited in Section 3.3. Then, just as labeled information algebras induce associated domain-free algebras, we shall show in Section 3.4 that from a domain-free algebra we may also construct a labeled one. Finally, a further important structural property will be introduced and discussed in Section 3.5.

3.2. Order

Can pieces of information be compared as to their information content? Suppose $(\mathrm{\Phi },E;·,\circ )$ to be a domain-free information algebra. If $\varphi ,\psi$ are two pieces of information such that $\varphi ·\psi =\varphi$, then we may say that ψ contains less information than ϕ, since knowing ϕ and ψ gives nothing more than ϕ. The piece of information ψ adds nothing to ϕ. So, in this case we write $\psi \le \varphi$, meaning that ψ contains less information than ϕ, is less informative than ϕ. This relation is indeed a partial order on Φ. This order has a number of simple consequences. So, $\varphi ,\psi \le \varphi ·\psi$. In fact, the combination of ϕ and ψ is the least upper bound, the supremum of the two elements, $\varphi ·\psi =sup\{\varphi ,\psi \}$. Further, 0 is an upper bound and 1 a lower bound for any element in Φ, $1\le \varphi \le 0$. This means that $(\mathrm{\Phi },\le )$ is a join-semilattice with minimal and maximal elements 1 and 0. We often write combination as join, when the order-theoretic point of view is to be stressed, $\varphi ·\psi =\varphi \vee \psi$. We mention also that in many cases, as for example in the string algebra and the algebra of partial maps, combination is defined primarily by order.

In any commutative and idempotent semigroup there exist exactly two orders compatible with the semigroup operation. Here, we have selected the one which represents the information order. This choice implies that $1\le \varphi \le 0$; the vacuous information is less informative than any other information, and the contradiction 0, which is not really an information, dominates in this order any piece of information.

In the same way we may define an order in the commutative, idempotent semigroup E of extraction operators: Here we define $ϵ\le \eta$ if $\eta \circ ϵ=ϵ$. So, $ϵ\le \eta$ means that once information is extracted according to ϵ, another extraction according to η does no more change anything. Under this order, composition of extraction corresponds to the infimum, $\eta \circ ϵ=inf\{\eta ,ϵ\}$. In this way, E becomes a meet-semilattice, and composition can be written as meet, $\eta \circ ϵ=\eta \wedge ϵ$. In the examples presented above, we have seen that the elements of E are often indexed by another set Q, $E=\{{ϵ}_x:x\in Q\}$. We may consider the elements x of Q as representing questions, and the maps ${ϵ}_x$ as extraction operators, which extract the information pertaining to question x from the pieces of information. The order in E induces an order in Q: $x\le y$ in Q, if ${ϵ}_x\le {ϵ}_y$ in E. Here $x\le y$ means that the question represented by x is coarser than the one represented by y; once the information relating to x is extracted, the information relating to the finer question y is lost and cannot be retrieved.

In this view, in domain-free information algebras, the system Q of questions is only assumed to be a meet-semilattice, whereas in labeled information algebras Q is assumed to be a lattice. This is a difference between the domain-free and the labeled view of information. In many cases however, even in domain-free information algebras, Q is in fact also a lattice, see the examples above.

We adopt in the sequel the view that the family of extraction operators E is indexed by the elements of a question semilattice Q. Here, we give a few elementary results on the interplay between the order in Φ and the one in Q in a domain-free information algebra:

(1)

${ϵ}_x\left(1\right)=1$ for all $x\in Q$,

(2)

$x\le y$ implies ${ϵ}_x\left(\varphi \right)\le {ϵ}_y\left(\varphi \right)$ for all $\varphi \in \mathrm{\Phi }$,

(3)

$\varphi \le \psi$ implies ${ϵ}_x\left(\varphi \right)\le {ϵ}_x\left(\psi \right)$ for all $x\in Q$,

(4)

${ϵ}_x\left(\varphi \right)\le \varphi$ for all $\varphi \in \mathrm{\Phi }$ and all $x\in Q$,

(5)

${ϵ}_x\left(\varphi \right)·{ϵ}_x\left(\psi \right)\le {ϵ}_x(\varphi ·\psi )$.

In particular, conditions (a) to (c) in item 2 of the definition of a domain-free information algebra above may be rewritten as follows:

(a)

${ϵ}_x\left(0\right)=0$,

(b)

${ϵ}_x\left(\varphi \right)\le \varphi$ for all $\varphi \in \mathrm{\Phi }$,

(c)

${ϵ}_x({ϵ}_x\left(\varphi \right)\vee \psi )={ϵ}_x\left(\varphi \right)\vee {ϵ}_x\left(\psi \right)$.

An operator on a semilattice, satisfying these conditions, is also called an existential quantifier in algebraic logic, see [12,13], although often the reversed order is used. This difference is explained by our choice of the order, dictated by the semantics of information. This is a first reference to the close connection between information algebras and logic, in particular algebraic logic. This subject will be taken up in the next subsection and in particular in Section 4.

To conclude our discussion of order, let’s mention that in just the same way an order may also be introduced between pieces of information in labeled information algebras. We take this for granted. This section can serve as a model how to do it. Many other concepts related to order, for instance ideal completion (Section 3.3) or compactness (Section 5) have their counterpart in the labeled version.

3.3. Ideal Completion

If an agent asserts a piece of information ϕ, then he should consistently also assert any lesser informative information $\psi \le \varphi$. So, the down set $\downarrow \varphi =\{\psi :\psi \le \varphi \}$ is then a consistent body of information deduced from ϕ. More generally, if an agent asserts two pieces of information ϕ and ψ, then he should also consistently assert their combination $\varphi ·\psi$. In other words, a consistent body of information should form an ideal in Φ. Assume $(\mathrm{\Phi },E;·,\circ )$ is a domain-free information algebra. An ideal in Φ is then defined as follows:

A non-empty set $I\subseteq \mathrm{\Phi }$ is called an ideal in Φ, if

(1)

$\varphi \in I$, $\psi \in \mathrm{\Phi }$ and $\psi \le \varphi$ jointly imply $\psi \in I$,

(2)

$\varphi ,\psi \in I$ imply $\varphi ·\psi \in I$.

This corresponds to the usual concept of an ideal in semilattices or lattices, see [14]. In our context, an ideal represents a coherent and consistent theory. The down set $\downarrow \varphi$ of an element ϕ of Φ is called a principal ideal, and if an ideal I is different from Φ it is called proper. Let $I_{\mathrm{\Phi }}$ denote the set of all ideals of Φ. We show that these ideals form an information algebra.

For this purpose we consider ideals I of Φ as pieces of information and define combination of two ideal $I_1$ and $I_2$ by

$$I_1·I_2=\{\varphi \in \mathrm{\Phi }:\varphi \le {\varphi }_1·{\varphi }_2 \text{for} \text{some} {\varphi }_1\in I_1,{\varphi }_2\in I_2\}$$

(23)

Clearly, $I_1·I_2$ is an ideal in Φ, so combination is well defined. It is commutative and idempotent. The ideal $\left\{1\right\}$ is the unit element and the ideal Φ the null element of this combination operation.

Next, assume extraction operators in E to be indexed by elements $x\in Q$, where Q is a meet-semilattice. Define for $x\in Q$ and an ideal I of Φ

$${\overline{ϵ}}_x\left(I\right)=\{\varphi \in \mathrm{\Phi }:\varphi \le {ϵ}_x\left(\psi \right) \text{for} \text{some} \psi \in I\}$$

(24)

Again, ${\overline{ϵ}}_x\left(I\right)$ is an ideal in Φ, so ${\overline{ϵ}}_x\left(I\right)$ is well defined. It can be shown that $\overline{E}=\{{\overline{ϵ}}_x:x\in Q\}$ is a commutative and idempotent semigroup under composition. It can also be shown that ${\overline{ϵ}}_x$ as a map from the set of ideals $I_{\mathrm{\Phi }}$ into itself satisfies conditions (a) to (c) of item 2 in the definition of a domain-free information algebra, that is, it is an existential quantifier, see [6,8]. Therefore, $(I_{\mathrm{\Phi }},\overline{E},·,\circ )$ is a domain-free information algebra. It is called the ideal extension of the information algebra $(\mathrm{\Phi },E;·,\circ )$. Note that in the information order we have $I_1\le I_2$ exactly if $I_1\subseteq I_2$. Thus $I_2$ is the larger theory than $I_1$, hence more informative!

The original algebra $(\mathrm{\Phi },E;·,\circ )$ is embedded into its ideal extension by the map $\varphi \mapsto \downarrow \varphi$, because $\downarrow (\varphi ·\psi )=\downarrow \varphi ·\downarrow \psi$, $\downarrow {ϵ}_x\left(\varphi \right)={\overline{ϵ}}_x(\downarrow \varphi )$ and the map is one-to-one. To simplify notation we identify henceforth Φ with its image under this map and also write ${ϵ}_x$ instead of ${\overline{ϵ}}_x$. Also $\varphi \in I$ is often written as $\varphi \le I$, referring to the order in $I_{\mathrm{\Phi }}$.

The intersection of any family of ideals is again an ideal. So the collection of all ideals is a ∩-system; and any ∩-system is a complete lattice under set-inclusion [14]. Meet is just set-interesection and join is defined by ideal-combination. So, $I_{\mathrm{\Phi }}$ is not just a join-semilattice under information order, but a complete lattice, where meet and join are defined for any family of ideals. The role of ideal completion will become clearer in Section 5.

3.4. Support

We claimed that an extraction operation ${ϵ}_x$ extracts exactly the information pertaining to a question $x\in Q$ from any piece of information $\varphi \in \mathrm{\Phi }$. Now, it may be that a piece of information $\varphi \in \mathrm{\Phi }$ just pertains fully to a question x, that is extracting information relative to x does not change ϕ, ${ϵ}_x\left(\varphi \right)=\varphi$. Then $x\in Q$ is called a support of ϕ. Note that, in general, not every element $\varphi \in \mathrm{\Phi }$ has a support. However, the identity map $id$ of Φ into itself satisfies condition (a) to (c) of item 2 in the definition of a domain-free information algebra, so $E\cup \left\{id\right\}$ is still a commutative, idempotent semigroup, containing E as a sub-semigroup. So, if $(\mathrm{\Phi },E;·,\circ )$ is a domain-free information algebra, then so is $(\mathrm{\Phi },E\cup \{id\},·,\circ )$. However, in this algebra every element of Φ has a support, namely $id$. To the extraction operator $id$ we may adjoin a new top element ⊤ in Q, satisfying $id={ϵ}_{\top }$. We mentioned already above that such a top element represents a finest question, which contains all possible answers. So, in this way, we may easily extend any domain-free information algebra in such a way that any element of Φ has a support. An algebra where each element has a support is called supported.

We list a few results concerning support:

(1)

${ϵ}_x\left({ϵ}_x\left(\varphi \right)\right)={ϵ}_x\left(\varphi \right)$, that is, x is a support of ${ϵ}_x\left(\varphi \right)$,

(2)

${ϵ}_x\left(\varphi \right)=\varphi$ implies ${ϵ}_x\left({ϵ}_y\left(\varphi \right)\right)={ϵ}_y\left(\varphi \right)$, that is, if x is a support of ϕ, then x is a also a support of any ${ϵ}_y\left(\varphi \right)$,

(3)

${ϵ}_x\left(\varphi \right)={ϵ}_y\left(\varphi \right)=\varphi$ implies ${ϵ}_{x\wedge y}\left(\varphi \right)=\varphi$, that is, if x and y are supports of ϕ, then so is $x\wedge y$,

(4)

${ϵ}_x\left(\varphi \right)=\varphi$ implies ${ϵ}_{x\wedge y}\left(\varphi \right)={ϵ}_y\left(\varphi \right)$, that is, if x is a support of ϕ, then $x\wedge y$ is a support of ${ϵ}_y\left(\varphi \right)$,

(5)

${ϵ}_x\left(\varphi \right)=\varphi$ and $x\le y$ imply ${ϵ}_y\left(\varphi \right)=\varphi$, that is, if x is a support ϕ, then any finer question y is a support of ϕ too,

(6)

${ϵ}_x\left(\varphi \right)=\varphi$ and ${ϵ}_x\left(\psi \right)=\psi$ imply ${ϵ}_x(\varphi ·\psi )=\varphi ·\psi$, that is, if x is a support of ϕ and ψ, then it is a support of $\varphi ·\psi$ too,

(7)

If Q is a lattice, then ${ϵ}_x\left(\varphi \right)=\varphi$ and ${ϵ}_y\left(\psi \right)=\psi$ imply ${ϵ}_{x\vee y}(\varphi ·\psi )=\varphi ·\psi$, that is, if x is a support of ϕ, and y a support of ψ, then $x\vee y$ is a support of $\varphi ·\psi$.

The last item reminds of the Labeling axiom of a labeled information algebra. In fact, if Q is a lattice, then we may associate a labeled information algebra to a supported domain-free information algebra $(\mathrm{\Phi },E;·,\circ )$. Here we assume as usual that E is labeled by the lattice Q. Consider the set of all pairs $(\varphi ,x)$, where $\varphi \in \mathrm{\Phi }$ and x is a support of ϕ. That is, we label all elements of Φ with all of its supports. Let then

$$\mathrm{\Psi }=\{(\varphi ,x):\varphi \in \mathrm{\Phi },{ϵ}_x\left(\varphi \right)=\varphi \}$$

(25)

Then we define a labeling operation on Ψ by

$$d(\varphi ,x)=x$$

(26)

Further, within Ψ, we define a combination operation by

$$(\varphi ,x)·(\psi ,y)=(\varphi ·\psi ,x\vee y)$$

(27)

Note that this operation is well defined by property (7) in the list above. Clearly, Ψ is a commutative and idempotent semigroup under this combination operation. It has $(1,x)$ and $(0,x)$ as unit and null elements on domains $x\in Q$. Finally, we define a projection operation for any $x\le y$ by

$${\pi }_x(\varphi ,y)=({ϵ}_x\left(\varphi \right),x)$$

(28)

It can be verified that the algebra $(\mathrm{\Psi },Q;d,·,\pi )$ indeed satisfies all axioms of a labeled information algebra [6]. If we go back from this labeled information algebra to the domain-free algebra by forming the quotient algebra $\mathrm{\Psi }/\sigma$, then $\mathrm{\Psi }/\sigma$ will be isomorphic to the initial one, namely $(\mathrm{\Phi },E;·,\circ )$. A formal definition of a morphism between information algebras will be given in Section 4.5, for the moment an intuitive understanding is sufficient. Similarly, if we start with a labeled information algebra $(\mathrm{\Psi },Q;d,·,\pi )$, construct the domain-free algebra on $\mathrm{\Psi }/\sigma$ and take the associated labeled algebra using the pair construction above, then this labeled algebra will be isomorphic to the initial one. In this sense, labeled and domain-free algebra are closely associated. Therefore, in the sequel, we shall freely select at convenience either the labeled or the domain-free version of an information algebra, and we take it for granted that any concepts and constructions may be developed in parallel in both versions of an information algebra—although typically one of the two settings will be preferable in most cases.

3.5. Atoms

In many cases there are maximal, most informative information elements; and often any piece of information is somehow composed from such elements. This issue will be discussed in this section. Here we consider a labeled information algebra $(\mathrm{\Psi },Q;d,·,\pi )$, although a similar development would also be possible for domain-free algebras.

An element $\alpha \in \mathrm{\Psi }$ with label $d\left(\alpha \right)=x$ is called an atom on domain x, if

(1)

$\alpha \ne 0_x$,

(2)

For all $\psi \in \mathrm{\Psi }$, $d\left(\psi \right)=x$ and $\alpha \le \psi$ jointly imply either $\psi =\alpha$ or $\psi =0_x$.

Therefore, atoms are the most informative elements on domain x (with the exception of the non-information $0_x$, the contradiction). In a pure order-theoretic view atoms would be called co-atoms, since atoms there are the smallest nonzero elements. However, in the view of information order, the present notion of an atom is justified, since it turns out that a piece of information may be identified in many important cases just with a set of atoms implying the information. In relational information algebras (see Section 2.2) the one-element relations, the relations containing exactly one tuple, are atoms in the domain of the relation. Of course, information algebras may have no atoms. Note also that in a subset information algebra (see Section 2.2) the individual blocks of a partition $\mathcal{P}$ are atoms on the domain represented by this partition.

Here are a few properties of atoms:

(1)

If α is an atom on domain x, $\psi \in \mathrm{\Psi }$ and $d\left(\psi \right)=x$, then either $\psi \le \alpha$ or $\psi ·\alpha =0_x$.

(2)

If α is an atom on domain x, and $y\le x$, then ${\pi }_y\left(\alpha \right)$ is an atom on domain y.

(3)

If α and β are atoms on domain x, then either $\alpha =\beta$ or $\alpha ·\beta =0_x$.

For a proof we refer to [6]. Two pieces of information ϕ and ψ on a same domain x are called contradictory (or disjoint), if $\varphi ·\psi =0_x$. So atoms are mutually contradictory and an atom and a piece of information on the same domain are either contradictory or the information is smaller than the atom.

Let $At\left({\mathrm{\Psi }}_x\right)$ denote the set of all atoms with domain x, $At\left(\mathrm{\Psi }\right)$ the set of all atoms, and for $\psi \in \mathrm{\Psi }$ with label $d\left(\psi \right)=x$, let $At\left(\psi \right)$ be the set of all atoms implying ψ,

$$At\left(\psi \right)=\{\alpha \in At\left({\mathrm{\Psi }}_x\right):\psi \le \alpha \}$$

(29)

If $\alpha \in At\left(\psi \right)$ we say that α is contained in ψ. For instance, in relational algebras, the atoms are the one-tuple relations: $\left\{\alpha \right\}$ is an atom contained in R, exactly if $\alpha \in R$.

We introduce a few notions with respect to atoms and labeled information algebras:

(1)

A labeled information algebra is called atomic if for all $\psi \in \mathrm{\Psi }$ different from $0_x$ the set $At\left(\psi \right)$ is not empty, i.e., if every piece of information contains an atom.

(2)

A labeled information algebra is called atomistic, if for all $\psi \in \mathrm{\Psi }$ different from $0_x$, ψ is the infimum of the atoms it contains,

$$\psi =\wedge At\left(\psi \right)$$

(30)

(3)

A labeled information algebra is called completely atomistic, if it is atomistic and if for all $x\in Q$ and for every subset of atoms $A\subseteq At\left({\mathrm{\Psi }}_x\right)$, the infimum $\wedge A$ exists and belongs to Φ.

A relational information algebra is completely atomistic. If in a relational information algebra V is the set of real numbers and only convex subsets in each domain are considered, then we still have a labeled information algebra (a subalgebra of the relational one), which is atomistic but not completely atomistic, and this is the case for many other subalgebras of a relational information algebra.

For atoms we have essentially two basic operations, namely Labeling: $\alpha \mapsto d\left(\alpha \right)$ and Projection: $\alpha \mapsto {\pi }_x\left(\alpha \right)$. In an atomic information algebra the system $\left(At\right(\mathrm{\Psi }),Q;d,\pi )$ has the following properties, inherited from the corresponding labeled information algebra (see [6]):

(1)

If $x\le d\left(\alpha \right)$, then $d\left({\pi }_x\left(\alpha \right)\right)=x$.

(2)

If $x\le y\le d\left(\alpha \right)$, then ${\pi }_x\left({\pi }_y\left(\alpha \right)\right)={\pi }_x\left(\alpha \right)$.

(3)

If $d\left(\alpha \right)=x$, then ${\pi }_x\left(\alpha \right)=\alpha$.

(4)

If $d\left(\alpha \right)=x$, $d\left(\beta \right)=y$ and ${\pi }_{x\wedge y}\left(\alpha \right)={\pi }_{x\wedge y}\left(\beta \right)$, then there exists a γ such that $d\left(\gamma \right)=x\vee y$ and ${\pi }_x\left(\gamma \right)=\alpha$, ${\pi }_y\left(\gamma \right)=\beta$.

(5)

If $d\left(\alpha \right)=x$ and $x\le y$, then there exists a γ such that $d\left(\gamma \right)=y$ and ${\pi }_x\left(\gamma \right)=\alpha$.

Note that the ordinary tuples of a relational algebra also satisfy these conditions. Therefore we call a system like $\left(At\right(\mathrm{\Psi }),Q;d,\pi )$ a tuple system. More on such systems will be said in Section 4.1.

Still considering an atomic labeled information algebra $(\mathrm{\Psi },Q;d,·,\pi )$. We call subsets of $At\left({\mathrm{\Psi }}_x\right)$generalized relations on domain x and denote the family of all such relations by $\mathcal{R}\left(At\right(\mathrm{\Psi }\left)\right)$.

In $\mathcal{R}\left(At\right(\mathrm{\Psi }\left)\right)$, we may define an operation of combination among generalized relations R and S on x and y respectively, according to the model of relational join:

$$R\bowtie S=\{\alpha \in At\left(\mathrm{\Psi }\right):d\left(\alpha \right)=x\vee y,{\pi }_x\left(\alpha \right)\in R,{\pi }_y\left(\alpha \right)\in S\}$$

We may also define projection of a generalized relation R on domain x to some domain $y\le x$ as in relational algebra:

$${\pi }_y\left(R\right)=\{{\pi }_y\left(\alpha \right):\alpha \in R\}$$

(31)

Then, exactly as in relational information algebras (see Section 2.2), we may define a generalized relational information algebra of pairs $(R,x)$ of generalized relations in $At\left(\mathrm{\Psi }\right)$ with domain $x\in Q$. We call this algebra $\left(\mathcal{R}\right(At\left(\mathrm{\Psi }\right)),Q;d,·,\pi )$ (see [6,8]). As before, we may drop the pair notation, and denote the pair $(\varnothing ,x)$ by ${\varnothing }_x$.

The map $\psi \mapsto At\left(\psi \right)$ between Ψ and $\mathcal{R}\left(At\right(\mathrm{\Psi }\left)\right)$ is a homomorphism (see Section 4.5 and [6]) between the atomic information algebra $(\mathrm{\Psi },Q;d,·,\pi )$ and the generalized relational algebra $\left(\mathcal{R}\right(At\left(\mathrm{\Psi }\right)),Q;d,·,\pi )$, that is,

$$\begin{array}{ccc}\hfill At(\psi ·\varphi )& =& At\left(\psi \right)\bowtie At\left(\varphi \right)\hfill \end{array}$$

(32)

$$\begin{array}{ccc}\hfill At\left({\pi }_x\left(\psi \right)\right)& =& {\pi }_x\left(At\left(\psi \right)\right)\hfill \end{array}$$

(33)

$$\begin{array}{ccc}\hfill At\left(1_x\right)& =& At\left({\mathrm{\Psi }}_x\right)\hfill \end{array}$$

(34)

$$\begin{array}{ccc}\hfill At\left(0_x\right)& =& {\varnothing }_x\hfill \end{array}$$

(35)

If the labeled information algebra $(\mathrm{\Psi },Q;d,·,\pi )$ is atomistic, then this map is one-to-one, hence an embedding of $(\mathrm{\Psi },Q;d,·,\pi )$ into the relational algebra $\left(\mathcal{R}\right(At\left(\mathrm{\Psi }\right)),Q;d,·,\pi )$. If the algebra $(\mathrm{\Psi },Q;d,·,\pi )$ is completely atomistic, then the map is even onto $\mathcal{R}\left(At\right(\mathrm{\Psi }\left)\right),$ and hence an isomorphism [6].

A similar analysis can be carried out in the domain-free version of an information algebra, which links atomic algebras to information algebras of subsets. These results are partial answers to the Representation Problem: Is any information algebra (labeled or domain-free) isomorphic to an information algebra of sets, and if yes, how? More on this question will be said in Section 6.

4. Generic Constructions

Where do information algebras come from? Are there general ways to construct information algebras? There are! In this Section we describe several generic ways to construct information algebras. There are at least three different approaches: First, similarly to atomic algebras, information algebras may be obtained from abstract tuple systems as generalized relational algebras. This is presented in Section 4.1. Secondly, logic provides a rich source for information algebras. There are several ways information algebras are related to logic, especially algebraic logic. Section 4.2 to Section 4.4 are devoted to this important subject. Finally, new information algebras may be obtained from given ones by constructions used in Universal Algebra and Category Theory. This will be discussed in Section 4.5.

4.1. Generalized Relational Algebras

Regarding atoms of an atomic information algebra, we have said that atoms form a tuple system. Here we give a formal definition of the concept of an abstract tuple system. Let Q be a lattice and T a set, whose generic elements will be denoted by $t,s,\dots$ . Suppose that two operations representing labeling and projection are defined:

(1)

Labeling: $d:T\to Q$, $t\mapsto d\left(t\right)$,

(2)

Projection:$\pi :T\times Q$, $t,x\mapsto {\pi }_x\left(t\right)$, defined for $x\le d\left(t\right)$.

The system $(T,Q;d,\pi )$ is called a tuple system, if the following conditions are satisfied for $t,s\in T$ and $x,y\in Q$:

(1)

If $x\le d\left(t\right)$, then $d\left({\pi }_x\left(t\right)\right)=x$,

(2)

If $x\le y\le d\left(t\right)$, then ${\pi }_x\left({\pi }_y\left(t\right)\right)={\pi }_x\left(t\right)$,

(3)

If $d\left(t\right)=x$, then ${\pi }_x\left(t\right)=t$,

(4)

If $d\left(t\right)=x$, $d\left(s\right)=y$ and ${\pi }_{x\wedge y}\left(t\right)={\pi }_{x\wedge y}\left(s\right)$, then there exists $r\in T$ such that $d\left(r\right)=x\vee y$ and ${\pi }_x\left(r\right)=t$, ${\pi }_y\left(r\right)=s$,

(5)

If $d\left(t\right)=x$ and $x\le y$, then there exists $s\in T$ such that $d\left(s\right)=y$ and ${\pi }_x\left(s\right)=t$.

The elements of T are called generalized tuples, since ordinary tuples evidently satisfy the conditions of a tuple system. The label $d\left(t\right)$ of a tuple t is called its domain.

As usual, the elements of Q may be thought of as representing questions. The tuples with domain x are the possible answers to the question represented by $x\in Q$. Projection of a tuple t to some coarser domain y links the answers to a question with the answers to a coarser question. If ${\pi }_y\left(t\right)={\pi }_y\left(s\right)$, then s and t belong to the same answer to the coarser question. As we have seen, the atoms of an atomic labeled information algebra form a tuple system. So do the blocks of a partition lattice satisfying condition in Equation (6). In fact, the elements of any labeled information algebra $(\mathrm{\Psi },Q;d,·,\pi )$ form a tuple system $(\mathrm{\Psi },Q;d,\pi )$ (see [6]).

If we fix $x\in Q$, then any subset R of tuples with domain x represents a partial answer to the question x. Such a subset R is called a generalized relation with domain x. Denote by $\mathcal{R}\left(T\right)$ the family of all such generalized relations. In the case of the tuple system of the blocks of a partition lattice, generalized relations are simply sets of blocks of a same partition. As for atomic labeled information algebras in Section 3.5, we may define generalized relational joins between generalized relations R and S with domain x and y respectively, as well as relational projection to $y\le x$ of a generalized relation relation R with domain x:

$$\begin{array}{ccc}\hfill R\bowtie S& =& \{t\in T:d\left(t\right)=x\vee y,{\pi }_x\left(t\right)\in R,{\pi }_y\left(t\right)\in S\}\hfill \end{array}$$

(36)

$$\begin{array}{ccc}\hfill {\pi }_y\left(R\right)& =& \{{\pi }_y\left(t\right):t\in R\}\hfill \end{array}$$

(37)

As in the atomic case (Section 3.5) it can be shown that the pairs $(R,x)$, where R is a generalized relation with domain x, form a labeled information, where the operations are defined as follows:

(1)

Labeling:$d(R,x)=x$,

(2)

Combination:$(R,x)·(S,y)=(R\bowtie S,x\vee y)$,

(3)

Projection:${\pi }_x(R,y)=({\pi }_x\left(R\right),x)$.

We denote this algebra by $\left(\mathcal{R}\right(T),Q;d,·,\pi )$ and call it the generalized relational algebra over the tuple system $(T,Q,d,\pi )$. Note that this is an information algebra whose elements are sets of tuples; it is a set algebra.

4.2. Quantifier Algebras

A monadic algebra [15,16] is a Boolean algebra Φ together with a map $\exists :\mathrm{\Phi }\to \mathrm{\Phi }$, which is an existential quantifier (see Section 3.2), that is,

(1)

$\exists \left(1\right)=1$,

(2)

$\exists \left(\varphi \right)\le \varphi$,

(3)

$\exists (\exists (\varphi )\vee \psi )=\exists \left(\varphi \right)\vee \exists \left(\psi \right)$.

Now, since only the join operation is involved in the definition of an existential quantifier, this concept applies to any join-semilattice, see [12]. Therefore, let Φ be a commutative, idempotent semigroup, hence a semilattice. We add a family of extraction operators in the following way: Let $\{{\exists }_i:i\in V\}$ be a finite set of commuting existential quantifiers, that is:

$${\exists }_i\left({\exists }_j\left(\varphi \right)\right)={\exists }_j\left({\exists }_i\left(\varphi \right)\right)$$

(38)

for all $i,j\in V$ and all $\varphi \in \mathrm{\Phi }$. Then we may define a map ${\exists }_s$ for any subset $s=\{i_1,\dots ,i_n\}\subseteq V$ by

$${\exists }_s\left(\varphi \right)={\exists }_{i_n}\left({\exists }_{i_{n-1}}\left(\dots {\exists }_{i_{i1}}\left(\varphi \right)\dots \right)\right)$$

(39)

Clearly, ${\exists }_s$ is an existential quantifier, and for subsets s and t of V,

$${\exists }_{s\cup t}\left(\varphi \right)={\exists }_s\left({\exists }_t\left(\varphi \right)\right)$$

(40)

In this way, we obtain a quantifier algebra $(\mathrm{\Phi },\mathcal{P}(V);·,\exists )$, where $\exists :\mathrm{\Phi }\times \mathcal{P}\left(V\right)\to \mathrm{\Phi }$, and where $\mathcal{P}\left(V\right)$ is the power set of V. This is closely related to an information algebra: We may define extraction operators ${ϵ}_s$ for subsets s of V by

$${ϵ}_s\left(\varphi \right)={\exists }_{V-s}\left(\varphi \right)$$

(41)

Then, $(\mathrm{\Phi },\mathcal{P}(V);·,ϵ)$ is a domain-free information algebra. In specific applications of this construction, existential quantification usually corresponds to the elimination of variables, which amounts to extract information pertaining to the remaining variables (see [6]). For this type of algebras we refer to cylindric algebras [17], which are an important concept in algebraic logic.

4.3. Consequence Operators

How can we express information? One possibility is to do this with propositions expressed by sentences of some appropriate formal language. The internal syntactic structure of this language is of no concern here. The essential point is that there is an entailment relation which permits to deduce further sentences from a set of given sentences. This captures the essence of a logic. The present section is essentially based on [18].

Let then $\mathcal{L}$ be a set of sentences, representing the language to be considered. An entailment relation relates sets of sentences $X\subseteq \mathcal{L}$ to single sentences $s\in \mathcal{L}$, such that the following conditions are satisfied:

(1)

$X⊢s$ for all $s\in X$,

(2)

If $X⊢s$ for all $s\in Y$ and $Y⊢t$, then $X⊢t$.

Such entailment relations are used, for instance, in information systems introduced by Scott [14]. The relation $X⊢s$ means that the sentence s can be deduced from X, or is entailed by X.

We may ask for the set of all sentences entailed by a set X. This leads to an operator C mapping the power set of $\mathcal{L}$ into itself,

$$C\left(X\right)=\{s\in \mathcal{L}:X⊢s\}$$

(42)

This mapping is a consequence operator (or also a closure operator); that is, it satisfies the following three conditions [6]:

(1)

$X\subseteq C\left(X\right)$,

(2)

$C\left(C\right(X\left)\right)=C\left(X\right)$,

(3)

if $X\subseteq Y$, then $C\left(X\right)\subseteq C\left(Y\right)$.

Sets X such that $X=C\left(X\right)$ are called closed; in particular any set of the form $C\left(X\right)$ is closed. We consider the closed sets as our pieces of information. Thus let Φ denote the family of all closed sets. Note that, in general, any closed set A can be written as $A=C\left(X\right)$ with $X\subseteq A$ in many different ways. Our interest usually is in finding small, at best finite sets X generating A in this way. So a piece of information A is coded nonuniquely by a set of sentences X contained in A. If we have two pieces of information coded by X and Y respectively, then the combined information is the closure of the set $X\cup Y$. So, we define a combination operation within Φ by

$$C\left(X\right)·C\left(Y\right)=C(X\cup Y)$$

(43)

Since $C(X\cup Y)=C\left(C\right(X)\cup C(Y\left)\right)$, this operation is well defined among closed sets. Clearly, under this operation Φ is a commutative, idempotent semigroup. The set $C\left(\varnothing \right)$ of all tautologies is the unit element of this combination and $\mathcal{L}$ is its null element. In fact, closed sets form a ∩-system, since the intersection of any family of closed systems is closed [14]. So Φ is a complete lattice under inclusion as order, intersection is the meet and the join is given by our combination operation.

In order to obtain an information algebra of closed sets, we need to introduce extraction operators into the semigroup Φ. For this purpose, we consider sublanguages $\mathcal{M}\subseteq \mathcal{L}$ of the original language. The information $C\left(X\right)$ restricted to $\mathcal{M}$ is simply $C\left(X\right)\cap \mathcal{M}$. However, this set is not necessarily closed. Thus a candidate for an extraction operator relative to the sublanguage $\mathcal{M}$ is obtained by taking the closure of this set,

$$C_{\mathcal{M}}\left(C\left(X\right)\right)=C(C\left(X\right)\cap \mathcal{M})$$

(44)

This is a map $C_{\mathcal{M}}:\mathrm{\Phi }\to \mathrm{\Phi }$.

We want this map to be an existential quantifier on the semilattice Φ of closed sets (see Section 3.2) in order to obtain an information algebra. This means that conditions (a), (b) and (c) in Section 3.2 must be satisfied, which imposes additional conditions on the consequence operator C.

Condition (a) requires that $C\left(\mathcal{M}\right)=\mathcal{L}$, that is, the sublanguage $\mathcal{M}$ by itself should not yet convey any information. Then $C_{\mathcal{M}}\left(\mathcal{L}\right)=C(\mathcal{L}\cap \mathcal{M})=C\left(\mathcal{M}\right)=\mathcal{L}$. So condition (a) of an existential quantifier is satisfied. Condition (b) is also satisfied, since $C_{\mathcal{M}}\left(C\left(X\right)\right)\subseteq C\left(X\right)$.

Condition (c) requires that for any two closed sets $E,F\in \mathrm{\Phi }$,

$$C_{\mathcal{M}}\left(C(C_{\mathcal{M}}\left(E\right)\cup F)\right)=C(C_{\mathcal{M}}\left(E\right)\cup C_{\mathcal{M}}\left(F\right))$$

(45)

This is equivalent to the following condition on the consequence operator C, for any subsets E and F of $\mathcal{L}$ [6,18,

$$C\left(C\right((E\cap \mathcal{M})\cup F)\cap \mathcal{M})=C\left(\right(E\cap \mathcal{M})\cup (F\cap \mathcal{M}\left)\right)$$

(46)

Next, we are going to consider a family $\mathcal{S}$ of sublanguages $\mathcal{M}$ of $\mathcal{L}$ which is closed under finite intersection, and such that each operator $C_{\mathcal{M}}$ is an existential quantifier, i.e., satisfies in particular Equation (46) for all $\mathcal{M}\in \mathcal{S}$. These sublanguages $\mathcal{M}$ may be thought of representing questions, namely what propositions in $\mathcal{M}$ are asserted by a piece of information $C\left(X\right)\in \mathrm{\Phi }$. Further, we require that $E=\{C_{\mathcal{M}}:\mathcal{M}\in \mathcal{S}\}$ becomes a commutative, idempotent semigroup of extraction operators $C_{\mathcal{M}}:\mathrm{\Phi }\to \mathrm{\Phi }$. A sufficient condition for this is

$$C_{\mathcal{M}}\circ C_{\mathcal{N}}=C_{\mathcal{M}\cap \mathcal{N}}$$

(47)

for any pair $\mathcal{M},\mathcal{N}\in \mathcal{S}$. Then in this way $(\mathrm{\Phi },E;·,\circ )$ becomes a domain-free information algebra; an algebra associated with an entailment system $(\mathcal{L},⊢)$ and an intersection-semilattice $\mathcal{S}$ of sublanguages of $\mathcal{L}$. This shows one way a system of logic may induce an information algebra. So, an information algebra can be obtained from a consequence operator satisfying the additional conditions in Equations (46) and (47) on some language $\mathcal{L}$.

Condition in Equation (47) means, in terms of the consequence operator C, that for any closed set E

$$C\left(C\right(E\cap \mathcal{N})\cap \mathcal{M})=C\left(C\right(E\cap \mathcal{M})\cap \mathcal{N})=C(E\cap \mathcal{M}\cap \mathcal{N})$$

(48)

This last condition has a natural characterization in terms of interpolation [18]: A consequence operator C has the interpolation property with respect to a class of sublanguages $\mathcal{S}$, if, for all sublanguages $\mathcal{M}$ and $\mathcal{N}$ of $\mathcal{S}$, from $s\in \mathcal{M}$, $X\subseteq \mathcal{N}$ and $s\in C\left(X\right)$ it follows that there exists a set of sentences $Y\subseteq \mathcal{M}\cap \mathcal{N}$ such that $Y\subseteq C\left(X\right)$ and $s\in C\left(Y\right)$. The set Y is called the interpoland between s and X. It can be shown [18] that a consequence operator C satisfies Equation (48) if and only if it has the interpolation property with respect to the family of sublanguages $\mathcal{S}$. A similar characterization of condition in Equation (46) is not known so far. In [18] a sufficient condition implying Equation (46) is given. Further, we refer to [19] for a discussion of information algebras related to various systems of logic.

4.4. Language and Models

In the previous section, information was defined on a purely syntactical level as closed theories. No semantics was considered. In this section we add semantics to language. As before, consider a set $\mathcal{L}$ of sentences. Add now another set $\mathcal{M}$, whose elements are considered to be models or possible worlds. We consider triples $(\mathcal{L},\mathcal{M},\vDash )$, where ⊧ is a binary relation $\vDash \subseteq \mathcal{L}\times \mathcal{M}$ between sentences and models. We write $m\vDash s$ (m models s) instead of $(s,m)\in \vDash$. Such triples are called classifications in the book [20] on information flow, and the terms types and tokens are used instead of our terms of sentences and models. Further, they occur in formal concept analysis [14,21] where the elements of $\mathcal{L}$ and $\mathcal{M}$ are called attributes and objects.

In our discussion, which is based on [9], we assume that the relation $m\vDash s$ means that model msatisfies sentence s or that m is a model, or a possible interpretation, of sentence s. An example is provided by propositional logic, where $\mathcal{L}$ is a propositional language and the elements of $\mathcal{M}$ are valuations, and $m\vDash s$ means that m satisfies s in the usual definition in propositional logic. Another example is provided by predicate logic. The language $\mathcal{L}$ is given by formulae of predicate logic and models $\mathcal{M}$ are structures used to interpret formulae. We refer to [6,22] for a discussion of information algebras relating to propositional and predicate logic, which are special cases of the structures considered here.

In a triple $(\mathcal{L},\mathcal{M},\vDash )$ a set X of sentences determines a set of models by

$$\widehat{r}\left(X\right)=\{m\in \mathcal{M}:m\vDash s \text{for} \text{all} s\in X\}$$

(49)

We call $\widehat{r}\left(X\right)$ the models of X. We assume here that there is no model which satisfies all sentences of $\mathcal{L}$, that is, $\widehat{r}\left(\mathcal{L}\right)=\varnothing$. Similarly, a set A of models determines a set X of sentences by

$$\check{r}\left(A\right)=\{s\in \mathcal{L}:m\vDash s \text{for} \text{all} m\in A\}$$

(50)

The set $\check{r}\left(A\right)$ is called the theory of A. Note that $\check{r}\left(\varnothing \right)=\mathcal{L}$. The significance of $\widehat{r}$ and $\check{r}$ will become clear in a moment. We note that

$$\widehat{r}\left(X\right)\supseteq A⟺X\subseteq \check{r}\left(A\right)$$

(51)

This means that $\widehat{r}$ and $\check{r}$ form a Galois connection [14]. We define further for $X\subseteq \mathcal{L}$ and $A\subseteq \mathcal{M}$,

(52)

It is well known that $C_{\vDash }$ and $C^{\vDash }$ are consequence operators in $\mathcal{L}$ and $\mathcal{M}$ respectively [14]. We consider the family of $C_{\vDash }$-closed sets ${\mathrm{\Phi }}_{\vDash }\subseteq \mathcal{L}$ and $C^{\vDash }$-closed sets ${\mathrm{\Phi }}^{\vDash }\subseteq \mathcal{M}$. Again, it is well known that ${\mathrm{\Phi }}_{\vDash }$ and ${\mathrm{\Phi }}^{\vDash }$ are complete lattices under inclusion [14].

We now construct two closely related information algebras based on the lattices ${\mathrm{\Phi }}_{\vDash }$ and ${\mathrm{\Phi }}^{\vDash }$. Note that the models of a set of sentences X, $\widehat{r}\left(X\right)$, are $C^{\vDash }$ closed sets in $\mathcal{M}$ [14]. We consider the models of X as the information about possible worlds expressed by a set of sentences X. Similarly, the theory $\check{r}\left(A\right)$ of a set of models is $C_{\vDash }$-closed. This is the theory of A. In ${\mathrm{\Phi }}_{\vDash }$ we define combination, as in the preceding Section 4.3 by

$$C_{\vDash }\left(X\right)·C_{\vDash }\left(Y\right)=C_{\vDash }(X\cup Y)$$

(53)

This corresponds to the information order $C_{\vDash }\left(X\right)\le C_{\vDash }\left(Y\right)$ if $C_{\vDash }\left(X\right)\subseteq C_{\vDash }\left(Y\right)$. The larger a theory, the more informative it is. In ${\mathrm{\Phi }}^{\vDash }$ however, we define

$$C^{\vDash }\left(A\right)·C^{\vDash }\left(B\right)=C^{\vDash }\left(A\right)\cap C^{\vDash }\left(B\right)$$

(54)

This corresponds to the information order $C^{\vDash }\left(A\right)\le C^{\vDash }\left(B\right)$ if $C^{\vDash }\left(A\right)\supseteq C^{\vDash }\left(B\right)$. The smaller a model set is the more precise and the more informative it is. This makes sense, if the questions we consider search for an unknown possible world, an unknown model. In both cases ${\mathrm{\Phi }}_{\vDash }$ as well as ${\mathrm{\Phi }}^{\vDash }$ become commutative, idempotent semigroups. In the first case the vacuous information is the tautology $C_{\vDash }\left(\varnothing \right)$, in the second case the whole model set $\mathcal{M}$. The null element in the first case is the whole language $\mathcal{L}$, in the second case it is the empty set.

Next, we introduce extraction operators. We do this here first with respect to ${\mathrm{\Phi }}^{\vDash }$; corresponding extraction operators for ${\mathrm{\Phi }}_{\vDash }$ will be derived later. For this purpose we consider a partition of $\mathcal{M}$ and use the associated equivalence relation defined by $m\equiv n$ iff m and n belong to the same block of the partition. Extraction then means to restrict an element $\varphi \in {\mathrm{\Phi }}^{\vDash }$ to this partition. This is expressed by the saturation operatorsϵ with respect to its partition. If A is any subset of $\mathcal{M}$, then

$$ϵ\left(A\right)=\{m\in \mathcal{M}:\exists n\in A,m\equiv n\}$$

(55)

We require that the saturation of a $C^{\vDash }$-closed set is still $C^{\vDash }$-closed. This requirement is fulfilled if and only if closure and saturation commute, $C^{\vDash }\left(ϵ\left(A\right)\right)=ϵ\left(C^{\vDash }\left(A\right)\right)$. Under this assumption it turns out that ϵ satisfies conditions (a) to (c) of item 2 of the definition of a domain-free algebra automatically. Or, in other words, ϵ is an existential quantifier with respect to the intersection semilattice of $C^{\vDash }$-closed sets (see Section 3.2),

(a)

$ϵ\left(\varnothing \right)=\varnothing$,

(b)

$ϵ\left(\varphi \right)\supseteq \varphi$,

(c)

$ϵ\left(ϵ\right(\varphi )\cap \psi )=ϵ\left(\varphi \right)\cap ϵ\left(\psi \right)$.

Note that here the information order is the reverse of set inclusion, join in the information order is therefore set intersection.

Consider now a lattice Q of partitions of $\mathcal{M}$, more precisely, a sublattice of the lattice $part\left(\mathcal{M}\right)$ of partitions of $\mathcal{M}$. We require that the following independence condition is satisfied (see Section 2.2, Equation (6)): If $\mathcal{P},\mathcal{Q}$ are two partitions from the lattice Q, then

$$m{\equiv }_{\mathcal{P}\wedge \mathcal{Q}}n\Rightarrow \exists l:m{\equiv }_{\mathcal{P}}l,n{\equiv }_{\mathcal{Q}}l$$

(56)

Let ${ϵ}_{\mathcal{P}}$ be the saturation operator relative to the partition $\mathcal{P}$ and E the family of all operators ${ϵ}_{\mathcal{P}}$ for all $\mathcal{P}\in Q$. Then the independence condition above is sufficient for E to be a commutative, idempotent semigroup under composition, or what is the same:

$${ϵ}_{\mathcal{P}}\circ {ϵ}_{\mathcal{Q}}={ϵ}_{\mathcal{P}\wedge \mathcal{Q}}$$

(57)

So, $({\mathrm{\Phi }}^{\vDash },E,·,\circ )$ defined in this way is a domain-free information algebra.

To any $\varphi \in {\mathrm{\Phi }}^{\vDash }$ we may associate its theory $\check{r}\left(\varphi \right)\in {\mathrm{\Phi }}_{\vDash }$. We have already defined combination in ${\mathrm{\Phi }}_{\vDash }$. Now, we also define extraction operators for $\mathcal{P}\in Q$ by

$${\tau }_{\mathcal{P}}\left(\psi \right)=\check{r}\left({ϵ}_{\mathcal{P}}\left(\widehat{r}\left(\psi \right)\right)\right)$$

(58)

for $\psi \in {\mathrm{\Phi }}_{\vDash }$. Then, with $T=\{{\tau }_{\mathcal{P}}:\mathcal{P}\in Q\}$, the system $({\mathrm{\Phi }}_{\vDash },T,·,\circ )$ also becomes a domain-free information algebra. In fact, the maps $\varphi \mapsto \check{r}\left(\varphi \right)$ and ${ϵ}_{\mathcal{P}}\mapsto {\tau }_{\mathcal{P}}$ define an isomorphism from $({\mathrm{\Phi }}^{\vDash },E,·,\circ )$ onto $({\mathrm{\Phi }}_{\vDash },T,·,\circ )$, see [9].

We refer also to [9] for a discussion of the associated labeled construction and a discussion of the relation to the notions of infomorphism and channels in [20].

This is a generic way to construct information algebras from models of propositional and predicate logic, of systems of linear equations and inequalities, etc. It extends and generalizes semantic information theory [23,24,25,26,27,28,29]. In all cases mentioned the lattice Q of partitions of $\mathcal{M}$ is usually obtained from a multivariate system (see Section 2.1), since these model are based on sets of variables and models are either vectors or sequences, see [6,9]. We refer also to [19], where structures called similarity models are introduced, using a similar approach to the one presented here to construct commutative and idempotent semigroups of extraction operators.

4.5. Morphisms

Usually, new algebraic structures are constructed from algebras of the same type. For example, Cartesian products of information algebras should be again an information algebras. This will by shown in Section 5.3. Here we consider maps between information algebras. This is also a step towards a category theoretical study of information algebra. More on this subject is to be found in Section 5.3.

Consider two domain-free information algebras $({\mathrm{\Phi }}_1,E_1,·,\circ )$ and $({\mathrm{\Phi }}_2,E_2,·,\circ )$. To simplify notation we use the same symbol for combination and composition in both algebras; it will always be clear from the context which operation is meant. In the same way, ≤ denotes the order either in ${\mathrm{\Phi }}_1$ or in ${\mathrm{\Phi }}_2$. A map $f:{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2$ is called order preserving, if $\varphi \le \psi$ in ${\mathrm{\Phi }}_1$ implies $f\left(\varphi \right)\le f\left(\psi \right)$ in ${\mathrm{\Phi }}_2$. Let $[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]$ denote the set of order preserving maps from ${\mathrm{\Phi }}_1$ into ${\mathrm{\Phi }}_2$. We define in $[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]$ a combination operation pointwise by the combination in ${\mathrm{\Phi }}_2$,

$$(f·g)\left(\varphi \right)=f\left(\varphi \right)·g\left(\varphi \right)$$

(59)

Clearly $f·g$ is still an order-preserving map and this combination operation of order-preserving maps is associative, commutative and idempotent. So $[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]$ becomes a commutative and idempotent semigroup under combination. The unit element of combination is given by the map $\varphi \mapsto 1$, where 1 is the unit in ${\mathrm{\Phi }}_2$ and the null element is given by the map $\varphi \mapsto 0$, where here 0 is the null element in ${\mathrm{\Phi }}_2$. The associated order is defined by $f\le g$ if and only if $f\left(\varphi \right)\le g\left(\varphi \right)$ for all $\varphi \in {\mathrm{\Phi }}_1$.

In order to introduce extraction operations on $[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]$, we consider the cartesian product $E_1\times E_2$ and define maps $({ϵ}_1,{ϵ}_2)f:{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2$ by

$$\left(({ϵ}_1,{ϵ}_2)f\right)\left(\varphi \right)={ϵ}_2(f\left({ϵ}_1\left(\varphi \right)\right)$$

(60)

Clearly , the map $({ϵ}_1,{ϵ}_2)f$ is still order preserving. It further satisfies conditions (a) to (c) of item 2 in the definition of a domain-free information algebra. The family $E_1\times E_2$ of these maps is further commutative and idempotent under composition. So, $([{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2],E_1\times E_2,·,\circ )$ with combination and extraction as defined above is a domain-free information algebra. We shall argue in Section 5.3 that these maps are the correct choice of morphisms for making the class of domain-free information algebras a category.

5. Compact Algebras

5.1. Finiteness

Computers can only treat “finite” information, and “infinite” information can often be approximated by “finite” elements. This point of view is developed in particular in domain theory, a subject at the interface between order theory and computer science [14,30,31,32,33]. Here, we treat the problem of finiteness in a similar way and show that in this respect information algebras extend domain theory and that many results of this theory can be extended to information algebras.

We discuss finiteness in the context of domain-free information algebras, although it could also be studied in the context of labeled algebras. The relation between finiteness in domain-free and labeled information algebras is interesting and it is not yet fully understood, see [8].

Consider then a domain-free information algebra $(\mathrm{\Phi },E;·,\circ )$. In the set Φ of information elements we single out a subset ${\mathrm{\Phi }}_f$ of elements which we consider to be finite. To qualify for this, they must of course satisfy a number of requirements, which we shall enumerate below. Before, we need a new concept, which serves to express the idea that any element of Φ should be approximated by finite elements. This is the concept of a directed set: A nonempty subset $X\subseteq \mathrm{\Phi }$ is called directed, if for every pair $\varphi ,\psi \in X$ there is an element $\chi \in X$ such that $\varphi ,\psi \le \chi$.

Here are now the conditions we impose on ${\mathrm{\Phi }}_f$:

(1)

Subalgebra: The system $({\mathrm{\Phi }}_f,E,·,\circ )$ is a subalgebra of $(\mathrm{\Phi },E,·,\circ )$, that is it is closed under combination and extraction and the unit 1 and the null element 0 belong to ${\mathrm{\Phi }}_f$.

(2)

Convergence: If $X\subseteq {\mathrm{\Phi }}_f$ is directed, then the supremum $\vee X$ exists in Φ.

(3)

Density: For all $\varphi \in \mathrm{\Phi }$ and $ϵ\in E$,

$$ϵ\left(\varphi \right)=\bigvee \{\psi \in {\mathrm{\Phi }}_f:\psi =ϵ\left(\psi \right)\le \varphi \}$$

(4)

Compactness: If $X\subseteq {\mathrm{\Phi }}_f$ is directed, and $\psi \in {\mathrm{\Phi }}_f$ satisfies $\psi \le \vee X$, then there is a $\varphi \in X$ such that $\psi \le \varphi$.

If these conditions are satisfied, we call the system $(\mathrm{\Phi },{\mathrm{\Phi }}_f,E;·,\circ )$ a compact information algebra. Sometimes weaker conditions are sufficient: [6,8]: For instance it is often sufficient that ${\mathrm{\Phi }}_f$ is closed under combination, but not under extraction. Further, if the information algebra is supported, then from the density condition it follows also that

$$\varphi =\bigvee \{\psi \in {\mathrm{\Phi }}_f:\psi \le \varphi \}$$

(61)

This says that any element of Φ can be approximated by the finite elements below it. Conversely, this is not sufficient for the density condition above to hold, which says that any element supported by some domain can be approximated by finite elements supported by the same domain. Therefore it is called the weak density. We shall not discuss these refinements here, see [8].

As examples of compact information algebras we cite the algebra of strings, where the finite strings are the finite elements. In the information algebra of convex sets, the convex polyhedra are the finite elements, since any convex set can be approximated by the convex polyhedra it is contained in. Further a domain-free information algebra $(\mathrm{\Phi },E;·,\circ )$, where Φ is a finite set is always compact. However, in a set algebra, the cofinite sets (the complements of finite sets) do not qualify as finite elements, since only weak density is satisfied.

Important examples of compact information algebras are provided by consequence operators. Consider an entailment system $(\mathcal{L},⊢)$ and its associated consequence operator C (see Section 4.3). Assume that C satisfies conditions in Equations (46) and (48) specificd there, such that the system $(\mathrm{\Phi },E;·,\circ )$ associated with the entailment system $(\mathcal{L},⊢)$ and an intersection-semilattice $\mathcal{S}$ of sublanguages is an information algebra. Assume now further that C satisfies the following condition:

$$C\left(X\right)=\bigcup \left\{C\right(Y):Y\subseteq X,Y \text{finite}\}$$

(62)

This is equivalent to the condition that if $X⊢s$, then there is a finite subset Y of X such that $Y⊢s$. With this additional condition on C, the information algebra $(\mathrm{\Phi },E;·,\circ )$ of closed sets becomes a compact algebra whose finite elements are the sets $C\left(X\right)$, where X is a finite set (see [6]).

Another example of a compact information algebra is the ideal completion of a domain-free information algebra, see Section 3.3. In fact, the elements of Φ or rather the principal ideals $\downarrow \varphi$ for $\varphi \in \mathrm{\Phi }$ are the finite elements in the compact algebra $(I_{\mathrm{\Phi }},E;·,\circ )$, see [6]. So, embedding an information algebra into its ideal completion is a way to compactify this algebra. We refer to [34] for a variant of this compactification, yielding an analoguous result. On the other hand the following theorem is an extension of a well-known representation theorem of domain theory [35]: If $(\mathrm{\Phi },{\mathrm{\Phi }}_f,E;·,\circ )$ is a compact information algebra, then the ideal completion $(I_{{\mathrm{\Phi }}_f},{\mathrm{\Phi }}_f,E;·,\circ )$ is a compact information algebra, isomorphic to $(\mathrm{\Phi },{\mathrm{\Phi }}_f,E;·,\circ )$. For a proof we refer to [8].

The ideal completion can also be seen in another perspective. The set

$$I\left(X\right)=\{\varphi \in \mathrm{\Phi }:\varphi \le {\varphi }_1·\dots ·{\varphi }_m \text{for} \text{some} \text{finite} \text{set} {\varphi }_1,\dots ,{\varphi }_m\in X\}$$

(63)

is the ideal generated by the set $X\subseteq \mathrm{\Phi }$. This is a consequence operator in Φ. If we take Φ to be a language, and the information order as an entailment relation, that is, $X⊢\varphi$ if $\varphi \le \vee X$, then $I\left(X\right)$ is the consequence operator associated with the entailment relation. Further, for any $ϵ\in E$, let ${\mathrm{\Phi }}_{ϵ}$ be the set of all element of Φ, supported by ϵ, that is ${\mathrm{\Phi }}_{ϵ}=\{\varphi \in \mathrm{\Phi }:\varphi =ϵ\left(\varphi \right)\}$. The family ${\mathrm{\Phi }}_{ϵ}$ for $ϵ\in E$ is closed under finite intersections. It turns out that the family of I-closed sets is just the set $I_{\mathrm{\Phi }}$ of ideals of Φ. Moreover, the consequence operator I satisfies conditions in Equations (46) and (48) and also Equation (62) above, see [6]. Therefore, the information algebra generated by the consequence operator I together with the family of sublanguages $\{{\mathrm{\Phi }}_{ϵ}:ϵ\in E\}$ is just the ideal completion of the domain-free information algebra $(\mathrm{\Phi },E;·,\circ )$. In this way, any compact information algebra is generated by a consequence operator, as described in Section 4.3.

If $(\mathrm{\Phi },{\mathrm{\Phi }}_f,E;·,\circ )$ is a compact information algebra, then it follows that Φ is a complete lattice under the information order, see [6]. Further, it follows that $\psi \in {\mathrm{\Phi }}_f$ if and only if, for every directed set X contained in Φ, $\psi \le \vee X$ implies that there is a $\varphi \in X$ such that $\psi \le \varphi$. In order theory, elements of a complete partially ordered (cpo) set satisfying this condition are called finite elements,14]. Thus the order theoretic concept of finiteness corresponds with our concept. Also $\psi \in {\mathrm{\Phi }}_f$ if and only if, for every set $X\subseteq \mathrm{\Phi }$, $\psi \le \vee X$ implies that there is a finite subset F of X such that $\psi \le \vee F$. In order theory, elements of a cpo satisfying this condition are called compact elements,14]. Thus the order theoretic concept of compactness corresponds also to our concept of finite elements. Finally, all of this together means that Φ is an algebraic lattice, see [14] and $\mathrm{\Phi }-\left\{0\right\}$ is an algebraically complete order such that $\varphi \vee \psi \in \mathrm{\Phi }-\left\{0\right\}$ if $\varphi \vee \psi \ne 0$. Such a structure is called a Scott-Ershov domain. This links domain-free information algebras to domain theory [35]. Compact information algebras are domains with an additional operation, namely the extraction. Many results from domain theory extend to compact information algebras. In particular, compactness could be generalized to continuity by use of the way-below relation. see [8,34].

5.2. Continuous Maps

Maps between sets of pieces of information also represent information. In fact, we have seen in Section 4.5 that order preserving maps between domain-free information algebras themselves form an information algebra. In the case of compact information algebras the associated morphisms are not simply order preserving maps, but continuous maps. The concept of an continuous map is inherited from $cpos$: Let $({\mathrm{\Phi }}_1,{\mathrm{\Phi }}_{1,f},E_1;·,\circ )$ and $({\mathrm{\Phi }}_2,{\mathrm{\Phi }}_{2,f},E_2;·,\circ )$ be two compact information algebras. A map $f:{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2$ is called continuous, if for all $\varphi \in {\mathrm{\Phi }}_1$,

$$f\left(\varphi \right)=\bigvee \{f\left(\psi \right):\psi \in {\mathrm{\Phi }}_{1,f},\psi \le \varphi \}$$

(64)

This is also equivalent to the condition that $f(\vee X)=\vee f\left(X\right)$ for every directed set $X\subseteq \mathrm{\Phi }$. Extraction operators of a compact information algebra $(\mathrm{\Phi },{\mathrm{\Phi }}_f,E;·,\circ )$ are continuous, that is, $ϵ(\vee X)=\vee ϵ\left(X\right)$ for any directed set X and any $ϵ\in E$, see [6]. Further, a continuous map is order preserving.

Just in the same way as we defined an information algebra of order preserving maps, we can also obtain an information algebra of continuous maps. Let ${[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]}_c$ be the set of continuous maps between the two compact algebras $({\mathrm{\Phi }}_1,{\mathrm{\Phi }}_{1,f},E_1;·,\circ )$ and $({\mathrm{\Phi }}_2,{\mathrm{\Phi }}_{2,f},E_2;·,\circ )$. Define combination and extraction as in Section 4.5 for the case of order preserving maps, for $f,g\in {[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]}_c$ and ${ϵ}_1\in E_1$, ${ϵ}_2\in E_2$,

(1)

Combination: $(f·g)\left(\varphi \right)=f\left(\varphi \right)·g\left(\varphi \right)$,

(2)

Extraction: $\left(({ϵ}_1,{ϵ}_2)f\right)\left(\varphi \right)={ϵ}_2\left(f\left({ϵ}_1\left(\varphi \right)\right)\right)$.

With these operations $({[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]}_c,E_1\times E_2;·,\circ )$ becomes a domain-free information algebra, a subalgebra of the algebra of order preserving maps, Section 4.5. In fact, this is a compact algebra. Its finite elements are defined using maps $s:Y\to {\mathrm{\Phi }}_{2,f}$, where Y is a finite subset of ${\mathrm{\Phi }}_{1,f}$. These maps are called simple. Let S be the set of simple maps. For any $s\in S$ let $Y\left(s\right)$ be its domain and define

$$\widehat{s}\left(\varphi \right)=\bigvee \{s\left(\psi \right):\psi \in Y\left(s\right),\psi \le \varphi \}$$

(65)

If the set on the right hand side is empty, then $\widehat{s}\left(\varphi \right)=1$. It turns out that these maps $\widehat{s}$ are the finite elements of the information algebra of continuous maps, see [6].

5.3. Categories of Information Algebras

To conclude this review of information algebras we are going to consider the following two categories of information algebras:

(1)

The category IA has as its objects domain-free information algebras $(\mathrm{\Phi },E;·,\circ )$ and as its morphisms order preserving maps.

(2)

The category CompIA has as its objects compact information algebras $(\mathrm{\Phi },{\mathrm{\Phi }}_f,E;·,\circ )$ and as its morphisms continuous maps.

The category CompIA is a subcategory of IA. It turns out that these categories are both Cartesian Closed.

This means that these categories satisfy the following three conditions:

(1)

both have terminal objects,

(2)

both have direct products,

(3)

both have exponentials.

An object T is a terminal object if there is exactly one morphism from any other object to T. In our case this is simply the information algebra $\left(\right\{1\},\{id\};·,\circ )$, where Φ and E both consist of only one element.

Secondly, the Cartesian product $({\mathrm{\Phi }}_1\times {\mathrm{\Phi }}_2,E_1\times E_2;·,\circ )$ of two information algebras, where both combination and extraction are defined component-wise, belongs to IA. If the two algebras are compact, then $({\mathrm{\Phi }}_1\times {\mathrm{\Phi }}_2,{\mathrm{\Phi }}_{1,f}\times {\mathrm{\Phi }}_2,E_{1,f}\times E_2;·,\circ )$ is compact too, hence belongs to CompIA. In fact, these Cartesian products are the categorial direct product. That is, if the projections $p_1$ and $p_2$ are defined in the usual way, $p_1({\varphi }_1,{\varphi }_2)={\varphi }_1$ and $p_2({\varphi }_1,{\varphi }_2)={\varphi }_2$, then, for any pair of morphisms (order preserving or continuous maps) $f_1$ and $f_2$ from a third information algebra $(\mathrm{\Phi },E;·,\circ )$ into $({\mathrm{\Phi }}_1,E_1,·,\circ )$ and $({\mathrm{\Phi }}_2,E_2,·,\circ )$, respectively, there is a morphisms f from $(\mathrm{\Phi },E;·,\circ )$ into $({\mathrm{\Phi }}_1\times {\mathrm{\Phi }}_2,E_1\times E_2;·,\circ )$ such that $p_1\circ f=f_1$ and $p_2\circ f=f_2$.

Finally both categories IA and CompIA have exponentials. In fact, the information algebras $([{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2],E_1\times E_2;·,\circ )$ and $({[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]}_c,E_1\times E_2;,·,\circ )$ of order preserving or continuous maps respectively are the exponentials in their respective categories. Define the map $eval:{[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]}_c\times {\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2$ by

$$eval(f,\varphi )=f\left(\varphi \right).$$

(66)

This map is continuous, hence a morphism in CompIA. Next let f be a morphism $f:\mathrm{\Phi }\times {\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2$ for a third compact information algebra $(\mathrm{\Phi },{\mathrm{\Phi }}_f,E;·,\circ )$. Then define a map $\lambda f:\mathrm{\Phi }\to {[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]}_c$ by

$$\left(\lambda f\right(\chi \left)\right)\left(\varphi \right)=f(\chi ,\varphi )$$

(67)

It turns out that the map $\lambda f$ is continuous, hence a morphism of the category CompIA and that $eval\circ (\lambda f,i{d}_{{\mathrm{\Phi }}_1})=f$. The same holds with respect to ordinary information algebras. This shows that the information algebras $([{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2],E_1\times E_2;·,\circ )$ and $({[{\mathrm{\Phi }}_1\to {\mathrm{\Phi }}_2]}_c,E_1\times E_2;·,\circ )$ are the exponentials in their respective categories.

So, indeed both categories IA and CompIA are Cartesian closed. For proofs we refer to the manuscript [8].

6. Outlook

In this section we address a few issues not discussed above. For a start, we consider the question what part information algebras could have in a general theory of information as sketched, for instance, in [36]. It seems that the basic pragmatic purpose of information is to answer questions. This is already implicit in Shannon’s theory. The question there is what symbol or sequence of symbols is transmitted over a channel. This implies that one should be concerned with extracting the part of the information relevant to the question one is interested in. Information extraction is indeed a common activity in internet search, query answering in databases, constraint solving etc. Surely also information comes usually in pieces, from different sources and sensors, from varying files or data bases. Therefore, aggregation is an unavoidable operation in processing information. Any theory of information must in one or another way cover these issues. Whether it does it the way information algebras proposes is another question. The fact that classical information structures of Computer Science like relational databases, different systems of logic, systems of equations, constraints, etc. can be modeled by information algebras is a strong argument for its relevance. Also many studies of semantic information turn around the concept of questions [26,27,28,29,37,38,39,40,41,42], although the algebraic structure of extraction is not explicitly discussed. These studies are also closely connected to semantic information theory, as discussed for instance in [23,24,25,43]. These last papers study language and information in a way similar to Section 4.4 where it is shown how language expresses information and how information algebras arise from language and its semantics. Finally we have shown that information algebras are also closely related to domain theory, another important part of Computer Science modeling information and computation. So, information algebras are very tightly linked to important pragmatic information concepts and information theories of Computer Science and Philosophy. This will be underlined even more strongly below.

The classical information theories are those of Shannon [44] and Solomonov, Kolmogorov and Chaitin [45,46,47,48]. All these theories address the question of how to measure information content of a message or an object (both seen as a sequence of symbols). In this sense, they are very limited theories of information. In information algebras we have an information order, that is a qualitative comparison of information content. We may wonder whether we may associate a quantitative measure of information content to the elements of an information algebra. This measure should respect the information order. This seems only possible if the information considered is in some sense “finite”, that is, can be represented by a finite number of bits. In [9] it is proposed to use Hartley measure to measure uncertainty of a partial answer to a given question in an atomistic information algebra, and then to measure information content by the reduction of uncertainty. As an illustration, consider the very particular case of a string algebra, see Section 3.1. We consider only binary strings to simplify matters. Suppose we look for a string of length n. There are $2^n$ possible strings of this length. If we know nothing about our string, then the measure of the uncertainty according to Hartley [49] is the logarithm of possible cases, that is n. If as an information we are given the prefix of length m of the string, there remain only $2^{n-m}$ possible strings, that is the remaining uncertainty is $n-m$. The reduction of uncertainty is then $n-(n-m)=m$. So the measure of information content in the string of length m about the string of length n is exactly m bits. This makes sense in this particular case as it does also in more general situations [9]. Shannon’s theory can be seen as an extension of Hartley’s approach to a probabilistic situation. Probabilistic information will be shortly touched below, although we claim that information is primarily not probabilistic. It turns out that Shannon’s way to measure information content can consistently be applied to information algebras, if some finiteness conditions are satisfied. It is an open question whether and how information algebra can be related to algorithmic information theory.

It has been mentioned at the beginning that the axiomatics of information algebra were originally motivated by local computation schemes [1,2]. The inference problem to be solved in a labeled information algebra $(\mathrm{\Psi },Q;d,\circ ,\pi )$ can be formulated as follows: Given ${\varphi }_1,\dots ,{\varphi }_n\in \mathrm{\Psi }$ and $x\in Q$ compute

$${\pi }_x({\varphi }_1·\dots ·{\varphi }_n)$$

(68)

Query answering in relational databases, constraint solving, solving systems of linear equations, computing Fourier transforms and many other problems can be put into this form, see [7] for many further examples. The factors in this combination have domains $x_i=d\left({\varphi }_i\right)$. Naive combination leads to an element on domain

$$d({\varphi }_1·\dots ·{\varphi }_n)=x_1\vee \dots \vee x_n$$

(69)

This is usually computationally infeasible, since the complexity of extraction and combination grows often (although not always) exponentially in the size of the domain [7]. Local computation schemes organize the computation in such a way that all operations can be executed “locally” on the domains $x_1,\dots ,x_n$ of the factors of the combination. This is already possible in valuation algebras where the idempotency property is missing [5,6,7]. However, idempotency allows to simplify the computational architectures further. Local computation is also related to the idea of (conditional) independence of information, a concept known especially from probability theory [50,51], but also from relational database theory [52,53]. This indicates that this concept can be generalized to valuation or information algebras [6,54,55]. In general this subject is treated in the framework of multivariate systems. However, the general framework would be the one of a lattice of questions or domains. In this respect not much has been done so far, indicating a fruitful topic of research.

In Section 3.5 we have seen that atomistic labeled information algebras can be embedded into a generalized relational information algebra, that is an algebra of subsets. The question arises, whether any information algebra (labeled or domain-free) can be represented by an information algebra of sets, more particularly a generalized relational algebra in the labeled case. And how can this set algebra be characterized? This is the subject of representation theory. In lattice theory, the well-known Stone respectively Priestley duality theories solve exactly this problem, by giving precise topological characterizations of the representing set algebras for the case of Boolean algebras respectively distributive lattices (see [14]). Is it possible to extend these theories to distributive lattice or Boolean information algebras, i.e., algebras where Φ is a distributive lattice or a Boolean information algebra? These questions also remain open to a large extent so far. Similar questions have been treated for special information algebras like cylindrical algebras, see [17].

Finally, information, undoubtedly, is often uncertain. In the framework of information algebras uncertain information can be modeled as random maps with values in an information algebra, see [6,56]. These random maps form themselves again information algebras. A possible interpretation of a random map is that an information may depend on some unknown assumptions. One may then formulate hypotheses and study, given a random map (an uncertain information), under what assumptions and hence how likely it is that the hypothesis holds. This induces a kind of distribution function for the random map; it is a function monotone of order ∞. Such functions are studied in capacity theory [57,58]. Many results from capacity theory can be used to study random maps and uncertain information [6,8,59]. The theory of uncertain information as understood here is a generalization of what is know as Dempster-Shafer theory [60,61]; and in fact, information algebras provide the natural general framework for this theory. Much of this theory, especially the beautiful Theory of Allocations of Probabilities (see [62]) can be extended to information algebras. Random maps are also related the Theory of Hints [63] and finally also to Probabilistic Argumentation Systems [64]. Finally, from the point of view of measuring information content, this probabilistic approach puts also Shannon’s theory of information in a more general perspective, see [65] for a preliminary discussion. Much remains to be done here.

To conclude we only mention a few other issues, where so far little or nothing is known to the best of our knowledge: One is the introduction of topology into information algebras, somehow like it is done for domains (see [66]). Another one is the question of effective information algebras, where combination and extraction are effectively computable. This should bring information algebras into the focus of computability. The theory is yet relatively new and much remains to be done.