Asymptotic Behavior of the Maximum Entropy Routing in Computer Networks

Abstract

Maximum entropy method has been successfully used for underdetermined systems. Network design problem, with routing and topology subproblems, is an underdetermined system and a good candidate for maximum entropy method application. Wireless ad-hoc networks with rapidly changing topology and link quality, where the speed of recalculation is of crucial importance, have been recently successfully investigated by maximum entropy method application. In this paper we prove a theorem that establishes asymptotic properties of the maximum entropy routing solution. This result, besides being theoretically interesting, can be used to direct initial approximation for iterative optimization algorithms and to speed up their convergence.

1. Introduction

1.1. Maximum Entropy Method

The maximum entropy method (MEM) was recently used with great success in many different areas where underdetermined systems are involved. It is most frequently used in crystallography [1], chemistry [2] and physics [3,4], but also in many other very diverse areas: natural language processing [5], transportation [6], character recognition [7], image processing [8,9], economy [10]. Theoretical developments also continue [11,12,13].

The basic idea of the maximum entropy method is to get a unique solution from the underdetermined system by introducing the additional constraint that the entropy function should be maximized. The other methods that were used for solving underdetermined systems use the same technique: they introduce additional, artificial constraints that make the number of constraints equal to the number of unknowns. The difference is that the maximum entropy method introduces the most natural additional constraint: one that does not introduce any new, arbitrary and unwarranted information. It uses only the information that is given and makes no assumptions about missing information.

It is interesting to mention that besides very pragmatic uses (like one proposed in this paper, network design) there were very extensive philosophical discussions about the real meaning of the maximum entropy principle. The predecessor of the maximum entropy principle is the principle of insufficient reason (James Bernoulli: “Ars Conjectandi,” 1713). It states that in the absence of any information (knowledge), all outcomes should be considered equally possible. This principle of insufficient reason was involved in the discussions about prior probabilities (probabilities of one event, state of the knowledge) and relative frequencies. Relative frequencies become predominant and some useful works from Laplace and Bayes were criticized. Shannon’s works on information theory opened a new opportunity for revitalization of the principle of insufficient reason, this time as a more sophisticated maximum entropy principle that was introduced by Jaynes [14,15]. These philosophical discussions about the real meaning of the maximum entropy method are interesting, but since the method was successfully applied in many areas, for any new area of application the most important criterion is not how well can we explain the relation between the MEM and that area, but how useful are the results we get by applying the method.

1.2. Definition of the MEM

The formal definition of the maximum entropy method is:

Suppose that for a discrete random variable X the values $x_1,x_2,...,x_n$ that it can take are known, but the corresponding probabilities $p_1,p_2,...,p_n$ are not known. Also, the expected values for $k<n-1$ functions of X (for example, the first k moments) are known:

$$E\left[f_r\left(X\right)\right]=m_{r}r=1,2,...,k$$

In fact, we do not need to know the values $x_1,x_2,...,x_n$, or analytical expressions for functions $f_r,r=1,2,...,k$. It is sufficient to know the values $f_r\left(X_i\right);r=1,2,...,k;i=1,2,...,n$. Also, we do not have to start with probabilities $p_1,p_2,...,p_n$. We can start with any set of numbers $t_1,t_2,...,t_n$. Then we introduce

$$p_i=\frac{t_i}{\sum t_j}$$

This gives us (together with $\sum p_i=1$ ) $k+1<n$ constraints for n unknown variables $p_1,p_2,...,p_n$. This system is underdetermined and has an infinite number of solutions. We want to find the unique solution that maximizes the entropy of the system. That is the best solution, in the sense that it uses only the information given. It is neutral to the missing information (it does not introduce any hidden assumptions). This additional constraint can be expressed as:

Maximize the entropy function

$$H(p_1,p_2,...,p_n)=-K\sum _{i=1}^n{p}_{i}ln\left(p_i\right)$$

If $K=1$ is selected, entropy will be expressed in natural units (rather than in bits).

This system can be solved by method of Lagrange multipliers and there is a standard algorithm to solve this system. However, the function that is to be minimized is not convex even in the simplest case when there is only one constraint: expected value. The standard Newton-Raphson procedure does not work, but the Jacobian matrix for the system is symmetric and positive definite. This gives a scalar potential function which is strictly convex and its minimum is easy to find. The use of the second order Taylor expansion is recommended, as well as heuristics that avoid inverting the Jacobian matrix on each iteration [16].

1.3. Network Design Problem

The network design problem (NDP) is an old well known NP-hard problem. It is a very interesting problem because it has great practical value and since it is intractable, heuristics and suboptimal solutions have been used for decades. It is an open problem and since unique best solution cannot be found, every new approach, including MEM, is promising in the sense that solution obtained can be better then previous ones, at least in some cases. This problem is intractable if full and exact solution is required. Networks can have many hundreds of nodes (computers). Fortunately, experience has shown that network design can be done hierarchically and still be near optimal. An example is a network for a country. First, it can be decided where to put trunks between major cities, then connect small cities to nearest major cities, then make local networks inside the cities. This approach allows to work with networks of at most 50 nodes at a time. This is a great help, but the problem is still intractable.

Computer networks consist of computers, called nodes, and communication lines, called links, that interconnect them. All data that is exchanged among nodes is divided into packets. Destination address is added to messages and these packets are sent to neighboring computers that send to their neighbors and so on, until the message reaches its destination.

The network design problem is:

• For given locations of nodes, traffic matrix (offered traffic for each pair of nodes) and cost matrix (cost to transfer a message for each pair of nodes)
• With performance constraints: reliability, delay (time that a message spend in the network), throughput
• Find values for variables: topology (which nodes will be connected directly with a line and which will have to communicate indirectly, using other nodes as intermediate stations), line capacities (how much traffic will each link be able to carry), flow assignment - routing (which paths messages between any pair of nodes will follow)
• Minimize the cost (of building and maintaining the whole network).

Other formulations of the problem are: minimize delay for the given cost or maximize throughput for given cost and delay. It has been shown that all these problems are similar and that the same techniques can be applied.

The network design problem, that was for many decades investigated with emphasis on wide area networks, has been recently revitalized with application to wireless ad-hoc networks and subclasses mobile ad-hoc networks (MANET) [17,18], wireless mesh networks (WMN) and wireless sensor networks (WSN) [19,20].

1.4. MEM for NDP

The problem of network design is to find a topology, routing and capacity assignment such that cost or delay is minimized. Once the topology and the routing are decided, there are exact methods for capacity assignment that will minimize delay or cost. However, there are few theoretical results on how to select topology and routing. Most of the algorithms that are used are heuristic and many of them do not even have intuitive justification other then “easy to calculate” or “only simple thing we can do.” Here is presented an attempt to use the maximum entropy method to select (initial) topology and routing.

Network design and analysis almost always involve underdetermined systems, especially when routing policy has to be determined. The number of possible routings grows with the factorial of the number of the nodes in the network and the number of possible topologies is exponential in the number of links. The number of constraints (such as “everything that goes in must go out” for each node that is neither source nor sink) is typically polynomial in the number of nodes in the network. That makes the network design problem a good candidate for the maximum entropy method application.

To solve the NDP, most methods that are currently used introduce new, artificial constraints. These artificial constraints do not have any justification other than that they make the number of unknowns and the number of constraints equal. The maximum entropy method has the nice property that it solves underdetermined systems without introducing any new, unwarranted information. The other advantage of the maximum entropy method is that it makes things as equal as possible. It is intuitively appealing that a network should not have overloaded or underutilized links i.e., traffic should be distributed as equally as possible along all lines. The same goal can be attained by using some other function that has maximum when all variables are equal. One very simple such function is the product of all variables. The product function expression seems simpler then entropy function expression which involves logarithms, but when the fact that partial derivatives are needed is considered, we see that entropy function is better since it separates variables.

It is possible to apply the MEM if analysis is started with totally interconnected network of n nodes. Some lines will be dropped later in the process of improving utilization or reducing the cost. Once the topology becomes sparse enough, other exact routing optimization methods can be introduced. To apply the maximum entropy method it has to be determined what will be the variables of the system. Some combination of required traffic values can be used for that [21,22] since for MEM application it is not necessary to start with probabilities, but an arbitrary set of numbers which can be normalized (by dividing each of these numbers by their sum).

The network design problem, primarily the routing feasibility, has been adjusted to the maximum entropy method requirements [16]. Computationally feasible algorithm was developed which implements the standard maximum entropy method, includes adjustments for problems that do not involve probabilities initially, calculates a function that substitutes large sparse matrix, includes heuristic that speeds up calculations by avoiding to invert Jacobian matrix at each iteration, determines variables that define constraints for the routing feasibility, includes additional constraints that direct uniformity of the solution in the desirable direction [23], cancels opposing traffic and excludes underutilized links. Mentioned additional constraints are “soft", which is a unique feature of this algorithm, in the sense that they do not have to be satisfied; the solution will be pulled in the direction of satisfying them as much as possible.

Proposed algorithm computes a reasonable solution that is robust with respect to often required dynamic changes of the cost function. The maximum entropy solution can be a good starting point for further optimization considering that the cost function with delay penalties involves queuing theory that is usually computationally expensive. Alternatively, guided version of the MEM can be used dynamically.

To improve mentioned algorithm some theoretical results can help direct initial approximation.

2. Asymptotic Behavior of the MEM Routing

The algorithm explained in the previous section uses maximum entropy method as a basis but it starts with a totally connected network and iteratively eliminates lines to decrease the objective function (combination of cost and delay). Since this is computationally expensive algorithm, it helps to always select better starting point to reduce convergence time. Here we prove a theorem that establishes maximum entropy routing behavior in a network.

Theorem 1.

For a totally interconnected network of n nodes and given offered load $L_{ij};i,j=1,n;i\ne j;$ (required traffic from node i to node j) the maximum entropy solution will asymptotically (when total traffic on the network approaches infinity) distribute net traffic (traffic that remains after the traffic for same message types between same nodes in opposing directions is canceled) along direct paths and paths of length two in ratio 2:1. That means that offered load $L_{ij}$ for any pair of nodes i and j will be send as follows: $\frac{2}{n}{L}_{ij}$ will be sent along direct path $[i,j]$ and $\frac{1}{n}{L}_{ij}$ will be sent along each of $n-2$ paths of length two: $[i,k]-[k,j],k=1,n;k\ne i,j$ (all paths of length two from node i to node j through any intermediary node k).

To prove the theorem we need three lemmas:

Lemma 1.

Consider a system with n probabilities and two independent sets of constraints where the first set applies to $p_1,...,p_k$ and the second set of constraints apply to $p_{k+1},...,p_n,1\le k<n$. If the sums for the two groups of probabilities are kept constant

$$\sum _{i=1}^k{p}_i=c\sum _{i=k+1}^n{p}_i=1-c$$

the maximization of the entropy for the system can be done as two separate maximizations, one for each group of probabilities. The proof is trivial [24].

Lemma 2.

To maximize the entropy for totally interconnected network of n nodes where offered load is $L_{ij}$ (the only traffic on the network is from node i to node j) and total network traffic is kept constant, we only need to consider paths of length 1 and 2 (direct path $[i,j]$ and $n-2$ paths of length two $[i,k]-[k,j];k=1,n;k\ne i,j$. All other lines will not carry any net traffic (opposing traffics will be equal).

Proof.

For totally interconnected network of n nodes there are $n-1$ lines going from node i and $n-1$ lines going to node j. All the traffic going from node i and all the traffic coming into node j has to go along these lines. These lines contain all paths from node i to node j of length one and two. We know that maximization of entropy favors equal traffic everywhere. In order to push traffic $L_{ij}$ from node i to node j we have to disturb this equality by adding some traffic on lines going from node i and subtracting some traffic on lines going to node i. Similar is true for lines adjacent to node j. We want to make this disturbance as small as possible and we distribute offered load along all possible lines (lines adjacent to node i) and route it along lines adjacent to node j. Lemma 3 will prove that we need to do this and show how to do this. What we are proving here is that we do not need any lines of length greater then two. Indeed, regardless of what paths we will use later, we have to disturb lines going from node i for the amount of offered load $L_{ij}$ (the offered load must be sent from node i). Also, regardless of paths that are used, we have to disturb lines going to node j for the amount of offered load $L_{ij}$ (the offered load has to come to node j). By introducing the new paths (of length greater then two), we can not decrease the amount of disturbance that we already have on lines that are adjacent to nodes i and j. All we can do is to disturb some other lines that were undisturbed (opposing traffics were equal) and separating two probabilities that were equal (sum has to remain unchanged) will only decrease the entropy.

Lemma 3.

For a totally connected network of n nodes, offered load $L_{ij}$ and total traffic on the network T, the entropy is maximized if traffic is routed along paths of the length 2 and 1 in the following quantities:

$$\begin{array}{cc}\hfill e_2& ={[R+\sqrt{Q}]}^{\frac{1}{3}}+{[R-\sqrt{Q}]}^{\frac{1}{3}}+\frac{d}{3(n-2)}\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill e_1& =d-(n-2)e_2\hfill \end{array}$$

(2)

where

$$\begin{array}{cc}\hfill R& =\frac{d^3+9(n-2)(n-3)d{p}^2}{27{(n-2)}^3}\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill Q& =\frac{p^2{d}^4+(2n^2-18n+27)p^4{d}^2+n^3(n-2)p^6}{27{(n-2)}^4}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill d& =\frac{L_{ij}}{2T}p=\frac{1}{n(n-1)}\hfill \end{array}$$

(5)

Proof.

From Lemma 2 we know that we only need to consider direct path and paths of length two. Since all paths of length two are equivalent there cannot be any reason to treat any of them differently: all paths of length two will carry the same amount of traffic. The only thing that we have to decide is how much of the traffic to send along direct path and how much along paths of length two.

Totally interconnected network of n nodes has $n(n-1)$ lines (two between any two nodes). Let us first satisfy the condition that total traffic be equal to T. Put on all lines equal amount of traffic, that means each line carries traffic $\frac{T}{n(n-1)}$. When we normalize traffic to probabilities, each probability will be $p=\frac{1}{n(n-1)}$. This system has maximum entropy, but there is no net traffic on this network: all traffic cancels and there is no flow from node i to node j. To push some net flow on line $[k,l]$ we have to increase probability $p_{kl}$ and decrease, by the same amount, the probability $p_{lk}$. The new probabilities will be $p_{kl}=p+e$ and $p_{lk}=p-e$. The net flow on line $[k,l]$ is now $2eT$. We want to push traffic $L_{ij}$ from node i to node j. Suppose that traffic $T_1$ will go along direct path, and traffic $T_2$ will go along each path of length two. Then

$$L_{ij}=T_1+(n-2)T_2$$

(6)

Corresponding changes in probabilities, $e_1$ and $e_2$, can be easily calculated:

$$e_1=\frac{T_1}{2T}{e}_2=\frac{T_2}{2T}$$

(7)

We have one direct path that now has probabilities $p+e_1$ and $p-e_1$ and $n-2$ paths of length two, i.e., $2(n-2)$ lines, each with probabilities $p+e_2$ and $p-e_2$. From the previous two equations we get

$$e_1=d-(n-2)e_{2}whered=\frac{L_{ij}}{2T}$$

(8)

The entropy function is now:

$$\begin{array}{c}\hfill H\left(e_2\right)=-[p+d-(n-2)e_2]ln[p+d-(n-2)e_2]-\\ \hfill -[p-d+(n-2)e_2]ln[p-d+(n-2)e_2]-\\ \hfill -2(n-2)\left[(p+e_2)ln(p+e_2)+(p-e_2)ln(p-e_2)\right]\\ \hfill -(n-1)(n-2)pln\left(p\right)\end{array}$$

(9)

The last term is for other lines that are not on any path of length one or two. This lines do not carry any net flow, do not depend on $e_1$ or $e_2$ and are not considered during optimization (the term vanishes when derivative is taken).

We want to maximize entropy, so we take derivative and equate it to zero:

$$\frac{dH}{d{e}_2}={(p+e_2)}^2[p-d+(n-2)e_2]-{(p-e_2)}^2[p+d-(n-2)e_2]=0$$

(10)

This gives third degree equation

$$e_2^3-\frac{d}{n-2}{e}_2^2+\frac{n{p}^2}{n-2}{e}_2-\frac{d{p}^2}{n-2}=0$$

(11)

To use Cardano’s formulas, in this equation of the form

$$e_2^3+a{e}_2^2+b{e}_2+c=0$$

(12)

we put

$$e_2=t-\frac{a}{3}$$

(13)

and get equation

$$t^3+pt+q=0$$

(14)

where

$$\begin{array}{cc}\hfill p& =\frac{-d^2+3(n-2)n{p}^2}{3{(n-2)}^2}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill q& =\frac{-2[d^3+9(n-2)(n-3)d{p}^2]}{27{(n-2)}^3}\hfill \end{array}$$

(16)

The solution for t will be

$$t=A+B$$

(17)

where

$$\begin{array}{cc}\hfill A& ={[-\frac{q}{2}+\sqrt{Q}]}^{\frac{1}{3}}\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill B& ={[-\frac{q}{2}-\sqrt{Q}]}^{\frac{1}{3}}\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill Q& =\frac{p^2{d}^4+(2n^2-18n+27)p^4{d}^2+n^3(n-2)p^6}{27{(n-2)}^4}\hfill \end{array}$$

(20)

It is easy to see that for $n>=8Q$ is always greater then zero. The solution we were looking for is:

$$\begin{array}{cc}\hfill e_2& ={[R+\sqrt{Q}]}^{\frac{1}{3}}+{[R-\sqrt{Q}]}^{\frac{1}{3}}+\frac{d}{3(n-2)}\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill e_1& =d-(n-2)e_2\hfill \end{array}$$

(22)

where

$$\begin{array}{cc}\hfill R& =\frac{d^3+9(n-2)(n-3)d{p}^2}{27{(n-2)}^3}\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill Q& =\frac{p^2{d}^4+(2n^2-18n+27)p^4{d}^2+n^3(n-2)p^6}{27{(n-2)}^4}\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill d& =\frac{L_{ij}}{2T}p=\frac{1}{n(n-1)}\hfill \end{array}$$

(25)

Proof of the Theorem 1.

To prove the theorem we only need to investigate asymptotic behavior for the solutions from Lemma 3. When $T->\infty ,d->0$. In that case

$$\begin{array}{cc}\hfill R& =\frac{(n-3)p^2}{3{(n-2)}^2}d+o\left(d\right)\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill Q& =\frac{n^3{p}^6}{27{(n-2)}^3}+o\left(d\right)\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill A& =\frac{\sqrt{n}p}{\sqrt{3}(n-2)}+\frac{n-3}{3n(n-2)}d+o\left(d\right)\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill B& =-\frac{\sqrt{n}p}{\sqrt{3}(n-2)}+\frac{n-3}{3n(n-2)}d+o\left(d\right)\hfill \end{array}$$

(29)

From these equation we get:

$$\begin{array}{cc}\hfill e_2& =\frac{2(n-3)}{3n(n-2)}d+\frac{1}{3(n-2)}d+o\left(d\right)\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill e_1& =d-(n-2)e_2\hfill \end{array}$$

(31)

or

$$\begin{array}{cc}\hfill e_2& =\frac{d}{n}+o\left(d\right)\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill e_1& =2\frac{d}{n}+o\left(d\right)\hfill \end{array}$$

(33)

From here we see that direct path should carry two times more traffic than each of paths of length two in order that entropy is maximized.

3. Conclusions

Maximum entropy method can be applied to the network design problem since it is an underdetermined problem. The algorithm that, by using the maximum entropy method, iteratively improves network topology and objective function can be enhanced by knowledge of the theoretical behavior of the maximum entropy routing. A theorem is proven which establishes that for a totally interconnected network and given offered load the maximum entropy solution will asymptotically (when the total traffic on the network approaches infinity) distribute net traffic (traffic that remains after the traffic for same message types between same nodes in opposing directions is canceled) along direct paths and paths of length two in ratio 2:1. This is an interesting theoretical result and it can also be used for selecting better initial solutions to improve convergence speed.