Entropy and Fractal Antennas

: The entropies of Shannon, Rényi and Kolmogorov are analyzed and compared together with their main properties. The entropy of some particular antennas with a pre-fractal shape, also called fractal antennas, is studied. In particular, their entropy is linked with the fractal geometrical shape and the physical performance.


Introduction
Classically, the concept of entropy arises from the analysis of physical problems in statistical physics and thermodynamics.Since the beginning, it was a measure of uncertainty in a physical system, and based on this, C.E. Shannon [1] proposed to extend this concept for the analysis of complexity in signals, thus giving rise to the emerging information theory [2].Several year later than Shannon, A. Rényi showed that a valid measure of entropy has to be defined in accordance with a measure of diversity [3].A step forward in this direction was given by A. Kolmogorov (1958), who used the concept of entropy to define a fundamental measure for chaotic evolution and loss of information in the course of time [4].
Indeed, this entropy is an extension of a known concept in information entropy (for time-dependent processes), which is used to characterize dynamical systems between regular, chaotic and purely random evolution.
Chaotic motions and, in particular, attractors can be also described by iterative maps, which belong to the fundamental methods of fractal geometry.
Chaos, complexity and fractals have many common features, and recently, they have attracted the interest of scholars for their application in science and engineering.
Fractal sets are abstract objects that cannot be physically implemented.However, some related geometries known as pre-fractals have been shown to be very useful in engineering and applied science [5,6].In particular, some fractal models have been used to design some fractal antenna with very special properties: about one-tenth of a wavelength (p.231, [7]) and a pre-fractal geometrical configuration.
An antenna is a complex device, characterized by different parameters (resonant frequency, gain, directivity, radiation pattern, etcetera), which define the performance of the radiator.The chaoticity of the fractal antenna will be studied in the following by an entropy measure based on the computation of the fractal dimension, according to the analysis of a radiating structure given by [8,9].
Since the Rényi entropy H α and generalized fractal dimension D α are connected by a well-known relation (see Equation (25)), in this paper, the quantity D α was used to compute the Rényi entropy H α of a pre-fractal structure and to describe the electromagnetic behavior of an antenna together with the corresponding performance.
The results of Best [10,11] show how antenna geometry alone (pre-fractal or otherwise) is not a significant factor to determine the performance of small antennas.Yet, this may be a good clue.
In the literature, there are only a few articles about how the self-similarity property of a pre-fractal radiator can influence its performance (see again [8,9]).
In order to investigate in this direction, the values of Rényi entropy H α through Equation ( 25) for the classical Sierpinski gasket were determined.The quantity D α was numerically estimated (see Paragraph 4.3).

Concept of Entropy
There are basically three definitions of entropy in this article.The Kolmogorov entropy K, which measures the chaoticity of a dynamical system (Chapter 6, [12]), can be estimated as the Rényi entropy H α .In information theory, the Shannon entropy is a special case of Rényi entropy for α = 1.
Definition 1 (Shannon entropy).The Shannon entropy [1,13] of a discrete-type RV X is defined as: where N is the number of possible states and p i is the probability of the event {X = x i }, and it is assumed p i > 0; the most common used values are b = 2 and b = e.
This entropy may be defined in a simple way for a continuous-type RV, as well [13].Yet, this is not the scope of this article.As is well known, Shannon entropy satisfies different properties, which will not be treated herein [14].
Moreover, it is possible to show that it represents the measure of uncertainty about a suitable partition [13].
A first generalization of this kind of entropy is the so-called Rényi entropy [3,15]: it represents one family of functionals to describe the uncertainty or randomness of a given system.Definition 2 (Rényi entropy).Let α be a positive real number.The Rényi entropy of order α is defined as [3]: where X is discrete-type RV and p i > 0 is the probability of the event {X = x i }, and the most common used values are the same as the Shannon entropy.
If the events {X = x i } are equiprobable, then H α is maximal and H α (X) = log b N for every α > 0, i.e., the so-called Hartley entropy . It is clear that they do not depend on the probability, but only on the number of events with non-zero probability.
In order to understand the meaning of the last definition, it is necessary to observe that at α = 1, the quantity: generates the indeterminate form 0 0 .By L'Hôpital's rule [17], it is easy to show that: i.e., the Shannon entropy, so lim α→1 H α (X) = H 1 (X) = H(X), as shown in Figure 1.
Therefore, the Rényi entropy may be considered as a generalization of the Shannon entropy.It can be shown that the Rényi entropies decrease as a function of α [3].Let x(t) = [x 1 (t), ..., x d (t)] be the trajectory of a dynamical system on a strange attractor.Let the d-dimensional phase space be partitioned into boxes of size l d and sampled at the discrete time intervals τ.Let p i 0 ,...,i n be the joint probability that the trajectory x(t = 0) is in the box i 0 , x(t = τ) is in the box i 1 , ..., and x(t = nτ) is in the box i n [18,19]; for example: According to Equation (1), the quantity: gives the expected amount of information needed to locate the system on a special trajectory i * 0 , ..., i * n , i.e., if it is known a priori that our system was in i * 0 , ..., i * n , then K n+1 − K n is the necessary information to predict in which box i * n+1 this system will be included.Using the language of information theory, this means that K n+1 − K n measures the loss of information for the system from n to n + 1 (Chapter 6, [12]).Therefore: The definition of this new kind of entropy will be provided at this point.
Definition 3 (Kolmogorov entropy).The Kolmogorov entropy K is defined as [19]: where l and τ have the same meaning as above.
From this definition, it can be seen immediately that it is nothing other than the average rate of loss of information.K is independent of the particular partition (thanks to the limit l → 0). Figure 2 reveals how K represents a measure of chaos: indeed (Chapter 6, [12]): for random systems C > 0 for chaotic-deterministic systems 0 for regular motion (7) where the definition of a random system can be found in Chapter 3 of [20].By describing the chaos in a dynamic system, the Kolmogorov entropy is expected to be strongly connected with the Lyapunov exponent λ; see [21].For more information about the theoretical aspects of entropy, its generalizations and entropy-like measures, which can be used to measure the complexity of a system, see [22][23][24][25][26].

Remarks on Fractal Geometry
A fractal is characterized by the property that each enlargement of this set reveals further details, so it has a structure that is too irregular to be described by a classic mathematical theory (even if a fractal can often be described recursively).Furthermore: it is self-similar, i.e., each very small portion of it is exactly or approximately similar to itself (this property has to be understood in the statistical or approximated sense, because a random element can be introduced in the construction of the fractal); 2.
it is a space-filling curve [28].

Hausdorff-Besicovitch and Box-Counting Dimensions
Among the different definitions of fractal dimensions in use, the Hausdorff-Besicovitch dimension is probably the most important, even if it is not usually used for experimental procedures to find the fractal dimensions of real objects.
Fractal dimensions are very important because they provide a measure of the degree to which new details are revealed at different scales.For example, the fractal dimension of the coastline of Great Britain is about 1.2 (Chapter 2, [29]).In order to define the Hausdorff-Besicovitch dimension, some remarks on fractal geometry are given [5,6].
Theorem 4 (See Figure 3).If n ∈ Z + and A is a bounded subset of Euclidean metric space R n , then there exists a unique number D H ∈ [0, n], such that: Proof.See Chapter 3 in [6].
Definition 5 (Hausdorff-Besicovitch dimension).Under the hypotheses of Theorem (4), the correspondent real number D H present in Equation ( 8) is called the Hausdorff-Besicovitch dimension of A, and it is generally indicated with D H (A).
From this last definition, it follows that the Hausdorff-Besicovitch dimension of a bounded subset A ∈ R n is a non-negative number D H , such that: Therefore, the Hausdorff measure of A, i.e., M s (A), might be equal to zero, infinity or such that 0 < M s (A) < ∞.In Figure 3, the plot of M s (A) is presented as a function of s, which shows us that D H (•) is the critical value of the variable s in the jump of M s (A) from ∞ to zero.
At this point, the definition of the fractal set can be provided.It is to be recalled that D T (A) ≤ D H (A), where D T (A) represents the topological dimension of the bounded subset A of R n (p. 3, [30]).The Hausdorff-Besicovitch dimension is not particularly useful in engineering or applied sciences, because its calculation is not very easy, so another definition of the fractal dimension more suitable to compute the fractal dimension for problems of mathematical modeling was introduced [6,31].
Definition 7 (Box-counting dimension).Let (X, d) be a metric space and A ∈ H(X), where H(X) denotes the space of non-empty compact subsets of X.Let N δ (A), ∀δ > 0 be the smallest number of closed balls of radius δ needed to cover A. The lower and upper box-counting dimensions of A, denoted D UB (A) and D LB (A), respectively, are defined as: When D UB (A) = D LB (A), the following limit exists and is called the box-counting dimension of A, denoted D B (A): The box-counting dimension of an object does not exactly have to be equal to the Hausdorff-Besicovitch dimension, even though they can be really close at times.This new definition of the fractal dimension is given by the minimum number of objects needed to cover the fractal set.
In Figure 4, it is shown how the box-counting dimension works to compute the length of England's coastline: looking at the first iterations, the meaning of the adjective box-counting is clear.

Iterated Function System and Pre-Fractals
It is to be recalled that a contraction on a metric space (X, d) is a transformation f : X → X, such that: where the number s, called the contractivity factor for f , belongs to (0, 1).The famous contraction mapping theorem states that every contraction f on a complete metric space (X, d) has exactly one fixed point x f ∈ X, and the sequence of iterations { f n (x)} n≥0 converges to x f , ∀x ∈ X (pp.76-77, [5]).
Clearly any contraction is continuous.If the equality holds in Equation (11), f is called a contracting similarity, because it transforms sets into geometrically similar sets.
It is now time to give the definition of an important procedure concerning fractals.
Definition 8 (Iterated function system).The iterated function system (IFS) is a couple {X, F}, where F : H(X) → H(X) is defined through a finite family of contractions {S i } m i=1 on the complete metric space (X, d), with m ≥ 2, and H(X) denotes again the space of non-empty compact subsets of X.Moreover, the set A ∈ H(X) is called the attractor (or sometimes invariant set) for the IFS if: Technically speaking, the operator F given by Equation ( 12) is called the Hutchinson operator associated with the IFS {S 1 , S 2 ,..., S m } [5,32].From the definition above, it is clear that the attractor for the IFS is also its unique fixed point.This is the fundamental property of an IFS, because this attractor is often a fractal.An IFS has a unique (non-empty compact) attractor (Chapter 9, [6]), but its introduction brings with it two main problems: the first one shows the way to represent a given set as the attractor of some IFS, while the second is to reconstruct the IFS starting from its attractor (p.126, [6]).
Both of these two problems can often be solved by inspection, especially if F has a self-similar structure (see Figure 5).For the majority of the fractals suitable for an application in antenna theory, the thesis of the Moran-Hutchinson theorem (pp.130-132, [6]) holds true, so: where A is the attractor of the IFS with contraction factors c 1 , ..., c m and s = D H (A) = D B (A).
This theorem provides us the possibility to compute the fractal dimension of many self-similarity fractals.Indeed, let us consider the von Koch curve and the middle third Cantor set (see Figure 5): for the first one, it is: while for the other set, we get: IFS can be applied to all self-similarity structures, especially for the simulation of real objects with fractal properties, like fractal antennas.
It is well known that fractals are only mathematical abstractions (because it is impossible to iterate indefinitely in the real word).In addition, numerical simulations show how the fractal modeling in antenna theory provides substantial advantages within a certain value of the iteration m * (typically, for fractal antennas, it is not greater than six).Beyond this value, the benefits are negligible.It is clear that all self-similar structures in nature are nothing other than fractals arrested at a prefixed iteration, i.e., pre-fractals (geometrical objects characterized by a finite number of fractal iterations).

Fractal Antennas
In order to minimize the antenna size holding a high radiation efficiency, a fractal approach to model its geometrical configuration can be considered.
The two fundamental properties of a fractal (i.e., self-similarity and space-filling) allow fractal antennas to have an efficient miniaturization and multiband characteristics.
The well-known log-periodic antennas, introduced by DuHamel and Isbell around the 1950s and closely paralleling the independent-frequency concept [7], might be considered the first fractal antenna of history.Another example of a self-similar antenna discovered in the same period is the spiral antenna (see Figure 6).However, their true origin may be traced back to 1988, when Nathan L. Cohen, a Boston University radio astronomer, published a paper about this new type of antenna [33].
Fractal antennas have not only a large effective length, but also a simple matching circuit, thanks to the contours of their geometrical shape, which is able to generate a capacity or an inductance.For instance, a quarter-wavelength monopole may be transformed into a smaller antenna using the von Koch curve (see Figure 5).
A big part of the research on fractal antennas has been done by Fractal Antenna Systems Inc., an American company founded by Cohen.
Carles Puente Baliarda (Polytechnic University of Catalonia) was the first to treat these antennas as multiband antennas.In 1998, he won the award of "innovative IT products with evident market potential" due to his pioneering research in fractal antennas (for a total of e 200,000), while he and his company (Fractus S.A.) were the finalists for the European Inventor Award 2014, showing the great potentials of these antennas.
In 2011, 9.7 billion fractal-based antenna units were supplied worldwide (a report by BCC Research).

Sierpinski Gasket and Hilbert Antenna
The Sierpinski triangle T can be constructed from an equilateral triangle by the removal of inverted equilateral triangles (see Figure 7).It is a fractal and attractive fixed set.Considering Figure 7, it is: since all of the contraction factors c 1 , c 2 , c 3 are equal to 1 2 .Therefore: This fractal may be also generated by an IFS (Chapter 9, [6]).There exist different versions of the Sierpinski triangle.The shape can be modified in many ways, and they are often used in engineering and applied sciences.
The Sierpinski (gasket) antenna belongs to the class of multiband fractal antennas based on the Sierpinski triangle.The classical Sierpinski dipole is shown in Figure 7.It is probably a fractal antenna with more applications, from wireless communication systems (GSM, UMTS and WLAN) through RF MEMS (radio frequency microelectromechanical system) to get to space probe design and ANN (artificial neural network) theory [34,35].
The famous Hilbert curve is a continuous fractal space-filling curve, i.e., it fills the plane without leaving any gaps.Hilbert introduced it as a modification of the Peano curve [28].
There are important differences between these two curves.Indeed, it is not possible to construct the Hilbert curve H through the IFS (while for the other one, this procedure is applicable).The reason is that the steps in the Hilbert curve's construction are not self-similar, i.e., they are not divisible in a number of parts similar to the initial figure.
The original construction of the Hilbert curve is extraordinarily elegant: it starts with a square A 0 , while in the first step (see Figure 7), the curve A 1 connects the centers of the quadrants by three line segments (having a size of one).In the second step, four copies (reduced by 1/2) of this initial stage are made and placed into the quarters (see again Figure 7).In this way, the first copy is clockwise rotated and the last one counter-clockwise rotated by 90 degrees.After this, the start and end points of these four curves are connected using three line segments (of a size of 1/2), and we call the resulting curve A 2 .
In the third iteration, the scaling is done again by 1/2, and four copies are placed into the quadrants of the square (as in the first step).They are again connected by three line segments (of a size of 1/4) obtaining A 3 , and so on.
In Figure 7, it can be noticed that each successive stage consists of four copies of the previous one, connected with additional line segments.Therefore, the curve is scaled down by the ratio 1 2 , and four copies are made; so: hence: Naturally, the topological dimension of H is one, since it consists only of line segments.Therefore, the Hilbert curve is a fractal for all intents and purposes.An alternative procedure to IFS is that of the so-called L-systems [36].In Figure 7, a Hilbert dipole is also shown, where the feed source point is placed at the point of symmetry for these two pre-fractals.
The Hilbert antenna is especially used in spatial communications, like RF MEMS design [37] and, generally speaking, in each (telecommunication) system where the space available for the antenna is limited [38].

The Results of Best and HRC Conditions
A fractal approach is not the only way to miniaturize an antenna; indeed, there exist few particular non-fractal modifications of the classical von Koch dipole that could have the same performance [10].
In addition, it is clear that fractal geometry does not uniquely translate the electromagnetic behavior of the antenna.The geometrical configuration alone (fractal or non-fractal) could not be the only significant factor that determines the resonant behavior of wire antennas: indeed, a fractal configuration does not represent alone a guarantee of the highest antenna efficiency [11].
The same applies to the loop antennas.It is well known that the main advantage of the fractal loop antennas is that they have a high radiation resistance on a "small" physical area.In Figure 8 (top side), three examples of non-fractal antennas are shown.They offer similar or, in some cases, improved performance over their fractal-antenna counterparts, like the Minkowski antenna.The reason is that the radiation resistance of an electrically-small loop, given by [10]: where λ is the working wavelength, is generally not valid for a loop antenna with complex geometry.However, there are few small non-fractal loop antennas with similar or better performance than their fractal counterparts (see Figure 8, top side).In order to investigate the significance of self-similarity in determining the multiband behavior of the fractal antennas, Steven R. Best has presented a comparison of the multiband behavior of the Sierpinski gasket and several modified gaskets where the major portions of the self-similar structure were modified or eliminated [11].
His numerical simulations reveal that many of the self-similar fractal gap structures can be eliminated from the Sierpinski gasket, without modifying its multiband behavior.
Best showed how the total self-similar fractal gap structure is not the primary factor that determines the multiband behavior, because the Sierpinski gasket and modified Parany gasket antenna have the same behavior [11].Therefore, for all of the Sierpinski-based antennas, the multiband behavior depends on the small isosceles trapezia located in the center of the modified Parany gasket antenna, as shown in Figure 8 (bottom side).
It would seem that some non-fractal geometries could be a good substitute for their fractal counterparts, but this is manifestly untrue.
Indeed, the results obtained by Best represent only a very few special cases, and these antennas do not belong to a family of radiators.Furthermore, the so-called HCR conditions can be considered [39] (they provide a necessary and sufficient condition for all frequency independent antennas).This criterion reveals that an antenna satisfies this property if and only if the radiating structure is both self-similar and origin symmetric about a point.It is clear that some non-fractal radiators might belong to this second one, potentially giving them the same performance of a fractal antenna that is non-symmetric about a point.

The Entropy of a Fractal Antenna
In addition to the box-counting dimension, another convenient way to estimate the fractal dimension is the so-called generalized fractal dimension (or Rényi dimension) D α , given by [40]: In this definition, N = N(δ) is the total number of boxes with p i > 0, where also here, the most commonly-used values are b = 2 and b = e.
Considering the definition of Rényi entropy (7), it is clear that: As α → 0, which is nothing but the fractal dimension.The numerator of the last equation is simply the Hartley entropy.It can be shown similarly, as for the definition of Rényi entropy, that lim α→1 D α (X) = D 1 (X) Therefore, the dimension D α is a generalization of D 1 , which is called the information dimension.Indeed, D 1 characterizes the information required for the determination of the point location in some cell i.
According to Equation (21), it also follows that: This quantity is called the correlation dimension, because it is very useful to detect chaotic behavior.
Therefore, the generalized fractal dimension D α provides a direct measurement of the fractal properties of an object.Several values of the momentum order α correspond to well-known generalized dimensions.
Equation ( 22) cannot be applied practically, and it is only possible to get an approximation fixing a small value of δ, but strictly greater than zero.Therefore, in applied sciences and engineering, Equation (22) becomes: where H α = H α (δ).
This equation shows us that the entropy of a region of size δ is a function of the box-counting fractal dimension = 1/δ: the entropies of analyzed regions (with size δ) can be calculated from the three spatial dimensions through Equation ( 25) [16].
Right now, the Rényi entropy has to be computed for the geometric configuration of each fractal antenna.It is easy to create an algorithm for its computation using Equation (25).This procedure consists of the classical algorithm for numerical estimating D α of affine RIFS-invariant measures; see [41,42].The Rényi entropy will be computed through Equation ( 25), considering the logarithm of the cell size (see Figure 9 below).With this procedure, it is possible to compute the entropy for a fractal radiator, but it must be completely modified for each class of fractal antennas.
However, the general definition of entropy for a small antenna, in order to better understand how the chaoticness of the structure may affect its performance, remains an open problem.

Figure 1 .
Figure 1.Rényi and Shannon entropies for a binomial distribution with N = 20: they converge for α → 1, in accord with Equation (3).The computation of both entropies was done for b = 2.

Figure 3 .
Figure 3. Graph of the M s (A), where A is a bounded subset of the Euclidean metric space R n .It takes only two possible values, and the Hausdorff-Besicovitch dimension of A is given by the value of s in which there is the jump from ∞ to zero.

Figure 4 .
Figure 4. Here, the first steps of the box-counting procedure about England's coastline are represented.

Figure 5 .
Figure 5. Here, the von Kock curve (on the left) and the middle third Cantor set (on the right) are shown: A 0 is the initiator of length equal to one in both cases; in the generator A 1 for the von Kock curve, the middle third of the unit interval is replaced by the other two sides of an equilateral triangle, while that of the middle third Cantor set is obtained removing the middle third of the interval.

Figure 6 .
Figure 6.Archimedean spiral antenna (on the left) and commercial log-periodic dipole antenna of 16 elements (on the right) [7].

Figure 7 .
Figure 7.A Sierpinski triangle (on the left) and a Hilbert curve (on the right) are shown: as in Figure5, A 0 is the initiator, and A 1 is the generator.The Sierpinski triangle is constructed using the iterated function system (IFS), while the other construction is that of David Hilbert.The two relative antennas are shown below with their feed points.

Figure 8 .
Figure 8. Examples of non-fractal antennas that offer similar performance over their fractal counterparts.Three different non-fractal antennas are presented above: they outperform their fractal counterparts, while the current distribution on the Sierpinski gasket antenna at the first three resonance frequencies is shown in the middle of the page.The modified Parany antenna (starting from the classical Sierpinski gasket antenna) is represented below.

Figure 9 .
Figure 9.The Rényi entropy H α of a Sierpinski gasket (Figure7) with α = 0, 1, 2: the plot shows us that Hartley entropy H 0 is an upper to both Shannon entropy H 1 and collision entropy H 2 .The main limit of this procedure is clearly the precision of the triangulation.