Geometry-Based Bounds on the Capacity of Peak-Limited and Band-Limited Signals over the Additive White Gaussian Noise Channel at a High SNR

Abstract

We present a new computable geometry-based upper bound on the capacity of peak-power-limited and band-limited signal over the Additive White Gaussian Noise Channel. The peak limit applies at continuous time. The bound is a function of the volume and shape of the transmitted signal set, namely the body, in the space of Nyquist-rate samples, comprising all of the points the transmitted signal can reach. At a high SNR, the bound is tight, better than previously known upper bounds and, together with a known lower bound, provides the capacity at an asymptotically high SNR. We found, using a numerical evaluation, the high-SNR capacity of signals with the structure used in Cyclic Prefix assisted Frequency Domain Equalization (CP-FDE) and OFDM for sequence length of up to 100 Nyquist intervals, and we present a conjecture that this result is correct for any sequence length and does not depend on the CPA-FDE structure. This paper extends the methodology developed in previous works. The penalty in power efficiency at a high SNR due to the peak power constraint relative to an average power constraint is about 7.5 dB in the low-pass case and about 5.4 dB in the band-pass case.

1. Introduction

We investigate the Peak-Power Limited and Band Limited (PPBL) Additive White Gaussian Noise (AWGN) channel in which the signal is band-limited and its instantaneous power cannot exceed the power P. Equivalently, the instantaneous amplitude cannot exceed $\sqrt{P}$. The peak power (peak amplitude) limit applies continuously, not only at the Nyquist-rate sampling times. This model is relevant to many systems in which the peak power is limited by the power amplifier at the transmitter. The model fits real system requirements even better since the introduction of Digital Pre-Distortion (DPD), e.g., [1,2], which linearizes the power amplifier up to its maximal transmit power, thus ensuring high and predictable performance with peak-power-limited signals. Clearly, the capacity limits of this channel are of major practical interest, e.g., the discussion of advances in new physical layer technologies in [3], the optimization in Section 3.6 of [2], and discussion in [4]. The importance of limiting the peak power is also reflected in many works analyzing and reducing the Peak to Average Power Ratio (PAPR), e.g., [5,6,7,8]. The impact of the peak power limit is classical in many communications settings since the beginning of the wireless communication era [9], and it is relevant to a variety of practical communication models, as, for example, fading channels and the like [4], and the optimization of constellations under the peak and average limits over the discrete-time channel [10]. The new results on the PPBL channel capacity may benefit recent and modern research on coding for this channel, e.g., [11], which investigates PAPR reduction for single carrier communication by transmit filter impulse-response optimization and [12], which induces a peak limit on 16-QAM Single-Carrier modulation by trellis shaping and exploits the shaping trellis as a component code of a turbo-coded scheme.

With the exact capacity of the classical Average-Power-Limited and Band-Limited (APBL) channel found by Shannon [9] and used widely for decades, the continuous-time PPBL channel capacity has only been studied sparsely, yielding lower and upper bounds on capacity with a wide gap in between. We believe the reason for this is the difficulty of analyzing this channel, as suggested already in [9]. Shannon analyzed this channel and presented lower and upper bounds on capacity in the low-pass case already in [9]. His lower bound was based on an achievability scheme with a low-pass signaling impulse response; his upper bound, valid at an asymptoticly high SNR, releases the peak limit to be applicable to Nyquist-rate samples only and then upper-bounds the entropy of the transmitted signals. The work by [13] used an improved impulse response to slightly improve the lower bound of [9] and provided an improved upper bound valid at all SNRs by releasing the peak limit to be applicable to Nyquist-rate samples only while applying the capacities of the corresponding sampled discrete-time channels available in [14,15]. It also extended the bounds to the more practical band-pass case. The capacity of the discrete-time peak-limited channel and analytical bounds on it are further investigated in [16,17]. Smith [14] derived the exact capacity-achieving distribution of signals over the discrete-time peak-limited channel and proved that it comprises discrete amplitudes. The capacity of this channel upper-bounds that of our continuously peak-limited channel, and both channels are peak-limited; still, the question whether the optimal signal in our case is similar to that in [14] remains unanswered. Recently, we presented a computable lower bound [18] on the capacity of the time-continuous PPBL channel at any Signal to Noise Ratio (SNR) applicable to the low-pass and the band-pass cases. The bound there was evaluated numerically over the Cyclic Prefix-assisted Frequency Domain Equalization (CP-FDE) or Orthogonal Frequency Division Multiplexing (OFDM) signaling. The CP-FDE signals are not strictly band-limited because they are limited in time; however, they are practically band-limited in the sense of zero inter-channel interference between users if the rules for cyclic prefix are adhered to, thus enabling the assignment of adjacent users to channels with no frequency gaps in between. This is applied, for example, in the multiuser uplink of the Long-Term Evolution (LTE) mobile communications system using Single-Carrier FDMA (SC-FDMA) [19].

The problem investigated here is related to communication over the Constrained Gaussian Channel (CGC) [20] in which a wideband peak-limited signal is fed into a transmit filter in the transmitter. As shown in [18], the capacity of the CGC is an upper bound of the capacity of the PPBL channel. The work by [20] uses the results of [21] on the Power Spectral Density (PSD) of unit processes, which are the inputs to the CGC channel, to derive the upper-bound capacity of the CGC channel, which is also valid for the PPBL channel. In [22,23], the approach is specified to the PPBL channel, gaining additional insights. The upper bound on the CCG channel was tightened in [23]. However, the CGC-based approach cannot provide an upper bound on capacity tight at high SNRs, as explained in Section 3.3.2 below. The review by [24] presents and categorizes a wide range of modulation schemes with different types of peak limits, including the CGC and the PPBL models.

Contributions: We derive a computable upper bound on the capacity of peak-limited CP-FDE and OFDM-type signals over the AWGN band-limited channel based on the volume and the shape of the signal set in the space of Nyquist-rate samples, comprising all of the points the transmitted signal can reach. This technique extends the geometric analysis of the transmitted signal set in [18] to analyze the geometric region into which the additive noise extends the transmit signal set to become the received signal set. The new upper bound is tighter at high SNRs than the known upper bounds. The existence of such a bound was mentioned in [18]; here, the bound is proved and its gap to the known lower bound is evaluated. The new upper bound approaches the lower bound at an asymptotically high SNR, thus providing capacity at an asymptotically high SNR. We also verify the lower bound in [18] by evaluating it using a more straightforward numerical method which enables us to control the standard deviation of the results and to present confidence intervals, unlike the more powerful but more complex method used in [18].

The methodology used in the current paper evolved relatively to that in the previous paper [18] as follows: In [18], we developed a lower bound on the capacity. The bound is a function of the volume of the transmitted signal set and uses the relation between maximal entropy and the volume of the signal set [9]. Since the signal set is N-dimensional, there was a need to develop a numerical technique to estimate the volume in the N-dimensional space, including a complex investigation of its accuracy. In the current paper, we developed an upper bound on the capacity to complement the lower bound [18]. To do so, we had to evaluate the entropy of the noisy received signal. Unlike the transmit signal [18], the received noisy signal is not confined to a bounded volume, and its entropy cannot be bounded by a function of the volume of the transmit signal set alone. Therefore, a new method bounding the contribution to the entropy of sections of space, each section extending over an unbounded volume, occupied by the received signal set, was developed. We utilized the volume-estimation core from [18] as an element of the new method.

Notation: Bold italic letters denote vectors. Log is the natural logarithm, unless stated otherwise. Differential entropy is denoted by h; E denotes the statistical expectation. The N-dimensional vector space of real variables is denoted R^N. The Probability Density Function (PDF) of x is p_x(x) or p(x). Convex bodies are denoted by calligraphic capital letters such as $\mathcal{X}$. The unity-radius N-ball centered at 0 is $\mathcal{B}$. The addition of convex bodies is performed by the Minkowski addition and is denoted by the usual +.

2. System Model

We begin with real-valued signals representing a low-pass channel. The system is presented in Figure 1. The encoder produces a real-valued low-pass signal x(t) in the frequency band |f| < B. The signal is constrained to be peak-limited, that is, $\left|x\left(t\right)\right|\le \sqrt{P}$. The signal passes an AWGN channel and is decoded. The channel output y(t) is

$$y\left(t\right)=x\left(t\right)+n\left(t\right)$$

(1)

where n(t) is a white Gaussian noise with power spectral density N₀ (0.5N₀ two-sided) and in-band power ${{\sigma }_n}^2=N_{0}B$. The Nyquist interval is T = 0.5/B. The channel and/or the decoder may include a brick-wall low-pass filter with bandwidth B; this does not influence the capacity, since x(t) is already band-limited. We seek the bounds on the capacity, which is the maximal Mutual Information (MUI) denoted I(x;y) per Nyquist interval. The Nyquist-rate-sampled x(t) is denoted by the vector x = (x₁ … x_k … x_N) of length N, x_k = x(kT). The vectors y and n denote the sampled received signal and noise, respectively. The sets of signals x and y are denoted $\mathcal{X}$ and $\mathcal{Y}$, respectively.

The signal to noise ratio (SNR) is defined as $\rho ={\displaystyle \frac{P}{{BN}_0}}$, as in the classical APBL channel. The capacity in bits per Nyquist interval of an APBL channel is the following famous [9] equation:

$$C_a=0.5{\log }_2\left({\displaystyle \frac{P}{N_{0}B}}+1\right)$$

(2)

To render the numerical analysis feasible, N needed to be finite. To this end, we modelled x(t) as cyclic with a period of NT and then limited to a duration of NT. This is identical to the CP-FDE and OFDM signaling formats before adding the cyclic prefix, so our analysis is applicable to CP-FDE and OFDM. In this setting, x(t) can be expressed in terms of its Nyquist-rate samples, e.g., [25], as

$$x\left(t\right)={\sum }_{i=0}^{N-1}{x}_i·\phi \left(t-iT\right); 0\le t<NT$$

(3)

where φ(t) is the periodic sinc function (4), also known as the Dirichlet kernel, instead of the usual sinc function (see, e.g., [25]):

$$\phi \left(t\right)={\displaystyle \frac{\mathrm{s}\mathrm{i}\mathrm{n}(\pi t/T)}{N·\mathrm{s}\mathrm{i}\mathrm{n}\left[\pi t/(N·T)\right]}}$$

(4)

The expression is valid for odd N and has a counterpart for even N (see, e.g., [25]). Note that $\phi \left(t\right)$ is periodic, with a period of NT. The functions $\phi \left(t-iT\right), i=0\dots N-1$ over 0 < t < NT form an orthogonal basis of our band-limited periodic signals, as can be shown easily using Equation (11) in [25].

The definition of capacity, which requires long sequences, is applicable if each error-correction block comprises many N-samples long CP-FDE signal segments (3), with each segment different and generated jointly by the error-correcting encoder. In the case of CP-FDE and OFDM signaling, each signal segment (3) of duration NT is to be prepended by its cyclic prefix and concatenated with the other segments to form the error-correction block. Thus, our model directly applies to CP-FDE and OFDM.

If N is increased, the cyclic prefixes become negligible and the signal with large and finite N is similar to a standard low-pass signal by the similarity of the sinc and periodic sinc (4) impulse responses, as follows: The standard low-pass signal x(t) is determined at time t by its Nyquist-rate samples x_i weighted by the sinc impulse response. We shall partition those samples into two groups: The first group comprises the N-neighboring samples x_i centered around t, such that |iT-t| < NT/2. Each such sample has one parallel sample in the finite N case (3), weighted by the periodic sinc impulse response (4). Comparison of the two impulse responses shows that they are similar at the relevant times; the differences are smaller than approximately ${\displaystyle \frac{\pi -2}{\pi N}}$ relative to the peak of the impulse response, which is 0.0071 for N = 51 and smaller for larger N. The second group comprises all the other more distant samples. These have no parallel in the finite N case; however, the sinc impulse response relevant to those samples is low, with a maxima of about 2 / (πN) relative to the peak of the impulse response; thus, their influence decreases with growing N.

3. Bounds on Capacity

3.1. On the Shape of the Signal Set

The transmit signal set $\mathcal{X}$ of vectors x of Nyquist-rate samples has the following attributes, illustrated in Figure 2: The set $\mathcal{X}$ is convex, that is, if signals ${\mathit{x}}_1$ and ${\mathit{x}}_2$ are in the signal set, so is $a{\mathit{x}}_1+(1-a){\mathit{x}}_2$ for any real positive scalar a < 1 (see [18] for proof and discussion). The signal region is symmetric, that is, if x is permissible, then −x is too. The set $\mathcal{X}$ is time-invariant, that is, invariant with respect to a time-shift in an integer number of Nyquist-rate samples; the shift becomes circular in the case of the CP-FDE channel model. The set $\mathcal{X}$ is confined to be in an N-cube, defined by $\left|x_i\right|\le \sqrt{P}$ by the peak limit when applied to the samples. The set $\mathcal{X}$ touches the N-cube at the center of each of its faces at the points $\mathit{x}=\dots 0, 0, 0 \mp \sqrt{P } , 0, 0, \dots$. Thus, all the faces of the hypercube are supporting hyperplanes of $\mathcal{X}$. There are many points of $\mathcal{X}$ on each such supporting hyperplane, since there are many different band-limited signals attaining maximal absolute value at any t = t_m. The set $\mathcal{X}$ does not reach most of the N-cube vertexes (corners); however, it does touch the special vertexes $\mathit{x}=\mp \sqrt{P }[1, 1, \dots 1]$, which produce a constant $x\left(t\right)=\pm \sqrt{P}$. The distance from the origin to any vertex of the N-cube is larger by a factor of $\sqrt{N }$ relative to the distance from the origin to the centers of the N-cube faces. Using (3) and (4) and shifting the Nyquist-rate sampling times by a fraction of the sampling interval from kT, k = 1…N, to kT + τ transforms $\mathit{x}$ from one set of orthogonal coordinates $\phi \left(t-iT\right)$ to another $\phi \left(t-\tau -iT\right)$ and creates an additional hypercube, confining $\mathcal{X}$ and having the same properties of the faces.

3.2. Volume-Based Lower Bound

The geometrical lower bound on the capacity C, based on the Entropy Power Inequality (EPI), was presented in [18]. We summarize the relevant expressions below.

$$C\ge 0.5{\log }_2\left({\displaystyle \frac{\gamma ·P}{N_{0}B}}+1\right)=0.5{\log }_2\left(\gamma ·\rho +1\right)$$

(5)

The power factor $\gamma \le 1$ is

$$\gamma ={\displaystyle \frac{V_x^{\frac{2}{N}}}{2\pi e}}$$

(6)

where V_x is the volume of the signal set $\mathcal{X}$ evaluated for signals with power P = 1.

An updated numerical method with an accuracy analysis added is presented next. The accuracy analysis is independent of the shape of $\mathcal{X}$, rendering it vastly simpler than its counterpart in [18]. As stated above, all the vectors x of samples comprising $\mathcal{X}$ are in an N-cube. To evaluate V_x, the components of x are generated at random u.i.i.d in $-\sqrt{P}\le x_i\le \sqrt{P}$. This covers the volume of the hypercube uniformly and is repeated N_t times. Each x is tested if being in $\mathcal{X}$, that is, not violating the peak limit in the transmission at any time t. Then V_x is the probability p_c of x being inside $\mathcal{X}$ multiplied by the volume ${\left(2\sqrt{P}\right)}^N$ of the hypercube, yielding

$$V_x=p_c·{\left(2\sqrt{P}\right)}^N$$

(7)

and, from (6), the power factor, evaluated without loss of generality at P = 1, is

$$\gamma ={\displaystyle \frac{{2p_c}^{2/N}}{\pi e}}$$

(8)

next, from (5)

$$C\ge 0.5\log \left({\displaystyle \frac{P}{{{\sigma }_n}^2}}·{\displaystyle \frac{2{p_c}^{2/N}}{\pi e}}+1\right),$$

(9)

thus, the power factor is a product of ${\displaystyle \frac{2}{\pi e}}$ present in the asymptotic high SNR upper bound [9] based on the discrete-time channel and of the new term ${p_c}^{2/N}$, accounting for the peak constraint applied also to all times between the Nyquist-rate samples. This is a somewhat different method of evaluating the power factor γ relative to the method used in [18], so it is a partial verification. Furthermore, using this new and simpler method, the accuracy of the numerical evaluation can be controlled better, showing that the results in Figure 3 are accurate, as follows: The value of p_c is evaluated by generating x(t) at random for N_t times. The estimated p_c is a binomial random variable, scaled by 1/N_t, of mean p_c and standard deviation of

$${\sigma }_p=\sqrt{p_c(1-p_c)/N_t}$$

(10)

To plot a confidence interval of the lower bound (9), we can plot the bound along with lines offsetting p_c by $\mp 4{\sigma }_p$. This is shown in Figure 3 using an oversampling factor N_k = 30.

Figure 3. Lower bound on the power factor as a function of the sequence length. The red lines are the confidence interval $\mp 4{\sigma }_p$. The number of simulations N_t was increased with increasing N to yield about 1000 hits inside the signal set.

Figure 3. Lower bound on the power factor as a function of the sequence length. The red lines are the confidence interval $\mp 4{\sigma }_p$. The number of simulations N_t was increased with increasing N to yield about 1000 hits inside the signal set.

Thus, the power factor is 0.1785 and is very stable with respect to increasing N. This is the same value as the one derived in Figure 12 of [18] by a different numerical method. The stability with rising N, the narrow confidence interval of less than 1% of the power factor itself, and the similarity, at sufficiently large N, of the standard low-pass signals to the CP-FDE signals used in the analysis (presented at the end of the system model section) lead to the following conjecture:

Conjecture 1.

The result γ = 0.1785 is also valid for infinite N and for general low-pass signals that are not necessarily a CP-FDE type.

3.3. Geometry-Based Upper Bound

3.3.1. Analysis

The capacity per sample is

$$C={\displaystyle \frac{1}{N}}\left[h\left(\mathit{y}\right)-h\left(\mathit{y}|\mathit{x}\right)\right]={\displaystyle \frac{1}{N}}\left[h\left(\mathit{y}\right)-h\left(\mathit{n}\right)\right]={\displaystyle \frac{1}{N}}h\left(\mathit{y}\right)-0.5\mathrm{l}\mathrm{o}\mathrm{g}(2\pi e{{\sigma }_n}^2)$$

(11)

maximized over the PDF of the transmit signals x. An interesting progress on a geometry-based upper bound over a discrete-time MIMO channel was achieved in [26]. One of its results, section Asymptotic gap there on the tightness of the EPI-based lower bound, applies to our problem (explanation: Equation (7) of [26] is the parallel of our (1) in its vector form with a finite N provided H =I _NxN, and our problem conforms to the convexity constraint on the signal set $\mathcal{X}$ as required in [26]). Thus, at an asymptotically high SNR, the lower bound (5), based on EPI, is tight. However, for the continuous-time channel here, we could not derive the intrinsic volumes of $\mathcal{X}$ required in the Steiner formula as performed in discrete-time in [26], so we are not able to derive quantitative results based on those, e.g., how high the SNR needs to be for the upper bound to be tight. To achieve this, we used a different approach. We proceeded in two stages: First, we established the following Theorem 2. Then we developed a numerical procedure upper-bounding the volume V_w in (12) and upper bounding the capacity (11) using (12).

Theorem 2.

The entropy of the received signal is upper bounded as follows

$$h\left(\mathit{y}\right)\le \mathrm{l}\mathrm{o}\mathrm{g}\left(V_w\right)$$

(12)

where V_w is volume of $\mathcal{W}$ and $\mathcal{W}$ is the Minkowski sum

$$\mathcal{W}=\mathcal{X}+R_f·\mathcal{B},$$

(13)

$\mathcal{B}$ denotes an N-ball of unit radius and$R_f={\sigma }_n·\sqrt{2e} {\left[\mathrm{\Gamma }\left({\displaystyle \frac{N}{2}}+1\right)\right]}^{{\displaystyle \frac{1}{N}}}, which is approximately$ $R_f\approx {\sigma }_n·\sqrt{N}$.

Remark: The Minkowski sums, similar to (13), of signal sets and noise sets appear in previous works, e.g., [26] and the references within, usually with infinite N, leading to the “ball-hardening” of the noise. In our case, with a finite N, the received signal cannot be confined to a finite volume (see Figure 4) and the object $\mathcal{B}$ in (13) is not a noise set, but rather a body, illustrated in Figure 5, the addition of which to $\mathcal{X}$ ensures the bound(12); see the following proof:

Proof of Theorem 2.

We start with the attributes of the PDF p_y(y) of the received signal at the region outside of $\mathcal{X}$. For a pair x, y the corresponding noise is n(x, y) =y − x with N-dimensional normal distribution and variance per dimension of σ². Then

$$p(\mathit{y})=\underset{all \mathit{x} }{\int }p\left(\mathit{x}\right)p(\mathbf{y}|\mathit{x})d\mathit{x}=\underset{all\mathit{x} }{\int }p\left(\mathit{x}\right)p\left[\mathit{n}\left(\mathit{x},\mathit{y}\right)\right]d\mathit{x}$$

(14)

Now, let us introduce a supporting (tangent) hyperplane touching the convex body $\mathcal{X}$ at a point z on it (see Figure 4). Line G, starting at z, is perpendicular to the supporting hyperplane. Consider a point y on G.

Note that the supporting hyperplane is N-1 dimensional and the distance g from it to the point y is one-dimensional. Note that, for any z on the surface of $\mathcal{X}$, there is at least one supporting hyperplane containing it and the associated perpendicular line marked G in Figure 4. By the normal distribution of n,

$$p\left[\mathit{n}\left(\mathit{y},\mathit{x}\right)\right]=C_1(\mathit{x}) e^{-0.5{\left(\left(g+C_2\left(\mathit{x}\right)\right)/{\sigma }_n\right)}^2}$$

for some pair of positive scalars $C_1\left(\mathit{x}\right), C_2\left(\mathit{x}\right)$. Then, following (14), the slowest decaying p_y(y) with respect to g, used in the proof of Proposition 3 below, is

$${p_y^s\left(\mathit{y}\right)=C_4·e}^{-0.5{\left(g/{\sigma }_n\right)}^2}$$

(15)

for some constant C₄. The slowest decaying $p_y^s\left(\mathit{y}\right)$ denotes that, for any $p_y\left(\mathit{y}\right)$, the ratio $p_y\left(\mathit{y}\right)/p_y^s\left(\mathit{y}\right)$ is non-increasing in g. Denote ${p_y^s\left(g\right)=C_4·e}^{-0.5{\left(g/{\sigma }_n\right)}^2}$.

Next, we partition the volume of the received signals y outside of $\mathcal{X}$ into sections constructed as follows:

• The base of each section is a small area on the N-1 -dimensional surface of $\mathcal{X}$ around some point denoted z. or a small M-1—dimensional body, M < N, around z in the surface of $\mathcal{X}$ or the point zitself.
• The section extends from the base outward along a line G perpendicular to the supporting hyperplane at z. The section is narrow enough to render p_y(y) a function p_y(g) of g only where g is the distance of y in the section from the supporting hyperplane. There may be several supporting hyperplanes at any z, e.g., at the special vertexes presented in Section 3.1 above.
• The intersection between the section and a hyperplane parallel to the supporting hyperplane and at a distance g from it has an (N-1) volume S(g). The volume S(g) is a non-decreasing function of g.

Figure 5 illustrates examples of such sections. A section, a base of which is on a flat part of the surface of $\mathcal{X}$, may be a cylinder with a constant S(g). A section the base of which is the vertex of the N-cube will include cones (single-nappe infinitely extending cones, that is, corresponding to values of g from zero to infinity), with S(g) proportional to

$${S\left(g\right)=C_3·g}^{N-1}.$$

(16)

The growth of S(g) in (16) is the fastest needed one, since such sections can cover an N-ball from a single center point z; see also the relation to the Steiner formula below.

Each such section extends to infinity, that is, g extends from 0 to infinity. We want to calculate the entropy contributed to h(y) by a single section, indexed i. This contribution to entropy is the N-dimensional integral

$$h_i=-{\int }_{section i}{p}_y(\mathit{y})\log \left(p_y\left(\mathit{y}\right)\right)d\mathit{y}$$

Denote the probability of y to be in a section i as p_i. The volume of each section can be integrated in slices taken at each value of g, the volume of each slice being $dg·S\left(g\right)$; the volume of the section is

$$V_i={\int }_0^{\infty }S\left(g\right)dg$$

(17)

which is infinite, and

$$h_i={\int }_0^{\infty }{p}_y(g)S(g)\log {\displaystyle \frac{1}{p_y\left(g\right)}}dg$$

(18)

and

$$p_i={\int }_0^{\infty }{p}_y(g)S(g)dg$$

(19)

Note that, by(15), p_y(g) is a decreasing function of g and S(g) is an increasing one by the definition of the sections construction above. Let us consider a situation in which p_i is fixed. Then the following proposition holds:

Proposition 3.

The p_y(g) which maximizes h_i for a given p_i and S(g) is the slowest decreasing one $p_y^s\left(g\right) in$ (15).

Proof. 

Let there be two different functions p_y(g), $p_y^s\left(g\right)$, both upholding (19). Such two functions must have crossings due to (19) and due to identical p_i, and there can be only one such crossing due to $p_y^s\left(g\right)$ being the slowest decreasing. The function $p_y^s\left(g\right)$ can be obtained from p_y(g) by incremental steps described below, each step moving some of the function mass from lower to higher region of g. This modification will enlarge h_i, since this operation moves probability mass from regions with lower values of $\log {\displaystyle \frac{1}{p_y\left(g\right)}}$ to regions with higher values (see (18)). Each step transforms p_y1(g) into p_y2(g) so that at the first step p_y1(g) = p_y (g) and at the last step $p_{y2}\left(g\right)=p_y^s\left(g\right)$. At each step, p_y1(g) is reduced in a narrow region dg of g around g₁ and increased around a narrow region of g around g₂, such that g_1< g₂, adjusting the increment to keep p_i in (19) constant while accounting for S(g). Thus, to produce p_y2(g), p_y1(g₁) will be decremented by some dp and p_y1(g₂) will be incremented by $dp{\displaystyle \frac{S\left(g_1\right)}{S\left(g_2\right)}}$. Define p_y3(g) as similar to p_y2(g), except that dp is increased so as to yield p_y3(g₁) = p_y3(g₂). Clearly, h_i, associated with p_y3, will be at least as large as that associated with p_y1, since uniform distribution maximizes entropy. Since p_y2(g) is a linear combination of p_y1(g) and of p_y3(g), the transition from p_y1 to p_y2 increases h_i in (18) by the concavity of the entropy function. □

So, inserting(15) and (16) into (18) will yield the upper bound on the entropy in a section with S(g) as given in(16). Consider evaluating (18) after such an insertion parametrized by p_i = 1 and by S(g) being the surface of an N-ball of radius g. In this case, p_y(g) is the PDF of the N-dimensional Gaussian noise; thus, h_i will be equal to the corresponding entropy, which is

$$h_{\mathit{n}}={\displaystyle \frac{N}{2}}\mathrm{l}\mathrm{o}\mathrm{g}(2\pi e{{\sigma }_n}^2).$$

Now let us evaluate (18) under the same conditions and using the same p_y(g), except for setting an arbitrary p_i and with S(g) reduced by multiplying it by a small factor α. This yields a narrow conical section. The changes will scale p_y(g) by ${\displaystyle \frac{p_i}{\alpha }}$. Then (18) yields

$$h_i={\int }_0^{\infty }{{\displaystyle \frac{p_i}{\alpha }}p}_y(g)\alpha S(g)\log {\displaystyle \frac{\alpha }{{p_i·p}_y\left(g\right)}}dg$$

$$h_i=p_i{\int }_0^{\infty }{p}_y(g)S(g)\log {\displaystyle \frac{1}{p_y\left(g\right)}}dg+p_i\log {\displaystyle \frac{\alpha }{p_i}}{\int }_0^{\infty }{p}_y(g)S(g)dg$$

$$h_i=p_i{h}_{\mathit{n}}+p_i\log {\displaystyle \frac{\alpha }{p_i}}$$

(20)

For the sake of upper-bounding h(y), we shall replace each section i by an equivalent section having the same $h_i$, the same probability p_i, and the same shape S_i(g) as the original section, but limited to the region 0 < g < R_f. The volume of the equivalent section is denoted $V_i^{\prime }$. The vector y occurs in the equivalent section with probability p_i with the uniform PDF $p_i\left(\mathit{y}\right)=p_i/V_i^{\prime }$. R_f is set so that h_i is preserved. The equivalence of p_i and h_i in each section renders the global new PDF legitimate. The new PDF has support of a limited volume V_W, the logarithm of which is an upper bound on entropy, and (12) follows. The contribution of the equivalent section to entropy is

$$h_i=-p_i\log (p_i/V_i^{\prime })=p_i\mathrm{l}\mathrm{o}\mathrm{g}\left(V_i^{\prime }\right)-p_i\mathrm{l}\mathrm{o}\mathrm{g}\left(p_i\right)$$

(21)

Let $V_i^{\prime }$ fit into the conical section (16) above up to a radius R_f. Then

$$V_i^{\prime }={\alpha V}_B{R_f}^N$$

where V_B is the volume of a unity radius N-ball and (21) becomes

$$h_i=p_i\mathrm{l}\mathrm{o}\mathrm{g}(V_B{R_f}^N)+p_i\log {\displaystyle \frac{\alpha }{p_i}}$$

So, to upper bound the section entropy (20) by that of the uniformly distributed $V_i^{\prime }$ we need

$$\log \left(V_B{R_f}^N\right){=h}_{\mathit{i}}{=h}_{\mathit{n}}$$

(22)

and this is independent of α and p_i. As presented in [9] and later termed the ‘noise ball hardening effect’, this yields, for large N, the radius of the noise sphere of

$$R_f\approx {\sigma }_n\sqrt{N}.$$

This is the approximate value of R_f stated in Theorem 2, and is easily verified using Stirling’s formula, as presented in [27]. So, placing a cone limited to radius R_f with the signal y distributed uniformly within provides at least the same entropy gain h_i as the original cone with unlimited g; thus, the conical sections which extend to infinity in Figure 5 can be replaced by the same conical sections, but with limited radius R_f, as illustrated by the dashed line in Figure 5 and with uniform pdf of y in each section. The exact evaluation of R_f from (22), without the large N approximation, starts with the exact volume of an N-ball with a unity radius (e.g., [27]):

$$V_B={\displaystyle \frac{{\pi }^{N/2}}{\mathrm{\Gamma }(\frac{N}{2}+1)}}$$

from (22)

$$\log \left({\displaystyle \frac{{\pi }^{N/2}}{\mathrm{\Gamma }(\frac{N}{2}+1)}}{R_f}^N\right)=N\log \left({\sigma }_n·\sqrt{2\pi e}\right)$$

$${R_f}^N={\displaystyle \frac{\left({\sigma }_n·\sqrt{2\pi e}\right)N}{{\pi }^{N/2}}}\mathrm{\Gamma }({\displaystyle \frac{N}{2}}+1)$$

$$R_f={\sigma }_n·\sqrt{2e} {\left[\mathrm{\Gamma }\left({\displaystyle \frac{N}{2}}+1\right)\right]}^{1/N}$$

(23)

as stated in the theorem. The other extreme case is the cylindrical section, that is, constant S(g) = S₀; we shall have from (15) and (19)

$$p_y\left(g\right)={\displaystyle \frac{2p_i}{S_0}}{p}_G\left(g\right)$$

where $p_G(g)$ is the PDF of the normal variable with zero mean and standard deviation of σ_n. Then, from (18),

$$h_i=S_0{\int }_0^{\infty }{\displaystyle \frac{2p_i}{S_0}}{p}_G(g)\log {\displaystyle \frac{1}{\frac{2p_i}{S_0}{p}_G(g)}}dg=p_i\log \left({\sigma }_n·\sqrt{2\pi e}\right)+p_i\log \left({\displaystyle \frac{S_0}{2p_i}}\right)$$

while the volume of a cylinder of length R_f is

$$V_i=S_0{R}_f.$$

Then, equating the last $h_i$ with the entropy (21) of y distributed uniformly in V_i,

$$p_i\log \left({\sigma }_n·\sqrt{2\pi e}\right)+p_i\log \left({\displaystyle \frac{S_0}{2p_i}}\right)=p_i\mathrm{l}\mathrm{o}\mathrm{g}(S_0{R}_f))-p_i\mathrm{l}\mathrm{o}\mathrm{g}(p_i)$$

$$R_f={\sigma }_n·\sqrt{{\displaystyle \frac{\pi e}{2}}}$$

(24)

which is much smaller than for the conical section. Another trackable section type is a cone-like shape with ${S\left(g\right)=C_5·g}^{M-1}, 2\le M<N$ and with a volume being M-th power of R_f by (17). Its radius R_f is readily derived as in (23) with N replaced by M; such a radius is smaller than R_f in (23). Indeed, inserting N = 1 into (23) yields(24), as expected, considering that (16) with unity N yields the cylindrical shape. The combination of the conical and cylindrical sections above yields an equation of the volume of the part external to $\mathcal{X}$ of $\mathcal{X}$ + R$\mathcal{B}$_N for any positive R since, using the Steiner formula (e.g., [28]), such volumes are polynomials of degree N in R. It is possible that additional shape types are needed for the partition of the whole volume into the sections; if so, arbitrary additional types conforming to the sections description above can be formed with no increase in R_f, as presented in Appendix A. End of proof of Theorem 2. □

Theorem 2 has some similarity to section V in [26]; the main difference is that Theorem 2 does not involve the limit of an infinite number of channel uses. In principle, the radius R_f can be reduced relative to (23) according to the local curvature of the surface of $\mathcal{X}$ (see Figure 5). However, to control the complexity and reliability of the numerical procedures, we shall use the maximal radius (23) all over, this will render the bound less tight. Equivalently, using the ball of the radius given by (23) is a tight bound around the vertexes and it exaggerates the entropy in the vicinity of flat faces, and the Minkowski addition of a ball in Theorem 2 overestimates the capacity.

Note that, at an infinite SNR, the set $\mathcal{W}$ is identical to $\mathcal{X}$, as evident from the vanishing R_f in (23), thus,, the upper bound becomes equal to the lower bound (5), regardless of the shape of $\mathcal{X}$, as predicted also by [26].

3.3.2. Numerical Analysis of the Upper Bound

We need to upper bound the volume V_w of $\mathcal{W}$ to upper-bound the entropy (12) and then the capacity(11). Since we do not have a simple expression of the shape of $\mathcal{X}$ (see Figure 2), we need a numerical evaluation, as presented next.

The numerical evaluation of V_w is based on the method used to evaluate V_X presented in [18] with an extension. As in [18], we partition the space of the signals y and x into narrow conical sections (N-cones) with apexes at the origin, as illustrated by the red lines in Figure 2 and Figure 6. The conical sections are visited at random by generating random x(t), as in [18]; each x(t) is scaled so that $\underset{t}{\max }\left|x\left(t\right)\right|=\sqrt{P}$ so the vector x of samples is on the surface of $\mathcal{X}$. Then, we evaluate r = |x|, the local radius of $\mathcal{X}$, that is, the distance from the origin to the point x on the surface of $\mathcal{X}$. The volume V_x in [18] is evaluated from many values of r. Here, instead of using r directly to evaluate V_x, as in [18], we use r_w, which is the distance from the origin to the surface of $\mathcal{W}$, to evaluate V_w. The radius r_w is upper bounded as

$$r_w=r{\displaystyle \frac{\sqrt{P}+R_f}{\sqrt{P}}}$$

(25)

justified as follows: At some t = t_m, we have $\left|x\left(t_m\right)\right|=\sqrt{P}$. We represent x(t) in its vector form by the sequence of the Nyquist-rate samples at times kT + τ shifted by 0< τ < T, such that t_m is one of the sampling times. Then, as explained in Section 3.1 above, x is a point on a supporting hyperplane of $\mathcal{X}$. Next, we upscale x to produce the point $\mathit{y}=\mathit{x}{\displaystyle \frac{\sqrt{P}+R_f}{\sqrt{P}}}$. Clearly $\left|y\left(t_m\right)\right|=\sqrt{P}+R_f$; thus, y is at a distance of at least R_f from x. And this also holds for all other signals x(t) due to the peak power constraint at time t_m. Then, by the Minkowski sum (13), y is on the surface of $\mathcal{W}$ or outside W. Since r scales identically to x(t), r_w in (25) is guaranteed to be a local radius reaching the surface of $\mathcal{W}$ or beyond.

Additionally to (25), r_w is limited at each trial not to exceed an N-ball with a radius of ${\sqrt{PN}+R}_f$. The evaluation of V_W is performed as the evaluation of V_x in [18] including the importance sampling while r in [18] is replaced by r_w(25).

The results are presented in Figure 7 compared to the EPI-based lower bound in (5) and (9), to the average power-limited capacity (2), and to the discrete-time upper bound using Equation (1) in Kellips [16]. We also compared to the unit process upper bound [23], which is equivalent to a power loss by a ratio of 0.9259 relative to (2), which might be improved, but not below a ratio of 0.5 due to lemma 3 in [22]; thus, the CGC-based approach cannot provide an upper bound on a capacity that is tight at a high SNR.

The gap between the upper and the lower bounds at an intermediate SNR is wider at high N and decreases at lower N as seen in Figure 8 with N = 31. Thus, the upper bound may be less tight at high N. Further improvement of the upper bound might be addressed in a future work that would refine the inequalities used in developing the upper bound, e.g., adapting R_f to the local curvature of the signal set $\mathcal{X}$ and evaluating h_i in each section, depicted in Figure 5, by a new method yet to be developed.

To verify the method and the software, we applied the peak limit on the Nyquist samples only. The results (see Figure 9) were expected to match the works on peak-limited discrete-time channels in [14,16,17], as presented in the Introduction. Indeed, the new upper bound and the upper bound in [16] on discrete-time channels plotted in Figure 9 do match at a high SNR while showing that, in this simple case, the tight upper bound in Kellips [16] is superior to ours at low and intermediate SNRs.

4. Band-Pass Signals

4.1. Signal Representation

The analysis is readily extended to band-pass signals represented in the complex-valued baseband. We reuse the previous notation with modifications, as follows. The encoder produces a complex-valued low-pass signal x(t) in the frequencies |f| < 0.5B. The noise is complex-valued with a power spectral density N₀ (two-sided). The signal is peak-limited, that is, $\left|x\left(t\right)\right|\le \sqrt{P} \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} t$. The signal to noise ratio is defined as $\rho ={\displaystyle \frac{P}{{BN}_0}}$. The classical capacity per Nyquist-rate sample of the APL channel is

$$C_a={\log }_2\left({\displaystyle \frac{P}{N_{0}B}}+1\right)$$

(26)

The real and imaginary components of the noise have a power of ${\sigma }_n^2$/2 = N₀B/2 each, and the per-symbol noise entropy is $h\left(n\right)=\log \left({\sigma }_n^2·\pi e\right)$. The complex samples are represented by pairs of real numbers, and the volume V_x is computed in an 2N-dimensional real-valued space. The N-cubes containing $\mathcal{X}$ in the low-pass case, which can be considered Cartesian products of line segments each representing one peak-limited real-valued sample, are replaced by Cartesian products of N disks with a radius of $\sqrt{P}$, each representing one complex sample peak-limited to $\sqrt{P}$. Denote the set defined by this product for P = 1 by $\mathcal{D}$.

4.2. Lower Bound

The power factor and the EPI-based lower bound (see [18]) are now

$$\gamma ={\displaystyle \frac{1}{\pi e}}{V}_x^{{\displaystyle \frac{1}{N}}}$$

(27)

$$C\ge {\log }_2\left({\displaystyle \frac{\gamma ·P}{N_{0}B}}+1\right)$$

(28)

The volume V_x is evaluated numerically as in the low-pass case while the N-cube with its volume of 2^N is replaced by the body $\mathcal{D}$ confining the signal set, the volume of which is $V_D={\pi }^N$. Equation (7) is replaced by $V_x=p_c·{\left(\pi \sqrt{P}\right)}^N$ accordingly. The 4σ confidence interval is calculated by the same procedure as in the low-pass case. The results are presented in Figure 10.

The estimated power factor is γ = 0.2907, identical to that in [18]; here, we were able to evaluate the 4σ confidence interval, which is less than 1% of the power factor.

4.3. Upper Bound

The upper bound in the band-pass case is evaluated similarly to the low-pass case. Both are evaluated in spaces of real numbers. The number of dimensions rises from N to 2N and the noise power in each real dimension is reduced by a factor of 2. This results in Corollary 4.

Corollary 4.

In the band-pass case,

$$h\left(\mathit{y}\right)\le \mathrm{l}\mathrm{o}\mathrm{g}\left(V_w\right)$$

(29)

where V_w is volume of $\mathcal{W}$ and $\mathcal{W}$ is the Minkowski addition $\mathcal{W}=\mathcal{X}+R_f·\mathcal{B}. \mathcal{B}$ is an 2N-ball of unit radius. R_f is

$$R_f={\sigma }_n·\sqrt{e} {\left[\mathrm{\Gamma }(N+1)\right]}^{1/\left(2N\right)},$$

(30)

the approximate value is $R_f\approx {\sigma }_n·\sqrt{N}$.

Proof .

Note that the proof of Theorem 2 was independent of the shape of $\mathcal{X}$. The difference in Corollary 4, relative to Theorem 2, is just replacing σ and N by $\sigma /\sqrt{2}$ and 2N, respectively. The approximate expression of R_f follows immediately. The exact expression (30) follows by inserting $\sigma /\sqrt{2}$ and 2N into (23). □

The numerical evaluation of the upper bound closely follows the low-pass case described above. The capacity of our continuous time channel is upper-bounded if the peak constraint is only imposed on the Nyquist-rate samples. The bounds on such a channel are presented in [17]. For simplicity, we present a simple EPI-based lower bound on the capacity of such a channel, Equation (38) in [15], which is

$$C_d\ge {\log }_2\left({\displaystyle \frac{SNR}{e}}+1\right)$$

(31)

It is close to capacity at a high SNR and is, of course, lower than any upper bound. The results are presented in Figure 11.

As evident from Figure 11, the new upper bound is tighter at high enough SNR, relative to any upper bound based on the discrete-time channel. At even higher SNRs, the new upper bound meets the lower EPI-based bound (28). The bound is slightly nearer to the EPI-based lower bound at reduced N (see Figure 12).

The method was verified by applying the peak limit on the Nyquist-rate samples only with results qualitatively similar to those in the low-pass case in Figure 9.

5. Conclusions

The important problem of the capacity of Peak-Power Limited and Band Limited (PPBL) signals over the AWGN channel was investigated. We present Theorem 2 upper-bounding the entropy of the received signal leading to an upper bound on capacity; the upper bound is geometry-based and dependent on the shape of the signal set, not only on its volume. We computed an upper bound on capacity applicable to Cyclic Prefix-assisted Frequency Domain Equalization (CP-FDE) and OFDM-type signals. At a high SNR, the new bound is tighter than the previously known upper bounds and approaches the lower bound derived in previous work. We found that, at a high SNR, the change from average to peak power constraints with the same power parameters incurs a cost of about 7.5 dB in the low-pass case and about 5.4 dB in the band-pass case, as reflected by the power factor γ. The previous upper bounds on γ were 2/πe in [9] in the low-pass case and 1/e in [15] and in (31) in the band-pass case, corresponding to a smaller power cost of 6.3 dB and 4.34 dB, respectively. We verified the previous lower bound using a simpler numerical method and established the accuracy of the result. Previous bounds based on the volume of the transmitted signal set yielded EPI-based lower bounds, e.g., [18]. Here, we utilize the effective volume of the received signal set, which is influenced by the volume and the shape of the transmit signal set to derive an upper bound.

Future directions: The upper-bounding technique might be improved and the bound may be tightened, e.g., by more detailed exploitation of the geometry of the problem. The same geometry-based bounding technique should be applicable over the AWGN channel to other signal sets which are convex, such as some cases of MIMO, e.g., [26], the constrained Gaussian channel [20] and PPBL frequency-selective channels. The effort to find effective and practical modulation and coding techniques, e.g., [11,12], should be continued.