Time-Limited Codewords over Band-Limited Channels: Data Rates and the Dimension of the W-T Space

We consider a communication system whereby T-seconds time-limited codewords are transmitted over a W-Hz band-limited additive white Gaussian noise channel. In the asymptotic regime as WT→∞, it is known that the maximal achievable rates with such a scheme converge to Shannon’s capacity with the presence of 2WT degrees of freedom. In this work we study the degrees of freedom and the achievable information rates for finite values of WT. We use prolate spheroidal wave functions to obtain an information lossless equivalent discrete formulation and then we apply Polyanskiy’s results on coding in the finite block-length regime. We derive upper and lower bounds on the achievable rates and the corresponding degrees of freedom and we numerically evaluate them for sample values of 2WT. The bounds are asymptotically tight and numerical computations show the gap between them decreases as 2WT increases. Additionally, the possible decrease from 2WT in the available degrees of freedom is upper-bounded by a logarithmic function of 2WT.


Introduction
Wireless communication technologies that use radio waves impose tight requirements on the used spectrum as adjacent radio bands may be used by other users or technologies. A specific transmit spectral mask is usually required, a mask that allows the communication system to transmit data within a specific radio band while guaranteeing an acceptable level of interference on the other users or technologies. It is therefore reasonable to consider that in a wireless communication system the transmitter confines its transmitted data within its radio band, and the receiver looks for the transmitted data in this band. The allocated band, the corresponding channel statistics and the available transmit power determine the maximal reliable possible data rate. In his pioneering work, Shannon [1] derived the capacity of band-limited real additive white Gaussian noise (AWGN) channels, which for the complex channel is C Shannon = 2W log 2 where W is the bandwidth of the baseband channel, P is the average transmit power and N 0 is the spectrum of the additive circularly symmetric complex white Gaussian noise. Shannon derived first the channel capacity for the discrete time channel as the block length of the codewords grows towards infinity. He then used the sampling theorem which provides a one-to-one relation between the continuous time and discrete time signals: For W-Hz band-limited signals when sampled at a rate of 2W samples per second, the operation is invertible and information lossless making use of the "sinc" function defined in this manuscript as sinc(2Wt) def = sin (2πWt) 2πWt .
While the derived results are mathematically rigorous, some of the made assumptions do not hold in practical settings.
First: the conversion from discrete time to continuous time and vice versa is not practical since the "sinc" function needs an infinite time support. Moreover, the use of any band-limited function with non-zero finite energy is not possible in practice. In the literature, Wyner [2,3] and Gallager [4] tackled this issue and considered the use of T-seconds time-limited codewords. However, they derived asymptotic results as T → ∞ and reached the same formula (1) derived by Shannon. In [2], Wyner considered four different physical models and derived the asymptotic channel capacity for each model. Wyner stated that the first two models suffer from some physical difficulties; the assumed noise in the first model results in an infinite noise power at the receiver, and the use of strictly band-limited signals in the second model is not practical and may produce interference between consecutive codewords. On the other hand, he proved that using a noise model with finite power results in infinite capacity. Wyner made some assumptions to avoid these issues in [3] and in the third and fourth models in [2]. He derived the channel capacity of the different models by relating the continuous time to discrete time models as Shannon did in [1], but by using the prolate spheroidal wave functions (PSWFs) and their property that as 2WT → ∞, the first 2WT PSWFs form asymptotically a complete orthonormal (CON) set for the time-limited and approximately band-limited signals.
Gallager [4] Section 8.5 considered transmitting time-limited signals over an additive real Gaussian noise channel with impulse response h(t). He used an arbitrary power spectral density (PSD) for the noise S N ( f ) and arbitrary filter h(t). As T → ∞, he derived the channel capacity to be where H( f ) is the Fourier transform of the filter h(t) and p( f ) is the transmit spectrum at frequency f subject to In [4] Section 8.3 he considered the special case where H( f ) is the ideal low pass filter and the noise is white and found the capacity to be the same as the one derived by Shannon, while avoiding the issues of infinite noise power and infinite capacity. In that special case (where H( f ) is the ideal low pass filter) the transformation between continuous time and discrete time was also done through the use of the PSWFs [4] Sections 8.4 and 8.5. Finally, Gallager provided an intuitive argument regarding inter-codeword interference [4] Section 8.5; one can introduce a large guard time, say T 1− for some > 0. Asymptotically, inter-codeword interference is avoided without affecting the data rates since A related question is that of determining the degrees of freedom when using T-seconds time-limited codewords over a W-Hz band-limited channel. The space of finite-energy functions that are band-limited and time-limited contains only the zero function. It is nevertheless commonly accepted in the literature that the dimension of the W-T space is approximately 2WT. This argument is supported by results in [5][6][7][8][9] that were also derived in the asymptotic regime as WT → ∞, something that we intend to relax hereafter.
Recently, in [10,11], we studied the use of T-seconds time-limited pulses over W-Hz real band-limited Gaussian channels, and derived the channel capacity by allowing the time duration of the codewords to grow towards infinity. In [10], we considered a pulse amplitude modulation (PAM) system and we studied optimal signaling; we showed that one can approach Shannon's capacity by signaling at faster than the Nyquist rate. In [11], we considered a combined PAM-orthogonal multi-pulse modulation scheme (PAM-OMM) and derived the achievable rates and evaluated them numerically. We showed that these rates can be made arbitrarily close to the Shannon's capacity by using a finite number of parallel filters. We also established that there are 2WT degrees of freedom when using such system.
Second: the second questionable assumption in practical settings is the use of infinite block-length codewords which is not feasible. In [12], Polyanskiy derived an approximation for the maximal data rates when given a target probability of error in the finite block-length regime, where only discrete time channels are considered.
In this work, we consider transmitting continuous time and finite duration codewords over a band-limited Gaussian channel. We use the ideal low pass filter as a model for the channel to force the transmitter to confine its transmitted information and energy in the allocated band and also model good receiver designs: Given any practical low pass filter, one can implement a sharper low pass filter that is closer to the ideal one. In such a model, there are no issues when it comes to infinite noise power and/or infinite capacity, however inter-codeword interference is inevitable.
Our main goal is to investigate the degrees of freedom and the achievable pairs of data rates and probability of error. We use a similar approach to [2][3][4] (using the PSWFs) to transform the problem from continuous time to discrete time and vice versa, and then we apply the adapted results by Polyanskiy for parallel discrete time AWGN channels.
Recently, in [13], we investigated the 'dual' problem where we derived an upper bound and a lower bound on the rates of source coding a T-seconds finite duration piece from a W-Hz band-limited real white Gaussian process.
The paper is organized as follows: in Section 2, we provide a brief overview of the PSWFs, some relevant properties in addition to some numerical computations. In Section 3, we present our system model and formulate the problem. We derive upper and lower bounds for the data rates and the corresponding degrees of freedom in Section 4. In Section 5, we present the results of our numerical computations. In Section 6, we present some possible enhancements on the bounds, and we summarize the results and conclude in Section 7.

Preliminaries: The Prolate Spheroidal Wave Functions
In [14], Slepian and Pollak showed that the PSWFs possess properties that make them useful in the Fourier analysis of band-limited functions and time-limited functions. For any c = πWT > 0, they defined the PSWFs as an infinitely countable set of real functions ϕ c,l (t) l∈N , normalized solutions of the integral equation where for every l ∈ N, The PSWFs form a CON set for band-limited functions [14] with where δ lm is the Kronecker delta. Additionally, these functions are orthogonal over the time window T: where λ c,l l∈N are the eigenvalues that are all in the range 0 < λ c,l < 1 and decreasing in l [14]. The eigenvalue λ c,l may be hence viewed as the energy concentration of ϕ c,l (t) in the time interval − T 2 , T 2 and ϕ c,0 (t) has the highest energy concentration. Additionally, the PSWFs are real continuous functions that are even when l is even and odd when l is odd.
In his book [4] Section 8.4, Gallager proved that the PSWFs are the desirable functions when sending time-limited signals over a band-limited channel. He made use of one important property of the PSWFs, namely the Fourier transform Φ c,l ( f ) of the PSWF ϕ c,l (t) is a scaled version of a time-limited PSWF: In this paper we define c def = 2WT as index for the PSWFs, which is different from the one used by Slepian and Pollak. More specifically ϕ 1,l (t) here is the same as ϕ π 2 ,l (t) in [14]. We also denote here by "Dϕ c,l (t)" the T-seconds time-limited version of a PSWF. More specifically, and since λ c,l is its energy, the normalized time-limited PSWF is Based on (2)-(4), when We have used a debugged version of the software package by Adelman et al. [15] to compute the eigenvalues of the PSWFs. While it is known [4] Section 8.4 that for any > 0 Figure 1 shows the eigenvalues λ c,l of the PSWFs for c = 100 and 84 ≤ l ≤ 116. Note that This transition region between the "extreme" eigenvalues (very close to 1 or 0) is known to have a length proportional to the logarithm of c [4] Section 8.4. For example, for c = 2000, λ 2000,l is between 10 −5 and 1 − 10 −5 only in the range 1988 < l < 2011. In this manuscript, shifted (in time and frequency) PSWFs will come in handy. Denote by α k,h c,l,m the inner product between the l th normalized time-limited PSWF already shifted in time and frequency by k T-seconds and h 2W-Hz, respectively, and the band-limited version of the m th normalized time-limited PSWF. By Equation (5) and using Parseval, which can be interpreted as the inner product between the bandlimited shifted normalized l th PSWF and the m th normalized time-limited PSWF. In Appendix A, we study the magnitude of α k,h c,l,m and derive various bounds that are used throughout this document.

System Model and Problem Formulation
We consider a system model where a T-seconds time-limited codeword is transmitted over a linear Gaussian channel with transfer function H( f )-assumed to be an ideal W-Hz low-pass filter, and an additive complex Gaussian noise N(t), assumed to be a stationary W-Hz band-limited "white" process with mean zero and PSD S N ( f ) = N o for f ∈ [−W, W]. With a system in mind whereby other codewords may be transmitted-possibly by other users-consecutively and/or in neighboring bands, we denote by C 0,0 (t) the codeword carrying the data packet of interest, and by {C k,h (t)} (k,h)∈Z 2 \{(0,0)} those carrying other data packets, possibly transmitted by other devices and interfering with the message of interest as illustrated in Figure 2. Our model is based on the reasonable assumption that all codewords follow the same modulation techniques, since every frequency band is usually allocated to a unique technology which abides by specific standards, and the neighboring bands are more likely to be used by the same technology.
In what follows, we consider various scenarios where some or all of those interfering codewords are present and we denote by I ⊂ Z 2 \ {(0, 0)} the set of other present codewords. The overall signal going through the channel can hence be written as the sum of the codeword of interest and the other interfering codewords: On the receiver side, the data packet of interest is to be recovered from y(t), t ∈ [−T/2, T/2], a T-seconds time-limited version of the output of the channel r(t). We assume that, whenever present, a transmitted codeword satisfies the power constraint, As Gallager proved that the PSWFs are the desirable CON set when sending a time-limited codeword over a band-limited channel [4] Section 8.4, we use the normalized time-limited PSWF as orthogonal pulses to send the data symbols. Hence the codewords can be written as where c = 2WT. Representing the continuous time signal C k,h (t) by the symbols {a k,h,l } is known as "signal space representation" in the context of digital communications. The {a k,h,l }'s are chosen from a given complex signal constellation and by Plancherel and (7) they satisfy When it comes to the noise, N(t) is band-limited and can be hence decomposed as where {n l } l∈Z are independent zero-mean complex circular Gaussian random variables with variance N 0 . At the receiver, sufficient statistics are clearly obtained by projecting y(t) on the set of normalized time-limited PSWF to extract the data symbols. It is worth noting that since r(t) is band-limited and has finite energy, it is necessarily analytic and it is therefore sufficient to know r(t) over any open interval to determine it fully. As a consequence, from an information-theoretic perspective, whether r(t) as whole is available or only y(t), the information rates are identical.
The problem at hand is to maximize the information rates given a maximal probability of error. This is naturally related to the available degrees of freedom when sending time-limited codewords over a band-limited channel, which is the maximal number of independent data symbols that can be transmitted to the receiver.
In the following section, we consider various scenarios and derive upper and lower bounds for the data rates and the degrees of freedom.

An Upper Bound
To derive an upper bound, we consider the case where only C 0,0 (t) is transmitted over the channel. By ignoring the other transmitted codewords we ignore the effect of inter-codeword interference, and we obtain upper bounds on the rates and the degrees of freedom since interference can only be harmful. In this scenario, the input to the channel can be written as , and the received signal r(t) is band-limited and can be written as where y m = λ c,m a 0,0,m + n m . From an information theoretic perspective, the considered system model is equivalent to the discrete time system model in Figure 3 where r m = y m / λ c,m = a 0,0,m + n m / λ c,m .  The noise components {n m / λ c,m } are independent, zero-mean complex circular Gaussian random variables and each complex channel is equivalent to two usages of independent real (real and imaginary) channels with additive Gaussian noise with variance N 0 /2λ c,m per dimension.
If E |a 0,0,m | 2 = 2P m T (where P m T is the second moment per dimension), the power constraint in (8) can be written as 2 ∞ ∑ m=0 P m ≤ P and the signal to noise ratio per dimension for r m is 2λ c,m P m T N 0 .
One can notice that it is possible to send infinitely many independent symbols in such a system. However, only a finite number of them, say L, is useful because the energy per symbol is finite and the noise energy of the m th channel is increasing towards infinity as m increases to infinity (λ c,m tends to 0 as m tends to ∞ as shown in Section 2). Note that L depends on c = 2WT since there are approximately c eigenvalues λ c,m that are close to 1, and naturally L grows to infinity with c.
Polyanskiy [12] derived upper and lower bounds and an approximation for the achievable rates at a given probability of error in the finite block-length regime. In [16], he studied the parallel Gaussian channel set-up where N memoryless parallel channels of different noise power are used each n times. Following the methodology in [12] and applying the Berry-Esseen inequality [12] Lemma 14 over n × N real independent variables shows that the maximal number of bits that can be transmitted is where • ∈ (0, 1) is the probability of error, • and L is the number of non-zero {P m }'s, which is less or equal N.
In Polyanskiy's work [16] N is constant and n grows towards infinity and hence [16] Theorem 4 shows an O(log 2 n) term instead of the O(log 2 nL) term here in (9). In our setup, each channel m in Figure 3 has a different noise power, but each has two dimensions with the same noise power. In our scenario therefore n = 2, L is of the order of (and grows with) c and the term O(log 2 nL) becomes O(log 2 L) as n is constant and equal to 2. These derivations yield the following lemma: Lemma 1. An upper bound on the data rates is given by, where • ∈ (0, 1) is the probability of error,

Lower Bounds
In what follows, we derive a lower bound on the rates by jointly: • Finding an upper bound on the interference. • Using only the first N PSWFs to transmit data.
Subsequently, we optimize over the value of N to obtain tighter bounds as well as lower bounds on the degrees of freedom.
We study below three scenarios for the interference and derive a lower bound for these scenarios.

Consecutive Single Band Codewords (CSB)
We consider first the case where a single band is used and only the codewords {C k,0 (t)} k∈Z are transmitted over the channel, i.e., I = {(k, 0), k ∈ Z * def = Z \ {0}} as shown in Figure 4. In the case where only the first N PSWFs are used, , and the received signal is the sum of an information bearing signal, an interfering signal and additive channel noise, Projecting r(t) on the m th normalized time-limited PSWF-which can be done using an appropriate matched filter-results in Since the {a k,0,l } N−1 l=0 are not necessarily uncorrelated for a fixed k, we use the upper bound and the fact that E a k,0,l 2 = 2P l T to upper bound the second moment Using the bound (A11) derived in Appendix A Using the alternative bound (A7) and the fact that Therefore the second moment of the interference term is upper-bounded by and the signal to noise and interference ratio per dimension in y m is lower-bounded by These derivations yield the following lemma:

Lemma 2.
A lower bound on the data rates is given by where L is the number of non-zero {P m }s.

Single Time-Slot Multi-Band Codewords (STMB)
In this scenario multiple bands in a single time-slot are used and only the codewords {C 0,h (t)} h∈Z are transmitted over the channel, i.e., I = {(0, h), h ∈ Z * } as shown in Figure 5 below.
Alternatively, using the bound (A8) Therefore the second moment of the interference term is upper-bounded by These derivations yield the following lemma: The corresponding lower bound on the data rates is given by where L is the number of non-zero {P m }'s and

Consecutive Multi-Band Codewords (CMB)
We consider now the case where all the codewords {C k,h } (k,h)∈Z 2 are transmitted over the channel. The analysis follows as above and the interference plus noise term w m is Upper bounds on the second moments of the first two interference terms have been derived in Sections 4.2.1 and 4.2.2, respectively, and it remains to derive one for the third term. By Equation (11), Using bound (A13) and Equations (12) and (14), Alternatively, using bound (A14), Therefore, the second moment on the interference terms is upper-bounded by These derivations yield the following lemma: Lemma 4. The corresponding lower bound on the data rates is given by where L is the number of non-zero {P m }s and

Numerical Results
In what follows, we use W = 1 KHz and = 10 −3 in our computations. Since we consider the complex base-band channel, an equivalent real pass-band channel will have 2W-Hz bandwidth (i.e., 2 KHz in our case), and L complex degrees of freedom in base-band is equivalent to 2L real degrees of freedom in passband.
In the following we evaluate the degrees of freedom and the data rates for different SNR levels (in dB) as L depends on the SNR; for example, since the water-filling algorithm is used for Equation (10), L can be increased by increasing the "water level" in the water-filling algorithm, which means that the SNR level must be increased. Note that for a fixed c, the ratio of the derived bounds to Shannon's capacity depends only on P, N o and W through the ratio P N o W , and the results for W = 1 KHz can be extracted from our presented results for an appropriate range of SNR = 2P/N o .
To evaluate the bounds in Equations (10), (13), (15) and (16), we used the optimization toolbox in MATLAB to search for the optimal solution in {P m } m . When it comes to the upper bound (10), the water-filling choice used by Polyanskiy only maximizes the term ∑ ∞ m=0 log 2 1 + 2λ c,m P m T N 0 and not the whole expression. However, the water-filling choice almost achieves the same performance as the optimization routine with negligible differences. When it comes to the term O(log 2 (L)), we specialize it to 1 2 log 2 (2L) for the sake of numerical computations; we used the constant 1 2 as Polyanskiy conjectured in Equation (4.218) in his thesis [12], and we used the term 2L inside log 2 (.) since the number of real independent variables is 2L (as explained in Section 4.1).

Upper Bound
To solve Equation (10), we maximize the following quantity using the optimization tool in MATLAB for different values of N. The solution being decreasing with m, L is the value of N where R N UB saturates (which is the same as the number of non-zero P m 's after saturation). We adopt similar method and notations for all the bounds presented hereafter. Figure 6 shows the obtained R N UB for c = 1000 and at SNR = 50 dB, and in this example the obtained degrees of freedom are L = 1004. The upper bound on the rates is  We compute L and the corresponding upper bound R UB for different values of c and for different levels of SNR. Figure 7 shows the difference (L − c) between the obtained degrees of freedom and c = 2WT, and as expected, L increases as the SNR increases (in a manner akin to the water-filling solution: as the water level increases, it is possible to use additional PSWFs). However, the additional degrees of freedom (beyond c) increase slowly with c and L/c → 1 as c increases towards infinity. In Figure 8 we plot the obtained upper bound on the rates together with the Shannon capacity. We notice that the gap between the upper bounds and the Shannon capacity decreases as c increases. The ratio of the bounds to the Shannon capacity can be seen in Figure 10 below.

Lower Bounds
In this section, we present the numerical results for Sections 4.2.1-4.2.3 (Equations (13), (15) and (16)), and we apply the same numerical method we used in the previous section. We notice that the lower bounds when using either CSB or STMB are almost the same with no significant differences and we omit the results for CSB. Figure 9 shows the obtained lower bounds on the degrees of freedom. We note that for given scenario (CSB, STMB or the general lower bound), the results are almost the same for different SNR levels; increasing the signal power will only lead to increasing the power of the interference (see Equations (13), (15) and (16)), and the effect on the signal to interference ratio remains negligible. Moreover, the results for the different scenarios are very close (±1 on average). Although L − c decreases as c increases, it decreases slowly and L/c → 1 as c increases towards infinity.
We propose approximating the degrees of freedom for the general lower bound by the following equation L LB= c − 1.35 log 2 (c) + 3.25, and we draw "L LB − c" in Figure 9. It is expected that "L LB − c" is a logarithmic function of c since the transition region of the eigenvalues of PSWFs is a logarithmic function of c (as shown in Section 2).  Figure 10 shows the ratio of the obtained bounds to the Shannon capacity. For a fixed c and as SNR increases, the bounds get relatively closer to capacity and the gap between the upper bound and the lower bound increases. In addition, for a fixed SNR, as c increases, the gap between the bounds decreases.

Possible Enhancements on the Bounds
Since the obtained upper and lower bounds on the rates ( Figure 10) are very close for SNR = 30 dB, the bounds are tight and a good approximation for the optimal data rates is reached. However, the gap between the upper and lower bounds increases as the SNR increases. For instance, the gap between the upper and lower bounds for c = 200 and SNR = 70 dB is 5.5% of Shannon's capacity, which means that one or both bounds are loose. In the following we present some possible improvements on the bounds.

A Tighter Upper Bound
To derive the upper bound, we ignored the interference due to other codewords and the obtained degrees of freedom surpassed 2WT for SNR ∈ {50 dB,70 dB}. However, the results in the literature shows that the asymptotic dimension of the W-T space is 2WT, and thus adding the constraint that the the codewords must be time-limited will not increase the available degrees of freedom. So one can conclude that our upper bound on the degrees of freedom is not tight. One possible way to improve this upper bound is to force the degrees of freedom to be at most 2WT, and hence, L in constrained to be less or equal to 2WT in Equation (10). In other words, the normalized time-limited PSWFs with m ≥ 2WT will not be used to transmit data and their allocated power P m 's are forced to be zero. The obtained upper bound will decrease and hence the gap between the upper and lower bounds will decrease.
In Figure 11 we present the numerical values for the tighter upper bound (TUB) in addition to those of the upper bound. When c = 200, we note that the tighter upper bound achieves improvements of 0.8% and 1.5% for SNR = 50 dB and SNR = 70 dB, respectively.

Introducing a Guard Time or a Guard Band between the Codewords
The use of a guard time or a guard band is expected to decrease the effect of inter-codeword interference which drives towards increasing the achievable rates. On the other hand a portion of the time resources or frequency resources will not be used which decreases the data rates. Therefore, there must be an optimal guard time and an optimal guard band that maintain a good trade-off between the lost resources and the improvement of the inter-codeword interference.
Using a guard time T G reduces the upper bound to R UB (T G ) = R UB T T+T G , but introducing a guard band does not affect it. When it comes to the lower bound, deriving a closed form expression has proven to be difficult.
Nevertheless, if adding a guard time is beneficial, we can state that the optimal guard time T * G must satisfy R UB (T * G ) ≥ R LB where R UB and R LB are given by Equations (10) and (16) and hence T * G T ≤ R UB R LB − 1. In Figure 12, we draw the obtained upper bound on T * G . Naturally, it increases as the SNR increases since the relative difference between the bounds increases as the SNR increases.

Summary and Conclusions
In this work, we studied the maximal achievable rates and the available degrees of freedom when transmitting T-seconds time-limited codewords over a W-Hz band-limited AWGN channel. We made use of the prolate spheroidal wave functions to switch to discrete time and then applied the results by Polyanskiy. We derived the upper bound by ignoring the interference due to other codewords, and we derived lower bounds by deriving upper bounds on the inter-codeword interference. The derived bounds were found to be tight for for low values of SNR/W (for example W = 1 KHz and SNR = 30 dB or equivalently W = 100 KHz and SNR = 50 dB). However, the gap between the bounds increases as this SNR/W ratio increases.
When it comes to the available degrees of freedom, the numerical results showed that the potential decrease from 2WT is upper bounded by a logarithmic function of 2WT and hence the relative reduction is asymptotically negligible.
Based on the results of this work, one can approximate the achievable data rates by (θ+1) 2 log 2 2 e and L is in the range c − 1.35 log 2 (c) + 3.25 ≤ L ≤ c. This approximation is guaranteed to be between the derived bounds, and hence is a "good" one for low values of SNR/W. By a change of variable ν = T f /2W, α k,h c,l,m = j l−m ϕ c,l (ν − hT), ϕ c,m (ν)rect ν T e j2πk2Wν = j l−m ϕ c,l (ν), Dϕ c,m (ν + hT)e j2πk2W(ν+hT) = j l−m λ c,m λ c,l