Capacity-Achieving Input Distributions of Additive Vector Gaussian Noise Channels: Even-Moment Constraints and Unbounded or Compact Support

Abstract

We investigate the support of a capacity-achieving input to a vector-valued Gaussian noise channel. The input is subjected to a radial even-moment constraint and is either allowed to take any value in ${\mathbb{R}}^n$ or is restricted to a given compact subset of ${\mathbb{R}}^n$. It is shown that the support of the capacity-achieving distribution is composed of a countable union of submanifolds, each with a dimension of $n-1$ or less. When the input is restricted to a compact subset of ${\mathbb{R}}^n$, this union is finite. Finally, the support of the capacity-achieving distribution is shown to have Lebesgue measure 0 and to be nowhere dense in ${\mathbb{R}}^n$.

1. Introduction

In this paper, we consider the support of the capacity-achieving input to a vector-valued channel that is subject to additive non-degenerate Gaussian noise. Vector-valued channels are used in a variety of applications, including the complex-valued inputs and outputs of quadrature channels, which have alternate representations as two-dimensional real vectors. Larger antenna arrays enable Multiple-Input Multiple-Output (MIMO) channels, which have inputs of $n\ge 1$ complex components. Additionally, noise with memory can be expressed by correlated noise components in a vector-valued channel.

Throughout the paper, the average input power is bounded to limit the consumption of environmental, battery, and monetary resources. Since the output of the amplifiers used in transmitters is severely distorted when the input is too large [1,2,3] and signals that are too small can be challenging to produce, it is also of practical interest to restrict the input to an arbitrary compact set.

There has been appreciable prior effort dedicated to understanding the capacity-achieving input to vector-valued channels subject to average power constraints and restrictions to compact sets. Nevertheless, there are significant technical challenges in working with vector-valued inputs. Therefore, much of the work to this point has been limited to either one-dimensional channels [4] or spherically symmetric channels [5,6,7,8], where the latter case ensures that the capacity-achieving distribution can be expressed as a univariate function of the radius. However, this restriction limits the scope of study to channels in which the input is constrained to a ball and the noise components are independent and identically distributed. In this paper, the only assumption made on the Gaussian noise distribution is that it is non-degenerate. We consider both cases, those in which inputs are restricted to arbitrary compact sets and those in which inputs are allowed to take any value in ${\mathbb{R}}^n$.

The power of a vector-valued signal is equivalent to the second moment of its Euclidean norm. A constraint on the fourth moment then has the practical interpretation of limiting the second moment of the instantaneous power. Furthermore, imposing a moment constraint of order $2k$ ensures that the tails of the input distribution decay at least as quickly as a degree $2k$ monomial. Therefore, increasing the even-moment constraint penalizes large inputs without imposing a strict cutoff. This motivates us to generalize the average power constraint by limiting some even moment of the input’s Euclidean norm.

The results in this paper apply to any combination of input constraints described above, except for the special case where the input is allowed to take any value in ${\mathbb{R}}^n$ and is subject to a second-moment constraint. This case reduces to a classical result in which the capacity-achieving distribution is known to be Gaussian. For all other cases, we show that the capacity-achieving distribution is contained in a countable union of i-dimensional submanifolds, where i ranges $\{0,\dots ,n-1\}$. Furthermore, this union is finite when the input is restricted to a compact set. We then show that the support of the capacity-achieving distribution is a nowhere dense set with Lebesgue measure 0.

The paper is organized as follows. We first give a review of prior work in Section 2. Section 3.1, Section 3.2 and Section 3.3 provide intermediary results prior to the main results in Section 3.4. Section 4 concludes the paper.

2. Prior Work

Dating back to Shannon’s work in [9], much of the research on continuous channels has focused on average power (equivalently, second-moment) constraints on the input. A transmitter’s inability to produce arbitrarily large powers then led to the consideration of additional peak power constraints, modeled by restricting the input almost surely to compact sets.

The first major result on amplitude-constrained channels considers a scalar Additive White Gaussian Noise (AWGN) model, both with and without a variance constraint [4]. In each case, the support of the capacity-achieving distribution has a finite number of points.

The use of the Identity Theorem for functions of a single complex variable is key to the argument of [4] and many papers that follow. The theorem can be applied to any univariate analytic function that has an accumulation point of zeros. By contrast, the Identity Theorem in n complex dimensions requires an analytic function with an open set of zeros in ${\mathbb{C}}^n$. Therefore, to apply the Identity Theorem directly for $n>1$, a random vector with support containing an open subset of ${\mathbb{C}}^n$ must be considered. It was suspected by some authors that, since ${\mathbb{R}}^n$ is not open in ${\mathbb{C}}^n$, no topological assumption on the support of the capacity-achieving distribution would be sufficient for this purpose [5,10,11]. Therefore, many papers restrict their models to ones that maintain spherical symmetry so that the capacity-achieving distribution can be expressed as a one-dimensional function of radius (e.g., Refs. [5,6,7,8]).

In [5] and [6], the results of [4] are extended to multivariate spherically symmetric channels, in n dimensions and 2 dimensions, respectively. In both papers, the inputs are subject to average and peak radial constraints. It is shown that it is optimal to concentrate the input on a finite number of concentric shells.

In [7] and [8], the number and positioning of optimal concentric shells under a peak radial constraint is studied. In [7], the least restrictive amplitude constraint for which the optimal distribution is concentrated on a single sphere is found. In [8], it is shown that the number of shells grows at most quadratically in the amplitude constraint. A similar result is found for $n=1$ under an additional average power constraint.

While much of the prior work has focused on spherically symmetric channels, some research has considered spherically asymmetric channels. The case of inputs constrained to arbitrary compact sets and subject to a finite number of quadratic cost constraints as well as non-degenerate multivariate Gaussian noise is considered in [12]. It is concluded that the support of the capacity-achieving distribution must be “sparse”—that is, there must exist a not identically zero analytic function that is 0 on the support of the capacity-achieving distribution. Assuming otherwise leads to a contradiction by the n-dimensional Identity Theorem and Fourier analysis. These results, while quite general, do not consider either inputs of unbounded support or inputs subject to higher-moment constraints. Furthermore, outside of the special cases of $n=1$ and spherically symmetric channels, they do not explore a characterization of sparse sets in ${\mathbb{R}}^n$.

In [13], MIMO channels with inputs that are restricted to compact sets, yet have no average power constraints, are considered. Using the Real-Analytic Identity Theorem and steps similar to [4], it is determined that the support of the optimal input distribution is nowhere dense in ${\mathbb{R}}^n$ and has Lebesgue measure 0. For the case considered in [12] that coincides with this setup, [13] gives an instance of sparsity in terms of subsets of ${\mathbb{R}}^n$, rather than analytic functions.

There has also been work dedicated to generalizing the classic quadratic average cost constraint. In [14], a scalar channel with the input subject to a combination of even-moment constraints and restrictions to compact or non-negative subsets of $\mathbb{R}$ is studied. It is shown that, in most of the cases considered, the support has a finite number of points.

In [15], a complex-valued non-dispersive optical channel is considered, where the input is subject to an average cost that grows super-quadratically in radius, a peak constraint, or both. The noise is taken to be circularly symmetric and, under these conditions, so is the optimal input. The number of concentric circles composing the support of the distribution is shown to be finite.

In this paper, we study an n-dimensional channel subject to non-degenerate Gaussian noise. The input can either take any value in ${\mathbb{R}}^n$ or is restricted to a compact subset of ${\mathbb{R}}^n$, and its norm is subject to even-moment constraints. The noise need not be spherically symmetric.

This paper gives a characterization of the capacity-achieving distribution to spherically asymmetric channels under peak and average power constraints that improves on prior work in three respects. Firstly, when our cases overlap with [12], our characterization of the capacity-achieving distribution is more detailed than the notion of sparsity used there. Secondly, our results apply to multivariate channels with inputs subject to even-moment constraints greater than 2. Thirdly, we consider both inputs that are restricted to compact sets and those that are allowed to take any value in ${\mathbb{R}}^n$.

3. Results

In this section, we consider ${\mathbb{R}}^n$-valued inputs subject to additive non-degenerate multivariate Gaussian noise. In Section 3.1, the capacity-achieving distribution, $F^{*}$, is formulated as the objective of an optimization problem; its support is then framed in terms of the zero set of a certain real-analytic function, which is dependent on $F^{*}$ and referred to as $s(·;F^{*})$. Section 3.2 finds an equivalent expression for $s(·;F^{*})$, which is an intermediary step to showing in Section 3.3 that $s(·;F^{*})$ is non-constant. Section 3.4 uses the result that $s(·;F^{*})$ is non-constant to show that the support of the capacity-achieving distribution is contained in a countable union of submanifolds of dimensions in the range $\{0,\dots ,n-1\}$. This union is finite when the input is constrained to a compact subset of ${\mathbb{R}}^n$. It is then shown that the support of the capacity-achieving input has Lebesgue measure 0 and is nowhere dense in ${\mathbb{R}}^n$.

Appendix A is dedicated to showing the convexity and compactness of the optimization space used in Section 3.1. Appendix B establishes a pointwise characterization of $supp\left(F^{*}\right)$, which justifies the definition of $s(·,F^{*})$. Appendix C provides integrability results, which are used throughout the paper. Appendix D shows that the objective functional is weak continuous, strictly concave, and weak differentiable on the optimization space. Appendix E shows that $s(·,F^{*})$ has an analytic extension to ${\mathbb{C}}^n$. Finally, Appendix F supports Section 3.3 by finding bounds for certain functions.

As a first step towards defining the set of feasible input distributions, let $\mathcal{F}\left({\mathbb{R}}^n\right)$ be the set of finite Borel measures on ${\mathbb{R}}^n$. Note that $\mathcal{F}\left({\mathbb{R}}^n\right)$ is contained in the set of finite signed Borel measures on ${\mathbb{R}}^n$, which has an intrinsic vector space structure and can be equipped with a norm [16]. Since $\mathcal{F}\left({\mathbb{R}}^n\right)$ lies within a normed vector space, the convexity and compactness of its subsets can be discussed.

The possibility that the transmitter is unable to produce arbitrary signals in ${\mathbb{R}}^n$ is modeled by restricting the input to an alphabet $\mathcal{A}\subseteq {\mathbb{R}}^n$. Denote the set of distributions for which the associated random variable is almost surely in $\mathcal{A}$ by

$${\mathcal{F}}_n\left(\mathcal{A}\right)\triangleq \{F\in \mathcal{F}\left({\mathbb{R}}^n\right)\mid F\left(\mathcal{A}\right)=1\}.$$

(1)

Two cases for $\mathcal{A}$ are considered:

• $\mathcal{A}={\mathbb{R}}^n$;
• $\mathcal{A}⊊{\mathbb{R}}^n$ is compact.

In addition to the restriction to $\mathcal{A}$, a radial even-moment constraint is associated with the input. For $k\in {\mathbb{Z}}_{>0}$, the input must belong to the set

$${\mathcal{P}}_n(\mathcal{A},k,a)\triangleq \{F\in {\mathcal{F}}_n\left(\mathcal{A}\right)\mid {\mathbb{E}}_{\mathit{X}\sim F}{[\parallel \mathit{X}\parallel }^{2k}] \le a\}.$$

(2)

The resulting channel model, with input $\mathit{X}\sim F\in {\mathcal{P}}_n(\mathcal{A},k,a)$, is

$$\mathit{Y}=\mathit{A}\mathit{X}+\mathit{N},$$

(3)

where $\mathit{Y}$ and $\mathit{N}\sim \mathcal{N}(0,\Sigma )$ are output and noise, respectively, and $\mathit{A}$ is an invertible matrix known to the transmitter and receiver. It is assumed that the noise covariance matrix $\Sigma$ is positive-definite.

We will simplify the analysis of (3) by showing that no generality is lost in assuming that $\mathit{A}={\mathit{I}}_n$, the n-dimensional identity matrix, and $\Sigma$ is diagonal. Since $\Sigma$ is positive-definite and $\mathit{A}$ is invertible, the positive-definite matrix ${\mathit{A}}^{-1}\Sigma {\left({\mathit{A}}^{-1}\right)}^T$ can be diagonalized by the orthogonal matrix $\mathit{Q}$.

Now, multiplying the output $\mathit{Y}$ in (3) by $\mathit{Q}{\mathit{A}}^{-1}$, the receiver obtains

$$\begin{array}{cc}\hfill \tilde{\mathit{Y}}& \triangleq \mathit{Q}{\mathit{A}}^{-1}\mathit{Y}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill & =\tilde{\mathit{X}}+\tilde{\mathit{N}},\hfill \end{array}$$

(5)

where $\tilde{\mathit{X}}=\mathit{Q}\mathit{X}$ and the covariance matrix of $\tilde{\mathit{N}}=\mathit{Q}{\mathit{A}}^{-1}\mathit{N}$ is diagonal. Since $\mathit{Q}{\mathit{A}}^{-1}$ is invertible, $I(\mathit{X};\mathit{Y})=I(\tilde{\mathit{X}};\tilde{\mathit{Y}})$. Furthermore, since $\mathit{Q}$ is orthogonal,

$$\mathbb{E}[\parallel \tilde{\mathit{X}}{\parallel }^{2k}]{=\mathbb{E}[\parallel \mathit{X}\parallel }^{2k}]$$

(6)

and the set $\left\{\mathit{Q}\mathit{x}\right|\mathit{x}\in \mathcal{A}\}$ is merely a rotated version of $\mathcal{A}$. Hence, no generality is lost by dropping the $“\sim ”$ and adopting the following channel model for the remainder of the paper:

$$\mathit{Y}=\mathit{X}+\mathit{N},$$

(7)

where $\mathit{N}\sim \mathcal{N}(0,\Sigma )$ and $\Sigma$ is diagonal with entries $0<{\sigma }_1^2\le {\sigma }_2^2\le \dots \le {\sigma }_n^2$. The density of $\mathit{N}$ is denoted by $p_{\mathit{N}}(·)$.

3.1. Optimization Problem

By Theorem 3.6.2 of [17], the capacity of the channel in (7) is given by the optimization problem

$$C=\underset{\mathit{X}\sim F\in {\mathcal{P}}_n(\mathcal{A},k,a)}{sup}I(\mathit{X};\mathit{Y})=\underset{{\mathbb{E}}_{\mathit{X}\sim F}{[\parallel \mathit{X}\parallel }^{2k}]\le a}{\underset{F\in {\mathcal{F}}_n\left(\mathcal{A}\right)}{sup}}I(\mathit{X};\mathit{Y}).$$

(8)

Since the relationship between $\mathit{Y}$, $\mathit{X}$, and $\mathit{N}$ is known, the mutual information is a function of the distribution of $\mathit{X}$ alone. Thus, the mutual information induced between $\mathit{X}\sim F$ and $\mathit{Y}$ will be denoted by $I\left(F\right)$. Similarly, we express the even-moment constraint in terms of a functional $g_k:{\mathcal{F}}_n\left(\mathcal{A}\right)\to \mathbb{R}\cup \{\infty \}$ given by

$$g_k\left(F\right)\triangleq {\int }_{\mathcal{A}}{\parallel \mathit{x}\parallel }^{2k}dF\left(\mathit{x}\right)-a,$$

(9)

where $g_k\left(F\right)\le 0$ is equivalent to ${\mathbb{E}}_{\mathit{X}\sim F}{[\parallel \mathit{X}\parallel }^{2k}] \le a$. Rewriting (8) in terms of $I(·)$ and $g_k(·)$ yields

$$C=\underset{F\in {\mathcal{P}}_n(\mathcal{A},k,a)}{sup}I\left(F\right)=\underset{g_k\left(F\right)\le 0}{\underset{F\in {\mathcal{F}}_n\left(\mathcal{A}\right)}{sup}}I\left(F\right).$$

(10)

Much of the appendix is dedicated to understanding properties of the problem presented in (10). It is shown in Theorem A1 that ${\mathcal{P}}_n(\mathcal{A},k,a)$ is convex and compact. Furthermore, by Theorems A3 and A4, $I(·)$ is a weakly continuous and strictly concave function on ${\mathcal{P}}_n(\mathcal{A},k,a)$. Therefore, the supremum is achieved by a unique input distribution $F^{*}\in {\mathcal{P}}_n(\mathcal{A},k,a)$ (see, e.g., Appendix C of [14])—that is,

$$C=\underset{F\in {\mathcal{P}}_n(\mathcal{A},k,a)}{max}I\left(F\right)=\underset{g_k\left(F\right)\le 0}{\underset{F\in {\mathcal{F}}_n\left(\mathcal{A}\right)}{max}}I\left(F\right)=I\left(F^{*}\right).$$

(11)

We use the notation ${\mathit{X}}^{*}$ to describe a capacity-achieving input directly (i.e., ${\mathit{X}}^{*}\sim F^{*}$).

Before proceeding, we require some definitions and notations. In the first definition, and throughout the paper, for $\mathit{x}\in {\mathbb{R}}^n$ and $r>0$, we denote the ball of radius r centered at $\mathit{x}$ by $B_r\left(\mathit{x}\right)\subseteq {\mathbb{R}}^n$. We will denote the closure of $B_r\left(\mathit{x}\right)$ by $\overline{B_r}\left(\mathit{x}\right)$. The output density induced by an input $\mathit{X}\sim F$ is denoted $p(·;F)$.

Definition 1. 

Let $\mathit{V}$ be a random variable with alphabet $\mathcal{A}\subseteq {\mathbb{R}}^n$. Then, the support of $\mathit{V}$ is the set given by

$$supp\left(\mathit{V}\right)\triangleq \{\mathit{x}\in \mathcal{A}\mid \forall r>0,P\{\mathit{V}\in B_r\left(\mathit{x}\right)\}>0\}.$$

(12)

If $\mathit{V}$ has distribution $F_{\mathit{V}}$, we may alternatively refer to $supp\left(F_{\mathit{V}}\right)\triangleq supp\left(\mathit{V}\right)$.

Definition 2. 

For $F\in {\mathcal{F}}_n\left(\mathcal{A}\right)$, the output differential entropy is given by

$$h_{\mathit{Y}}\left(F\right)\triangleq -{\int }_{{\mathbb{R}}^n}p(\mathit{y};F)lnp(\mathit{y};F)d\mathit{y}$$

(13)

and the marginal entropy density at $\mathit{x}\in {\mathbb{R}}^n$ is given by

$$h(\mathit{x};F)\triangleq -{\int }_{{\mathbb{R}}^n}{p}_{\mathit{N}}(\mathit{y}-\mathit{x})lnp(\mathit{y};F)d\mathit{y},$$

(14)

whenever the integrals exist.

The relationship between the differential entropy and marginal entropy density can be seen as follows. For any $b>0$ and $F_1,F_2\in {\mathcal{P}}_n(\mathcal{A},k,b)$, we have that

$$\begin{array}{cc}\hfill {\int }_{\mathcal{A}}h(\mathit{x};F_1)d{F}_2\left(\mathit{x}\right)& =-{\int }_{\mathcal{A}}{\int }_{{\mathbb{R}}^n}{p}_{\mathit{N}}(\mathit{y}-\mathit{x})lnp(\mathit{y};F_1)d\mathit{y}d{F}_2\left(\mathit{x}\right)\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill & =-{\int }_{{\mathbb{R}}^n}{\int }_{\mathcal{A}}{p}_{\mathit{N}}(\mathit{y}-\mathit{x})d{F}_2\left(\mathit{x}\right)lnp(\mathit{y};F_1)d\mathit{y}\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill & =-{\int }_{{\mathbb{R}}^n}p(\mathit{y};F_2)lnp(\mathit{y};F_1)d\mathit{y}.\hfill \end{array}$$

(17)

The equivalence of (15)–(17) is due to Fubini–Tonelli [18] and Lemma A4. If $F=F_1=F_2$, then

$${\int }_{\mathcal{A}}h(\mathit{x};F)dF\left(\mathit{x}\right)=h_{\mathit{Y}}\left(F\right).$$

(18)

Lastly, define

$$\begin{array}{cc}\hfill {\mathcal{Q}}_n(\mathcal{A},k,a)& \triangleq \bigcup _{b\ge a}{\mathcal{P}}_n(\mathcal{A},k,b)\hfill \end{array}$$

(19)

and let $s(·;F^{*}):{\mathbb{R}}^n\to \mathbb{R}$ be given by

$$\begin{array}{cc}\hfill s(\mathit{x};F^{*})& \triangleq {\gamma (\parallel \mathit{x}\parallel }^{2k}-a)+C+h\left(\mathit{N}\right)-h(\mathit{x};F^{*})\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill & ={\gamma (\parallel \mathit{x}\parallel }^{2k}-a)+C+h\left(\mathit{N}\right)+{\int }_{{\mathbb{R}}^n}{p}_{\mathit{N}}(\mathit{y}-\mathit{x})lnp(\mathit{y};F^{*})d\mathit{y}.\hfill \end{array}$$

(21)

Since $F^{*}\in {\mathcal{P}}_n(\mathcal{A},k,a)$, we have that $h(\mathit{x};F^{*})$ is finite for all $\mathit{x}\in {\mathbb{R}}^n$ and conclude that $s(\mathit{x};F^{*})$ is also finite for all $\mathit{x}\in {\mathbb{R}}^n$. Furthermore, by Lemma A8, $s(·;F^{*})$ can be extended to a complex analytic function for all $\mathit{z}\in {\mathbb{C}}^n$; hence, it is continuous.

The remainder of Section 3.1 consists of two steps:

• We show that $F^{*}$ solves (11) if and only if there exists $\gamma \ge 0$ such that

  $${\int }_{{\mathbb{R}}^n}h(\mathit{x};F^{*})dF\left(\mathit{x}\right)-C-h\left(\mathit{N}\right)-\gamma g_k\left(F\right)\le 0,$$

  (22)

  for all $F\in {\mathcal{Q}}_n(\mathcal{A},k,a)$.
• We show that, for the choice of $\gamma$ in (22), the inequality in (22) is equivalent to the condition that for all $x\in \mathcal{A}$,

  $$h(\mathit{x};F^{*})\le {\gamma (\parallel \mathit{x}\parallel }^{2k}-a)+C+h\left(\mathit{N}\right),$$

  (23)

  and if $\mathit{x}\in supp\left(F^{*}\right)$, then

  $$h(\mathit{x};F^{*})={\gamma (\parallel \mathit{x}\parallel }^{2k}-a)+C+h\left(\mathit{N}\right).$$

  (24)
  For $\mathit{x}\in \mathcal{A}$, (24) is satisfied if and only if $s(\mathit{x};F^{*})=0$.

To establish (22), we will first use a Lagrange multiplier to reformulate (11) as an unconstrained problem over ${\mathcal{Q}}_n(\mathcal{A},k,a)$. We will then obtain (22) by taking the weak derivative of the resulting objective functional and applying a Karush–Kuhn–Tucker condition for optimality. We choose to work in the space ${\mathcal{Q}}_n(\mathcal{A},k,a)$ since, when $\mathcal{A}={\mathbb{R}}^n$, the functionals $I(·)$ and $g_k(·)$ are not weakly differentiable on the larger space ${\mathcal{F}}_n\left(\mathcal{A}\right)={\mathcal{F}}_n\left({\mathbb{R}}^n\right)$.

Since $\{F\in {\mathcal{Q}}_n(\mathcal{A},k,a)|g_k\left(F\right)\le 0\}={\mathcal{P}}_n(\mathcal{A},k,a)$, (11) can equivalently be written as

$$C=\underset{g_k\left(F\right)\le 0}{\underset{F\in {\mathcal{Q}}_n(\mathcal{A},k,a)}{max}}I\left(F\right)=I\left(F^{*}\right),$$

(25)

where $F^{*}$ is the same as in (11). By Theorem A5, $g_k(·)$ is convex. Moreover, letting $F_s\in {\mathcal{Q}}_n(\mathcal{A},k,a)$ be a Heaviside step function at $\mathbf{0}\in {\mathbb{R}}^n$, $F_s$ is an interior point of the feasible region since $g_k\left(F_s\right)=-a<0$. Since ${\mathcal{Q}}_n(\mathcal{A},k,a)$ is convex by Lemma A1, there exists $\gamma \ge 0$ such that

$$C=\underset{F\in {\mathcal{Q}}_n(\mathcal{A},k,a)}{max}{J}_{\gamma }\left(F\right)=J_{\gamma }\left(F^{*}\right),$$

(26)

where

$$J_{\gamma }\left(F\right)\triangleq I\left(F\right)-\gamma g_k\left(F\right),$$

(27)

and $\gamma g_k\left(F^{*}\right)=0$ (see e.g., Appendix C of [14]). Furthermore, for an arbitrary $b\ge a$, $F^{*}\in {\mathcal{P}}_n(\mathcal{A},k,b)\subseteq {\mathcal{Q}}_n(\mathcal{A},k,a)$. Therefore, for this choice of $\gamma$, we also have

$$C=\underset{F\in {\mathcal{P}}_n(\mathcal{A},k,b)}{max}{J}_{\gamma }\left(F\right)=J_{\gamma }\left(F^{*}\right).$$

(28)

By Theorems A5 and A6, $J_{\gamma }(·)$ has a weak derivative at $F^{*}$ in the direction of any $F\in {\mathcal{P}}_n(\mathcal{A},k,b)$ given by

$$\begin{array}{cc}\hfill J_{\gamma }^{\prime }(F^{*},F)& =I^{\prime }(F^{*},F)-\gamma g_k^{\prime }(F^{*},F)\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill & ={\int }_{\mathcal{A}}h(\mathit{x};F^{*})dF\left(\mathit{x}\right)-h_{\mathit{Y}}\left(F^{*}\right)-\gamma (g_k\left(F\right)-g_k\left(F^{*}\right))\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill & ={\int }_{\mathcal{A}}h(\mathit{x};F^{*})dF\left(\mathit{x}\right)-h_{\mathit{Y}}\left(F^{*}\right)-\gamma g_k\left(F\right),\hfill \end{array}$$

(31)

where (31) is due to $\gamma g_k\left(F^{*}\right)=0$. Substituting $C=h_{\mathit{Y}}\left(F^{*}\right)-h\left(\mathit{N}\right)$ gives

$$J_{\gamma }^{\prime }(F^{*},F)={\int }_{\mathcal{A}}h(\mathit{x};F^{*})dF\left(\mathit{x}\right)-C-h\left(\mathit{N}\right)-\gamma g_k\left(F\right),$$

(32)

where the differential entropy of the noise $h\left(\mathit{N}\right)=\frac{1}{2}ln\left|2\pi e\Sigma \right|$ is finite since $\Sigma$ is positive definite.

Now, $J_{\gamma }(·)$ is the difference between a strictly concave function (see Theorem A4) and a convex function (due to Theorem A5 and the non-negativity of $\gamma$). Therefore, $J_{\gamma }(·)$ is strictly concave and $F^{*}$ is optimal if and only if, for all $F\in {\mathcal{P}}_n(\mathcal{A},k,b)$,

$$J_{\gamma }^{\prime }(F^{*},F)={\int }_{\mathcal{A}}h(\mathit{x};F^{*})dF\left(\mathit{x}\right)-C-h\left(\mathit{N}\right)-\gamma g_k\left(F\right)\le 0.$$

(33)

However, $b\ge a$ is arbitrary and each $F\in {\mathcal{Q}}_n(\mathcal{A},k,a)$ satisfies $F\in {\mathcal{P}}_n(\mathcal{A},k,b)$ for some $b\ge a$. Therefore, $F^{*}$ is optimal if and only if, for all $F\in {\mathcal{Q}}_n(\mathcal{A},k,a)$,

$$J_{\gamma }^{\prime }(F^{*},F)={\int }_{\mathcal{A}}h(\mathit{x};F^{*})dF\left(\mathit{x}\right)-C-h\left(\mathit{N}\right)-\gamma g_k\left(F\right)\le 0,$$

(34)

which is the statement we sought to show in (22).

The condition (34) is on the capacity-achieving distribution $F^{*}$ itself. Since our objective is to characterize $supp\left(F^{*}\right)$, we find an equivalent condition to (34) to describe the behavior of $F^{*}$ at individual points in the input alphabet $\mathcal{A}$. Thus, by (34) and Theorem A2, for all $\mathit{x}\in \mathcal{A}\subseteq {\mathbb{R}}^n$,

$$s(\mathit{x};F^{*})={\gamma (\parallel \mathit{x}\parallel }^{2k}-a)+C+h\left(\mathit{N}\right)+{\int }_{{\mathbb{R}}^n}{p}_{\mathit{N}}(\mathit{y}-\mathit{x})lnp(\mathit{y};F^{*})d\mathit{y}\ge 0,$$

(35)

and if $\mathit{x}\in supp\left(F^{*}\right)$, then

$$\begin{array}{c}\hfill s(\mathit{x};F^{*})={\gamma (\parallel \mathit{x}\parallel }^{2k}-a)+C+h\left(\mathit{N}\right)+{\int }_{{\mathbb{R}}^n}{p}_{\mathit{N}}(\mathit{y}-\mathit{x})lnp(\mathit{y};F^{*})d\mathit{y}=0.\end{array}$$

(36)

The rest of Section 3 is dedicated to exploiting the relationship

$$supp\left(F^{*}\right)\subseteq \mathcal{Z}(s;F^{*})\cap \mathcal{A},$$

(37)

where $\mathcal{Z}(s;F^{*})$ is the zero set of $s(·,F^{*})$.

3.2. Hilbert Space and Hermite Polynomial Representation

In this subsection, an equivalent expression for (36) is found by viewing the integral as an inner product in a Hilbert space and writing $lnp(·;F^{*})$ in terms of a Hermite polynomial basis for that space. Hermite polynomial bases are well-suited to analysis of Gaussian noise channels and they have been used in a number of information-theoretic papers (see, e.g., Refs. [14,19]).

Consider the Hilbert space

$$L_{p_{\mathit{N}}}^2\left({\mathbb{R}}^n\right)\triangleq \{\xi :{\mathbb{R}}^n\to \mathbb{R}|{\int }_{{\mathbb{R}}^n}{\xi }^2\left(\mathit{x}\right)p_{\mathit{N}}\left(\mathit{x}\right)d\mathit{x}<\infty \},$$

(38)

equipped with inner product

$${⟨\xi ,\psi ⟩}_{L_{p_{\mathit{N}}}^2\left({\mathbb{R}}^n\right)}\triangleq {\int }_{{\mathbb{R}}^n}\xi \left(\mathit{x}\right)\psi \left(\mathit{x}\right)p_{\mathit{N}}\left(\mathit{x}\right)d\mathit{x}.$$

(39)

The inner product’s subscript is omitted when the space can be inferred.

Since the components of $\mathit{N}$ are independent, with $N_i$ having variance ${\sigma }_i^2>0$, the density of $\mathit{N}$ factors into

$$\begin{array}{cc}\hfill p_{\mathit{N}}\left(\mathit{y}\right)& =\prod _{i=1}^n{p}_{N_i}\left(y_i\right)\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill & =\prod _{i=1}^n\frac{1}{\sqrt{2\pi {\sigma }_i^2}}{e}^{-\frac{1}{2{\sigma }_i^2}{y}_i^2}.\hfill \end{array}$$

(41)

We will construct an orthogonal basis for $L_{p_{\mathit{N}}}^2\left({\mathbb{R}}^n\right)$ from orthogonal bases for the spaces $L_{p_{N_i}}^2\left(\mathbb{R}\right)$.

First, with $Z\sim \mathcal{N}(0,1)$, an orthogonal basis for $L_{p_Z}^2\left(\mathbb{R}\right)$ is given by the Hermite polynomials ${\left\{H_m\right\}}_{m=0}^{\infty }$ [20], which are defined through the generating function

$$e^{-\frac{1}{2}{x}^2+yx}=\sum _{m=0}^{\infty }{H}_m\left(y\right)\frac{x^m}{m!}.$$

(42)

For any $m\in {\mathbb{Z}}_{\ge 0}$, the mth Hermite polynomial has degree m and a positive leading coefficient. Next, for each $i\in \{1,\dots ,n\}$ and $m_i\in {\mathbb{Z}}_{\ge 0}$, define the stretched Hermite polynomials

$$H_{m_i}(y;{\sigma }_i^2)\triangleq \frac{1}{{\sigma }_i^{m_i}}{H}_m\left(\frac{y}{\sqrt{{\sigma }_i^2}}\right).$$

(43)

The inner product of $L_{p_{N_i}}^2\left(\mathbb{R}\right)$ is related to that of $L_{p_Z}^2\left(\mathbb{R}\right)$ by

$$\begin{array}{cc}\hfill {⟨\xi ,\psi ⟩}_{L_{p_{N_i}}^2\left(\mathbb{R}\right)}& ={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi {\sigma }_i^2}}{e}^{-\frac{1}{2{\sigma }_i^2}{y}^2}\xi \left(y\right)\psi \left(y\right)dy\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill & ={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }}{e}^{-\frac{1}{2}{u}^2}\xi \left(\sqrt{{\sigma }_i^2}u\right)\psi \left(\sqrt{{\sigma }_i^2}u\right)du\hfill \end{array}$$

(45)

$$\begin{array}{cc}\hfill & ={⟨{\xi }_{\sqrt{{\sigma }_i^2}},{\psi }_{\sqrt{{\sigma }_i^2}}⟩}_{L_{p_Z}^2\left(\mathbb{R}\right)},\hfill \end{array}$$

(46)

where ${\xi }_{\sqrt{{\sigma }_i^2}}$ and ${\psi }_{\sqrt{{\sigma }_i^2}}$ are versions of $\xi$ and $\psi$ that have been stretched horizontally by a factor of $1/\sqrt{{\sigma }_i^2}$. Substituting any $H_{m_i}(·,{\sigma }_i^2)$ and $H_{l_i}(·,{\sigma }_i^2)$ into (44) shows that the set ${\left\{H_{m_i}(·,{\sigma }_i^2)\right\}}_{m_i=0}^{\infty }$ is orthogonal. Furthermore, if there was a non-zero function (in an $L_{p_{N_i}}^2\left(\mathbb{R}\right)$ sense) that had a zero $L_{p_{N_i}}^2\left(\mathbb{R}\right)$ inner product with $H_{m_i}(·,{\sigma }_i^2)$ for each $m_i$, then a stretched version of this function would also have a zero $L_{p_Z}^2\left(\mathbb{R}\right)$ inner product with $H_m$ for each m. This would contradict the completeness of the Hermite polynomials in $L_{p_Z}^2\left(\mathbb{R}\right)$; hence, ${\left\{H_{m_i}(·,{\sigma }_i^2)\right\}}_{m_i=0}^{\infty }$ forms a basis for $L_{p_{N_i}}^2\left(\mathbb{R}\right)$. Lastly, the stretched Hermite polynomials ${\left\{H_{m_i}(·,{\sigma }_i^2)\right\}}_{m_i=0}^{\infty }$ have the generating function

$$e^{-\frac{1}{2{\sigma }_i^2}{x}^2+\frac{1}{{\sigma }_i^2}yx}=\sum _{m_i=0}^{\infty }{H}_{m_i}(y;{\sigma }_i^2)\frac{x^{m_i}}{m_i!}.$$

(47)

Now, $L_{p_{\mathit{N}}}^2\left({\mathbb{R}}^n\right)$ is isomorphic to the tensor product of the $L_{p_{N_i}}^2\left(\mathbb{R}\right)$ spaces, $i\in \{1,\dots ,n\}$; consequently, $\left\{H_{\mathit{m}}(·;{\mathit{\sigma }}^{\mathbf{2}})\right|\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n\}$ forms an orthonormal basis for $L_{p_{\mathit{N}}}^2\left({\mathbb{R}}^n\right)$ [21], where

$$H_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})\triangleq \prod _{i=1}^n{H}_{m_i}(y_i;{\sigma }_i^2).$$

(48)

Since, by Lemma A2, $lnp(·;F^{*})\in L_{p_{\mathit{N}}}^2\left({\mathbb{R}}^n\right)$, there exist constants ${\left\{c_{\mathbf{i}}^{\prime }\right\}}_{\mathbf{i}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}$ for which

$$lnp(\mathit{y};F^{*})=\sum _{\mathbf{i}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}{c}_{\mathbf{i}}^{\prime }{H}_{\mathbf{i}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}}),$$

(49)

where equality is in an $L_{p_{\mathit{N}}}^2\left({\mathbb{R}}^n\right)$ sense. Then, substituting (47), and using the notations

$$\mathit{m}!\triangleq m_1!\cdots m_n!,$$

(50)

and

$${\mathit{x}}^{\mathit{m}}\triangleq x_1^{m_1}\cdots x_n^{m_n}$$

(51)

for $\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n$ and $\mathit{x}\in {\mathbb{R}}^n$, we write

$$\begin{array}{cc}\hfill e^{-\frac{1}{2}{\mathit{x}}^T{\Sigma }^{-1}\mathit{x}+{\mathit{x}}^T{\Sigma }^{-1}\mathit{y}}& =\prod _{i=1}^n{e}^{-\frac{1}{2{\sigma }_i^2}{x}_i^2+\frac{1}{{\sigma }_i^2}{y}_i{x}_i}\hfill \end{array}$$

(52)

$$\begin{array}{cc}\hfill & =\prod _{i=1}^n\sum _{m_i=0}^{\infty }{H}_{m_i}(y_i;{\sigma }_i^2)\frac{x_i^{m_i}}{m_i!}\hfill \end{array}$$

(53)

$$\begin{array}{cc}\hfill & =\sum _{\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}\prod _{i=1}^n{H}_{m_i}(y_i;{\sigma }_i^2)\frac{x_i^{m_i}}{m_i!}\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill & =\sum _{\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}{H}_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})\frac{{\mathit{x}}^{\mathit{m}}}{\mathit{m}!}.\hfill \end{array}$$

(55)

Substituting (49) and (55) into the integral term in (36) yields

$$\begin{array}{cc}\hfill {\int }_{{\mathbb{R}}^n}{p}_{\mathit{N}}(\mathit{y}-\mathit{x})lnp(\mathit{y};F^{*})d\mathit{y}& ={\int }_{{\mathbb{R}}^n}{p}_{\mathit{N}}\left(\mathit{y}\right)e^{-\frac{1}{2}{\mathit{x}}^T{\Sigma }^{-1}\mathit{x}+{\mathit{x}}^T{\Sigma }^{-1}\mathit{y}}\sum _{\mathbf{i}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}{c}_{\mathbf{i}}^{\prime }{H}_{\mathbf{i}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})d\mathit{y}\hfill \end{array}$$

(56)

$$\begin{array}{cc}\hfill & ={\int }_{{\mathbb{R}}^n}{p}_{\mathit{N}}\left(\mathit{y}\right)\sum _{\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}\frac{{\mathit{x}}^{\mathit{m}}}{\mathit{m}!}{H}_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})\sum _{\mathbf{i}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}{c}_{\mathbf{i}}^{\prime }{H}_{\mathbf{i}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})d\mathit{y}\hfill \end{array}$$

(57)

$$\begin{array}{cc}\hfill & =\sum _{\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}\frac{{\mathit{x}}^{\mathit{m}}}{\mathit{m}!}\sum _{\mathbf{i}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}{c}_{\mathbf{i}}^{\prime }⟨H_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}}),H_{\mathbf{i}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})⟩\hfill \end{array}$$

(58)

$$\begin{array}{cc}\hfill & =\sum _{\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}{c}_{\mathit{m}}{\mathit{x}}^{\mathit{m}},\hfill \end{array}$$

(59)

where $c_{\mathit{m}}=c_{\mathit{m}}^{\prime }/{\rho }_{\mathit{m}}$, with

$$\infty >{\rho }_{\mathit{m}}\triangleq \frac{\mathit{m}!}{⟨H_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}}),H_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})⟩}>0.$$

(60)

This simplification to a polynomial will be helpful since the cost function associated with the even-moment constraint is also a polynomial. This relationship is exploited in Section 3.3.

3.3. Non-Constancy of $s(·;F^{*})$

Recall the relationship $supp\left(F^{*}\right)\subseteq \mathcal{Z}(s;F^{*})\cap \mathcal{A}$ from (37). Since $supp\left(F^{*}\right)\ne \varnothing$, $s(·;F^{*})$ has at least one zero, it is constant if and only if $\mathcal{Z}(s;F^{*})={\mathbb{R}}^n.$ This subsection is dedicated to showing the latter equivalent condition. The immediate implication is that $supp\left(F^{*}\right)$ is a strict subset of ${\mathbb{R}}^n$; however, the fact that $s(·;F^{*})$ is a non-zero real-analytic function will be used in Section 3.4 to prove the main results.

By way of contradiction, suppose that, for all $\mathit{x}\in {\mathbb{R}}^n$, $s(\mathit{x};F^{*})=0$. Substituting (59) into (36), this is equivalent to

$$\begin{array}{cc}\hfill [\gamma a-C-h\left(\mathit{N}\right)]-\gamma (\sum _{i=1}^n{x}_i^2{)}^k& =\sum _{\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}{c}_{\mathit{m}}{x}_1^{m_1}\dots x_n^{m_n},\hfill \end{array}$$

(61)

for all $\mathit{x}\in {\mathbb{R}}^n$. The discussion proceeds in two cases: $k=1$ and $k>1$.

• Case $k=1$:

In the case that $k=1$ and $\mathcal{A}={\mathbb{R}}^n$, ${\mathit{X}}^{*}$ is known to be Gaussian [22] and there is no contradiction with (61). Therefore, for $k=1$, we focus only on compact input alphabets $\mathcal{A}⊊{\mathbb{R}}^n$.

With $k=1$ and $\mathcal{A}⊊{\mathbb{R}}^n$, ref. (61) reduces to

$$\begin{array}{cc}\hfill [\gamma a-C-h\left(\mathit{N}\right)]-\gamma \sum _{i=1}^n{x}_i^2& =\sum _{\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}{c}_{\mathit{m}}{x}_1^{m_1}\dots x_n^{m_n},\hfill \end{array}$$

(62)

for all $\mathit{x}\in {\mathbb{R}}^n$. Let ${\mathit{e}}_i$ be the i′th row of the $n\times n$ identity matrix and let $\mathbf{0}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n$ be the all zero vector. Since (62) holds for all $\mathit{x}\in {\mathbb{R}}^n$, matching coefficients gives

$$c_{\mathit{m}}=\left\{\begin{array}{cc}\gamma a-C-h\left(\mathit{N}\right),\hfill & \mathrm{if} \mathit{m}=\mathbf{0}\hfill \\ -\gamma ,\hfill & \mathrm{if} \mathit{m}=2{\mathit{e}}_i,i\in \{1,\dots ,n\}\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(63)

Since, for each $i\in \{1,\dots ,n\}$ and $\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n$, $H_{m_i}(y_i;{\sigma }_i^2)$ has degree $m_i$ and a positive leading coefficient,

$$H_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})=\prod _{i=1}^n{H}_{m_i}(y_i;{\sigma }_i^2)$$

(64)

also has degree $m_i$ in $y_i$ and the unique term with total degree $d\left(\mathit{m}\right)=m_1+\dots +m_n$ is $y^{m_1}\cdots y^{m_n}$, which has a positive coefficient. Therefore, the polynomials present in the sum are of the form

$$H_{\mathbf{0}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})={\kappa }_{\mathbf{0}}>0$$

(65)

and, for $i\in \{1,\dots ,n\}$,

$$H_{2{\mathit{e}}_i}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})={\kappa }_{2{\mathit{e}}_i}{y}_i^2+{\alpha }_{2{\mathit{e}}_i}{y}_i+{\beta }_{2{\mathit{e}}_i}.$$

(66)

The constants ${\kappa }_{\mathbf{0}}$ and ${\kappa }_{2{\mathit{e}}_i}$ are positive, while ${\alpha }_{2{\mathit{e}}_i}$ and ${\beta }_{2{\mathit{e}}_i}$ are real. Substituting this and the identity $c_{\mathit{m}}^{\prime }=c_{\mathit{m}}{\rho }_{\mathit{m}}$ into (49) yields

$$\begin{array}{cc}\hfill lnp(\mathit{y};F^{*})& =c_{\mathbf{0}}{\rho }_{\mathbf{0}}{H}_{\mathbf{0}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})+\sum _{i=1}^n{c}_{2{\mathit{e}}_i}{\rho }_{2{\mathit{e}}_i}{H}_{2{\mathit{e}}_i}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})\hfill \end{array}$$

(67)

$$\begin{array}{cc}\hfill & ={\kappa }_{\mathbf{0}}{\rho }_{\mathbf{0}}[\gamma a-C-h\left(\mathit{N}\right)]-\gamma \sum _{i=1}^n{\rho }_{2{\mathit{e}}_i}({\kappa }_{2{\mathit{e}}_i}{y}_i^2+{\alpha }_{2{\mathit{e}}_i}{y}_i+{\beta }_{2{\mathit{e}}_i}),\hfill \end{array}$$

(68)

or equivalently,

$$p(\mathit{y};F^{*})=e^{{\kappa }_{\mathbf{0}}{\rho }_{\mathbf{0}}[\gamma a-C-h\left(\mathit{N}\right)]}\prod _{i=1}^n{e}^{-\gamma {\rho }_{2{\mathit{e}}_i}({\kappa }_{2{\mathit{e}}_i}{y}_i^2+{\alpha }_{2{\mathit{e}}_i}{y}_i+{\beta }_{2{\mathit{e}}_i})}.$$

(69)

By definition, $\gamma \ge 0$; however, $\gamma =0$ results in a constant density on ${\mathbb{R}}^n$, which is invalid. Then, it must be the case that $\gamma >0$. Thus, the output achieved by ${\mathit{X}}^{*}$, ${\mathit{Y}}^{*}\triangleq {\mathit{X}}^{*}+\mathit{N}$ has independent Gaussian components. Since ${\mathit{X}}^{*}$ and $\mathit{N}$ are independent and $\mathit{N}$ is an n-variate Gaussian random variable, ${\mathit{X}}^{*}$ must either be an n-variate Gaussian random variable or be almost surely equal to some ${\mathit{x}}_0\in {\mathbb{R}}^n$. In the former case, ${\mathit{X}}^{*}$ violates the stipulation that the input alphabet $\mathcal{A}$ is compact and contradicts the assumption that $s(·;F^{*})$ is identically 0. In the latter case, $supp\left(F^{*}\right)=\left\{{\mathit{x}}_0\right\}$ is trivial and satisfies the main results of the paper.

• Case $k>1$:

For the case $k>1$, we derive a contradiction to (61) using results on the rate of decay of a function compared with that of its Fourier transform to conclude that $s(·;F^{*})$ is not identically 0.

Lemma 1. 

Let $\mathit{U}\in {\mathbb{R}}^n$ have, for some $\beta >0$, a characteristic function satisfying

$$\left|{\varphi }_{\mathit{U}}\left(\mathit{\omega }\right)\right|\triangleq \left|\mathbb{E}\left[e^{i{\mathit{\omega }}^T\mathit{U}}\right]\right|\le e^{-\frac{{\beta \parallel \mathit{\omega }\parallel }^2}{2}}$$

(70)

for all $\mathit{\omega }\in {\mathbb{R}}^n$. Let $\mathit{V}$ be a random variable independent of $\mathit{U}$. Then, the characteristic function of $\mathit{W}=\mathit{U}+\mathit{V}$ satisfies, for all $\mathit{\omega }\in {\mathbb{R}}^n$,

$$\left|{\varphi }_{\mathit{W}}\left(\mathit{\omega }\right)\right|\le e^{-\frac{{\beta \parallel \mathit{\omega }\parallel }^2}{2}}.$$

(71)

Proof. 

By the independence of $\mathit{U}$ and $\mathit{V}$, and the fact that characteristic functions have pointwise moduli upper-bounded by 1,

$$\begin{array}{cc}\hfill |{\varphi }_{\mathit{W}}\left(\omega \right)|& = |{\varphi }_{\mathit{U}}\left(\omega \right)\left|\right|{\varphi }_{\mathit{V}}\left(\omega \right)|\hfill \end{array}$$

(72)

$$\begin{array}{cc}\hfill & \le |{\varphi }_{\mathit{V}}\left(\omega \right)|e^{-\frac{{\beta \parallel \mathit{\omega }\parallel }^2}{2}}\hfill \end{array}$$

(73)

$$\begin{array}{cc}\hfill & \le e^{-\frac{{\beta \parallel \mathit{\omega }\parallel }^2}{2}}.\hfill \end{array}$$

(74)

□

Lemma 2. 

Let $\mathit{U}\in {\mathbb{R}}^n$ have, for some constant $\beta >0$, a characteristic function satisfying

$$\left|{\varphi }_{\mathit{U}}\left(\mathit{\omega }\right)\right|\triangleq \left|\mathbb{E}\left[e^{i{\mathit{\omega }}^T\mathit{U}}\right]\right|\le e^{-\frac{{\beta \parallel \mathit{\omega }\parallel }^2}{2}}$$

(75)

for all $\mathit{\omega }\in {\mathbb{R}}^n$. Let $\mathit{V}$ be a random variable independent of $\mathit{U}$ and $\mathit{W}=\mathit{U}+\mathit{V}$ have density $p_{\mathit{W}}(·)$. If there exist positive constants α and K such that, for all $\mathit{x}\in {\mathbb{R}}^n$,

$$p_{\mathit{W}}\left(\mathit{x}\right)\le K{e}^{-{\alpha \parallel \mathit{x}\parallel }^2},$$

(76)

then $\alpha \beta \le 0.5$.

Proof. 

Apply Lemma 1 and Theorem 4 of [23], noting that an identically 0 function cannot be a density. □

We make use of Lemma 2 by setting $\mathit{U}=\mathit{N}$, $\mathit{V}={\mathit{X}}^{*}$, and $\mathit{W}={\mathit{Y}}^{*}$ and deriving a contradiction to the assumption that $s(·;F^{*})$ is identically 0. Note that, using Rayleigh quotients, the modulus of the characteristic function of $\mathit{N}$ can be upper-bounded for any $\mathit{\omega }\in {\mathbb{R}}^n$ by

$$\left|\mathbb{E}\left[e^{i{\mathit{\omega }}^T\mathit{N}}\right]\right|=e^{-\frac{1}{2}{\mathit{\omega }}^T\Sigma \mathit{\omega }}\le e^{-\frac{1}{2}{\sigma }_1^2{\parallel \mathit{\omega }\parallel }^2}.$$

(77)

That is, the characteristic function of $\mathit{N}$ satisfies (75) with $\beta ={\sigma }_1^2$.

To complete the contradiction, we show that there exists $\alpha >0.5/\beta$ and $K_{\alpha }>0$ such that $p(·;F^{*})$ satisfies the bound in (76). The assumption that $s(·;F^{*})$ is identically 0 yields (61); substituting the Multinomial Theorem,

$$[\gamma a-C-h\left(\mathit{N}\right)]-\gamma \sum _{k_1+\dots +k_n=k}\frac{k!}{k_1!\cdots k_n!}{x}_1^{2k_1}\cdots x_n^{2k_n}=\sum _{\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n}{c}_{\mathit{m}}{x}_1^{m_1}\cdots x_n^{m_n}.$$

(78)

By coefficient matching in (78), the set of non-zero coefficients, other than $c_{\mathbf{0}}$, is indexed by the set

$$\mathcal{M}\triangleq \{\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n|\sum _{i=1}^n{m}_i=2k \mathrm{and} m_i \mathrm{is} \mathrm{even} \forall i\in \{1,\dots ,n\}\}.$$

(79)

Furthermore, for $\mathit{m}\in \mathcal{M}$,

$$c_{\mathit{m}}=-\gamma \frac{({\sum }_{i=1}^n{m}_i/2)!}{(m_1/2)!\cdots (m_n/2)!}$$

(80)

Therefore, substituting (80) into (49),

$$\begin{array}{cc}\hfill p(\mathit{y};F^{*})& =e^{{\kappa }_{\mathbf{0}}{\rho }_{\mathbf{0}}(\gamma a-C-h\left(\mathit{N}\right))}{e}^{{\sum }_{\mathit{m}\in \mathcal{M}}{c}_{\mathit{m}}{\rho }_{\mathit{m}}{H}_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})}.\hfill \end{array}$$

(81)

for some positive constant ${\kappa }_{\mathbf{0}}=H_{\mathbf{0}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})$. As with the case $k=1$, $\gamma =0$ results in a constant output density over ${\mathbb{R}}^n$ and can be disregarded as a possibility. Thus, for each $\mathit{m}\in \mathcal{M}$, we have

$$c_{\mathit{m}}<0.$$

(82)

With $\mathit{W}={\mathit{Y}}^{*}$ and $\alpha >0$, showing that there exists $K_{\alpha }$ for which (76) holds, is equivalent to showing that

$$\begin{array}{c}\hfill \frac{p(\mathit{y};F^{*})}{e^{-{\alpha \parallel \mathit{y}\parallel }^2}}=e^{{\kappa }_{\mathbf{0}}{\rho }_{\mathbf{0}}(\gamma a-C-h\left(\mathit{N}\right))}{e}^{{\sum }_{\mathit{m}\in \mathcal{M}}{c}_{\mathit{m}}{\rho }_{\mathit{m}}{H}_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})+\alpha {\sum }_{i=1}^n{y}_i^2}\end{array}$$

(83)

is bounded. This, in turn, is equivalent to showing that the polynomial in the exponent,

$$q_{\alpha }\left(\mathit{y}\right)\triangleq \sum _{\mathit{m}\in \mathcal{M}}{c}_{\mathit{m}}{\rho }_{\mathit{m}}{H}_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})+\alpha \sum _{i=1}^n{y}_i^2,$$

(84)

is upper bounded. We proceed by considering the degrees of the terms of $q_{\alpha }\left(\mathit{y}\right)$ to determine the behavior of (84) as $\parallel \mathit{y}\parallel$ increases.

For each $\mathit{m}\in {\mathbb{Z}}_{\ge \mathbf{0}}^n$ and $i\in \{1,\dots ,n\}$, $H_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})$ has degree $m_i$ in $y_i$. Furthermore, $H_{\mathit{m}}(\mathit{y};{\mathit{\sigma }}^{\mathbf{2}})$ has total degree

$$d\left(\mathit{m}\right)\triangleq \sum _{i=1}^n{m}_i$$

(85)

and the unique highest degree term, $y_1^{m_1}\cdots y_n^{m_n}$, has coefficient ${\kappa }_{\mathit{m}}>0$. Note that, since $k>1$, and by the definition of $\mathcal{M}$, $q_{\alpha }(·)$ has total degree $2k\ge 4$. Hence, ref. (84) can be rewritten as $q_{\alpha }\left(\mathit{y}\right)=q_{\alpha }^{\left(0\right)}\left(\mathit{y}\right)+q_{\alpha }^{\left(1\right)}\left(\mathit{y}\right)+q_{\alpha }^{\left(2\right)}\left(\mathit{y}\right)$, where

$$\begin{array}{cc}\hfill q_{\alpha }^{\left(0\right)}\left(\mathit{y}\right)& \triangleq \sum _{i=1}^n{c}_{2k{\mathit{e}}_i}{\rho }_{2k{\mathit{e}}_i}{\kappa }_{2k{\mathit{e}}_i}{y}_i^{2k}=-\gamma \sum _{i=1}^n{\rho }_{2k{\mathit{e}}_i}{\kappa }_{2k{\mathit{e}}_i}{y}_i^{2k},\hfill \end{array}$$

(86)

$$\begin{array}{cc}\hfill q_{\alpha }^{\left(1\right)}\left(\mathit{y}\right)& \triangleq \sum _{\begin{array}{c}\mathit{m}\in \mathcal{M}\\ \mathit{m}\notin \{2k{\mathit{e}}_i\mid i\in \{1,\dots ,n\}\}\end{array}}{c}_{\mathit{m}}{\rho }_{\mathit{m}}{\kappa }_{\mathit{m}}{\mathit{y}}^{\mathit{m}},\hfill \end{array}$$

(87)

and $q_{\alpha }^{\left(2\right)}\left(\mathit{y}\right)$ is the sum of the remaining terms, each with a total degree of at most $2k-2$.

Note the following:

• For each $\mathit{y}\in {\mathbb{R}}^n$, $q_{\alpha }^{\left(0\right)}\left(\mathit{y}\right)\le 0$ and, by Lemma A9, the minimal value of $|q_{\alpha }^{\left(0\right)}\left(\mathit{y}\right)|$—evaluated on a sphere $\parallel \mathit{y}\parallel =L$ of radius $L\ge 0$—is at least $\gamma {min}_{i\in \{1,\dots ,n\}}\left\{{\rho }_{2k{\mathit{e}}_i}{\kappa }_{2k{\mathit{e}}_i}\right\}{L}^{2k}/n^k$.
• For each $\mathit{y}\in {\mathbb{R}}^n$, we have that $q_{\alpha }^{\left(1\right)}\left(\mathit{y}\right)\le 0$. Indeed, for each $\mathit{m}\in \mathcal{M}$, $c_{\mathit{m}}<0$ by (82), ${\rho }_{\mathit{m}}>0$, and ${\kappa }_{\mathit{m}}>0$; further, for each $i\in \{1,\dots ,n\}$, $m_i$ is even.
• The maximum value of $|q_{\alpha }^{\left(2\right)}\left(\mathit{y}\right)|$, evaluated on a sphere $\parallel \mathit{y}\parallel =L$ of radius $L\ge 1$, is at most $A{L}^{2k-2}$ for some $A>0$—that is, each term of $q_{\alpha }^{\left(2\right)}\left(\mathit{y}\right)$ is either of the form $\alpha y_i^2$ or $c_{\mathit{m}}{\rho }_{\mathit{m}}{\nu }_{\mathit{m},\mathbf{l}}{\mathit{y}}^{\mathbf{l}}$ for some $\mathit{m}\in \mathcal{M},\mathbf{l}\in {\mathbb{Z}}_{\ge 0}^n$, and ${\nu }_{\mathit{m},\mathbf{l}}\in \mathbb{R}$, where $d\left(\mathbf{l}\right)\le 2k-2$. Lemma A10 shows that these are no more than $\alpha L^2$ or $c_{\mathit{m}}{\rho }_{\mathit{m}}{\nu }_{\mathit{m},\mathbf{l}}{L}^{d\left(\mathbf{l}\right)}$.

We conclude that, since $q_{\alpha }^{\left(0\right)}(\mathit{y})\le 0$ and $q_{\alpha }^{\left(1\right)}(\mathit{y})\le 0$ for all $\mathit{y}\in {\mathbb{R}}^n$,

$$\begin{array}{cc}\hfill \underset{\parallel \mathit{y}\parallel \to \infty }{lim}{q}_{\alpha }\left(\mathit{y}\right)& =\underset{\parallel \mathit{y}\parallel \to \infty }{lim}(q_{\alpha }^{\left(0\right)}\left(\mathit{y}\right)+q_{\alpha }^{\left(2\right)}\left(\mathit{y}\right))+\underset{\parallel \mathit{y}\parallel \to \infty }{lim}{q}_{\alpha }^{\left(1\right)}\left(\mathit{y}\right)\hfill \end{array}$$

(88)

$$\begin{array}{cc}\hfill & \le \underset{\parallel \mathit{y}\parallel \to \infty }{lim}(q_{\alpha }^{\left(0\right)}\left(\mathit{y}\right)+q_{\alpha }^{\left(2\right)}\left(\mathit{y}\right))\hfill \end{array}$$

(89)

$$\begin{array}{cc}\hfill & =-\infty .\hfill \end{array}$$

(90)

Thus, since $q_{\alpha }(\mathit{y})$ is a continuous function that satisfies (90), it is bounded from above. Let $M_{q,\alpha }={sup}_{\mathit{y}\in {\mathbb{R}}^n}{q}_{\alpha }\left(\mathit{y}\right)$ and

$$K_{\alpha }=e^{{\kappa }_{\mathbf{0}}{\rho }_{\mathbf{0}}(\gamma a-C-h\left(\mathit{N}\right))}{e}^{M_{q,\alpha }}.$$

(91)

Then, for all $\mathit{y}\in {\mathbb{R}}^n$,

$$p(\mathit{y};F^{*})\le K_{\alpha }{e}^{-{\alpha \parallel \mathit{y}\parallel }^2}.$$

(92)

Recall that, with ${\sigma }_1^2>0$, the smallest eigenvalue of $\Sigma$, and $\beta ={\sigma }_1^2$, the characteristic function of $\mathit{N}$ satisfies (75). Let $\alpha =1/{\sigma }_1^2$ and choose $K_{\alpha }$ according to (91). Then, $p(\mathit{y};F^{*})$ satisfies (92), yet $\alpha \beta =1>0.5$. Hence, the bound on the characteristic function of $\mathit{N}$ given by (77) and the bound on the density of ${\mathit{Y}}^{*}$ given in (92) contradict Lemma 2. Therefore, the coefficient matching equation (78) cannot hold for all $\mathit{x}\in {\mathbb{R}}^n$ and we conclude that, for $k>1$, $s(·;F^{*})$ cannot be identically 0 on ${\mathbb{R}}^n$.

We summarize the results of the two cases, $k=1$ and $k>1$, in a theorem.

Theorem 1. 

Suppose that either

1. 

$\mathcal{A}⊊{\mathbb{R}}^n$ is compact, or

2. 

$\mathcal{A}={\mathbb{R}}^n$, with $k\ne 1$.

Then, either $supp\left(F^{*}\right)=\left\{{\mathit{x}}_0\right\}$ for some ${\mathit{x}}_0\in {\mathbb{R}}^n$ or

$$\mathcal{Z}(s;F^{*})⊊{\mathbb{R}}^n.$$

(93)

An immediate consequence of Theorem 1 is that $supp\left(F^{*}\right)$ is a strict subset of ${\mathbb{R}}^n$. Recall from Section 3.1 that $s(·;F^{*})$ has an analytic extension to ${\mathbb{C}}^n$. Therefore, Theorem 1 shows that $supp\left(F^{*}\right)$ is “sparse" in the sense used by [12]—that is, there exists a non-zero function with an analytic extension to ${\mathbb{C}}^n$ that is zero on $supp\left(F^{*}\right)$. However, the primary importance of Theorem 1 is as an intermediary result that is used in Section 3.4 to obtain a better understanding of the structure of $supp\left(F^{*}\right)$.

3.4. Main Results

In this section, we use geometry to show that $supp\left(F^{*}\right)$ is contained in a countable disjoint union of submanifolds of dimensions ranging $0,\dots ,n-1$. Furthermore, this union is finite when $\mathcal{A}$ is compact. We then show that $supp\left(F^{*}\right)$ has Lebesgue measure 0 and is nowhere dense in ${\mathbb{R}}^n$.

The discussions in this section consider subsets of a vector’s components; so, for $\mathit{x}\in {\mathbb{R}}^n$ and $i\in \{1,\dots ,n\}$, we introduce the notation

$${\mathit{x}}^i\triangleq (x_1,\dots ,x_i)\in {\mathbb{R}}^i.$$

(94)

Recall from (37) in Section 3.1 that

$$\begin{array}{cc}\hfill supp\left(F^{*}\right)& \subseteq \mathcal{Z}(s;F^{*})\cap \mathcal{A}.\hfill \end{array}$$

(95)

Since, by Lemma A8, $s(·;F^{*})$ has an analytic extension to ${\mathbb{C}}^n$, it is real-analytic, which motivates us to study the geometry of zero sets of real-analytic functions. We start by restating Theorem 6.3.3 of [24] to the level that is needed in this paper.

Theorem 2 

(Structure Theorem). Let $\psi (·):{\mathbb{R}}^n\to \mathbb{R}$ be a real-analytic function, where $\psi (0,\dots ,0,x_n)$ is not identically 0 in $x_n$. After a rotation of the coordinates $x_1,\dots ,x_{n-1}$, there exist constants ${\delta }_m$, $m\in \{1,\dots ,n\}$ such that with

$$Q\triangleq \{\mathit{x}\in {\mathbb{R}}^n\mid |x_m|<{\delta }_m\forall m\in \{1,\dots ,n\}\},$$

(96)

we have

$$\{\mathit{x}\in Q\mid \psi \left(\mathit{x}\right)=0\}=\bigcup _{i=0}^{n-1}{V}_i,$$

(97)

where $V_0$ is either empty or contains only the origin and $V_i$, $i\in \{1,\dots ,n-1\}$, is a finite disjoint union of i-dimensional submanifolds—that is, for each $i\in \{1,\dots ,n-1\}$, there exists $n_i$ for which

$$V_i=\bigcup _{j=0}^{n_i}{\Gamma }_i^j,$$

(98)

where each ${\Gamma }_i^j$ is an i-dimensional submanifold. Furthermore, letting

$$Q_i\triangleq \{{\mathit{x}}^i\in {\mathbb{R}}^i\mid |x_m|<{\delta }_m\forall m\in \{1,\dots ,i\}\},$$

(99)

there exists an open set ${\Omega }_i^j\subseteq Q_i$ and real-analytic functions ${\alpha }_i^{j,m}(·)$, $m\in \{i+1,\dots ,n\}$, on ${\Omega }_i^j$ for which

$${\Gamma }_i^j=\{({\mathit{x}}^i,{\alpha }_i^{j,i+1}\left({\mathit{x}}^i\right),\dots ,{\alpha }_i^{j,n}\left({\mathit{x}}^i\right))\in {\mathbb{R}}^n\mid {\mathit{x}}^i\in {\Omega }_i^j\}.$$

(100)

We apply Theorem 2 to characterize the zero set of $s(·;F^{*})$ in the form of (97) and obtain the following result.

Theorem 3. 

Suppose that either

1. 

$\mathcal{A}⊊{\mathbb{R}}^n$ is compact, or

2. 

$\mathcal{A}={\mathbb{R}}^n$, with $k\ne 1$.

Then,

$$supp\left(F^{*}\right)\subseteq \mathcal{Z}(s;F^{*})\cap \mathcal{A}=\mathcal{A}\cap \left(\bigcup _{i=0}^{n-1}{T}_i\right),$$

(101)

where $T_0$ is a countable union of isolated points and $T_i$, $i\in \{1,\dots ,n-1\}$, is a countable disjoint union of i-dimensional submanifolds. Furthermore, if $\mathcal{A}$ is compact, these unions are finite.

Proof. 

First, note that, by Theorem 1, either $supp\left(F^{*}\right)=\left\{{\mathit{x}}_0\right\}$ for some ${\mathit{x}}_0\in {\mathbb{R}}^n$ or $s(·;F^{*})$ is not identically 0 on ${\mathbb{R}}^n$. In the former case, the result is trivially true; so, assume that $s(·;F^{*})$ is not identically 0 on ${\mathbb{R}}^n$. Therefore, for any $\mathit{q}\in {\mathbb{Q}}^n$, we can translate $s(·;F^{*})$ by $\mathit{q}$ and rotate its coordinate system to apply Theorem 2—that is, there exists a sufficiently small open set $Q^{\mathit{q}}$ around $\mathit{q}$ such that

$$\mathcal{Z}(s;F^{*})\cap Q^{\mathit{q}}=\bigcup _{i=0}^{n-1}{V}_i^{\mathit{q}},$$

(102)

where the $V_i$ values are as in Theorem 2.

Since ${\mathbb{Q}}^n$ is dense in ${\mathbb{R}}^n$,

$$\mathcal{A}\subseteq \bigcup _{\mathit{q}\in {\mathbb{Q}}^n}{Q}^{\mathit{q}}$$

(103)

Furthermore, if $\mathcal{A}$ is compact, the open cover ${\left\{Q^{\mathit{q}}\right\}}_{\mathit{q}\in {\mathbb{Q}}^n}$ has a finite subcover ${\left\{Q^{{\mathit{q}}_j}\right\}}_{j=1}^m$—that is,

$$\mathcal{A}\subseteq \bigcup _{j=1}^m{Q}^{{\mathit{q}}_j}.$$

(104)

Defining the index set

$$\mathcal{M}\triangleq \left\{\begin{array}{cc}{\mathbb{Q}}^n,\hfill & \mathcal{A}={\mathbb{R}}^n\hfill \\ {\left\{{\mathit{q}}_j\right\}}_{j=1}^m,\hfill & \mathcal{A}⊊{\mathbb{R}}^n\mathrm{is} \mathrm{compact},\hfill \end{array}\right.$$

(105)

we obtain

$$\begin{array}{cc}\hfill \mathcal{Z}(s;F^{*})\cap \mathcal{A}& =\mathcal{A}\cap \left(\bigcup _{\mathit{q}\in \mathcal{M}}(\mathcal{Z}(s;F^{*})\cap Q^{\mathit{q}})\right)\hfill \end{array}$$

(106)

$$\begin{array}{cc}\hfill & =\mathcal{A}\cap \left(\bigcup _{\mathit{q}\in \mathcal{M}}\bigcup _{i=0}^{n-1}{V}_i^{\mathit{q}}\right)\hfill \end{array}$$

(107)

$$\begin{array}{cc}\hfill & =\mathcal{A}\cap \left(\bigcup _{i=0}^{n-1}\bigcup _{\mathit{q}\in \mathcal{M}}{V}_i^{\mathit{q}}\right).\hfill \end{array}$$

(108)

Since, for each $\mathit{q}\in \mathcal{M}$, $V_0^{\mathit{q}}$ is either empty or a single point,

$$T_0\triangleq \bigcup _{\mathit{q}\in \mathcal{M}}{V}_0^{\mathit{q}}$$

(109)

is a countable set of points and is finite when $\mathcal{A}$ is compact. Furthermore, each $V_i^{\mathit{q}}$, where $i\in \{1,\dots ,n-1\}$, is itself a finite union of i-dimensional submanifolds. Hence,

$$T_i\triangleq \bigcup _{\mathit{q}\in \mathcal{M}}{V}_i^{\mathit{q}}$$

(110)

is a countable union of i-dimensional submanifolds. When $\mathcal{A}$ is compact, this union is also finite. □

Note that Theorem 3 agrees with the results of [5] when the cases overlap. Indeed, consider the case in which $\mathcal{A}$ is a ball centered at the origin, $k=1$, and the noise covariance matrix $\Sigma =t{\mathit{I}}_n$, where ${\mathit{I}}_n$ is the $n\times n$ identity matrix and $t>0$. Then, ref. [5] shows that the capacity-achieving distribution is supported on a finite number of concentric $(n-1)$-spheres. Each $(n-1)$-sphere is an $n-1$ dimensional submanifold.

In the next two theorems, we show that $supp\left(F^{*}\right)$ has Lebesgue measure 0 and is nowhere dense in ${\mathbb{R}}^n$.

Theorem 4. 

Suppose that either

1. 

$\mathcal{A}⊊{\mathbb{R}}^n$ is compact, or

2. 

$\mathcal{A}={\mathbb{R}}^n$, with $k\ne 1$.

Let $\mu (·)$ denote the n-dimensional Lebesgue measure. Then,

$$\mu \left(supp\left(F^{*}\right)\right)=0.$$

(111)

Proof. 

By Theorem 3, we have

$$\begin{array}{cc}\hfill supp\left(F^{*}\right)& \subseteq \mathcal{A}\cap \left(\bigcup _{i=0}^{n-1}{T}_i\right)\hfill \end{array}$$

(112)

$$\begin{array}{cc}\hfill & \subseteq \bigcup _{i=0}^{n-1}\bigcup _{\mathit{q}\in \mathcal{M}}{V}_i^{\mathit{q}},\hfill \end{array}$$

(113)

where $\mathcal{M}$ is countable. Note that for each $\mathit{q}\in \mathcal{M}$, $V_0^{\mathit{q}}$ is either empty or a single point; so, $\mu \left(V_0^{\mathit{q}}\right)=0$. Furthermore, for each $\mathit{q}\in \mathcal{M}$ and $i\in \{1,\dots ,n-1\}$, $V_i^{\mathit{q}}$ is a finite disjoint union of $n_i^{\mathit{q}}$i-dimensional submanifolds, and for $i\le n-1$, each submanifold has Lebesgue measure 0. Therefore, $\mu \left(supp\left(F^{*}\right)\right)=0$. □

We will now define the notion of a subset being nowhere dense in its superset and show that $supp\left(F^{*}\right)$ is nowhere dense in ${\mathbb{R}}^n$.

Definition 3. 

Let $B\subseteq {\mathbb{R}}^n$. A set $A\subseteq B$ is said to be dense in B if, for every $b\in B$, there exists a sequence ${\left\{a_i\right\}}_{i=0}^{\infty }\subseteq A$ such that

$$\underset{i\to \infty }{lim}{a}_i=b.$$

(114)

Definition 4. 

Let $B\subseteq {\mathbb{R}}^n$. A set $A\subseteq B$ is called nowhere dense in B if, for every open set $U\subseteq B$, $A\cap U$ is not dense in U.

Theorem 5. 

Suppose that either

1. 

$\mathcal{A}⊊{\mathbb{R}}^n$ is compact, or

2. 

$\mathcal{A}={\mathbb{R}}^n$, with $k\ne 1$.

Then, $supp\left(F^{*}\right)$ is nowhere dense in ${\mathbb{R}}^n$.

Proof. 

By Theorem 1, either $supp\left(F^{*}\right)=\left\{{\mathit{x}}_0\right\}$ for some ${\mathit{x}}_0\in {\mathbb{R}}^n$ or

$$\mathcal{Z}(s;F^{*})⊊{\mathbb{R}}^n.$$

(115)

Since, in the former, case $supp\left(F^{*}\right)$ is nowhere dense in ${\mathbb{R}}^n$, assume the latter and let (115) hold. Let $U\subseteq {\mathbb{R}}^n$ be a non-empty open set; we will show the result by proving that $\mathcal{Z}(s;F^{*})\cap U$ is not dense in U.

Fix $\mathit{x}\in U$. Translating $s(·;F^{*})$ by $\mathit{x}$, rotating the coordinate system and applying Theorem 2 shows that there exists a sufficiently small open set Q containing $\mathit{x}$ on which

$$\begin{array}{cc}\hfill \mathcal{Z}(s;F^{*})\cap Q& =\bigcup _{i=0}^{n-1}{V}_i\hfill \end{array}$$

(116)

$$\begin{array}{cc}\hfill & =V_0\cup \left(\bigcup _{i=1}^{n-1}\bigcup _{j=0}^{n_i}{\Gamma }_i^j\right).\hfill \end{array}$$

(117)

It suffices to show the existence of a point of the form $({\mathit{x}}^{n-1},u_n)\in U\cap Q$ that is not the limit of any sequence in $\mathcal{Z}(s;F^{*})\cap U\cap Q$. Let $({\mathit{y}}_m^{n-1},y_{n,m})$ be a convergent sequence in $\mathcal{Z}(s;F^{*})\cap U\cap Q$, indexed by m, for which

$$\begin{array}{c}\hfill \underset{m\to \infty }{lim}{\mathit{y}}_m^{n-1}={\mathit{x}}^{n-1}.\end{array}$$

(118)

Using the parameterization from (100), the n’th component of sequence index m satisfies one of the following:

• $y_{n,m}\in \{v_n\mid \mathit{v}\in V_0\}$, or
• for some $i_m\in \{1,\dots ,n-1\}$ and $j_m\in \{1,\dots ,n_{i_m}\}$,

  $$y_{n,m}={\alpha }_{i_m}^{j_m,n}\left({\mathit{y}}_m^{i_m}\right).$$

  (119)

  Since each ${\alpha }_{i_m}^{j_m,n}(·)$ is real-analytic, it is continuous. Then, for $y_{n,m}$ satisfying (119), if ${lim}_{m\to \infty }{y}_{n,m}$ exists, we have ${lim}_{m\to \infty }{y}_{n,m}={\alpha }_i^{j,n}\left({\mathit{x}}^i\right)$ for some $i\in \{1,\dots ,n-1\}$ and $j\in \{1,\dots ,n_i\}$. Since $V_0$ is either empty or a single point, the number of possible values for ${lim}_{m\to \infty }{\left(y_n\right)}_m$ is at most

  $$\begin{array}{cc}\hfill |V_0| +\sum _{i=1}^{n-1}{n}_i& <\infty .\hfill \end{array}$$

  (120)

However, since $\mathit{x}\in U\cap Q$, where $U\cap Q$ is open, the set $\{t\in \mathbb{R}\mid ({\mathit{x}}^{n-1},x_n+t)\in U\cap Q\}$ is uncountable. Thus, there exists t such that $({\mathit{x}}^{n-1},x_n+t)\in U\cap Q$ is not the limit of any sequence in $\mathcal{Z}(s;F^{*})\cap U\cap Q$. □

4. Discussion

This paper has considered vector-valued channels with additive Gaussian noise. Unlike much of the prior work in this area, the noise was not limited to having independent and identically distributed components. The support of the capacity-achieving input distribution was discussed when inputs were subjected to an even-moment radial constraint of order $2k$. Furthermore, the inputs were either allowed to take any value in ${\mathbb{R}}^n$ or restricted to a compact set. When the input alphabet was the entire space, ${\mathbb{R}}^n$, only the case $k\ge 2$ was considered since, for $k=1$, the optimal input distribution is well-known to be Gaussian.

The problem was framed as a convex optimization problem that was shown to be solved by a unique input distribution $F^{*}$. The conditions for optimality yielded a real-analytic function $s(·;F^{*})$ whose zero set contained $supp\left(F^{*}\right)$, the support of $F^{*}$. Using the framework of an $L^2$ space that was weighted by the noise density, $s(·;F^{*})$ was simplified and shown to be non-constant on ${\mathbb{R}}^n$. Through geometric analysis of the zero set of $s(·;F^{*})$, $supp\left(F^{*}\right)$ was shown to be contained in a countable union of single points and submanifolds of dimensions ranging $1,\dots ,n-1$. When the input alphabet was compact, this union was further shown to be finite. Finally, it was determined that $supp\left(F^{*}\right)$ has Lebesgue measure 0 and is nowhere dense in ${\mathbb{R}}^n$.

This paper is an expansion of the work concerning even-moment input constraints in [14] to vector-valued channels that are not necessarily spherically symmetric. Viewed as a generalization of [12], it considers order $2k$ rather than second-moment radial constraints and includes ${\mathbb{R}}^n$ as a possible input alphabet. Unlike prior work, it also provides geometric results on the supports of capacity-achieving inputs to spherically asymmetric channels.