A Gradient-Based Method for Robust Sensor Selection in Hypothesis Testing

Abstract

This paper considers the binary Gaussian distribution robust hypothesis testing under a Bayesian optimal criterion in the wireless sensor network (WSN). The distribution covariance matrix under each hypothesis is known, while the distribution mean vector under each hypothesis drifts in an ellipsoidal uncertainty set. Because of the limited bandwidth and energy, we aim at seeking a subset of p out of m sensors such that the best detection performance is achieved. In this setup, the minimax robust sensor selection problem is proposed to deal with the uncertainties of distribution means. Following a popular method, minimizing the maximum overall error probability with respect to the selection matrix can be approximated by maximizing the minimum Chernoff distance between the distributions of the selected measurements under null hypothesis and alternative hypothesis to be detected. Then, we utilize Danskin’s theorem to compute the gradient of the objective function of the converted maximization problem, and apply the orthogonal constraint-preserving gradient algorithm (OCPGA) to solve the relaxed maximization problem without 0/1 constraints. It is shown that the OCPGA can obtain a stationary point of the relaxed problem. Meanwhile, we provide the computational complexity of the OCPGA, which is much lower than that of the existing greedy algorithm. Finally, numerical simulations illustrate that, after the same projection and refinement phases, the OCPGA-based method can obtain better solutions than the greedy algorithm-based method but with up to $48.72\%$ shorter runtimes. Particularly, for small-scale problems, the OCPGA -based method is able to attain the globally optimal solution.

1. Introduction

Wireless sensor networks (WSNs) are extensively used to collect and transmit data in many applications, such as autonomous driving [1], disaster detection [2], target tracking [3], etc. In the WSN, it is usually unaffordable to collect and process all sensor data due to the limitations of power and communication resources [4,5]. Therefore, it is of great significance to choose an optimal subset of sensors such that the best performance is attained only based on data collected by the selected sensors, which is the so-called sensor selection problem.

In the past dozen years, sensor selection has been widely studied in various fields, e.g., estimation [6], target tracking [7], condition monitoring [8], to name a few. For parameter estimation in Kalman filtering dynamic system, [6] chose the optimal subset of sensors in each iteration via minimizing the error covariance matrix of the next iteration. The sensor selection problem for target tracking in large sensor networks was addressed in [7] based on generalized information gain. In [8], it provided an entropy-based sensor selection method for condition monitoring and prognostics of aircraft engine, which can describe the information contained in the sensor data sets.

Meanwhile, the sensor selection problem in hypothesis testings has also attracted a lot of attention [9,10,11]. For this type of hypothesis testings, only part of sensors in WSN are activated to transmit observation data, and then decisions are made based on the measurements of selected sensors to achieve the best detection performance. When the optimal sensor selection matrix is fixed, the corresponding hypothesis testing problem is reduced to a common one, which is easy to be dealt with. Hence, it is crucial to solve the involved sensor selection problem.

Work [9] studied the sensor selection for the binary Gaussian distribution hypothesis testing in the Neyman–Pearson framework, where the true distribution under each hypothesis is exactly known. It approximately converted the minimization of the false alarm error probability to the maximization of the Kullback-Leibler (KL) divergence between the distributions of the selected measurements under null hypothesis and alternative hypothesis to be detected. Additionally, [9] proposed a sensor selection framework of first relaxation and then projection for the first time, and provided the greedy algorithm to solve the relaxed problem by optimizing each column vector of the selection matrix.

In practical applications, the events to be detected (i.e., parameters of the hypothesis testing) are usually estimated from training data and affected by some uncertainty factors, such as poorly observation environment and system errors. Then, these parameters are not known precisely, but assumed to lie in some given uncertainty sets [10,11]. In these scenarios, the minimax robust sensor selection problem is formulated to cope with the parameter uncertainty. For the binary Gaussian distribution hypothesis testing under the Neyman–Pearson framework, following the framework in [9], work [10] investigated the involved sensor selection problem with distribution mean under each hypothesis falling in an ellipsoidal uncertainty set (the distribution covariance is known). Furthermore, [11] considered the sensor selection problems involved in the Gaussian distribution robust hypothesis testings with both Neyman–Pearson and Bayesian optimal criteria, where the distribution mean under each hypothesis drifts in an ellipsoidal uncertainty set. For the Bayesian framework, minimizing the maximum overall error probability is approximately converted to maximizing the minimum Chernoff distance. Then the corresponding greedy algorithm together with projection and refinement is also proposed to solve the robust sensor selection problem in the above hypothesis testing.

It has been shown in [11] that the robust sensor selection problem in the hypothesis testing is NP-hard under the Bayesian framework. Nevertheless, when the size of the sensor selection problem is small, its optimal solution can be obtained by the exhaustive method via traversing all possible choices. However, for a large-scale problem, the exhaustive method is not affordable due to its huge computation complexity. Although the aforementioned greedy algorithm-based method (i.e., greedy algorithm, projection and refinement) admits a lower computation complexity than the exhaustive method, it can not arrive at the globally optimal solution in many cases, and its computation complexity is still high for large-scale problems. Therefore, it is significant to seek a more efficient algorithm for solving the robust sensor selection problem in the hypothesis testing of WSN. To our surprise, even though other general sensor selection problems have been continuously investigated, for instance, sparse sensing [12] and sensor selection in sequential hypothesis testing [13], there is little progress for this type of sensor selection problems since [11] was published in 2011, which motivates our research.

In this paper, we consider the same binary Gaussian distribution robust hypothesis testings under a Bayesian framework as in [11], where the distribution mean under each hypothesis lies in an ellipsoidal uncertainty set. We attempt to select an optimal subset of sensors such that the maximum overall error probability is minimized. Following the similar idea in [11], minimizing the maximum overall error probability is approximated by maximizing the minimum Chernoff distance between two distributions under null hypothesis and alternative hypothesis to be detected. Then, our main contributions can be summarized as follows.

• First, we succeed in converting the maximinimization of the Chernoff distance to a maximization problem, and adopting the orthogonal constraint-preserving gradient algorithm (OCPGA) [14] to obtain a stationary point of the relaxed maximization problem without 0/1 constraints.
• Specifically, when implementing the OCPGA to the relaxed maximization problem, we utilize the Danskin’s theorem [15] to acquire its gradient. Furthermore, the efficient bisection is applied to get the means of distributions under null hypothesis and alternative hypothesis to be detected corresponding to the minimum Chernoff distance.
• The computational complexity of the OCPGA is shown to be lower than that of the greedy algorithm in [11] from the theoretical point of view, while numerical simulations show that the OCPGA-based method (i.e., OCPGA, projection and refinement) can obtain better solutions than the greedy algorithm-based method (i.e., R-C algorithm in [11]) with up to $48.72\%$ shorter runtimes. Therefore, better solutions are available for our proposed OCPGA-based method in some scenarios.

The remainder of this paper is organized as follows. Section 2 states the problem formulation. The proposed OCPGA, as well as projection and refinement phases, is characterized in Section 3. In Section 4, the existence and computation of gradient are provided. Section 5 presents some numerical experiments to corroborate our theoretical results, while Section 6 concludes the paper.

Notations: Denote ${\mathbb{R}}^m$ and ${\mathbb{R}}^{m\times p}$ as the m-dimensional real vector space and $m\times p$ dimensional real matrix space, respectively. Let $\mathbf{I}$ and $\mathbf{0}$ be the identity matrix and the zero matrix whose dimensions will be clear from the context; bold-face lower-case letters are used for vectors, while bold-face upper-case letters are for matrices. $\mathcal{N}(\mathit{\mu },\mathit{S})$ represents the Gaussian distribution with mean $\mathit{\mu }$ and covariance $\mathit{S}$. For matrix $\mathit{A}$, $\mathrm{tr}(\mathit{A})$, $|\mathit{A}|$, ${\parallel \mathit{A}\parallel }_F$, ${\mathit{A}}^H$ and ${\mathit{A}}_{i,j}$ denote its trace, determinant, Frobenius norm, conjugate transpose and the $(i,j)$-th entry, respectively. For square matrices $\mathit{A}$ and $\mathit{B}$, $\mathit{A}⪰(\succ )\mathit{B}$ represents that $\mathit{A}–\mathit{B}$ is semi-positive (positive) definite. For a semi-positive definite matrix $\mathit{A}$, ${\mathit{A}}^{\frac{1}{2}}$ stands for its square-rooting matrix.

2. Problem Formulation

2.1. System Model

Define $x={(x_1,x_2,\cdots ,x_m)}^T\in {\mathbb{R}}^m$ as the observation vector of all m sensors (each sensor corresponds to one-dimensional measurement). Consider the same binary Gaussian distribution robust hypothesis testing as in [11]:

$$\begin{array}{c}\hfill {\mathcal{H}}_0:\mathit{x}\sim \mathcal{N}({\mathit{m}}_{\mathbf{0}},{\mathit{S}}_{\mathbf{0}}),{\mathcal{H}}_1:\mathit{x}\sim \mathcal{N}({\mathit{m}}_{\mathbf{1}},{\mathit{S}}_{\mathbf{1}}),\end{array}$$

(1)

where mean vector ${\mathit{m}}_i$ falls in a given ellipsoidal uncertainty set $\mathcal{E}({\overline{\mathit{m}}}_{\mathit{i}},k_i{\mathit{S}}_i^{-1})=\{\mathit{x}\in {\mathbb{R}}^m|{(\mathit{x}-{\overline{\mathit{m}}}_{\mathit{i}})}^T{{\mathit{S}}_{\mathit{i}}}^{-1}(\mathit{x}-{\overline{\mathit{m}}}_{\mathit{i}})⩽\frac{1}{k_i}\}$, ${\overline{\mathit{m}}}_{\mathit{i}}$ denotes the mean estimated by training data, covariance ${\mathit{S}}_{\mathit{i}}$ is a known matrix, and $k_i\in (0,+\infty ]$ is the robustness parameter, $i=0,1$. Obviously, when $k_i=+\infty$, the ellipsoidal uncertainty set is reduced to a single point, and thereby ${\mathit{m}}_{\mathit{i}}={\overline{\mathit{m}}}_{\mathit{i}}$, i.e., there is no uncertainty.

Sensor Selection: In the WSN, sensors transmit their observations to a fusion node, which then performs the hypothesis testing based on its received measurements. Due to power constraints, suppose that only p out of m sensors are chosen ($p<m$) to transmit the observations to the fusion node. We aim at selecting p sensors to guarantee the best detection performance, that is, seeking a selection matrix $\mathit{E}\in {\mathbb{R}}^{m\times p}$ with 0/1 elements such that the best detection performance based on measurements $\mathit{y}={\mathit{E}}^T\mathit{x}\in {\mathbb{R}}^p$ is achieved. It is easy to see that, $\mathit{E}$ has exactly one unit entry per column (corresponds to a selected sensor) and at most one unit entry per row (each sensor is selected at most once). Therefore, $\mathit{E}$ should be a column orthogonal matrix, i.e., ${\mathit{E}}^T\mathit{E}=\mathbf{I}.$

Hypothesis Testing Induced by $\mathit{E}$: Owing to $\mathit{y}={\mathit{E}}^T\mathit{x}$, the original hypothesis testing (1) about $\mathit{x}$ in the high dimensional space ${\mathbb{R}}^m$ is converted to one about $\mathit{y}$ in a lower-dimensional space ${\mathbb{R}}^p$:

$$\begin{array}{c}\hfill {\mathcal{H}}_0:\mathit{y}\sim \mathcal{N}({\mathit{E}}^T{\mathit{m}}_{\mathbf{0}},{\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E}),{\mathcal{H}}_1:\mathit{y}\sim \mathcal{N}({\mathit{E}}^T{\mathit{m}}_{\mathbf{1}},{\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E}).\end{array}$$

(2)

Without loss of generality, we assume that

$$\mathcal{N}({\mathit{E}}^T{\mathit{m}}_{\mathbf{0}},{\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E})\ne \mathcal{N}({\mathit{E}}^T{\mathit{m}}_{\mathbf{1}},{\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E}).$$

(3)

Otherwise, the hypothesis testing (2) makes no sense. For hypothesis testing (2), when $\mathit{E}$ and ${\mathit{m}}_{\mathit{i}}$ are determined, the fusion node executes the following likelihood ratio (LR) test:

$$l(\mathit{y}):=\frac{f_1(\mathit{y};\mathit{E})}{f_0(\mathit{y};\mathit{E})}\underset{{\mathcal{H}}_0}{\overset{{\mathcal{H}}_1}{\gtrless }}\gamma ,$$

where $f_i(\mathit{y};\mathit{E})$ is the density of $\mathcal{N}({\mathit{E}}^T{\mathit{m}}_{\mathit{i}},{\mathit{E}}^T{\mathit{S}}_{\mathit{i}}\mathit{E})$, $i=0,1$, and $\gamma$ is the test threshold [16].

Sensor Selection Optimality Criteria: Under a Bayesian framework, detection performance is quantified by the overall error probabilities $P_e$, where $P_e:=P({\mathcal{H}}_0)P_F+P({\mathcal{H}}_1)P_M$ with $P_F:=P(l(\mathit{y})>\gamma |H_0)$ and $P_M:=P(l(\mathit{y})<\gamma |H_1)$ being the false alarm and the miss detection probabilities respectively. Then, the robust sensor selection problem in the hypothesis testing under a Bayesian framework is to seek the selection matrix $\mathit{E}$ such that the maximum overall error probability $P_e$ is minimized when making decisions with respect to hypothesis testing (2), that is, solving the following optimization problem:

$$\begin{array}{cc}\hfill & \underset{\mathit{E}\in {\mathbb{R}}^{m\times p}}{min}\underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{max}{P}_e\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.{\mathit{E}}^T\mathit{E}=\mathbf{I},{\mathit{E}}_{i,j}\in \{0,1\}.\hfill \end{array}$$

(4)

2.2. Problem Transformation

Since computing $P_e$ in problem (4) is usually difficult due to the involved integrals, we follow the popular approach in [11] to approximately optimize $P_e$. Based on the Chernoff lemma [17], when the number of independent identically distributed (i.i.d.) measurements increases, the rate of exponential decay of $P_e$ is equal to the Chernoff distance between the two distributions $\mathcal{N}({\mathit{E}}^T{\mathit{m}}_{\mathbf{0}},{\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E})$ and $\mathcal{N}({\mathit{E}}^T{\mathit{m}}_{\mathbf{1}},{\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E})$. On the other hand, according to the definition of Chernoff distance between two probability densities $f_0$ and $f_1$:

$$D_C(f_1,f_0):=\underset{s\in [0,1]}{max}log\int f_1^s(x)f_0^{1-s}(x)dx,$$

we derive that the Chernoff distance between distributions $\mathcal{N}({\mathit{E}}^T{\mathit{m}}_{\mathbf{0}},{\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E})$ and $\mathcal{N}({\mathit{E}}^T{\mathit{m}}_{\mathbf{1}},{\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E})$ is

$$f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}):=\underset{s\in [0,1]}{max}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$$

(5)

with

$$\begin{array}{cc}\hfill f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}):=& \frac{1}{2}s(1-s){({\mathit{m}}_{\mathbf{1}}-{\mathit{m}}_{\mathbf{0}})}^T\mathit{E}{({\mathit{E}}^T(s{\mathit{S}}_{\mathbf{0}}+(1-s){\mathit{S}}_{\mathbf{1}})\mathit{E})}^{-1}{\mathit{E}}^T\hfill \\ \hfill & \times ({\mathit{m}}_{\mathbf{1}}-{\mathit{m}}_{\mathbf{0}})-\frac{1}{2}log\frac{|{\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E}{|}^s{|{\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E}|}^{1-s}}{|{\mathit{E}}^T(s{\mathit{S}}_{\mathbf{0}}+(1-s){\mathit{S}}_{\mathbf{1}})\mathit{E}|}.\hfill \end{array}$$

(6)

Therefore, as in [11], minimizing the maximum overall error probability $P_e$ can be approximately converted to maximizing the minimum Chernoff distance $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$. Accordingly, problem (4) is transformed into

$$\begin{array}{cc}\hfill & \underset{\mathit{E}}{max}\underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{min}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.{\mathit{E}}^T\mathit{E}=\mathbf{I},{\mathit{E}}_{i,j}\in \{0,1\},\hfill \end{array}$$

(7)

or equivalently,

$$\begin{array}{cc}\hfill & \underset{\mathit{E}}{max}\underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{min}\underset{s\in [0,1]}{max}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.{\mathit{E}}^T\mathit{E}=\mathbf{I},{\mathit{E}}_{i,j}\in \{0,1\}.\hfill \end{array}$$

(8)

Work [11] has proven that problem (7) is NP-hard, and proposed a suboptimal greedy method along with projection and refinement phases to deal with problem (7). Although the greedy algorithm-based method (i.e., R-C algorithm in [11]) admits a lower computation complexity than the exhaustive method, it can not arrive at the globally optimal solution in many cases, and still remains high computation complexity for large-scale problems. Therefore, we endeavor to propose a more efficient method to obtain a better solution of problem (7).

It is not difficult to see that, solving problem (7) can be sequentially divided into the inner minimization and outer maximization. For a given selection matrix $\mathit{E}$, defining

$${\tilde{f}}_C(\mathit{E}):=\underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{min}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}),$$

(9)

and denoting $({\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ as the optimal solution of the following subproblem

$$\underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{min}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}),$$

(10)

then it holds ${\tilde{f}}_C(\mathit{E})=f_C(\mathit{E},{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$. Correspondingly, problem (7) can be equivalently transformed into

$$\begin{array}{cc}\hfill \underline{\mathbf{Primal\; Problem} (\mathbf{PP}):}& \underset{\mathit{E}\in {\mathbb{R}}^{m\times p}}{max}{\tilde{f}}_C(\mathit{E})\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.{\mathit{E}}^T\mathit{E}=\mathbf{I},{\mathit{E}}_{i,j}\in \{0,1\}.\hfill \end{array}$$

(PP)

Remarkably, the optimal solutions of problems (7) and (PP) are the same. Hence, we will discuss how to solve problem (PP) in the following. Taking the orthogonal constraint ${\mathit{E}}^T\mathit{E}=\mathbf{I}$ into account, we adopt the OCPGA-based method to deal with problem (PP).

3. OCPGA-Based Method

Referring to the greedy algorithm-based method proposed in [11], solving problem (PP) can be also successively divided into three phases: relaxation, projection and refinement. Then we will utilize the OCPGA-based method to solve problem (PP). That is, in the relaxation phase the OCPGA is used to handle the relaxed problem without 0/1 constraints, while the projection and refinement phases are the same as in the greedy algorithm-based method. First we provide the working flow of the OCPGA-based method for solving problem (PP) in Figure 1, and the details are shown in the following Section 3.1 and Section 3.2.

3.1. Relaxation Phase

By relaxing the 0/1 constraints, problem (PP) is reduced to

$$\begin{array}{ll}\underline{\mathbf{Relaxed\; Problem} (\mathbf{RP}):}& \underset{\mathit{E}\in {\mathbb{R}}^{m\times p}}{max}{\tilde{f}}_C(\mathit{E})\\ & \mathrm{s}.\mathrm{t}.{\mathit{E}}^T\mathit{E}=\mathbf{I}.\end{array}$$

(RP)

Taking into consideration the orthogonal constraint in problem (RP), once the gradient $\nabla {\tilde{f}}_C(\mathit{E})$ of the objective function ${\tilde{f}}_C(\mathit{E})$ exists and is computable, we can implement the OCPGA in [14] to solve problem (RP), which is presented in Algorithm 1.

Algorithm 1: OCPGA

The update formula of ${\mathit{Y}}_{\mathit{n}}(\tau )$ in Algorithm 1 is the Cayley transformation [18], and thereby ${\mathit{Y}}_{\mathit{n}}(\tau )$ always satisfies the orthogonal constraint ${\mathit{Y}}_{\mathit{n}}{(\tau )}^T{\mathit{Y}}_{\mathit{n}}(\tau )=\mathbf{I}$ for each iteration n. Meanwhile, the stepsize $\tau$ in Algorithm 1 is chosen by curvilinear search algorithms [19] combined with the Barzilai-Borwein (BB) [20] nonmonotonic line search [21]. It has been shown by Lemma 2.2 and Remark 2.3 in [22] that the sequence generated by the OCPGA is globally convergent to a stationary point. For clarity and integrity, we provide the convergence result of Algorithm 1 in the following theorem without proof.

Theorem 1

([22]). When ${\tilde{f}}_C(\mathit{E})$ in problem (RP) is differentiable and the gradient $\nabla {\tilde{f}}_C(\mathit{E})$ is derived, then ${\mathit{E}}^{*}$ obtained by the OCPGA in Algorithm 1 is a stationary point of problem (RP).

Moreover, the computation complexity of the OCPGA in Algorithm 1 is $\mathcal{O}(m{p}^2)$ [14]. Combined with the fact that p is generally much smaller than m, the computation complexity of the OCPGA is much lower than that $\mathcal{O}(m^{3}p)$ of the greedy algorithm in [11]. Hence, our proposed OCPGA is more efficient than the greedy algorithm, particularly for a quite large m.

3.2. Projection and Refinement Phases

Generally, the solution ${\mathit{E}}^{*}$ obtained by the OCPGA in Algorithm 1 is not a selection matrix, because the elements of ${\mathit{E}}^{*}$ are not guaranteed to be 0/1. Thus we need to further execute the projection and refinement phases as in [11].

First, we seek matrix $\tilde{\mathit{E}}$ closest to the range space of ${\mathit{E}}^{*}$ by solving the following problem

$$\begin{array}{cc}\hfill \underset{\mathit{E}}{min}& \parallel \mathit{E}{\mathit{E}}^T-{\mathit{E}}^{*}{({\mathit{E}}^{*})}^T{\parallel }_F\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathit{E}}_{i,j}\in \{0,1\},{\mathit{E}}^T\mathit{E}=\mathbf{I}.\hfill \end{array}$$

(11)

It has been shown in [11] that problem (11) admits a closed-form solution. Specifically, let $(j_1,\cdots ,j_p)$ be the indexes of the p largest entries on the diagonal of ${\mathit{E}}^{*}{({\mathit{E}}^{*})}^T$, and then $\tilde{\mathit{E}}=({\mathit{i}}_{j_1},\cdots ,{\mathit{i}}_{j_p})$ is the optimal solution of problem (11), where ${\mathit{i}}_j$ stands for the j-th column of the identity matrix ${\mathbf{I}}_m$.

Subsequently, after projecting to the set of 0/1 selection matrices, we further implement a refinement around $\tilde{\mathit{E}}$. Setting $\mathit{E}=\tilde{\mathit{E}}$, the first column of $\mathit{E}$ is viewed as the optimization variable, while all other columns are fixed. Then we sweep through all canonical vectors (i.e., columns of the identity matrix ${\mathbf{I}}_m$) different from the remaining $p-1$ columns of $\mathit{E}$, and choose the one corresponding to the maximum ${\tilde{f}}_C$ as the first column. In the next step, the procedure is repeated for the second column, and so on, up to the p-th step. Finally, one refinement is finished and the matrix $\widehat{\mathit{E}}=\mathit{E}$ is regarded as a solution of problem (PP). Obviously, the more we refine, the better solution we can achieve. When $p=1$, the solution achieved by one refinement is indeed the globally optimal selection matrix of problem (PP). If time permits, we can execute the refinement phase for several times until the objective function value ${\tilde{f}}_C$ keeps unchanged.

Notice that, the computation cost for projection is very small (as there exists an analytical solution), while in refinement we need to calculate the objective function value ${\tilde{f}}_C$ for $(m-p)p$ times. Since the OCPGA is more efficient than the greedy algorithm, and the remaining projection and refinement phases are the same, the OCPGA-based method naturally possesses higher efficiency than the greedy algorithm-based method.

Remark 1.

For the OCPGA in Algorithm 1, its initial point is randomly chosen from column orthogonal matrices. Experiments illustrate that different initial points may result in different outcomes. Therefore, in order to improve the performance, we implement the OCPGA with different initial points several times, and after projection and refinement, choose the best solution as the output.

Remark 2.

According to the above discussion, the key to implement the OCPGA-based method is computing the gradient $\nabla {\tilde{f}}_C(\mathit{E})$ in problem (RP). In the next section, we will show how to obtain the gradient $\nabla {\tilde{f}}_C(\mathit{E})$ and discuss when it exists.

4. Existence and Computation of the Gradient in Problem (RP)

As can be easily seen from Algorithm 1, it is essential to compute the gradient $\nabla {\tilde{f}}_C(\mathit{E})$ in problem (RP). Invoking the definition of ${\tilde{f}}_C(\mathit{E})$ in Equation (9), we have

$${\tilde{f}}_C(\mathit{E})=\underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{min}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}),$$

with $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=\underset{s\in [0,1]}{max}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$. To proceed, we first prove the strict concavity of $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ with respect to s, which forms a key ingredient of our later arguments.

Proposition 1.

Given $\mathit{E}$ and $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ defined by Equation (6) is a strictly concave function of s.

Proof. 

It is easily seen that $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is continuously differentiable with respect to s for given $\mathit{E}$ and $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$. Hence we turn to show the positiveness of the second derivative ${\nabla }_s^{2}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$.

For notation brevity, define $\mathsf{\Lambda }:=({\mathit{m}}_{\mathbf{1}}-{\mathit{m}}_{\mathbf{0}}){({\mathit{m}}_{\mathbf{1}}-{\mathit{m}}_{\mathbf{0}})}^T$ and $\mathit{B}(s):=\mathit{E}{({\mathit{E}}^T(s{\mathit{S}}_{\mathbf{0}}+(1-s){\mathit{S}}_{\mathbf{1}})\mathit{E})}^{-1}{\mathit{E}}^T$. It is clear that both $\mathsf{\Lambda }$ and $\mathit{B}(s)$ are semi-positive definite. By taking derivative of $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ with respect to s, we have

$$\begin{array}{cc}{\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=\hfill & (\frac{1}{2}-s)\mathrm{tr}\left[\mathit{B}(s)\mathsf{\Lambda }\right]+\frac{1}{2}\mathrm{tr}\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]\hfill \\ & -\frac{1}{2}s(1-s)\mathrm{tr}\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\mathit{B}(s)\mathsf{\Lambda }\right]-\frac{1}{2}log\frac{|{\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E}|}{|{\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E}|}.\hfill \end{array}$$

(12)

Then, taking derivative of ${\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ with respect to s once again, we obtain

$$\begin{array}{cc}\hfill {\nabla }_s^{2}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=& -\mathrm{tr}\left[\mathit{B}(s)\mathsf{\Lambda }\right]+(2s-1)\mathrm{tr}\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\mathit{B}(s)\mathsf{\Lambda }\right]\hfill \\ \hfill & +s(1-s)\mathrm{tr}\left\{{\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]}^2\mathit{B}(s)\mathsf{\Lambda }\right\}-\frac{1}{2}\mathrm{tr}\left\{{\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]}^2\right\}\hfill \\ \hfill \stackrel{(a)}{=}& -\mathrm{tr}\left[\mathit{B}(s)\mathsf{\Lambda }\right]+(s-1)\mathrm{tr}\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\mathit{B}(s)\mathsf{\Lambda }\right]\hfill \\ \hfill & +s\mathrm{tr}\left[\mathit{B}(s){\mathit{S}}_{\mathbf{0}}\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\mathit{B}(s)\mathsf{\Lambda }\right]-\frac{1}{2}\mathrm{tr}\left\{{\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]}^2\right\}\hfill \\ \hfill \stackrel{(b)}{=}& -\frac{1}{1-s}\mathrm{tr}\left[\mathit{B}(s){\mathit{S}}_{\mathbf{0}}\mathit{B}(s)\mathsf{\Lambda }\right]+\frac{s}{1-s}\mathrm{tr}\left[\mathit{B}(s){\mathit{S}}_{\mathbf{0}}\mathit{B}(s){\mathit{S}}_{\mathbf{0}}\mathit{B}(s)\mathsf{\Lambda }\right]-\frac{1}{2}\mathrm{tr}\left\{{\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]}^2\right\}\hfill \\ \hfill \stackrel{(c)}{=}& -\frac{1}{1-s}\mathrm{tr}\left\{{\mathsf{\Lambda }}^{\frac{1}{2}}\mathit{B}(s){\mathit{S}}_{\mathbf{0}}\mathit{E}\mathsf{\Gamma }(s){\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{B}(s){\mathsf{\Lambda }}^{\frac{1}{2}}\right\}-\frac{1}{2}\mathrm{tr}\left\{{\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]}^2\right\},\hfill \end{array}$$

(13)

where $\mathsf{\Gamma }(s)={({\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E})}^{-1}-{({\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E}+\frac{1-s}{s}{\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E})}^{-1}$, equalities $(a)$ and $(b)$ both result from $\mathit{B}(s)=\mathit{B}(s)[s{\mathit{S}}_{\mathbf{0}}+(1-s){\mathit{S}}_{\mathbf{1}}]\mathit{B}(s)$ and thus $(1-s)\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\mathit{B}(s)=\mathit{B}(s){\mathit{S}}_{\mathbf{0}}\mathit{B}(s)-\mathit{B}(s)$, and equality $(c)$ owing to the definition of $\mathit{B}(s)$ and the relationship $\mathrm{tr}(\mathit{AB})=\mathrm{tr}(\mathit{BA})$ for matrices $\mathit{A}$ and $\mathit{B}$.

If ${\mathit{E}}^T{\mathit{m}}_{\mathbf{0}}\ne {\mathit{E}}^T{\mathit{m}}_{\mathbf{1}}$, then we have ${\mathit{E}}^T{\mathsf{\Lambda }}^{\frac{1}{2}}\ne \mathbf{0}$. Because of ${\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E}\succ \mathbf{0}$, it holds

$$\begin{array}{c}\hfill {\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{B}(s){\mathsf{\Lambda }}^{\frac{1}{2}}={\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E}{\left[\mathit{A}(s)\right]}^{-1}{\mathit{E}}^T{\mathsf{\Lambda }}^{\frac{1}{2}}\ne \mathbf{0}\end{array}$$

with $\mathit{A}(s)={\mathit{E}}^T(s{\mathit{S}}_{\mathbf{0}}+(1-s){\mathit{S}}_{\mathbf{1}})\mathit{E}\succ \mathbf{0}$. Combined with $\mathsf{\Gamma }(s)\succ \mathbf{0}$, $\mathit{B}(s)⪰\mathbf{0}$, and $\mathrm{tr}\{{\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]}^2\}=\mathrm{tr}\{\mathit{B}(s)\left[({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]\}⩾0$, we conclude ${\nabla }_s^{2}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})<0$ from Equation (13).

On the other hand, when ${\mathit{E}}^T{\mathit{m}}_{\mathbf{0}}={\mathit{E}}^T{\mathit{m}}_{\mathbf{1}}$, then it follows from Equation (3) that ${\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E}\ne {\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E}$. If ${\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]}^2=\mathbf{0}$, Then we have $\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\mathit{B}(s)=\mathbf{0}$, that is

$$\begin{array}{c}\hfill \mathit{E}{\left[\mathit{A}(s)\right]}^{-1}({\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E}-{\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E}){\left[\mathit{A}(s)\right]}^{-1}{\mathit{E}}^T=\mathbf{0}.\end{array}$$

Then it immediately follows ${\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E}-{\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E}=\mathbf{0}$, which leads to a contradiction. Therefore, when ${\mathit{E}}^T{\mathit{m}}_{\mathbf{0}}={\mathit{E}}^T{\mathit{m}}_{\mathbf{1}}$, we deduce $\mathrm{tr}\left\{{\left[\mathit{B}(s)({\mathit{S}}_{\mathbf{0}}-{\mathit{S}}_{\mathbf{1}})\right]}^2\right\}>0$, which means ${\nabla }_s^{2}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})<0$ by Equation (13).

As a consequence, $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is a strictly concave function of s. □

Moreover, the following Proposition 2 shows that $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is continuously differentiable with respect to $\mathit{E}$ for given $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, which is also necessary for follow-up analysis.

Proposition 2.

Given $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ defined by (5) is a continuously differentiable function of $\mathit{E}$.

Proof. 

For given $\mathit{E}$ and $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, denote $s^{*}:=arg\underset{s\in [0,1]}{max}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ with $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ given by Equation (6). Note that $f(\mathit{E},0,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=f(\mathit{E},1,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=0$ and $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=\underset{s\in [0,1]}{max}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})>0$ under condition (3). Hence, $s^{*}\in (0,1)$, which, combined with the strict concavity of $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ in Proposition 1, implies that $s^{*}$ is unique with satisfying ${\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=0.$

Recalling the expression of ${\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ in (12), since all the involved terms $B(s)$, $\mathrm{tr}(·)$, $|·|$, $log(·)$ and $E^T{S}_{0}E\ne 0$ are continuously differentiable functions of E, then ${\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ as a composite function of the above functions is also continuously differentiable with respect to E. Similarly, ${\nabla }_s^{2}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ in (13) is a composition of functions $\frac{1}{1-s}$, $B(s)$ and $\mathrm{tr}(·)$, which are all continuous with respect to s. Hence, ${\nabla }_s^{2}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is continuous with respect to s, that is, ${\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is continuously differentiable with respect to s. Due to ${\nabla }_s^{2}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\ne 0$, it follows from the implicit function theorem [23] that $s^{*}$ is an implicit function of $\mathit{E}$. Hence, we rewrite $s^{*}$ as

$$s^{*}(\mathit{E}):=arg\underset{s\in [0,1]}{max}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}).$$

(14)

Furthermore, based on the implicit function theorem [23], $s^{*}(\mathit{E})$ is a continuously differentiable function of $\mathit{E}$, which means that $\nabla s^{*}(\mathit{E})$ is continuous with respect to $\mathit{E}$.

Because of $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=\underset{s\in [0,1]}{max}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, we have $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=f(\mathit{E},s^{*}(\mathit{E}),{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ immediately. In addition, $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is continuously differentiable with s and $\mathit{E}$. Combined with the fact that $s^{*}(\mathit{E})$ is a continuously differentiable function of $\mathit{E}$, thereby $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is differentiable with respect to $\mathit{E}$.

Moreover, by leveraging on the chain rule [24] to $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=f(\mathit{E},s^{*}(\mathit{E}),{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, it holds

$$\begin{array}{cc}\hfill & {\nabla }_{\mathit{E}}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=\left[{\nabla }_{\mathit{E}}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})+{\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\nabla s^{*}(\mathit{E})\right]{|}_{s=s^{*}(\mathit{E})},\hfill \end{array}$$

where

$$\begin{array}{cc}\hfill {\nabla }_{\mathit{E}}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})& =s(1-s)[\mathbf{I}-\mathit{W}(s)\mathit{E}{\mathit{A}}^{-1}(s){\mathit{E}}^T]\mathsf{\Lambda }\mathit{E}{\mathit{A}}^{-1}(s)\hfill \\ \hfill & -s{\mathit{S}}_{\mathbf{0}}\mathit{E}{({\mathit{E}}^T{\mathit{S}}_{\mathbf{0}}\mathit{E})}^{-1}-(1-s){\mathit{S}}_{\mathbf{1}}\mathit{E}{({\mathit{E}}^T{\mathit{S}}_{\mathbf{1}}\mathit{E})}^{-1}+\mathit{W}(s)\mathit{E}{\mathit{A}}^{-1}(s)\hfill \end{array}$$

(15)

with $\mathit{W}(s)=s{\mathit{S}}_{\mathbf{0}}+(1-s){\mathit{S}}_{\mathbf{1}}$, $\mathit{A}(s)={\mathit{E}}^T\mathit{W}(s)\mathit{E}$, and $\mathsf{\Lambda }=({\mathit{m}}_{\mathbf{1}}-{\mathit{m}}_{\mathbf{0}}){({\mathit{m}}_{\mathbf{1}}-{\mathit{m}}_{\mathbf{0}})}^T$. Since $\nabla s^{*}(\mathit{E})$, ${\nabla }_{s}f(\mathit{E},s,m_0,{\mathit{m}}_{\mathbf{1}})$ and ${\nabla }_{\mathit{E}}f(\mathit{E},s,m_0,{\mathit{m}}_{\mathbf{1}})$ are all continuous with respect to $\mathit{E}$, then ${\nabla }_{\mathit{E}}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is also continuous. In conclusion, $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is a continuously differentiable function of $\mathit{E}$. □

4.1. Compute the Gradient in Problem (RP) by Danskin’s Theorem

In the sequel, we will exploit Danskin’s theorem in Appendix A to compute the gradient $\nabla {\tilde{f}}_C(\mathit{E})$, where ${\tilde{f}}_C(\mathit{E})$ is defined by Equation (9).

On basis of Proposition 2, the following results hold true for function $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ and set $\mathit{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathit{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})$:

• $\mathit{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathit{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})$ is a compact set because it is a finite-dimensional bounded closed set. Meanwhile, $\forall ({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, mapping $(t,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\to f_C(\mathit{E}+t\mathit{h},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is continuous at point $(0,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ due to the continuity of $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$;
• For arbitrary given $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ and sufficiently small $t>0$, since $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is differentiable, there exists a bounded directional derivative

  $$\begin{array}{c}\hfill D_1{f}_C(\mathit{E}+t\mathit{h},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}};\mathit{h})=\underset{\tau \to 0^{+}}{lim}\frac{1}{\tau }\times \left[f_C(\mathit{E}+(t+\tau )\mathit{h},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})-f_C(\mathit{E}+t\mathit{h},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\right];\end{array}$$
• Mapping $(t,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\to D_1{f}_C(\mathit{E}+t\mathit{h},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is continuous at point $(0,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, which is from the continuity of ${\nabla }_{\mathit{E}}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$.

With identifications $\mathit{E}\sim u$, $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\sim v$, ${\mathbb{R}}^{m\times p}\sim U$, $\mathit{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathit{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})\sim V$, $-f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\sim J(u,v)$, and $-{\tilde{f}}_C(\mathit{E})\sim \overline{J}(u)$, all conditions of Danskin’s theorem in Appendix A are satisfied. Subsequently, for a given selection matrix $\mathit{E}$, if the optimal solution $({\widehat{m}}_0(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ of problem (10) is unique, then the gradient of ${\tilde{f}}_C(\mathit{E})$ exists. Furthermore, on basis of the Danskin’s theorem, we have

$$\nabla {\tilde{f}}_C(\mathit{E})={\nabla }_{\mathit{E}}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}){|}_{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=({\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))}.$$

(16)

Recall $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=\underset{s\in [0,1]}{max}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ from Equation (5). For given $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, we can again utilize the Danskin’s theorem in Appendix A to obtain ${\nabla }_{\mathit{E}}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$. Obviously, $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is continuously differentiable with respect to $\mathit{E}$. Therefore, it is easy to verify that conditions 1)−3) of Danskin’s theorem in Appendix Appendix A are all satisfied with identifications $\mathit{E}\sim u$, $s\sim v$, ${\mathbb{R}}^{m\times p}\sim U$, $[0,1]\sim V$, $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\sim J(u,v)$, and $f_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\sim \overline{J}(u)$. Combined with the uniqueness of $s^{*}(\mathit{E})$ defined by Equation (14), we can deduce

$${\nabla }_{\mathit{E}}{f}_C(\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})={\nabla }_{\mathit{E}}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}){|}_{s=s^{*}(\mathit{E})}.$$

(17)

On the other hand, due to the equivalence of Problems (7) and (8), for a given matrix $\mathit{E}$, a solution $({\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ of Problem (10) corresponds to a solution $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ of problem

$$\underline{\mathbf{Iner\; Problem} (\mathbf{IP}):}\underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{min}\underset{s\in [0,1]}{max}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}),$$

(IP)

satisfying $f_C(\mathit{E},{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))=f(\mathit{E},\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ and

$$\widehat{s}(\mathit{E})=arg\underset{s\in [0,1]}{max}f(\mathit{E},s,{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E})).$$

(18)

Combined with Equations (16) and (17), when the solution $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ is unique, we have

$$\begin{array}{cc}\hfill & \nabla {\tilde{f}}_C(\mathit{E})={\nabla }_{\mathit{E}}f(s,\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}){|}_{(s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))},\hfill \end{array}$$

(19)

where ${\nabla }_{\mathit{E}}f(s,\mathit{E},{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is given by Equation (15) and $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ is the optimal solution of problem (IP).

4.2. Compute the Optimal Solution of Problem (IP)

Owing to the Sion’s minimax theorem [25] and Lemma 5 in [11], for given $\mathit{E}$, we can exchange the orders of the minimization with respect to $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ and the maximization with respect to s in problem (IP). Correspondingly, problem (IP) can be equivalently converted into

$$\begin{array}{cc}\hfill & \underset{s\in [0,1]}{max}\underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{min}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}).\hfill \end{array}$$

(20)

Therefore, we will solve Problem (20) to attain the solution $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ of Problem (IP).

First, for given matrix $\mathit{E}$ and parameter s, denote

$$({\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s)):=arg\underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{min}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}).$$

Combining with the expression of $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ in Equation (6) and removing items irrelevant to $({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$, then we can get $({\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ by solving the following subproblem

$$\begin{array}{c}\hfill \underset{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})\in \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})}{min}{({\mathit{m}}_{\mathbf{1}}-{\mathit{m}}_{\mathbf{0}})}^T\mathit{B}({\mathit{m}}_{\mathbf{1}}-{\mathit{m}}_{\mathbf{0}})\end{array}$$

(21)

with $\mathit{B}:=\mathit{E}{({\mathit{E}}^T(s{\mathit{S}}_{\mathbf{0}}+(1-s){\mathit{S}}_{\mathbf{1}})\mathit{E})}^{-1}{\mathit{E}}^T⪰\mathbf{0}$. Clearly, Problem (21) is convex, which can be directly solved by the CVX toolbox [26] with computation complexity $\mathcal{O}(p^3)$ [27]. Thus, once s is fixed, the corresponding $({\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ follows. Particularly, $({\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))=({\mathit{m}}_{\mathbf{0}}^{*}(\widehat{s}(\mathit{E})),{\mathit{m}}_{\mathbf{1}}^{*}(\widehat{s}(\mathit{E})))$.

Next, we will show how to determine the optimal $\widehat{s}(\mathit{E})$ given by Equation (18). Based on Proposition 1, $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is concave with respect to s. Combined with the fact that $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ is the minimum of a family of functions $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ over the uncertainty set $\mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})\times \mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})$, we conclude that $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ is also a concave function of s [28]. Therefore, ${\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ is monotonically decreasing with respect to s. Subsequently, we apply the efficient bisection method to search $\widehat{s}(\mathit{E})$ such that ${\nabla }_{s}f(\mathit{E},\widehat{s}(\mathit{E}),{\mathit{m}}_{\mathbf{0}}^{*}(\widehat{s}(\mathit{E})),{\mathit{m}}_{\mathbf{1}}^{*}(\widehat{s}(\mathit{E})))=0.$

For given $\mathit{E}$, we can once again use the Danskin’s theorem in Appendix A to derive ${\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$. Since $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is continuously differentiable with respect to s, all conditions of Danskin’s theorem in Appendix A are met. Meanwhile, even if the optimal solution $({\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ of Problem (21) is not unique, ${\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ in Equation (12) is always the same no matter which $({\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ is substituted. Therefore, $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ is differentiable with respect to s, and we have

$$\begin{array}{c}\hfill {\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))={\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}}){|}_{({\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})=({\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))},\end{array}$$

(22)

where ${\nabla }_{s}f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ is given by Equation (12), and $({\mathit{m}}_{\mathbf{0}}^{*}(s),{\mathit{m}}_{\mathbf{1}}^{*}(s))$ is an arbitrary optimal solution of Problem (21).

After we get the optimal $\widehat{s}(\mathit{E})$, the corresponding $({\mathit{m}}_{\mathbf{0}}^{*}(\widehat{s}),{\mathit{m}}_{\mathbf{1}}^{*}(\widehat{s}))$, i.e., $({\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$, is acquired by solving Problem (21). That is, the optimal solution $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ of Problem (IP) is obtained. We summarize the process of solving $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ as the following Inner Procedure 1.

Inner Procedure 1: Computing the Optimal Solution of Problem (IP)

Remark 3.

If the true distribution under each hypothesis is exactly known, i.e., there is no uncertainty in the mean vector, then we omit the process of computing solution $({\mathit{m}}_{\mathbf{0}}^{*}(\widehat{s}),{\mathit{m}}_{\mathbf{1}}^{*}(\widehat{s}))$ of problem (21), and directly regard the true mean vector as $({\mathit{m}}_{\mathbf{0}}^{*}(\widehat{s}),{\mathit{m}}_{\mathbf{1}}^{*}(\widehat{s}))$. Consequently, the robust sensor selection problem is reduced to one without uncertainty.

Based on the previous discussion, if the optimal solution $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ of Problem (IP) is unique, then the gradient $\nabla {\tilde{f}}_C(\mathit{E})$ exists and can be computed by Equation (19). Next we will discuss the existence of the gradient $\nabla {\tilde{f}}_C(\mathit{E})$ detailedly.

4.3. Existence of the Gradient in Problem (RP)

It has been shown that the uniqueness of the optimal solution $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ leads to the existence of the gradient $\nabla {\tilde{f}}_C(\mathit{E})$. Therefore, in the following, we turn to show when $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ is unique. First the following lemma demonstrates the uniqueness of $\widehat{s}(\mathit{E})$.

Lemma 1.

For given $\mathit{E}$, let $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ be the optimal solution of Problem (IP). Then $\widehat{s}(\mathit{E})$ is unique.

Proof. 

We will prove by contradiction. Assume that $(\tilde{s}(\mathit{E}),{\tilde{m}}_0(\mathit{E}),{\tilde{m}}_1(\mathit{E}))$ is also the optimal solution of problem (IP) with $\tilde{s}(\mathit{E})\ne \widehat{s}(\mathit{E})$.

It is worth noting that, both $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ and $(\tilde{s}(\mathit{E}),{\tilde{m}}_0(\mathit{E}),{\tilde{m}}_1(\mathit{E}))$ are the saddle points of problem (IP). According to Proposition 1.4 in [29], the set of saddle points admits the Cartesian product form. Hence, $(\tilde{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ is also a saddle point. That is, for given $({\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$, both $\widehat{s}(\mathit{E})$ and $\tilde{s}(\mathit{E})$ are the optimal solutions of problem $\underset{s\in [0,1]}{max}f(\mathit{E},s,{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$, which contradicts with the strict concavity of $f(\mathit{E},s,{\mathit{m}}_{\mathbf{0}},{\mathit{m}}_{\mathbf{1}})$ with respect to s. Consequently, $\widehat{s}(\mathit{E})$ is unique. □

With the uniqueness of $\widehat{s}(\mathit{E})$, we additionally make the following assumption to guarantee the existence of $\nabla {\tilde{f}}_C(\mathit{E})$.

Assumption 1.

For given matrix $\mathit{E}$, the saddle point $(\widehat{s}(\mathit{E}),{\mathit{m}}_{\mathbf{0}}^{*}(\widehat{s}(\mathit{E})),{\mathit{m}}_{\mathbf{1}}^{*}(\widehat{s}(\mathit{E})))$ of Problem (IP) is unique.

Remark 4.

Because Lemma 1 has proven that $\widehat{s}(\mathit{E})$ is unique, if the solution $({\mathit{m}}_{\mathbf{0}}^{*}(\widehat{s}(\mathit{E})),{\mathit{m}}_{\mathbf{1}}^{*}(\widehat{s}(\mathit{E})))$ of Problem (21) is unique, then Assumption A1 is satisfied. Under some conditions (e.g., the distance of the two uncertainty sets $\mathcal{E}({\overline{\mathit{m}}}_{\mathbf{0}},k_0{{\mathit{S}}_{\mathbf{0}}}^{-1})$ and $\mathcal{E}({\overline{\mathit{m}}}_{\mathbf{1}},k_1{{\mathit{S}}_{\mathbf{1}}}^{-1})$ is large), $({\mathit{m}}_{\mathbf{0}}^{*}(\widehat{s}(\mathit{E})),{\mathit{m}}_{\mathbf{1}}^{*}(\widehat{s}(\mathit{E})))$ of Problem (21) is naturally unique. Therefore, Assumption 1 is not very restrictive.

Under Assumption 1, $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$ is unique and achievable by Inner Procedure 1. Hence, $\nabla {\tilde{f}}_C(\mathit{E})$ exists and can be obtained by Equation (19). Subsequently, Algorithm 1 can be executed. Furthermore, on basis of Theorem 1 in Section 3.1, we conclude that ${\mathit{E}}^{*}$ obtained by the OCPGA in Algorithm 1 is a stationary point of problem (RP), which lays a foundation to attain a better solution of the original problem (PP).

Remark 5.

If Assumption 1 is not satisfied, we could use the Clark generalized gradient [30] to replace the gradient in Algorithm 1. Based on the Danskin’s theorem in Appendix A, the Clark generalized gradient can also be attained by Equation (19), where an arbitrary solution of Problem (21) is used. Then Algorithm 1 is still applicable and preserves the orthogonal constraint in each iteration. Although the resulting solution is not guaranteed to be a stationary point of Problem (RP), however, after projection and refinement phases, numerical simulations illustrate that the performance of the final result is also acceptable.

5. Numerical Simulations

In this section, numerical examples are carried out to show that the OCPGA-based method can obtain better solutions than the greedy algorithm-based method in [11]. To this end, (1) for fixed-size sensor selection problems, i.e., the total number and selected number of sensors are fixed, with randomly generated 50 (or 20) cases with different uncertainty sets, we exhibit the proportions that the OCPGA-based method performs better than, the same as, and worse than the greedy algorithm-based method; (2) for small-scale sensor selection problems, we compare the OCPGA-based method with the greedy algorithm based method and the exhaustive method; (3) for larger-scale sensor selection problems, the OCPGA-based method is compared with the greedy algorithm based method; (4) for specific small-scale and larger-scale sensor selection problems, the corresponding receiver operating characteristic (ROC) curves are depicted. Notably, in cases of (1)–(3), the performance of the method is measured by the resulting Chernoff distance (i.e., ${\tilde{f}}_C(\mathit{E})$ in Problem (PP)), that is, the method with larger Chernoff distance admits better performance. All the procedures are coded in MATLAB R2014b on an ASUS notebook with the Intel(R) Core(TM) i3-2310M CPU of 2.10GHz and memory of 6GB.

Assume that we need to choose p out of m sensors. In all simulations, for given $(m,p)$, the ellipsoidal uncertainty sets $\mathcal{E}({\overline{\mathit{m}}}_{\mathit{i}},k_i{{\mathit{S}}_{\mathit{i}}}^{-1})$ in Problem (IP), $i=0,1$, which contain the true distribution under each hypothesis, are generated as follows. Elements of the estimated mean vectors ${\overline{\mathit{m}}}_{\mathbf{0}}$ and ${\overline{\mathit{m}}}_{\mathbf{1}}$ are randomly generated from $(0,1)$ and $(0,2)$, respectively. The covariance matrix $S_i$ is generated by ${\mathit{S}}_{\mathit{i}}={{\mathit{P}}_{\mathit{i}}{\mathsf{\Sigma }}_{\mathit{i}}{\mathit{P}}_{\mathit{i}}}^T$, where ${\mathit{P}}_{\mathit{i}}$ is an orthogonal basis of $m\times m$-dimensional matrices whose elements are generally drawn from $(0,1)$, and ${\mathsf{\Sigma }}_{\mathit{i}}$ is a diagonal matrix with diagonal entries randomly generated in $(0,2)$, $i=0,1$. The robustness parameters $k_0=k_1=1$.

When $(m,p)$ and the ellipsoidal uncertainty sets $\mathcal{E}({\overline{\mathit{m}}}_{\mathit{i}},k_i{{\mathit{S}}_{\mathit{i}}}^{-1})$ are given, we adopt Inner Procedure 1 to compute $(\widehat{s}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{0}}(\mathit{E}),{\widehat{\mathit{m}}}_{\mathbf{1}}(\mathit{E}))$, use Equation (19) to obtain the gradient $\nabla {\tilde{f}}_C(\mathit{E})$, and then apply the OCPGA in Algorithm 1 to get the stationary point ${\mathit{E}}^{*}$ of Problem (RP). After the projection and refinement phases described in Section 3.2, the final solution $\widehat{\mathit{E}}$ of Problem (PP) is achieved. Meanwhile, the greedy algorithm-based method is a deterministic approach, that is, for given $(m,p)$ and ellipsoidal uncertainty set, its outputs are all the same no matter how many times it is recalled. On the contrary, since the initial point of the OCPGA in Algorithm 1 is randomly chosen, the outputs of the OCPGA-based method may vary with initial points of the OCPGA. If time permits, the OCPGA-based method can be recalled for several times to achieve better performance. Moreover, the OCPGA and the greedy algorithm based methods both execute one refinement only.

Fixed-Size Examples: For 8 fixed pairs of small $(m,p)$, we give the proportions that the OCPGA-based method performs better than, the same as, and worse than the greedy algorithm-based method. For each $(m,p)$, by implementing the two methods with randomly generated 50 different ellipsoidal uncertainty sets (only 20 different ellipsoidal uncertainty sets for $(50,5)$, $(80,5)$, and $(100,5)$ due to their long runtimes), the corresponding results are listed in

Table 1. Performance Comparison Proportions.

$(\mathbf{m},\mathbf{p})$	Better	Same	Worse
$(10,3)$	20%	74%	6%
$(10,4)$	24%	72%	4%
$(10,5)$	20%	76%	4%
$(12,3)$	18%	74%	8%
$(15,3)$	22%	74%	4%
$(50,5)$	20%	70%	10%
$(80,5)$	15%	80%	5%
$(100,5)$	10%	85%	5%

. Here, with each ellipsoidal uncertainty set, the OCPGA-based method is recalled for two times and the better result is selected as the output. As shown in Table 1, for each $(m,p)$, the OCPGA-based method performs as well as the greedy algorithm-based method in most cases, while the “better” proportion is more than twice as many as the “worse” proportion. Actually, simulations show that, for “worse” cases, if we recall the OCPGA-based method for more times, then it can perform as well as even better than the greedy algorithm-based method.

Small-Scale Examples: we consider small-scale sensor selection problems, whose globally optimal solutions can be attained by the exhaustive method. Hence, we compare the optimal Chernoff distance obtained by the OCPGA-based method with those of the greedy algorithm-based method and the exhaustive method. Suppose that $p=3,4,5$ out of $m=10,12,15$ sensors are chosen. The corresponding outputs of the three methods are given in

Table 2. Optimal Chernoff Distances of Three Methods.

m	$\mathit{p}=3$			$\mathit{p}=4$			$\mathit{p}=5$
	Exh	Gre	OCP	Exh	Gre	OCP	Exh	Gre	OCP
10	0.15	0.11	0.15	0.34	0.28	0.34	0.29	0.26	0.29
12	0.51	0.35	0.51	0.25	0.21	0.25	0.62	0.34	0.62
15	0.30	0.15	0.30	0.77	0.76	0.77	0.20	0.19	0.20

Exh: Exhaustive Method; Gre: Greedy Algorithm Based Method; OCP: OCPGA Based Method.

, and the corresponding runtimes of the three methods are listed in

Table 3. Runtimes of Three Methods (s).

m	$\mathit{p}=3$			$\mathit{p}=4$			$\mathit{p}=5$
	Exh	Gre	OCP	Exh	Gre	OCP	Exh	Gre	OCP
10	2.59$\times {10}^3$	52.12	51.07	1.10$\times {10}^4$	204.77	105.01	6.49 $\times {10}^4$	87.07	80.39
12	4.17$\times {10}^3$	128.75	109.35	2.66$\times {10}^4$	82.41	75.94	2.04$\times {10}^5$	106.53	98.31
15	4.33$\times {10}^4$	92.68	89.28	1.50$\times {10}^5$	193.45	188.95	1.20$\times {10}^6$	153.09	150.41

Exh: Exhaustive Method; Gre: Greedy Algorithm Based Method; OCP: OCPGA Based Method.

. It can be easily seen from Table 2 that, the Chernoff distances achieved by the OCPGA-based method are larger than those of the existing greedy algorithm-based method. For the case of $(15,3)$, the Chernoff distance achieved by the OCPGA-based method is even $100\%$ larger than that of the greedy algorithm-based method. In particular, our proposed OCPGA-based method can attain the same performance as the exhaustive method. Meanwhile, it is shown in Table 3 that, the OCPGA-based method is more efficient than the greedy algorithm-based method, and both of them possess much shorter runtimes than the exhaustive method, which is coincident with the theoretical computation complexity analyses. Via simple computations, we can see from Table 3 that the runtime of the OCPGA-based method can be up to $48.72\%$ shorter than that of greedy algorithm-based method (for the case of $(10,4)$). Therefore, our proposed OCPGA-based method admits better performance in terms of not only the objective value but also the runtime.

Larger-Scale Examples: Now we consider larger-scale problems, where $m=50,80,100$ and $p=5,10,15$. Since m and p are large, which leads to the failure of the exhaustive method, we only compare the obtained Chernoff distances of our OCPGA-based method with those of the greedy algorithm-based method. The corresponding results are exhibited in

Table 4. Optimal Chernoff Distances of Two Methods.

m	$\mathit{p}=5$		$\mathit{p}=10$		$\mathit{p}=15$
	Gre	OCP	Gre	OCP	Gre	OCP
50	1.37	1.55	3.59	3.68	3.82	3.85
80	1.32	1.34	3.36	3.41	5.86	5.92
100	1.69	1.71	4.46	4.56	5.74	6.06

Gre: Greedy Algorithm Based Method; OCP: OCPGA Based Method.

, while the corresponding runtimes of the two methods are displayed in

Table 5. Runtimes of Two Methods (s).

m	$\mathit{p}=5$		$\mathit{p}=10$		$\mathit{p}=15$
	Gre	OCP	Gre	OCP	Gre	OCP
50	1499.76	866.96	2083.39	1956.96	3794.17	3635.68
80	1839.27	1718.73	5435.69	5251.96	5350.99	5240.56
100	2290.26	1752.71	5589.49	5273.85	11966.38	11433.33

Gre: Greedy Algorithm Based Method; OCP: OCPGA Based Method.

. As we can see from Table 4, the OCPGA-based method can attain a better solution than the greedy algorithm-based method. In the case of $(50,5)$, the Chernoff distance obtained by the OCPGA-based method can be $13.14\%$ larger than that of the greedy algorithm-based method. Moreover, Table 5 illustrates that, the OCPGA-based method admits higher efficiency than the greedy algorithm -based method, and the runtime of the OCPGA-based method can reduce up to $42.19\%$ in the case of $(50,5)$. Compared with small-scale cases in Table 3, we can see from Table 5 that the improvement in runtime is more obvious, which is due to the larger gap between m and p for larger-scale cases. Hence, for larger-scale cases, the OCPGA-based method also can achieve better solutions than the greedy algorithm-based method with shorter runtimes.

ROC Curves for Specific Examples: By Monte Carlo simulations with 200,000 instantiations of the LR tests and calculating $P_D/P_F$ with $P_D:=1-P_M$ being the detection probability, we depict the ROC curves to show the validity of the OCPGA-based method. It is well known that the higher the ROC curve, the better the detection performance. For the case of $(m,p)=(10,3)$ in Table 2, we display the corresponding ROC curves of the exhaustive method, the greedy algorithm-based method and the OCPGA-based method. It can be seen from Figure 2 that the OCPGA-based method is superior to the greedy algorithm-based method, while it can attain the same performance as that of the exhaustive method. Similarly, for the case of $(m,p)=(50,5)$ in Table 4, Figure 3 illustrates that our proposed approach performs better than the existing greedy algorithm-based method.

In summary, compared with the greedy algorithm-based method, the OCPGA-based method not only admits a lower theoretical computation complexity, but also can obtain better solutions with shorter runtimes in numerical simulations.

6. Conclusions

We address the minimax robust sensor selection in the binary Gaussian distribution hypothesis testing of WSN with the distribution mean vector under each hypothesis drifting in an ellipsoidal uncertainty set. Under a Bayesian optimal criterion, minimizing the maximum overall error probability with respect to the selection matrix is approximately converted to maximizing the minimum Chernoff distance between the distributions under a null hypothesis and alternative hypothesis to be detected. Then, the gradient of the objective function of the converted maximization problem is computed by Danskin’s theorem. Furthermore, we apply the OCPGA to solve the relaxed maximization problem without 0/1 constraints, which can get a stationary point of the relaxed problem with lower computational complexity than the existing greedy algorithm. Numerical simulations demonstrate that the OCPGA-based method can attain better solutions than the greedy algorithm-based method with up to $48.72\%$ shorter runtimes, and the OCPGA-based method is able to attain the globally optimal solution obtained by the exhaustive method for small-scale problems. In future work, we can consider cases where the distribution mean falls in other types of uncertainty sets such as the band model. In addition, the cases with not precisely known distribution covariance can also be a future research direction.