Efficient Sensor Node Selection for Observability Gramian Optimization

Abstract

Optimization approaches that determine sensitive sensor nodes in a large-scale, linear time-invariant, and discrete-time dynamical system are examined under the assumption of independent and identically distributed measurement noise. This study offers two novel selection algorithms, namely an approximate convex relaxation method with the Newton method and a gradient greedy method, and confirms the performance of the selection methods, including a convex relaxation method with semidefinite programming (SDP) and a pure greedy optimization method proposed in the previous studies. The matrix determinant of the observability Gramian was employed for the evaluations of the sensor subsets, while its gradient and Hessian were derived for the proposed methods. In the demonstration using numerical and real-world examples, the proposed approximate greedy method showed superiority in the run time when the sensor numbers were roughly the same as the dimensions of the latent system. The relaxation method with SDP is confirmed to be the most reasonable approach for a system with randomly generated matrices of higher dimensions. However, the degradation of the optimization results was also confirmed in the case of real-world datasets, while the pure greedy selection obtained the most stable optimization results.

1. Introduction

Collecting information from sensor measurements is often the only viable approach when estimating the internal state or hidden physical quantities. The optimization of sensor positions was intensively discussed in order to determine the most representative sensors and to reduce the resulting estimation error, such as when monitoring sensor networks [1,2,3,4], fluid flows around objects [5,6,7,8,9,10,11,12,13,14,15], plants and factories [16,17,18], infrastructures [19,20,21], circuits [22], and biological systems [23], estimating physical field [24,25,26,27], and localizing sources [28,29]. Recent advances in data science techniques have enabled us to extract reduced-order models from vastly large-scale measurements of complex phenomena [30,31,32,33,34,35,36,37,38,39]. Therefore, the optimized sensor measurement is gaining importance for the reconstruction of complex phenomena from sensor measurements based on the data-driven reduced-order models [40,41], as well as for model-free machine learning [42,43] and data assimilation [44,45,46].

The physical aspects of the phenomena are interpreted as state space models, while the assumed system representations are often nominal. Our study implements optimization frameworks for sensor node selections based on the criteria of expected estimation errors defined for these models. Sensor measurements in many nondynamical configurations are statistically described by the Fisher information matrix (FIM), which originated in the parameter sensitivity analysis [47,48]. This measure addresses the expected error ellipsoid of the estimation using instantaneous measurements, which corresponds to the inverse of the Cramer–Rao bound, see Chapter 8 of [49,50]. The combinatorial problem structure is transformed into small subproblems in a greedy formulation [40,41,51], of which, the solution quality is known to be ensured by the submodularity property in such optimization problems [52,53,54,55,56]. This property of the optimization problem helps to exploit other criteria from information theory, including entropy and various types of information distances [3,57,58]. Moreover, greedy selections are empirically confirmed to effectively work even in the absence of submodularity [51,59,60]. There are also other approaches that seek the global optimum by relaxing the integer constraints on the selection variable [50,61], or by employing proximal algorithms that form proximity operators [62,63,64].

The subset selection of sensor nodes for dynamical systems is similarly conducted based on the estimation error covariance, while the measurement quality evaluation tends to be more cumbersome. The observability Gramian is the counterpart of the FIM for linear time-invariant (LTI) representations with deterministic dynamics, as shown in [65]. The actuator node selection, which is a dual problem of the sensor node selection, is similarly obtained using the controllability Gramian [66,67]. It is worth emphasizing that a previous analysis used examples of random and regular networks of nodes of tens or hundreds due to the computational cost. For example, a previous study reported a tremendous increase in the computation time of the greedy method and the SDP relaxation method for a power grid system of such a size [66]. There are likely to be computational issues when the optimization objective is further extended to the observability for the nonlinear state space models [44,68,69], the error covariance of the Kalman filter [70,71,72,73,74], and the $H_2$ norm of an LTI system, which can be reduced through balanced truncation [75,76,77]. Accordingly, the main interest in this study is the applicability of the Gramian-based methods for the larger scale system constructed by the data-driven modeling methods.

The main claims of this paper are listed below:

Novel selection methods proposed: Two novel selection methods are proposed and implemented. Those two methods utilize the gradients of the objective functions while relaxing them to continuous functions. Computations in both methods are expected to be remarkably accelerated as compared to those of existing greedy or relaxation-based algorithms since the gradients of objective functions are mere quadratic forms of the dimensions of sensor candidates.

Comparison using high-dimensional and low-rank systems: A comparison of selection strategies is also provided in Section 4 to illustrate the characteristics of each selection method in terms of the execution time and the acquired optimization measure. This comparison also elaborates effective computational complexities of each selection method, which is distinct from the results of the previous study.

Figure 1 depicts the main frame of this manuscript. In the following sections, the basic formulation of the optimization problem is revealed in Section 2 after providing the dynamical system of interest. Section 3 subsequently addresses our novel algorithms for optimization while the previously presented approaches are briefly included. The systems are constructed in two different ways, for comparison; one is constructed via random numbers as a general case in Section 4.1, and in the other case, a data-driven method using proper orthogonal decomposition (POD) [78] is applied to real-world datasets, as seen in Section 4.2 and Section 4.3

2. LTI State Space and the Observability Gramian

This section provides rudimentary formulations that represent dynamics and optimization tasks based on the observability of the system. Problem settings are first presented with basic equations while notations for our formulation are defined. The term “observability” is omitted hereafter unless it causes confusion with the controllability Gramian or other Gram matrices. One may be reminded that the methodologies presented here are similarly adopted to controllability optimization since these properties are dual for the linear systems. A discrete-time LTI state space model is considered with $r(\in \mathbb{N})$ states and $n(\in \mathbb{N})$ measurements. Let $\left(\mathbf{C},\mathbf{A}\right)$ be an observable pair and an autonomous system

$${\mathbf{x}}_{k+1}=\mathbf{A}{\mathbf{x}}_k$$

(1)

$${\mathbf{y}}_k=\mathbf{C}{\mathbf{x}}_k+{\mathbf{v}}_k$$

(2)

generates a trajectory of state variables ${\mathbf{x}}_k\in {\mathbb{R}}^r$ and observations ${\mathbf{y}}_k\in {\mathbb{R}}^n$. The subscript $k\in \mathbb{N}$ refers to a snapshot at the instance k. Assume the observation Equation (2) is corrupted by a Gaussian noise ${\mathbf{v}}_k\sim \mathcal{N}\left(\mathbf{0}\right|\mathbf{R})$, which is independent of the state ${\mathbf{x}}_k$ and has the variance of the same amplitude, $\mathrm{E}\left[{\mathbf{v}}_k{\mathbf{v}}_k^{\top }\right]=\mathbf{R}={\sigma }^2{\mathbf{I}}_n$, where ${\mathbf{I}}_n$ is the identity matrix of size n. Here, $\mathrm{E}\left[◯\right]$ is taking the expectation over the random variable. Some data-driven techniques systematically provide this linear representation from the measured data, such as proper orthogonal decomposition and dynamic mode decomposition [37,38]. In the state space representation obtained by the data-driven techniques, the state vector represents coordinates of a low-dimensional subspace inherent in the high-dimensional space of the measurement vector and, therefore, $n\gg r$ is assumed hereafter. The observability of the system is quantified by the Gramian equation

$$\begin{array}{cc}\hfill & {\overline{\mathbf{W}}}_{\mathrm{O}}={\displaystyle \sum _{k=0}^{\infty }}{\left({\mathbf{A}}^{\top }\right)}^k{\mathbf{C}}^{\top }\mathbf{C}{\mathbf{A}}^k.\hfill \end{array}$$

(3)

In our formulation, this matrix is characterized by the linear least-squares estimation of the state vector that utilizes time-series measurements. The measurement equation is extended first, using from $k=0$ to $k=l-1$ of Equations (1) and (2):

$$\begin{array}{ccc}\hfill {\mathbf{y}}_{0:l-1}& :=& {\left[{\mathbf{y}}_0^{\top },{\mathbf{y}}_1^{\top },\dots ,{\mathbf{y}}_{l-1}^{\top }\right]}^{\top }\hfill \\ & =& \left[\begin{array}{c}\mathbf{C}\\ \mathbf{C}\mathbf{A}\\ ⋮\\ \mathbf{C}{\mathbf{A}}^{l-1}\end{array}\right]{\mathbf{x}}_0+\left[\begin{array}{c}{\mathbf{v}}_0\\ {\mathbf{v}}_1\\ ⋮\\ {\mathbf{v}}_{l-1}\end{array}\right]\hfill \\ & =:& {\mathbf{C}}_{0:l-1}{\mathbf{x}}_0+{\mathbf{v}}_{0:l-1},\hfill \end{array}$$

(4)

where ${\mathbf{C}}_{0:l-1}$ and ${\mathbf{v}}_{0:l-1}$ are the stacked components in the brackets in Equation (4). An estimate of ${\mathbf{x}}_0$ is obtained by the linear inversion as follows:

$$\begin{array}{cc}\hfill \therefore {\tilde{\mathbf{x}}}_0& ={\left({\mathbf{C}}_{0:l-1}\right)}^{†}{\mathbf{y}}_{0:l-1}\hfill \\ \hfill & ={\left({\mathbf{C}}_{0:l-1}^{\top }{\mathbf{C}}_{0:l-1}\right)}^{-1}{\mathbf{C}}_{0:l-1}^{\top }{\mathbf{y}}_{0:l-1}\hfill \\ \hfill & ={\mathbf{x}}_0+{\left({\mathbf{C}}_{0:l-1}^{\top }{\mathbf{C}}_{0:l-1}\right)}^{-1}{\mathbf{C}}_{0:l-1}^{\top }{\mathbf{v}}_{0:l-1},\hfill \end{array}$$

(5)

where ${\tilde{\mathbf{x}}}_0\in {\mathbb{R}}^r$ and ${◯}^{†}$ stand for the estimate of ${\mathbf{x}}_0$ and the Moore–Penrose pseudo inverse, respectively. The matrix ${\mathbf{C}}_{0:l-1}$ is assumed to have full column rank due to $n\gg r$. An error covariance matrix under the estimation ${\tilde{\mathbf{x}}}_0$ is proportional to the inverse matrix of the Gramian, as shown in the following equation:

$$\begin{array}{cc}\hfill & \mathrm{E}\left[\left({\mathbf{x}}_0-{\tilde{\mathbf{x}}}_0\right){\left({\mathbf{x}}_0-{\tilde{\mathbf{x}}}_0\right)}^{\top }\right]\hfill \\ \hfill & \propto {\left({\mathbf{C}}_{0:l-1}^{\top }{\mathbf{C}}_{0:l-1}\right)}^{-1}\hfill \\ \hfill & ={\left({\displaystyle \sum _{k=0}^{l-1}}{\left({\mathbf{A}}^{\top }\right)}^k{\mathbf{C}}^{\top }\mathbf{C}{\mathbf{A}}^k\right)}^{-1}\hfill \\ \hfill & \underset{l\to \infty }{⟶}{\left({\overline{\mathbf{W}}}_{\mathrm{O}}\right)}^{-1}.\hfill \end{array}$$

(6)

Here, let a small number of sensors be deployed while maintaining the quality of state estimation. The optimization problem is formulated as a selection of a subset ${\mathcal{I}}_p=\{i_1,\dots ,i_p\}$ from all available n measurement nodes, labeled as ${\mathcal{I}}_n=\{1,2,\dots ,n\}$, where a measure of the Gramian is optimized under some constraints. A permutation matrix $\mathbf{P}\left({\mathcal{I}}_p\right)$ that extracts a part of the measurement corresponding to the sensor indices $\left({\mathcal{I}}_p\right)$ is defined by

$$\begin{array}{c}\hfill \mathbf{P}\left({\mathcal{I}}_p\right)=\left[\begin{array}{c}{\mathbf{e}}_{i_1}^{\top }\\ ⋮\\ {\mathbf{e}}_{i_p}^{\top }\end{array}\right],\end{array}$$

(7)

where a unit vector ${\mathbf{e}}_i\in {\mathbb{R}}^n$ has unity in the i-th entry, and the rest is zero. The Gramian given by the selected sensors is rewritten as a subset function

$$\begin{array}{cc}\hfill {\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_p\right)& ={\displaystyle \sum _{k=0}^{\infty }}{\left({\mathbf{A}}^{\top }\right)}^k{\left(\mathbf{P}\mathbf{C}\right)}^{\top }\left(\mathbf{P}\mathbf{C}\right){\mathbf{A}}^k,\hfill \end{array}$$

(8)

and, therefore, ${\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_n\right)={\overline{\mathbf{W}}}_{\mathrm{O}}$. Here, ${\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_p\right)$ can be calculated by solving the following Lyapunov equation:

$$\begin{array}{cc}\hfill & {\mathbf{A}}^{\top }{\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_p\right)\mathbf{A}-{\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_p\right)+{\left(\mathbf{P}\mathbf{C}\right)}^{\top }\left(\mathbf{P}\mathbf{C}\right)=\mathbf{0},.\hfill \end{array}$$

(9)

Some measures for the optimization of ${\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_p\right)$ were presented in [66]. A maximization of the determinant of the Gramian is configured in the current study. The maximization problem is as follows:

$$\begin{array}{cc}\hfill & \underset{{\mathcal{I}}_p\subset \mathcal{I},\left|{\mathcal{I}}_p\right|=p}{\mathrm{maximize}}\log \det \left({\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_p\right)\right).\hfill \end{array}$$

(10)

The logarithmic form, which is monotone, is considered for the ease of calculations in the algorithms. This determinant maximization strategy is commonly used in the sensor placement and optimal design of interpolation methods [40,79].

3. Sensor Selection Strategies

The optimization problem (10) is a combinatorial problem and, therefore, finding the true optimum is computationally prohibitive. This section introduces four methods, as shown in

Table 1. Selection algorithms for each relaxed problem and the expected arithmetic complexity order based on the basic matrix operations. Several approaches are proposed in the presented paper.

Problem	Algorithm	Expected Complexity
Linear relaxation SDP [67]	Path-following method	$\left(\mathcal{O}\left(n^4\right)+\mathcal{O}\left(n^2{r}^2\right)+\mathcal{O}\left(n{r}^3\right)\right)$/iter.
Approximate convex relaxation	BRS-Newton [Algorithm 2]	$\left(\mathcal{O}\left(n^3\right)+\mathcal{O}\left(n^2{r}^2\right)+\mathcal{O}\left(n{r}^3\right)\right)$/iter.
Greedy [66]	Pure greedy [Algorithm 3]	$\mathcal{O}\left(pn{r}^3\right)$
Greedy	Gradient greedy [Algorithm 4]	$\mathcal{O}\left(pn{r}^2\right)+\mathcal{O}\left(p{r}^3\right)$

, for obtaining a suboptimal yet reasonable solution to (10). Section 3.1 deals with convex relaxation approaches to (10), where the notations of SDP and approximate convex relaxation stand for the selection problems based on the semidefinite programming (SDP) of [67] and an approximate smooth convex relaxation for the Newton method, which is extended from [50], respectively. Subsequently, Section 3.2 provides formulations with regard to the pure greedy and gradient greedy methods, as a simplified greedy implementation of [66] for the matrix determinant maximization, while the latter is its linear approximation. The computational complexities are discussed in Section 3.3.

3.1. Convex Relaxation Methods

This strategy replaces the Gramian ${\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_p\right)$ by the weighted sum of the Gramian for every sensor candidate, namely the relaxed Gramian. Let the sum of the selection weight $\mathbf{s}\in {\left[0,1\right]}^n$ be p, and the relaxed optimization problem of interest is:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \underset{\mathbf{s}}{\mathrm{maximize}}\log \det \left(\mathbf{Q}\left(\mathbf{s}\right)\right)\hfill \\ \hfill & \begin{array}{cc}\hfill \mathrm{subject\; to}& \mathbf{Q}\left(\mathbf{s}\right)={\displaystyle \sum _{i=1}^n}{s}_i{\mathbf{W}}_{\mathrm{O}}\left(\left\{i\right\}\right),\hfill \\ \hfill & s_i\in \left[0,1\right],{\mathbf{1}}^{\top }\mathbf{s}=p.\hfill \end{array}\hfill \end{array}\end{array}$$

(11)

This “weight” formulation is widely used in the various types of sensor selections, including linear inverse and Kalman filter estimations, [50,61]. The “gain” formulation, on the other hand, is employed to optimize the gains for yielding state variables from measurements. The group regularization is therein adopted to distinguish the representative measurement locations, which can also be found in the literature on sensor selection for linear inverse and Kalman filter estimations [62,63,64,80]. To the best of our knowledge, there is no existing study in which the gain formulation is applied to the sensor selection based on the observability Gramian. This may be because the Gramian is related to an infinite series of temporal measurements, and so are the gains for such measurements. Truncating Equation (6) to a finite time horizon is obviously a possible option, but dealing with a large solution vector and defining an appropriate finite time horizon as a hyperparameter are practically infeasible. Accordingly, the implementation of the gain formulation remains an interesting challenge.

3.1.1. Semidefinite Programming-Based Selection (SDP)

Convex relaxation for Gramian optimization was previously introduced in the section II-B of [67]; therefore, the readers should refer to the original work for more details. It should be emphasized that an optimization problem in the discrete-time form is briefly revisited in our study and is included in Section 4 for a comparison of selection methods. An optimization problem (11) is transformed into the following SDP representation:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \begin{array}{cc}\hfill \underset{\mathbf{s}}{\mathrm{maximize}}& \log \det \left(\mathbf{Q}\left(\mathbf{s}\right)\right)\hfill \end{array}\hfill \\ \hfill & \begin{array}{cc}\hfill \mathrm{subject\; to}& s_i\in \left[0,1\right],\mathbf{1}{}^{\top }\mathbf{s}=p,\mathbf{Q}\left(\mathbf{s}\right)⪰\mathbf{0},\hfill \\ \hfill & {\mathbf{A}}^{\top }\mathbf{Q}\left(\mathbf{s}\right)\mathbf{A}-\mathbf{Q}\left(\mathbf{s}\right)+{\displaystyle \sum _{i=1}^n}{s}_i{\mathbf{c}}_i^{\top }{\mathbf{c}}_i=\mathbf{0},\hfill \end{array}\hfill \end{array}\end{array}$$

(12)

where ${\mathbf{c}}_i$ is the i-th row of the measurement matrix $\mathbf{C}$, and $\mathbf{0}$ is a zero matrix with appropriate dimensions. It should be noted that the Lyapunov equation, imposed as a constraint in the problem, can be satisfied by SDP solvers introduced later. The complexity in Table 1 is based on a path-following method for a general primal–dual interior point method [81].

3.1.2. Newton Method for Approximate Convex Relaxation, and Its Customized Algorithm with Randomized Subspace Sampling (Approximate Convex Relaxation)

This novel method solves the convex relaxation problem by applying the Newton method and a customized randomization technique applied to Equation (11), with a penalty term that bounds the weight variables added.

The description of the proposed method starts with the extension of the formulation of sensor selection for static systems first introduced in [50].

In their approach, the Newton method solved a weight formulation of a determinant optimization for the FIM of the linear inverse problem, which returned the p-largest indices of $\mathbf{s}$ as a result of a heuristic sensor node selection. In this study, the idea above is straightforwardly extended to the Gramian. A smooth convex objective function approximates Equation (10) as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \begin{array}{cc}\hfill \underset{\mathbf{s}}{\mathrm{maximize}}& \log \det \left(\mathbf{Q}\left(\mathbf{s}\right)\right)+\kappa {\displaystyle \sum _{i=1}^n}\left(\log \left(s_i\right)+\log \left(1-s_i\right)\right)\hfill \end{array}\hfill \\ \hfill & \begin{array}{cc}\hfill \mathrm{subject\; to}& s_i\in \left(0,1\right),\mathbf{1}{}^{\top }\mathbf{s}=p\hfill \end{array}\hfill \end{array}\end{array}$$

(13)

with $\kappa >0$, which adjusts the smoothness of the objective function. A Newton step $\delta \mathbf{s}\in {\mathbb{R}}^n$ is determined by minimizing the second-order approximation of the objective function under a constraint $\delta \mathbf{s}=0$ as discussed in the Section 10.2 of [50,81], with notation f referring to the objective function in Equation (13), as shown below:

$$\begin{array}{cc}\hfill & \delta \mathbf{s}={\left({\nabla }^{2}f\right)}^{-1}\left(-\nabla f+\frac{{\mathbf{1}}^{\top }{\left({\nabla }^{2}f\right)}^{-1}\nabla f}{{\mathbf{1}}^{\top }{\left({\nabla }^{2}f\right)}^{-1}\mathbf{1}}\mathbf{1}\right).\hfill \end{array}$$

(14)

The first and second derivatives, with respect to the selection weight $\mathbf{s}$, are given by

$$\begin{array}{cc}\hfill {\left[\nabla f\right]}_i& ={\mathbf{c}}_i\left({\displaystyle \sum _{k=0}^{\infty }}{\mathbf{A}}^k{\mathbf{Q}}^{-1}\left(\mathbf{s}\right){\left({\mathbf{A}}^{\top }\right)}^k\right){\mathbf{c}}_i^{\top }+\frac{\kappa }{s_i}-\frac{\kappa }{1-s_i},\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill {\left[{\nabla }^{2}f\right]}_{i,j}& =-{\mathbf{c}}_i{\mathbf{H}}_j\left(\mathbf{s}\right){\mathbf{c}}_i^{\top }-{\delta }_{i,j}\left(\frac{\kappa }{s_i^2}+\frac{\kappa }{{\left(1-s_i\right)}^2}\right),\hfill \end{array}$$

(16)

where ${\delta }_{i,j}$ is the Kronecker delta, and

$$\begin{array}{c}\hfill {\mathbf{H}}_j\left(\mathbf{s}\right):={\displaystyle \sum _{k=0}^{\infty }}\left[\begin{array}{c}{\mathbf{A}}^k\mathbf{Q}{\left(\mathbf{s}\right)}^{-1}{\mathbf{W}}_{\mathrm{O}}\left(\left\{j\right\}\right)\mathbf{Q}{\left(\mathbf{s}\right)}^{-1}{\left({\mathbf{A}}^{\top }\right)}^k\hfill \end{array}\right]\end{array}$$

(17)

is the solution of the following Lyapunov equation;

$$\begin{array}{cc}\hfill & \mathbf{A}\mathbf{X}{\mathbf{A}}^{\top }-\mathbf{X}+\mathbf{Q}{\left(\mathbf{s}\right)}^{-1}{\mathbf{W}}_{\mathrm{O}}\left(\left\{j\right\}\right)\mathbf{Q}{\left(\mathbf{s}\right)}^{-1}=\mathbf{0}.\hfill \end{array}$$

(18)

The procedure is carried out according to Algorithm 1, where the iteration is completed when the ${\ell }_2$ norm of the solution update falls below the threshold given.

Algorithm 1 Newton algorithm for Equation (13)

Input: $\mathbf{C}\in {\mathbb{R}}^{n\times r},\mathbf{A}\in {\mathbb{R}}^{r\times r},p\in \mathbb{N}$

Output: Indices of chosen p sensor positions ${\mathcal{I}}_p$

Set an initial weight $\mathbf{s}\leftarrow \mathbf{1}p/n$

while convergence condition not satisfied do

Calculate $\nabla f$ by Equation (15) and ${\nabla }^{2}f$ by Equation (16)

Calculate $\delta \mathbf{s}$ by Equation (14)

Obtain step size t by backtracking line search

Set $\mathbf{s}\leftarrow \mathbf{s}+t\delta \mathbf{s}$

end while

Return the indices of the p-largest components of $\mathbf{s}$ as ${\mathcal{I}}_p$

One of the most computationally demanding steps of the Newton method in Section 3.1.2 is the inverse of the Hessian in Equation (14), approximately reaching $\mathcal{O}\left(n^3\right)$. The rest of this section describes an efficiency improvement for the Newton method of Algorithm 1, as the computational cost was relaxed using small sketches, as in [82,83]. The dimension of the sketched Newton step is $\tilde{n}<n$, where the results for $\tilde{n}/n=0.1$ in [83] showed a drastic reduction in the total computation time for convergence in spite of the increase in the number of iterations. In this study, a sketching matrix ${\mathbf{S}}_{\tilde{n}}\in {\mathcal{R}}^{\tilde{n}\times n}$ is constructed using a permutation matrix for ease of computation, as summarized in the following and in Algorithm 2. The subspace referring to the permutation is randomly assigned from the n components, which is indexed by;

$$\begin{array}{cc}\hfill & {\mathcal{I}}_{\tilde{n}}=\{i_1^{\prime },\dots ,i_{\rho }^{\prime },\dots i_{\tilde{n}}^{\prime }\}.\hfill \end{array}$$

(19)

This random selection is biased, according to the selection weight $\mathbf{s}$ in the previous iteration. This heuristic leads to a reasonable acceleration of the convergence because the weights of higher weighted sensors in the first few iterations are more likely updated frequently for convergence. In this study, half of the permutation coordinates correspond to the indices of the highest $\rho =\tilde{n}/2$ values of $\mathbf{s}$. The other half of the permutation coordinates were randomly selected from the remaining $n-\tilde{n}/2$ dimensions, and the exploration of sensor selection was further accelerated.

Accordingly, the calculations of the gradient (the first term in Equation (15)) and the Hessian (also the first term in Equation (16)) are simplified to the subspace indexed by ${\mathcal{I}}_{\tilde{n}}$. These subsampled derivatives and the derived Newton step are denoted by $\nabla \tilde{f}$, ${\nabla }^2\tilde{f}$, and $\delta \tilde{\mathbf{s}}.$ The criterion of the convergence is modified from that in Algorithm 1 due to the randomness, where the algorithm stops if the update size is less than a given threshold in $n/\tilde{n}$ consecutive iterations.

Algorithm 2 Customized algorithm of Algorithm 1 (BRS-Newton)

Input: $\mathbf{C}\in {\mathbb{R}}^{n\times r},\mathbf{A}\in {\mathbb{R}}^{r\times r},p>0,\tilde{n}>0$

Output: Indices of chosen p sensor positions ${\mathcal{I}}_p$

Set $\mathbf{s}\leftarrow \mathbf{1}p/n$

while convergence condition not satisfied do

Select ${\mathcal{I}}_{\tilde{n}}$ [Equation (19)] and set ${\mathbf{S}}_{\tilde{n}}$

Calculate subsampled derivatives $\nabla \tilde{f}$ and ${\nabla }^2\tilde{f}$

Calculate $\delta \tilde{\mathbf{s}}$

Obtain step size t by backtracking line search

Set $\mathbf{s}\leftarrow \mathbf{s}+t\left({\mathbf{S}}_{\tilde{n}}^{\top }\delta \tilde{\mathbf{s}}\right)$

end while

Return the indices of the p-largest components of $\mathbf{s}$ as ${\mathcal{I}}_p$

3.2. Greedy Algorithms

This type of algorithm sequentially adds the most effective sensor to the subset determined in the previous greedy iterations. The greedy algorithms, therefore, approximate the combinatorial aspect of the original optimization problem, yet preserve the discrete optimization structure. However, even the use of the greedy selection is empirically known to be prohibitive for the system of high-dimensional state vectors and observation vectors; therefore, the accelerating method is often applied [55,56]. Section 3.2.2 provides an approximated form of the greedy selection.

3.2.1. Greedy Selection with Simple Evaluation (Pure Greedy)

In the selection of the q-th sensor, where $1\le q\le p$, the Gramian ${\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_q\right)$ is obtained from the algebraic Lyapunov Equation (9). The submodularity in the objective function guarantees the quality of greedy solutions [52,53,84], and the derivation of the submodularity of several metrics related to the Gramian is discussed in detail in [66]. This method starts with finding the most effective single sensor; therefore, the determinant can be zero if the observability is not obtained by any single sensor. As mentioned in [66], one may evaluate the objective function for the observable subspace on such occasions. Algorithm 3 starts by decomposing the Gramian for each sensor candidate and identifying its subset ${\mathcal{I}}_{*}$, which results in the highest dimension of the observable subspace, then evaluates the determinant of the decomposed Gramian ${\widehat{\mathbf{W}}}_{\mathrm{O}}\left({\mathcal{I}}_{q-1}\cup \left\{i\right\}\right)$ into the observable space by multiplying their nonzero eigenvalues. Once the full-rank Gramian is achieved by an obtained subset, the greedy algorithm drops the computation of the rank of the candidate Gramian since the rank is monotone-increasing [66].

Algorithm 3 Determinant-based greedy algorithm (pure greedy)

Input: $\mathbf{C}\in {\mathbb{R}}^{n\times r},\mathbf{A}\in {\mathbb{R}}^{r\times r},p\in \mathbb{N}$

Output: Indices of chosen p sensor positions ${\mathcal{I}}_p$

${\mathcal{I}}_n\leftarrow \left\{1,\dots ,n\right\},{\mathcal{I}}_0\leftarrow \varnothing ,$

for $q=1,\dots ,p$ do

${\mathcal{I}}_{*}\leftarrow \left\{i:i\in \underset{i\in {\mathcal{I}}_n\backslash {\mathcal{I}}_{q-1}}{\mathrm{argmax}}\mathrm{rank}{\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_{q-1}\cup \left\{i\right\}\right)\right\}$

$i_q\leftarrow \underset{i\in {\mathcal{I}}_{*}}{\mathrm{argmax}}\log \det {\widehat{\mathbf{W}}}_{\mathrm{O}}\left({\mathcal{I}}_{q-1}\cup \left\{i\right\}\right)$

${\mathcal{I}}_q\leftarrow {\mathcal{I}}_{q-1}\cup \left\{i_q\right\}$

end for

3.2.2. Greedy Selection with Gradient Approximation (Gradient Greedy Proposed in This Study)

Algorithm 4 proposed in this paper is expected to accelerate the greedy evaluations by the linear approximation of the objective function, which is schematically explained in Figure 2.

The gradient greedy algorithm selects, in the current step q, a sensor corresponding to the element of the highest gradient of the objective function $f\left(\mathbf{s}\right)$ in the prior step. Recall the gradient with respect to the i-th sensor candidate of Equation (15);

$$\begin{array}{c}\hfill {\left[\nabla \log \det \left(\mathbf{Q}\left(\mathbf{s}\right)\right)\right]}_i={\mathbf{c}}_i\left({\displaystyle \sum _{k=0}^{\infty }}{\mathbf{A}}^k{\mathbf{Q}}^{-1}\left(\mathbf{s}\right){\left({\mathbf{A}}^{\top }\right)}^k\right){\mathbf{c}}_i^{\top },\end{array}$$

where ${\mathbf{c}}_i$ is the same notation as used in (13). In the selection of the q-th sensor, the i-th component of $\nabla \log \det \left(\mathbf{Q}\left(\mathbf{s}\right)\right)$ is given by replacing $\mathbf{Q}\left(\mathbf{s}\right)\to {\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_{q-1}\right)$ as follows:

$$\begin{array}{c}\hfill \underset{\mathbf{Q}\left(\mathbf{s}\right)\to {\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_{q-1}\right)}{\lim }{\left[\nabla \log \det \left(\mathbf{Q}\left(\mathbf{s}\right)\right)\right]}_i={\mathbf{c}}_i\left({\displaystyle \sum _{k=0}^{\infty }}{\mathbf{A}}^k{\mathbf{W}}_{\mathrm{O}}{\left({\mathcal{I}}_{q-1}\right)}^{-1}{\left({\mathbf{A}}^{\top }\right)}^k\right){\mathbf{c}}_i^{\top }.\end{array}$$

(20)

This approximation drops the number of evaluations of the Lyapunov equation and the matrix determinant from the greedy selection, and instead computes inner products of vectors with weighting matrix including the Gramian determined in the previous search. Note that the inverse in Equation (20) is singular when the system is unobservable. A small regularization term $\mathrm{diag}\left[\delta ,\delta ,\dots ,\delta \right]$ is therefore added to ${\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_{q-1}\right)$ to ensure its regularity.

Algorithm 4 Determinant-based gradient greedy algorithm (Gradient greedy)

Input: $\mathbf{C}\in {\mathbb{R}}^{n\times r},\mathbf{A}\in {\mathbb{R}}^{r\times r},p\in \mathbb{N},\delta >0$

Output: Indices of chosen p sensor positions ${\mathcal{I}}_p$

${\mathcal{I}}_n\leftarrow \left\{1,\dots ,n\right\},{\mathcal{I}}_0\leftarrow \varnothing ,$

for $q=1,\dots ,p$ do

${\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_{q-1}\right)\leftarrow {\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_{q-1}\right)+\mathrm{diag}\left[\delta ,\delta ,\dots ,\delta \right]\in {\mathbb{R}}^{r\times r}$

Find $\mathbf{M}$ s.t. $\mathbf{A}\mathbf{M}{\mathbf{A}}^{\top }-\mathbf{M}+{\mathbf{W}}_{\mathrm{O}}{\left({\mathcal{I}}_{q-1}\right)}^{-1}=\mathbf{0}$

$i_q\leftarrow \underset{i\in {\mathcal{I}}_n\backslash {\mathcal{I}}_{q-1}}{\mathrm{argmax}}{\mathbf{c}}_i\mathbf{M}{\mathbf{c}}_i^{\top }$

${\mathcal{I}}_q\leftarrow {\mathcal{I}}_{q-1}\cup \left\{i_q\right\}$

Calculate ${\mathbf{W}}_{\mathrm{O}}\left({\mathcal{I}}_q\right)$

end for

3.3. Expected Computational Complexity

This section describes the expected computational complexity of selection methods based on the matrix operations found in textbooks of basic linear algebra. The leading terms, with respect to the system dimension parameters $n,p,r$, are summarized in Table 1. This also gives the expected overhead term of the run time for each algorithm, while in practice, more compact expressions can be obtained, depending on the libraries for computation employed and the problem structures considered. Readers should also note that the notation of complexity $\mathcal{O}(◯)$ conventionally omits the constant factors for simplicity and, therefore, lower-order terms might overwhelm the others in actual computations.

Although the optimal subset ${\mathcal{I}}_p$ for Equation (10) can be found by calculating the objective function for all subsets of which the member size is p, the computational complexity of this brute-force search will be $\mathcal{O}\left(n^p{r}^3\right)$ and barely amenable. As for the existing approaches, a naive implementation of the linear convex relaxation method assuming the semidefinite problem (SDP) structure is a simplified form of [67]. The interior point method and path-following iterations should require $(\mathcal{O}\left(n^4\right)+\mathcal{O}\left(n^2{r}^2\right)+\mathcal{O}\left(n{r}^3\right))$ per iteration to construct the Newton direction. This is due to the linear matrix inequality (LMI) constraints, such as the Gramian, being semidefinite, or the selection variable being bounded, while it is known that certain problem structures ameliorate the overall costs ([81], Section 11.8). A simple greedy algorithm (pure greedy) requires solving the Lyapunov equation and calculating the determinant of the observability Gramian, of which, the cost is $\mathcal{O}\left(r^3\right)$, for all n sensor candidates and p sensor increment iterations. The overall computational complexity is, therefore, $\mathcal{O}\left(pn{r}^3\right)$.

The proposed methods will accelerate both of the existing approaches so far by comparing the theoretical complexities. The proposed convex relaxation method (denoted by approximate convex relaxation) utilizes the Newton method iterations for the sensor selection. The algorithm requires $\mathcal{O}\left(n^3\right)$ computations of the inversion of Hessian. Furthermore, Equation (18) will be solved for every $j\in \{1,2,\dots ,n\}$ for Equation (16), which requires computational complexities of $\mathcal{O}\left(n{r}^3\right)$ for solving the algebraic Lyapunov Equation (18) and obtaining the matrix multiplication inside, respectively. The first term in Equation (16) requires $\mathcal{O}\left(n^2{r}^2\right)$; therefore, the leading terms will be the sum $\left(\mathcal{O}\left(n^3\right)+\mathcal{O}\left(n^2{r}^2\right)+\mathcal{O}\left(n{r}^3\right)\right)$ per iteration. The sketching matrix, in Algorithm 2, compresses the dimension of the Newton system to $n\to \tilde{n}$, and reduces the costs of the above-mentioned computations by ratio. However, its computational complexity notation does not change. The proposed gradient greedy algorithm (gradient greedy) replaces the evaluations of the Lyapunov equation over n sensor candidates with inner products of vectors weighted by an $r\times r$ matrix. The overall computational complexity will be $\mathcal{O}\left(p{r}^3\right)+\mathcal{O}\left(pn{r}^2\right)$, where an evaluation of the algebraic Lyapunov equation is taken only p times.

4. Comparison and Discussion

Numerical examples illustrate the efficiency of the considered algorithms in Section 4. The tested systems, which are represented by $\left(\mathbf{C},\mathbf{A}\right)$, are generated by synthetic datasets of random numbers and real-world counterparts of large-scale measurements. As used in Section 2, the systems of interest are in discrete-time LTI forms. The entire computation was conducted on MATLAB R2022a, and CVX 2.2 [85,86]; the MOSEK solver was used for the SDP-based selection, as noted in Appendix A. The dlyap function was used for the solutions of the discrete-time Lyapunov equations, such as Equations (9) and (18), adopting the subroutine libraries from the Subroutine Library in Control Theory (SLICOT) [87,88,89,90]. It should also be noted that there is another approach to solving the equation, such as [91]. The MATLAB programs are available through the GitHub repository of the present authors [92].

4.1. Results of the Randomly Generated System

The characteristics of the sensor selection methods are investigated for different sets of system dimension parameters $(n,r,p)$ by varying one parameter while fixing the others. The abscissa of Figure 3a,b is the rank $r\in \{10,20,\dots ,60\}$, which is the dimension of the reduced state variable vector and, thus, of the Gramian. In Figure 3c,d, $n\in \{2^{10},2^{11},\dots ,2^{16}\}$ denote the sizes of node members comprising the original measurements, and accordingly the number of sensor candidates. In Figure 3e,f, $p\in \{1,2,\dots ,10,20,\dots ,100\}$ represents the varying number of sensors selected.

The problem setting for the system of random numbers is found in [39]. First, the conjugate complex numbers, of which the real parts are negative, are assigned to the eigenvalues of a damping continuous-time system matrix $\widehat{\mathbf{A}}$. A discrete-time system matrix can be obtained by $\mathbf{A}=\exp \left(\widehat{\mathbf{A}}\Delta t\right)$, which is stable and full-rank, whereas $\Delta t$ is the time step of the discrete system. The observation matrix $\mathbf{C}$ is a column-orthogonal matrix generated by the singular value decomposition for a matrix of Gaussian random numbers of appropriate dimensions. Sensors up to p are selected using the algorithms presented in the previous section, and the objective function, $\log \det {\mathbf{W}}_O\left({\mathcal{I}}_p\right)$, is calculated for each selected subset. Figure 3 shows the performance of the selection methods applied to the system of random numbers.

Figure 3a,c,e illustrate the total computation time of each algorithm in the tests, where the gradient greedy method is the least time-consuming, followed by the pure greedy and the SDP-based methods, which produce similar results. Unfortunately, the proposed approximate convex relaxation method took the longest time to solve the optimization in almost all of the conditions tested despite its acceleration owing to the customized randomization. However, it is also expected that the computation time of the SDP-based method and the pure greedy selection will exceed the others for even larger r and p, respectively.

The empirical orders of the computation time with regard to each parameter are analyzed herein. The growth rates of the computation time of each algorithm are evaluated by solely changing the system dimension parameters r, n, and p, as summarized in

Table 2. Practical orders of the computation times of selection methods, investigated with respect to each parameter individually.

Selection Method	r	n	p
SDP	$r^{\left[4\right]}$	n	$p^0$
Approx. conv. [Algorithm 2]	$r^{\left[3\right]}$	$n^{\left[2\right]}$	$p^{\left[1\right]}$
Pure greedy [Algorithm 3]	$r^3$	n	$p^1$
Gradient greedy [Algorithm 4]	N/A	n	$p^1$

, based on the results shown in Figure 3a,c,e. First, the gradient greedy method ran in time proportional to n, but it is not clear regarding r. Since the number of sensor candidates, n, is much larger than r in the first experiment, the term with $r^3$ is not significant for the gradient greedy method. The increase against p is on the order of unity, which is clearly reasonable. Second, the empirical orders of the pure greedy method when solely changing r, n, and p are $r^3$, n, and p, respectively, which agrees with the expected leading order. One can see a term with $r^3$ becomes dominant as r grows in Figure 3a. Meanwhile, the estimated orders of the SDP-based method when solely changing r and n are $r^{\left[4\right]}$ and n, while those of the approximate convex relaxation method are $r^{\left[3\right]}$ and $n^{\left[2\right]}$, respectively. Here, let the notation $\left[j\right]$ stand for (despite an obvious abuse) a real number x bounded by $j-1$ and j for an arbitrary positive integer j. These noninteger orders may be due to the optimized arithmetic employed in the software, such as the Strassen-like algorithm [93]. The dependency regarding p was not clear in the SDP-based method, while a slight increase was observed for the approximate convex relaxation method.

The interesting aspect is that the dominating order of the SDP-based method with respect to n was not $n^4$, which was initially expected but is approximately proportional to n. This is perhaps because the constraint posed as $0\le s_i\le 1$ in Appendix A was simplified in the tested implementation as a mere diagonal block of the semidefinite linear matrix inequality (LMI), and the MOSEK solver should have taken advantage of its structure during the Newton step calculation, despite the large n assumed. It should also be noted that this efficacy was not observed for other solvers, such as the SDPT3, although the CVX parser does not seem to change its output. Nonetheless, the complexity of solving the SDP would be enormous, as expected, if the LMI included such a semidefinite relaxation [61,94] of the selection variable vector $\mathbf{s}$, such as $\mathbf{s}{\mathbf{s}}^{\top }$ being a semidefinite matrix. This agrees well with the observations from the experiment for the SDP-based method regarding r, where the LMI of the dimension of $r\times r$ is included.

In addition to the computation time per step, the iteration numbers before convergence, as shown in

Table 3. Iteration numbers of the convex relaxation approaches, investigated with respect to each parameter individually. Averages are rounded to integers.

(a) Various r for Randomly Generated Systems, Average of 100 Trials
r	10	20	30	40	50	60
SDP	18	13	11	12	12	12
Approx. conv.	203	293	348	373	380	388
(b) Various n for randomly generated systems, average of 20 trials
n	$2^{10}$	$2^{11}$	$2^{12}$	$2^{13}$	$2^{14}$	$2^{15}$	$2^{16}$
SDP	18	21	23	27	28	31	33
Approx. conv.	209	167	147	141	139	139	141
(c) Various p for randomly generated systems, average of 100 trials
p	1	2	4	8	10	20	40
SDP	14	13	15	17	18	18	17
Approx. conv.	171	168	172	198	206	273	411

, also illustrate the computationally friendly features for the high-dimensional problems. The iteration numbers of the SDP-based and the approximate convex relaxation methods do not increase significantly as r, n, and p individually increase. Interestingly, the iteration numbers of these convex relaxation methods change in a different fashion. That of the SDP-based method slightly increases with increases in n and p, and slightly decreases with an increase in r, while that of the approximate convex relaxation shows opposite results. This leads to similar growth rates against r between these two methods, as shown in Figure 3a, where the empirical computational complexity is less than $\mathcal{O}\left(r^4\right)$ for the SDP-based method and more than $\mathcal{O}\left(r^3\right)$ for the approximate convex relaxation method. Moreover, Figure 3c shows that this difference leads to growth rates that are slightly larger than $\mathcal{O}\left(n\right)$ for the SDP-based method and smaller than $\mathcal{O}\left(n^3\right)$ for the approximate convex relaxation method, against n. This unexpected efficiency of the SDP-based method leads to better scalability with respect to n over the approximate convex relaxation, which is the other convex relaxation method. With regard to the increase in p, the iteration numbers of the SDP-based method increased only slightly, while those of the approximate convex relaxation method exhibited an increase around $p=20$, corresponding to the increase in the total run time in Figure 3e.

As shown in Figure 3b,d,f, almost the same objective values were obtained by the SDP-based and the approximate convex relaxation methods, which implies a good agreement between the solutions of these relaxation methods. They yielded better or comparable objective function values as compared to the greedy methods, which were up to twice as high, except for $p=1,2$ cases. Those obtained by the gradient greedy method, on the other hand, give an inferior impression as a selection method as compared with the other methods for $p\le r$ cases. This is possibly due to the difficulty in ensuring the observability with a limited number of sensors, especially $r>p$, which should lead to an unstable calculation of the gradient Equation (20). The gradient greedy algorithm is concluded to have poor performance in achieving observability, especially when the convex objective function is hardly approximated by its linear tangent due to large r.

The discussion above illustrates that the use of the SDP-based method seems to be the most favorable among the compared methods in this experiment. The use of the pure greedy method is also a reasonable choice due to its shorter computation time, straightforward implementation, and reasonable performance, which returns half the objective function value of the convex relaxation methods in the experiments.

4.2. Results for Data-Driven System Derived from Real-World Experiment

An example of a practical application was conducted using the experimental dataset of flow velocity distribution around an airfoil. The data used herein are found in [95,96], which were previously acquired by the particle image velocimetry (PIV) in a wind tunnel. A brief description of the experiment is shown in

Table 4. Brief descriptions of PIV data [95,96].

(a) Wind Tunnel Experiment
Airfoil	NACA0015 (Chord length 100 mm)
Wind tunnel	Recirculating low-speed wind tunnel
Flow speed	10 m/s
Angle of attack	${18}^{°}$
(b) Acquisition condition of velocity distribution
Sampling rate	5000 Hz
Spatial sample	9353 points
Snapshot sample	2000 snapshots × 5 sets

; refer to the original article by the authors for more specific information. The snapshots taken in the experiment quantify velocity vectors that span the visualized plane, i.e., two components on a two-dimensional grid of $n=9353$ points, as depicted in Figure 4.

Only the streamwise components (the direction shown by the arrow in Figure 4) are used, and the ensemble averages over m snapshots are subtracted at each measurement location; that is, averages for each pixel of the calculated velocity image.

As attempted in Section 4.1, a linear representation $\left(\mathbf{C},\mathbf{A}\right)$ is derived first. The data-driven system identification procedure is based on the modeling method of [30,78]. Here, the snapshots of the velocity field are reshaped to $(n\times 1)$-dimensional m vectors, ${\mathbf{y}}_1,\dots ,{\mathbf{y}}_m$, then the data matrix is defined by $\mathbf{Y}:=\left[{\mathbf{y}}_1,\dots ,{\mathbf{y}}_m\right]\in {\mathbb{R}}^{n\times m}$. The proper orthogonal decomposition (POD) is then adopted and the data are projected onto the subspace of the leading r POD modes [37], resulting in $\mathbf{Y}\approx \mathbf{U}\mathsf{\Sigma }{\mathbf{V}}^{\top }$ ($\mathbf{U}\in {\mathbb{R}}^{n\times r},\mathsf{\Sigma }\in {\mathbb{R}}^{r\times r},\mathbf{V}\in {\mathbb{R}}^{m\times r}$, respectively). The measurement matrix $\mathbf{C}$ is defined by $\mathbf{U}$, which consists of r left singular vectors related to the largest r singular values. These left singular vectors illustrate the dominant spatial coherent structures, while the right vectors represent temporal coherence, and the singular values are the amplitudes of these modes. The use of POD in our study is intended for a more fundamental discussion based on a linear system representation where arbitrary-order low-rank systems are derived from high-dimensional measurement data.

The r-dimensional state variable vector $\mathbf{x}$ is given by

$$\begin{array}{c}\hfill \left[{\mathbf{x}}_1,\dots ,{\mathbf{x}}_m\right]:=\mathsf{\Sigma }{\mathbf{V}}^{\top }\in {\mathbb{R}}^{r\times m}.\end{array}$$

(21)

After introducing ${\mathbf{X}}_1=\left[{\mathbf{x}}_1,\dots ,{\mathbf{x}}_{m-1}\right]$ and ${\mathbf{X}}_2=\left[{\mathbf{x}}_2,\dots ,{\mathbf{x}}_m\right]$, the system matrix is then expressed using linear least squares via the pseudo-inverse operation $\mathbf{A}={\mathbf{X}}_2{\left({\mathbf{X}}_1\right)}^{†}$. The system matrix $\mathbf{A}$ is manipulated to ensure that its eigenvalues are less than unity and, therefore, the considered systems are stable.

As discussed in Section 4.1, the comparisons are illustrated using the objective function values obtained. The sensor candidate set corresponds to each pixel obtained by the experimental visualization and, thus, its size is fixed to $n=9353$. The state variable vector is set to have $r=10,\dots ,60$ components as a result of the order reduction at different truncation thresholds, while p is fixed to 20. Moreover, the results with respect to various p values are also provided, where $p=1,2,\dots ,10,20,\dots ,100$ with $r=10$ being fixed. The original dataset consisting of the $10,000$ snapshots is divided into $m=2000$ consecutive snapshots and, therefore, the following results are the five-fold average.

The values of the objective function show similar trends to Figure 3b,f for the gradient greedy algorithm, while a large degradation is observed in the convex relaxation methods for $r>p$. Surprisingly, those of the convex relaxation methods are 2–100 times lower than that of the pure greedy method. This trend is also partially observed for the $p=1$ case in Figure 5b, where the pure greedy selection corresponds to the true optimum for this configuration by definition. This clear degradation in the convex relaxations has not yet been explained in this study and may be due to the ill condition of the low-order approximation or the numerical error. A similar discussion was conducted for the previous study [66]. The cause should be investigated in detail for a practical application with the real-world data in a future study.

The results above, together with those in the previous subsection, illustrate that the pure greedy method shows a more stable performance, while the convex relaxation methods show wide variations in their performance. Therefore, the pure greedy method can be considered the most appropriate method, taking into account the computation time. It should be noted that the present study also shows the prominent performance of the SDP-based method in terms of the objective function and the computation time in the randomly generated, general problem settings. As long as the additional computation time is acceptable, the present authors recommend trying both the pure greedy and the SDP-based methods and to use a better sensor set when selecting the sensors based on the observability Gramian.

4.3. Results for Data-Driven System Derived from Weather Observation

A data-driven linear model is obtained from the time-series data of the global sea surface temperature (SST), as conducted in Section 4.2.

Table 5. Brief descriptions of sea surface temperature data [99,100].

Data type	Weekly means from 31 December 1989 to 29 January 2023
Grid scale	1.0 degree lat × 1.0-degree long grid (180 × 360)
Spatial sample	44,219
Temporal sample	1727

and Figure 6 briefly describe the conditions of processing for the SST dataset. The satellite observation, along with in situ correction composites, interpolated the distribution of SST, and the weekly mean distribution was stacked in the data matrix as a time-series measurement spanning from 1989 to 2023. The data matrix consists of 1727 snapshots of 44,219 spatial components, which correspond to the number of columns and rows, respectively. The sensor selection application to the SST dataset is a common benchmark, as found in [2,40,41,63,97,98].

The linear model was constructed in the same data-driven manner as previously explained in Section 4.2. Moreover, with a dimension set of the system given by r = 10, n = 44,219 and $p=10$, the evaluation was conducted as in the previous section by varying r and p solely as $r=10,20,\dots ,60$, and $p=1,2,\dots ,10,20,\dots ,100$, respectively. The whole duration was used for the construction of the reduced-order model in this example.

The sensor selection results in Figure 7a,b support the observations of the real-world examples of PIV measurements in Section 4.2, as the optimization measures of the SDP-based method and the approximate relaxation method similarly returned deteriorated results as compared with those of the pure-greedy method at $p\le r$.

5. Conclusions

This research investigates efficient algorithms that give beneficial sensor positions based on the observability Gramian of a discrete-time linear dynamical system, and are essential to realizing effective state estimation and control. The selection methods were characterized using synthetic numerical examples and a data-driven system representation of a real-world dataset. An analysis of the increase in computation time with respect to the system dimension parameters showed the effectiveness of each selection method, along with comparisons of their optimization performance.

This study offers two novel approaches for sensor selection: an approximate convex relaxation solved by a customized Newton method and a linear approximation of a pure greedy evaluation. A comparison was conducted against conventional methods, including a convex relaxation to the semidefinite programming and the pure greedy method. The proposed gradient greedy method achieved a moderate solution with orders-of-magnitude speedups compared to the other methods when the dimensions of the system’s state variables did not exceed the number of sensors deployed. Meanwhile, this method was found to be incapable of ensuring the observability of the larger dimensions of the state variable. The convex relaxation methods, including the SDP-based and the approximate convex relaxation methods, achieved better solutions than the greedy methods in synthetic data, while the real-world example discloses their unstable optimization results, especially for a large-scale system. In less observable situations, the pure greedy selection is supposed to be the most reliable choice in terms of the optimization metric obtained.