On Application of the Ray-Shooting Method for LQR via Static-Output-Feedback

Abstract

In this article we suggest a randomized algorithm for the LQR (Linear Quadratic Regulator) optimal-control problem via static-output-feedback. The suggested algorithm is based on the recently introduced randomized optimization method called the Ray-Shooting Method that efficiently solves the global minimization problem of continuous functions over compact non-convex unconnected regions. The algorithm presented here is a randomized algorithm with a proof of convergence in probability. Its practical implementation has good performance in terms of the quality of controllers obtained and the percentage of success.

1. Introduction

Let a continuous-time system be given by

$$\left\{\begin{array}{c}\dot{x}\left(t\right)=Ax\left(t\right)+Bu\left(t\right)\hfill \\ y\left(t\right)=Cx\left(t\right)\hfill \end{array}\right.$$

(1)

where $A\in {\mathbb{R}}^{p\times p},B\in {\mathbb{R}}^{p\times q},C\in {\mathbb{R}}^{r\times p}$ and $x,u,y$ are the state, the input and the measurement, respectively. We assume that $\left(A,B\right)$ and $\left(A^T,C^T\right)$ are controllable (see [1] for a new reduction from the case where $\left(A,B\right)$ and $\left(A^T,C^T\right)$ are only stabilizable). Let

$$J\left(x_0,u\right):={\int }_0^{\infty }\left(x{\left(t\right)}^{T}Qx\left(t\right)+u{\left(t\right)}^{T}Ru\left(t\right)\right)dt,$$

(2)

denote the cost functional, where $Q>0$ and $R\ge 0$. Assuming that $x_0=x\left(0\right)$ is given, the LQR problem is to attenuate the disturbance $x_0$ using minimal control cost, i.e., to design a regulation input $u\left(t\right)$ that minimizes $J\left(x_0,u\right)$, subject to the constraints given by (1). Let $u=-Ky$ be the static-output-feedback (SOF) with the closed-loop matrix $A_{c\ell }\left(K\right):=A-BKC$. Let ${\mathbb{C}}_{-}$ denote the left-half plane, let $\alpha >0$ and let ${\mathbb{C}}_{\alpha }$ denote the set of all $z\in \mathbb{C}$ with $\Re \left(z\right)\le -\alpha$, where $\Re \left(z\right)$ is the real part of z. Let ${\mathcal{S}}^{q\times r}$ denote the set of all matrices $K\in {\mathbb{R}}^{q\times r}$, such that $A_{c\ell }$ is stable, i.e., $\sigma \left(A_{c\ell }\right)\subset {\mathbb{C}}_{-}$ (where $\sigma \left(A_{c\ell }\right)$ is the spectrum of $A_{c\ell }$). By ${\mathcal{S}}_{\alpha }^{q\times r}$, we denote the set of all matrices $K\in {\mathbb{R}}^{q\times r}$, such that $\sigma \left(A_{c\ell }\right)\subset {\mathbb{C}}_{\alpha }$. In this case, we say that $A_{c\ell }$ is $\alpha$-stable. Below, we will occasionally write ${\mathcal{S}}_{\alpha }$ instead of ${\mathcal{S}}_{\alpha }^{q\times r}$, when it is clear what the size of the related matrices is.

Let $K\in {\mathcal{S}}_{\alpha }^{q\times r}$ be given. Substitution of $u=-Ky=-KCx$ into (2) gives

$$J\left(x_0,K\right):={\int }_0^{\infty }x{\left(t\right)}^T\left(Q+C^T{K}^{T}RKC\right)x\left(t\right)dt.$$

(3)

Since $Q+C^T{K}^{T}RKC>0$ and since $A_{c\ell }\left(K\right)$ is stable, it follows that the Lyapunov equation

$$A_{c\ell }{\left(K\right)}^{T}P+P{A}_{c\ell }\left(K\right)=-\left(Q+C^T{K}^{T}RKC\right)$$

(4)

has a unique solution $P>0$, given by

$$P=-mat\left({\left(I_p\otimes A_{c\ell }{\left(K\right)}^T+A_{c\ell }{\left(K\right)}^T\otimes I_p\right)}^{-1}·vec\left(Q+C^T{K}^{T}RKC\right)\right),$$

(5)

where $vec$ puts all the columns of the given matrix in a single column and $mat$ is the inverse of $vec$. Let us denote the solution (5) as $P\left(K\right)$. Substitution of (4) into (3) and noting that $\dot{x}\left(t\right)=A_{c\ell }\left(K\right)x\left(t\right)$ with $A_{c\ell }\left(K\right)$ stable, leads to

$$\begin{array}{cc}\hfill J\left(x_0,K\right)& =-{\int }_0^{\infty }x{\left(t\right)}^T\left(A_{c\ell }{\left(K\right)}^{T}P\left(K\right)+P\left(K\right)A_{c\ell }\left(K\right)\right)x\left(t\right)dt\hfill \\ & =-{\int }_0^{\infty }\frac{d}{dt}\left(x{\left(t\right)}^{T}P\left(K\right)x\left(t\right)\right)dt=x_0^{T}P\left(K\right)x_0.\hfill \end{array}$$

Thus, we look for $K\in {\mathcal{S}}_{\alpha }^{q\times r}$ that minimizes the functional $J\left(x_0,K\right)=x_0^{T}P\left(K\right)x_0$. When $x_0$ is unknown, we seek $K\in {\mathcal{S}}_{\alpha }^{q\times r}$ for which

$${\sigma }_{max}\left(K\right):=max\left(\sigma \left(P\left(K\right)\right)\right)$$

(6)

is minimal. In this case, we get a robust LQR via SOF, in the sense that it minimizes $J\left(x_0,K\right)$ for the worst possible (unknown) $x_0$. Note that

$$x_0^{T}P\left(K\right)x_0={∥P{\left(K\right)}^{\frac{1}{2}}{x}_0∥}^2\le {∥P{\left(K\right)}^{\frac{1}{2}}∥}^2{∥x_0∥}^2=∥P\left(K\right)∥{∥x_0∥}^2={\sigma }_{max}\left(K\right){∥x_0∥}^2,$$

and that there exists $x_0\ne 0$ for which equality holds. Therefore $\frac{J\left(x_0,K\right)}{{∥x_0∥}^2}\le {\sigma }_{max}\left(K\right)$, where equality holds in the worst case.

Note that the functionals $J\left(x_0,K\right)$ and ${\sigma }_{max}\left(K\right)$ are generally not convex since their domain of definition ${\mathcal{S}}^{q\times r}$ (and therefore ${\mathcal{S}}_{\alpha }^{q\times r}$) is generally non-convex. Necessary conditions for optimality were given as three quadratic matrix equations in [2,3,4,5]. Necessary and sufficient conditions for optimality, based on linear matrix inequalities (LMI), were given in [6,7,8]. However, algorithms based on these formulations are generally not guaranteed to converge, seemingly because of the non-convexity of the coupled matrix equations or inequalities, and when they converge, it is to a local optimum only.

The application of SOFs in LQRs is appealing for several reasons: they are reliable and cheap, and their implementation is simple and direct. Moreover, the long-term memory of dynamic-feedbacks is useless for systems subject to random disturbances, to fast dynamic loadings or to impulses, and the application of state-feedbacks is not always possible, due to unavailability of full-state measurements (see [9], for example). On the other hand, in practical applications, the entries of the needed SOFs are bounded, and since the problem of SOFs with interval constrained entries is NP-hard (see [10,11]), one cannot expect the existence of a deterministic polynomial-time algorithm to solve this problem. Randomized algorithms are thus natural solutions to this problem. The probabilistic and randomized methods for the constrained SOF problem and robust stabilization via SOFs (among other hard problems) are discussed in [12,13,14,15]. The Ray-Shooting Method was recently introduced in [16], where it was utilized to derive the Ray-Shooting (RS) randomized algorithm for the minimal-gain SOF problem with regional pole-assignment, where the region can be non-convex and unconnected. For a survey of the SOF problem see [17] and for a recent survey of the robust SOF problem see [18].

The contribution of this research is as follows:

• The suggested algorithm is based on the Ray-Shooting Method (see [16]), which, as opposed to smooth optimization methods, has the potential of finding a global optimum of continuous functions over compact non-convex and unconnected regions.
• The suggested algorithm has a proof of convergence (in probability) and explicit complexity.
• Experience with the algorithm shows good quality of controllers, high percent of success and good run-time for real-life systems. Thus, the suggested practical algorithm efficiently solves the problem of LQR via SOF.
• The algorithm does not need to solve any Riccati equations and thus can be applied to large systems.
• The suggested algorithm is one of the few that deals with LQR via SOF and has the ability to deal with discrete-time systems under the same formulation.

The reminder of the article is organized as follows:

In Section 2, we introduce the practical randomized algorithm for the problem of LQR via SOF. In Section 3, we give the results of the algorithm for some real-life systems and we compare its performance with the performance of a well known algorithm that has a proof of convergence to local minimum (under some reasonable assumptions). Finally, in Section 4 we conclude with some remarks.

2. The Practical Algorithm for the Problem of LQR via SOF

Assume that $K^{\left(0\right)}\in int\left({\mathcal{S}}_{\alpha }\right)$ was found by the RS algorithm (see [16]) or by any other method (see [19,20,21]). Let $h>0$ and let $U^{\left(0\right)}$ be a unit vector w.r.t. the Frobenius norm, i.e., ${∥U^{\left(0\right)}∥}_F=1$. Let $L^{\left(0\right)}=K^{\left(0\right)}+h·U^{\left(0\right)}$ and let $\mathcal{L}$ be the hyperplane defined by $L^{\left(0\right)}+V$, where ${〈V,U^{\left(0\right)}〉}_F=0$. Let $r_{\infty }>0$ and let ${\mathcal{R}}_{\infty }$ denote the set of all $F\in \mathcal{L}$, such that ${∥F-L^{\left(0\right)}∥}_F\le r_{\infty }$. Let ${\mathcal{R}}_{\infty }\left(ϵ\right)={\mathcal{R}}_{\infty }+\overline{\mathbb{B}\left(0,ϵ\right)}$, where $\overline{\mathbb{B}\left(0,ϵ\right)}$ denotes the closed ball centered at 0 with radius $ϵ$ ($0<ϵ\le \frac{1}{2}$), with respect to the Frobenius norm on ${\mathbb{R}}^{q\times r}$. Let ${\mathcal{D}}^{\left(0\right)}=CH\left(K^{\left(0\right)},{\mathcal{R}}_{\infty }\left(ϵ\right)\right)$ denote the convex-hull of the vertex $K^{\left(0\right)}$ with the basis ${\mathcal{R}}_{\infty }\left(ϵ\right)$. Let ${\mathcal{S}}_{\alpha }^{\left(0\right)}={\mathcal{S}}_{\alpha }\cap {\mathcal{D}}^{\left(0\right)}$ and note that ${\mathcal{S}}_{\alpha }^{\left(0\right)}$ is compact (but generally not convex). We wish to minimize the continuous function ${\sigma }_{max}\left(K\right)$ (or the continuous function $J\left(x_0,K\right)$, when $x_0$ is known) over the compact set ${\mathcal{S}}_{\alpha }\cap \overline{\mathbb{B}\left(K^{\left(0\right)},h\right)}$. Let $K_{*}$ denote a point in ${\mathcal{S}}_{\alpha }\cap \overline{\mathbb{B}\left(K^{\left(0\right)},h\right)}$ where the minimum of ${\sigma }_{max}\left(K\right)$ is accepted. Obviously, $K_{*}\in {\mathcal{D}}^{\left(0\right)}$, for some direction $U^{\left(0\right)}$, as above.

The suggested algorithm in Algorithm 1 works as follows:

We start with a point $K^{\left(0\right)}\in int\left({\mathcal{S}}_{\alpha }\right)$, found by the RS algorithm.

Assuming that $K_{*}\in {\mathcal{D}}^{\left(0\right)}$, the inner-loop ($j=1,\dots ,n$) uses the Ray-Shooting Method in order to find an approximation of the global minimum of the function ${\sigma }_{max}\left(K\right)$ over ${\mathcal{S}}_{\alpha }^{\left(0\right)}$—the portion of ${\mathcal{S}}_{\alpha }$ bounded in the cone ${\mathcal{D}}^{\left(0\right)}$. The proof of convergence in probability of the inner-loop and its complexity (under the above mentioned assumption) can be found in [16] (see also [22]). In the inner-loop, we choose a search direction by choosing a point F in ${\mathcal{R}}_{\infty }\left(ϵ\right)$—the base of the cone ${\mathcal{D}}^{\left(0\right)}$. Next, in the most inner-loop ($k=1,\dots ,s$) we scan the ray $K\left(t\right):=\left(1-t\right)K^{\left(0\right)}+tF$ and record the best controller on it. Repeating this a sufficient number of times (as is given in (7) and in the discussion right after), we reach $K_{*}$ (or an $ϵ$- neighborhood of it) with high probability, under the assumption that $K_{*}\in {\mathcal{D}}^{\left(0\right)}$.

The outer-loop ($i=1,\dots ,m$) is used as a substitution for restarting the RS algorithm again and again, by taking $K^{\left(\mathrm{best}\right)}$ as the new vertex of the search cone instead of $K^{\left(0\right)}$ and by choosing a different direction $U^{\left(0\right)}$. The choice of a different direction is made as a backup to the case where the above mentioned assumption didn’t hold in the previous iterations (see Remark 1 below). The replacement of $K^{\left(0\right)}$ by $K^{\left(\mathrm{best}\right)}$ can be considered as a heuristic step, which is made instead of running the RS algorithm many times in order to generate “the best starting point”, which is relevant only if we actually evaluate ${\sigma }_{max}\left(K\right)$ on each such point and take the point with the best value as the best starting point. Since we, in any case, evaluate ${\sigma }_{max}\left(K\right)$ in the main algorithm, we could avoid the repeated execution of the RS algorithm. The outer-loop is similar to what is done in the Hide-And-Seek algorithm (see [23,24]). The convergence in probability of the Hide-And-Seek algorithm can be found in [25].

Remark 1.

The volume of $\overline{\mathbb{B}\left(K^{\left(0\right)},h\right)}$ is given by $\frac{{\pi }^{\ell /2}}{\mathsf{\Gamma }\left(\ell /2+1\right)}·h^{\ell }$ where $\ell :=qr$ and Γ is the known Γ-function. The volume of ${\mathcal{D}}^{\left(0\right)}$ is given approximately (and exactly when $ϵ=0$) by $\frac{h}{\ell }·\frac{{\pi }^{\left(\ell -1\right)/2}}{\mathsf{\Gamma }\left(\left(\ell -1\right)/2+1\right)}·r_{\infty }^{\ell -1}$. Thus, by taking $r_{\infty }=h$, the portion of $\overline{\mathbb{B}\left(K^{\left(0\right)},h\right)}$ covered by ${\mathcal{D}}^{\left(0\right)}$ (i.e., the probability that $K_{*}\in {\mathcal{D}}^{\left(0\right)}$) is given by $\frac{\mathsf{\Gamma }\left(\ell /2+1\right)}{\ell ·\sqrt{\pi }·\mathsf{\Gamma }\left(\left(\ell -1\right)/2+1\right)}$. Let Θ denote the known relation between functions $f,g:\mathbb{N}\to R$ defined by $f\left(n\right)=\mathsf{\Theta }\left(g\left(n\right)\right)$ if and only if ${lim}_{n\to \infty }\frac{f\left(n\right)}{g\left(n\right)}=1$. Since $\mathsf{\Gamma }\left(\ell /2+1\right)=\mathsf{\Theta }\left(\sqrt{2\pi }{e}^{-\ell /2}{\left(\frac{\ell }{2}\right)}^{\frac{\ell +1}{2}}\right)$ and since ${\left(\frac{\ell -1}{\ell }\right)}^{\ell /2}\to e^{-1/2}$ when $\ell \to +\infty$, it follows that

$$\frac{\mathsf{\Gamma }\left(\ell /2+1\right)}{\ell ·\sqrt{\pi }·\mathsf{\Gamma }\left(\left(\ell -1\right)/2+1\right)}=\mathsf{\Theta }\left(\frac{1}{e·\sqrt{2\pi \ell }}\right).$$

Therefore, by taking $m=⌈e·\sqrt{2\pi \ell }⌉$ iterations in the outer-loop, we have $K_{*}\in {\mathcal{D}}^{\left(0\right)}$ almost surely. Specifically, when $\ell \ge 12$, we suggest taking $m=2\ell$ and $\ell \times \ell$ orthogonal matrix $U=\left[\begin{array}{cccc}{u}_1& u_2& \cdots & u_{\ell }\end{array}\right]$, and to take the directions $U_j^{\left(0\right)}=\pm mat\left(u_j\right),j=1,\dots ,\ell$ in the outer-loop.

The complexity of the suggested practical algorithm measured as the number of its arithmetic operations is given as follows:

• computing the matrix $P\left(K\left(t\right)\right)$ as in (5) takes $O\left({\left(p^2\right)}^3\right)=O\left(p^6\right)$, since the dominant operation is the inversion of the $p^2\times p^2$ matrix there.
• checking $K\left(t\right)\in {\mathcal{S}}_{\alpha }$ by checking the $\alpha$-stability of $A_{cl}\left(K\left(t\right)\right)$ (as well as computing $\sigma \left(K\left(t\right)\right)$), takes $O\left(p^3\right)$ for computing the characteristic polynomial of $A_{c\ell }\left(K\left(t\right)\right)$ (of $P\left(K\left(t\right)\right)$, respectively) and $O\left(p{log}_2^2\left(p\right)\left({log}_2^2\left(p\right)+{log}_2^2\left(b\right)\right)\right)$ for computing approximations ${\tilde{\lambda }}_j$ for all the eigenvalues ${\lambda }_j,j=1,\dots ,p$ of $A_{c\ell }\left(K\left(t\right)\right)$ (of $P\left(K\left(t\right)\right)$, respectively), with accuracy $\left|{\tilde{\lambda }}_j-{\lambda }_j\right|<2^{2-\frac{b}{p}}$ where $b\ge p{log}_2\left(p\right)$. The approximated eigenvalues can be computed to the accuracy $2^{2-\frac{b}{p}}=ϵ$, with $b=\left(2+{log}_2\left(\frac{1}{ϵ}\right)\right)p$, by the algorithm of V. Y. Pan (see [26]). We end up with $O\left(p^3\right)$ for these operations.
• computing uniformly distributed $q\times r$ matrix takes $O\left(max{\left(q,r\right)}^3\right)$ operations.

We therefore have a total complexity of $O\left(mns\left(max{\left(q,r\right)}^3+p^6\right)\right)$.

Let a closed $ϵ$-neighborhood of $K_{*}$ in $D^{\left(0\right)}$ be defined by

$${\mathcal{S}}_{\alpha }^{\left(0\right)}\left(ϵ\right)=\left\{K\in {\mathcal{S}}_{\alpha }^{\left(0\right)}\left|\sigma \left(K\right)\le \sigma \left(K_{*}\right)+ϵ\right.\right\}.$$

Let the idealized algorithm be the algorithm that samples the search space $D^{\left(0\right)}$ until hitting ${\mathcal{S}}_{\alpha }^{\left(0\right)}\left(ϵ\right)$, where the sampling is according to a general p.d.f. g and a related generator G. For $0<\beta <1$, the number of iterations needed to guarantee a probability of at least $1-\beta$ to hit ${\mathcal{S}}_{\alpha }^{\left(0\right)}\left(ϵ\right)$ is given by

$$⌈\frac{M_{g}Vo\ell \left(D^{\left(0\right)}\right)}{m_{g}Vo\ell \left({\mathcal{S}}_{\alpha }^{\left(0\right)}\left(ϵ\right)\right)}\left|ln\left(\beta \right)\right|⌉,$$

(7)

where $\mathrm{Vo}\ell$ denotes the volume of the related set and $M_g,m_g$ are the essential-supremum and essential-infimum of g over $D^{\left(0\right)}$, respectively (see [16]). Similarly to what is done in [16], one can show that the last is $O\left(\frac{\left|ln\left(\beta \right)\right|h{M}_g}{ϵm_g}{\left(\frac{r_{\infty }}{r_{ϵ}}\right)}^{qr}\right)$, where $r_{ϵ}$ is a radius of a ball of a basis of a cone with height $ϵ$ and vertex $K_{*}$ that has a volume that equals to $Vo\ell \left({\mathcal{S}}_{\alpha }^{\left(0\right)}\left(ϵ\right)\right)$. This results in an exponential number of iterations, but if we restrict the input of the algorithm to systems with $q,r$ satisfying $q\le q_0,r\le r_0$ when $q_0,r_0$ are fixed, then, the number of iterations would be $O\left(\frac{\left|ln\left(\beta \right)\right|h{M}_g}{ϵm_g}{\left(\frac{r_{\infty }}{r_{ϵ}}\right)}^{q_0{r}_0}\right)$, i.e., polynomial in $\left(\frac{r_{\infty }}{r_{ϵ}}\right)$—which can be considered as the true size of the problem (for a fixed $p,q,r$). In this sense we can say that the algorithm is efficient. The total number of arithmetic operations of the idealized algorithm that guarantees a probability at least $1-\beta$ to hit ${\mathcal{S}}_{\alpha }^{\left(0\right)}\left(ϵ\right)$ is therefore given by $O\left(\frac{\left|ln\left(\beta \right)\right|h}{ϵ}{\left(\frac{r_{\infty }}{r_{ϵ}}\right)}^{q_0{r}_0}\left(max{\left(q,r\right)}^3+p^6\right)\right)$, since sampling points according to the uniform distribution g (and therefore $m_g=M_g=1$) and the related generator G, takes $O\left(max{\left(q,r\right)}^3\right)$.

For the sake of comparison, which will be presented in the next section, we bring here, in Algorithm 2, the algorithm of D. Moerder and A. Calise (see [5]) adjusted to our formulation of the problem, which we call: the MC Algorithm. To the best of our knowledge, this is the best algorithm for LQR via SOF published so far.

Algorithm 1: The practical randomized algorithm for the LQR via static-output-feedback (SOF) problem.

Input: $0<ϵ\le \frac{1}{2},\alpha ,h,r_{\infty }>0$, integers: $m,n,s>0$,

controllable pairs $\left(A,B\right)$ and $\left(A^T,C^T\right)$,

matrices $Q>0,R\ge 0$ and $K^{\left(0\right)}\in int\left({\mathcal{S}}_{\alpha }\right)$.

Output: $K\in {\mathcal{S}}_{\alpha }$ close as possible to $K_{*}$.

1. compute $P\left(K^{\left(0\right)}\right)$ as in (5)

2. $P^{\left(\mathrm{best}\right)}\leftarrow P\left(K^{\left(0\right)}\right)$

3. ${\sigma }_{max}^{\left(\mathrm{best}\right)}\leftarrow max\left(\sigma \left(P^{\left(\mathrm{best}\right)}\right)\right)$

4. for $i=1$ to m do

4.1. choose $U^{\left(0\right)}$ such that ${∥U^{\left(0\right)}∥}_F=1$,

uniformly at random

4.2. let $L^{\left(0\right)}\leftarrow K^{\left(0\right)}+h·U^{\left(0\right)}$

4.3. for $j=1$ to n do

4.3.1. choose $F\in {\mathcal{R}}_{\infty }\left(ϵ\right)$ uniformly at random

4.3.1.1. for $k=1$ to s do

4.3.1.1.1. $t\leftarrow \frac{k}{s}$

4.3.1.1.2. $K\left(t\right)\leftarrow \left(1-t\right)K^{\left(0\right)}+tF$

4.3.1.1.3. if $K\left(t\right)\in {\mathcal{S}}_{\alpha }$ then

4.3.1.1.3.1. compute $P\left(K\left(t\right)\right)$ as in (5)

4.3.1.1.3.2. ${\sigma }_{max}\left(K\left(t\right)\right)\leftarrow max\left(\sigma \left(P\left(K\left(t\right)\right)\right)\right)$

4.3.1.1.3.3. if $\left({\sigma }_{max}\left(K\left(t\right)\right)<{\sigma }_{max}^{\left(\mathrm{best}\right)}\right)$ then

4.3.1.1.3.3.1. $K^{\left(\mathrm{best}\right)}\leftarrow K\left(t\right)$

4.3.1.1.3.3.2. $P^{\left(\mathrm{best}\right)}\leftarrow P\left(K\left(t\right)\right)$

4.3.1.1.3.3.3. ${\sigma }_{max}^{\left(\mathrm{best}\right)}\leftarrow {\sigma }_{max}\left(K\left(t\right)\right)$

4.4. $K^{\left(0\right)}\leftarrow K^{\left(\mathrm{best}\right)}$

5. return $K^{\left(\mathrm{best}\right)},P^{\left(\mathrm{best}\right)},{\sigma }_{max}^{\left(\mathrm{best}\right)}$

Algorithm 2: The MC Algorithm.

Input: $0<ϵ,\alpha$, integers: $m,s>0$,

controllable pairs $\left(A,B\right)$ and $\left(A^T,C^T\right)$,

matrices $Q>0,R>0$ and $K_0\in int\left({\mathcal{S}}_{\alpha }\right)$.

Output: $K\in {\mathcal{S}}_{\alpha }$ close as possible to $K_{*}$.

1. $j\leftarrow 0$

2. $A_0\leftarrow A-B{K}_{0}C$

3. $P_0\leftarrow -mat\left(\begin{array}{c}{\left(I_p\otimes A_0^T+A_0^T\otimes I_p\right)}^{-1}·\\ vec\left(Q+C^T{K}_0^{T}R{K}_{0}C\right)\end{array}\right)$

4. $S_0\leftarrow -mat\left({\left(I_p\otimes A_0+A_0\otimes I_p\right)}^{-1}·vec\left(I_p\right)\right)$

5. ${\sigma }_{max}\left(K_0\right)\leftarrow max\left(\sigma \left(P_0\right)\right)$

6. $\mathrm{\Delta }{K}_0\leftarrow R^{-1}{B}^T{P}_0{S}_0{C}^T{\left(C{S}_{0}C\right)}^{-1}-K_0$

7. $flag\leftarrow 0$

8. for $k=1$ to s do

8.1. $t\leftarrow \frac{k}{s}$

8.2. $K\left(t\right)\leftarrow \left(1-t\right)K_0+t\mathrm{\Delta }{K}_0$

8.3. if $K\left(t\right)\in {\mathcal{S}}_{\alpha }$ then

8.3.1. $A\left(t\right)\leftarrow A-BK\left(t\right)C$

8.3.2. $P\left(t\right)\leftarrow -mat\left(\begin{array}{c}{\left(I_p\otimes A{\left(t\right)}^T+A{\left(t\right)}^T\otimes I_p\right)}^{-1}·\\ vec\left(Q+C^{T}K{\left(t\right)}^{T}RK\left(t\right)C\right)\end{array}\right)$

8.3.3. $S\left(t\right)\leftarrow -mat\left({\left(I_p\otimes A\left(t\right)+A\left(t\right)\otimes I_p\right)}^{-1}·vec\left(I_p\right)\right)$

8.3.4. ${\sigma }_{max}\left(K\left(t\right)\right)\leftarrow max\left(\sigma \left(P\left(t\right)\right)\right)$

8.3.5. if ${\sigma }_{max}\left(K\left(t\right)\right)<{\sigma }_{max}\left(K_0\right)$ then

8.3.5.1. $K_1\leftarrow K\left(t\right)$

8.3.5.2. $A_1\leftarrow A-B{K}_{1}C$

8.3.5.3. $P_1\leftarrow P\left(t\right)$

8.3.5.4. $S_1\leftarrow S\left(t\right)$

8.3.5.5. ${\sigma }_{max}\left(K_1\right)\leftarrow {\sigma }_{max}\left(K\left(t\right)\right)$

8.3.5.6. $flag\leftarrow 1$

9. if $flag==1$ then

9.1. while $\left(\left|{\sigma }_{max}\left(K_{j+1}\right)-{\sigma }_{max}\left(K_j\right)\right|\ge ϵ\right)$ and $\left(j<m\right)$ do

9.1.1. $\mathrm{\Delta }{K}_j\leftarrow R^{-1}{B}^T{P}_j{S}_j{C}^T{\left(C{S}_{j}C\right)}^{-1}-K_j$

9.1.2. for $k=1$ to s do

9.1.2.1. $t\leftarrow \frac{k}{s}$

9.1.2.2. $K\left(t\right)\leftarrow \left(1-t\right)K_j+t\mathrm{\Delta }{K}_j$

9.1.2.3. if $K\left(t\right)\in {\mathcal{S}}_{\alpha }$ then

9.1.2.3.1. $A\left(t\right)\leftarrow A-BK\left(t\right)C$

9.1.2.3.2. $P\left(t\right)\leftarrow -mat\left(\begin{array}{c}{\left(I_p\otimes A{\left(t\right)}^T+A{\left(t\right)}^T\otimes I_p\right)}^{-1}·\\ vec\left(Q+C^{T}K{\left(t\right)}^{T}RK\left(t\right)C\right)\end{array}\right)$

9.1.2.3.3. $S\left(t\right)\leftarrow -mat\left({\left(I_p\otimes A\left(t\right)+A\left(t\right)\otimes I_p\right)}^{-1}·vec\left(I_p\right)\right)$

9.1.2.3.4. ${\sigma }_{max}\left(K\left(t\right)\right)\leftarrow max\left(\sigma \left(P\left(t\right)\right)\right)$

9.1.2.3.5. if ${\sigma }_{max}\left(K\left(t\right)\right)<{\sigma }_{max}\left(K_j\right)$ then

9.1.2.3.5.1. $K_{j+1}\leftarrow K\left(t\right)$

9.1.2.3.5.2. $A_{j+1}\leftarrow A-B{K}_{j+1}C$

9.1.2.3.5.3. $P_{j+1}\leftarrow P\left(t\right)$

9.1.2.3.5.4. $S_{j+1}\leftarrow S\left(t\right)$

9.1.2.3.5.5. ${\sigma }_{max}\left(K_{j+1}\right)\leftarrow {\sigma }_{max}\left(K\left(t\right)\right)$

9.1.2.3.5.6. $j\leftarrow j+1$

10. return $K^{\left(\mathrm{best}\right)}\leftarrow K_j,A_j,P_j,S_j,{\sigma }_{max}^{\left(\mathrm{best}\right)}\leftarrow {\sigma }_{max}\left(K^{\left(best\right)}\right)$

3. Experiments

In the following experiments we applied the Algorithm 1 and Algorithm 2, on systems taken from the liberaries [27,28,29]. We took only the systems with controllable $\left(A,B\right),\left(A^T,C^T\right)$ pairs, for which the RS algorithm succeeded in finding SOFs (see [16], Table 8, p. 231). In order to initialize the MC Algorithm, we also used the RS algorithm to find a starting $\alpha$-stabilizing static-output-feedback, for known optimal value for $\alpha$. In all the experiments for the suggested algorithm we used $m=100$, $n=100$, $s=100$, $h=100,r_{\infty }=100,ϵ={10}^{-16}$, and for the MC Algorithm we used $m=10,000,s=100$ (in order to get the same number of ${10}^6$ overall iterations and the same number $s=100$ of iterations for the local search). In every case, we took $Q=I_p,R=I_q$. The Stability Margin column of

Table 1. Results of the suggested randomized algorithm for LQR via SOF compared with the results of the MC Algorithm.

System	Stability Margin	RS CPU Time	${\mathit{\sigma }}_{\mathbf{max}}^{\left(0\right)}$ for $\left(\mathit{A},\mathit{B},\mathit{C}\right)$	${\mathit{\sigma }}_{\mathbf{max}}^{\left(\mathbf{best}\right)}$ for $\left(\mathit{A},\mathit{B},\mathit{C}\right)$ Suggested Algorithm	${\mathit{\sigma }}_{\mathbf{max}}\left({\mathit{F}}_{*}\right)$ for $\left(\mathit{A},\mathit{B}\right)$	Suggested Algorithm CPU Time	${\mathit{\sigma }}_{\mathbf{max}}^{\left(\mathbf{best}\right)}$ for $\left(\mathit{A},\mathit{B},\mathit{C}\right)$ MC Algorithm	MC Algorithm CPU Time
AC1	$0.1$	0.0625	20.9727	$\begin{array}{c}13.3987\hfill \\ (56.52\%)\hfill \end{array}$	13.0686	43.1718	16.2315	0.1562
AC2	$0.1$	0.0312	20.9727	$\begin{array}{c}13.4217\hfill \\ (56.25\%)\hfill \end{array}$	13.0686	42.3906	16.2315	0.2500
AC5	$0.875$	0.0625	$2.2821\times {10}^6$	$\begin{array}{c}2.2208\times {10}^6\hfill \\ (2.76\%)\hfill \end{array}$	$8.4264\times {10}^5$	56.8125	$2.0608\times {10}^6$	0.1093
AC6	$0.875$	0.1093	463.8502	$\begin{array}{c}6.2413\hfill \\ (7.33\times {10}^3\%)\hfill \end{array}$	5.9721	43.8125	91.3276	0.0625
AC11	$0.1$	0.1093	$1.0661\times {10}^3$	$\begin{array}{c}7.4244\hfill \\ (1.42\times {10}^4\%)\hfill \end{array}$	5.8648	31.6406	$1.0661\times {10}^3$	0.0625
AC12	$0.1$	<${10}^{-4}$	$4.1640\times {10}^{19}$	$\begin{array}{c}5.5149\times {10}^3\hfill \\ (7.55\times {10}^{17}\%)\hfill \end{array}$	$2.7690\times {10}^3$	46.3437	$2.9950\times {10}^3$	3.0625
AC15	$0.25$	<${10}^{-4}$	112.0500	$\begin{array}{c}106.1312\hfill \\ (5.57\%)\hfill \end{array}$	104.8458	42.0000	111.9781	0.1562
HE1	$0.1$	<${10}^{-4}$	26.7550	$\begin{array}{c}9.1169\hfill \\ (1.93\times {10}^2\%)\hfill \end{array}$	2.9961	44.8125	12.9412	0.1250
HE3	$0.25$	0.0312	944.3921	$\begin{array}{c}632.1294\hfill \\ (49.39\%)\hfill \end{array}$	611.8468	100.7812	668.0894	0.4218
HE4	$0.01$	0.0625	$4.7440\times {10}^3$	$\begin{array}{c}474.7121\hfill \\ (8.99\times {10}^2\%)\hfill \end{array}$	229.9171	152.9531	$4.7440\times {10}^3$	<${10}^{-4}$
ROC1	${10}^{-16}$	0.0468	$1.4211\times {10}^4$	$\begin{array}{c}1.1368\times {10}^4\hfill \\ (25\%)\hfill \end{array}$	$1.1207\times {10}^3$	81.2500	$1.4211\times {10}^4$	0.0937
ROC4	${10}^{-16}$	0.1875	$1.7688\times {10}^4$	$\begin{array}{c}8.6513\times {10}^3\hfill \\ (1.04\times {10}^2\%)\hfill \end{array}$	$8.5454\times {10}^2$	79.7968	$1.7688\times {10}^4$	0.0468
DIS4	1	<${10}^{-4}$	$3.2208\times {10}^7$	$\begin{array}{c}1.7529\hfill \\ (1.83\times {10}^9\%)\hfill \end{array}$	1.7504	68.2812	2.0166	0.7031
DIS5	$0.1$	0.0312	$3.9985\times {10}^5$	$\begin{array}{c}2.3800\times {10}^5\hfill \\ (68\%)\hfill \end{array}$	$9.0756\times {10}^4$	$109.8906$	$2.8304\times {10}^5$	0.3593
REA1	$0.75$	0.0468	$9.3478\times {10}^5$	$\begin{array}{c}2.2790\hfill \\ (4.10\times {10}^7\%)\hfill \end{array}$	2.2265	58.3906	2.7385	0.5156
REA2	$0.1$	0.0312	9.8349	$\begin{array}{c}2.2640\hfill \\ (3.34\times {10}^2\%)\hfill \end{array}$	2.2443	69.9843	2.7770	0.3125
TMD	$0.05$	0.0312	45.0958	$\begin{array}{c}27.2080\hfill \\ (65.74\%)\hfill \end{array}$	16.7680	37.2031	27.6023	0.8281
TF1	${10}^{-16}$	0.1093	$9.1632\times {10}^3$	$\begin{array}{c}193.2880\hfill \\ (4.64\times {10}^3\%)\hfill \end{array}$	58.1296	51.3750	$9.1632\times {10}^3$	0.0312
HF2D10	${10}^{-16}$	0.0312	1.8090	$\begin{array}{c}1.4032\hfill \\ (28.91\%)\hfill \end{array}$	1.3832	118.2656	1.4269	2.0781
HF2D11	${10}^{-16}$	0.0312	0.4315	$\begin{array}{c}0.3699\hfill \\ (16.65\%)\hfill \end{array}$	0.3676	154.0312	0.3784	1
NN1	${10}^{-16}$	0.0312	$7.4631\times {10}^3$	$\begin{array}{c}1.6144\times {10}^3\hfill \\ (3.62\times {10}^2\%)\hfill \end{array}$	106.7801	50.4375	$2.3561\times {10}^3$	0.2500
NN5	$0.01$	0.0468	$3.9123\times {10}^4$	$\begin{array}{c}9.6741\times {10}^3\hfill \\ (3.04\times {10}^2\%)\hfill \end{array}$	$2.8787\times {10}^3$	129.2812	$9.8102\times {10}^3$	0.3125
NN9	$0.01$	0.0312	$4.0577\times {10}^3$	$\begin{array}{c}295.6797\hfill \\ (1.27\times {10}^3\%)\hfill \end{array}$	21.2937	40.6093	$3.7349\times {10}^3$	0.0937
NN12	$0.01$	0.0937	934.5127	$\begin{array}{c}236.4467\hfill \\ (2.95\times {10}^2\%)\hfill \end{array}$	30.3714	35.3281	934.5127	0.0625
NN13	$0.1$	0.0312	6.4782	$\begin{array}{c}1.7094\hfill \\ (2.66\times {10}^2\%)\hfill \end{array}$	0.6299	33.9218	1.8055	0.4531
NN14	$0.1$	0.0312	4.8907	$\begin{array}{c}1.6664\hfill \\ (1.93\times {10}^2\%)\hfill \end{array}$	0.6299	33.4687	1.7922	0.1718
NN15	$0.01$	<${10}^{-4}$	$1.3309\times {10}^5$	$\begin{array}{c}387.3778\hfill \\ (3.42\times {10}^4\%)\hfill \end{array}$	386.5741	94.2031	387.4183	0.3281
NN16	$0.1$	0.1093	35.9487	$\begin{array}{c}6.0864\hfill \\ (4.90\times {10}^2\%)\hfill \end{array}$	2.3276	45.0468	5.9982	0.7343
NN17	$0.1$	0.0312	$2.6404\times {10}^3$	$\begin{array}{c}36.7664\hfill \\ (7.08\times {10}^3\%)\hfill \end{array}$	3.1308	28.3281	666.7218	0.0937

relates to $\alpha >0$ for which the real part of any eigenvalue of the closed-loop is less than or equal to $-\alpha$. The values of $\alpha$ in Table 1 relates to the largest $\alpha$ for which the RS algorithm succeeded in finding $K^{\left(0\right)}$. The reason is that if $\lambda$ is an eigenvalue of $A_{c\ell }\left(K\right)$ with corresponding eigenvector v then, (4) implies

$$\Re \left(\lambda \right)=-\frac{v^{*}\left(Q+C^T{K}^{T}RKC\right)v}{2v^{*}P\left(K\right)v}\le -\frac{v^{*}Qv}{2v^{*}P\left(K\right)v}.$$

It follows that minimizing ${\sigma }_{max}\left(K\right)$ results in a larger abscissa. Thus, it is worth searching for a starting point $K^{\left(0\right)}$ that maximizes the abscissa $\alpha$. This can be done efficiently by running a binary search on the abscissa and using the RS algorithm as an oracle. Note that RS CPU time appearing in the third column of Table 1 relates to running the RS algorithm for known optimal value of the abscissa. The RS algorithm is sufficiently fast also for this purpose, but other methods such as the HIFOO algorithm (see [19]) can be applied for this purpose as well.

Let ${\sigma }_{max}\left(F\right)$ denote the functional (6) for the system $\left(A,B,I_p\right)$, where $A-BF$ is stable, i.e., $F\in {\mathcal{S}}^{q\times p}$. Let $P\left(F\right)$ denote the unique solution of (4) for the system $\left(A,B,I_p\right)$ with F as above. Let ${\sigma }_{max}\left(K\right)$ denote the functional (4) for the system $\left(A,B,C\right)$ with $K\in {\mathcal{S}}^{q\times r}$ and related Lyapunov matrix $P=P\left(K\right)$. Now, if $A-BKC$ is stable for some K then, $A-BF$ is stable for $F=KC$ (but there might exist a F such that $A-BF$ is stable, but which cannot be defined as $KC$ for some $q\times r$ matrix K). Therefore

$${\sigma }_{max}\left(F_{*}\right)=\underset{F\in {\mathcal{S}}^{q\times p}}{min}{\sigma }_{max}\left(F\right)\le \underset{K\in {\mathcal{S}}_{\alpha }^{q\times r}\cap \overline{\mathbb{B}\left(K^{\left(0\right)},h\right)}}{min}{\sigma }_{max}\left(K\right)={\sigma }_{max}\left(K_{*}\right),$$

(8)

where $F_{*}$ is an optimal LQR state-feedback controller for the system $\left(A,B,I_p\right)$. We conclude that ${\sigma }_{max}\left(F_{*}\right)\le {\sigma }_{max}\left(K_{*}\right)\le {\sigma }_{max}\left(K^{\left(\mathrm{best}\right)}\right)$. Thus, ${\sigma }_{max}\left(F_{*}\right)$ is a lower-bound for ${\sigma }_{max}\left(K^{\left(\mathrm{best}\right)}\right)$ and can serve as a good estimator for it (as is evidently seen from Table 1 in many cases) in order to stop the algorithm earlier, since ${\sigma }_{max}\left(F_{*}\right)$ can be calculated in advance.

The fourth column of Table 1 represents ${\sigma }_{max}\left(K^{\left(0\right)}\right)$. The fifth column stands for ${\sigma }_{max}\left(K^{\left(\mathrm{best}\right)}\right)$, where the number in the parentheses is the relative improvement over ${\sigma }_{max}\left(K^{\left(0\right)}\right)$, in percent. The sixth column is for ${\sigma }_{max}\left(F_{*}\right)$ and the seventh column is for the CPU time of the suggested algorithm, given in seconds. The eighth column stands for ${\sigma }_{max}\left(K^{\left(\mathrm{best}\right)}\right)$ found by the MC Algorithm and finally, the ninth column stands for the CPU time of the MC Algorithm.

Regarding the suggested algorithm, note that the relative improvement of ${\sigma }_{max}^{\left(\mathrm{best}\right)}$ over ${\sigma }_{max}^{\left(0\right)}$ is at least $1000\%$ for the systems: AC6, AC11, AC12, DIS4, REA1, TF1, NN9, NN15 and NN17 (i.e., in 9 out of 29 systems). The AC12 should be noted, for which the improvement is $7.55·{10}^{17}\%$! Note also that for the (13 out of 29) systems: AC1, AC2, AC6, AC11, AC12, AC15, HE3, DIS4, REA1, REA2, HF2D10, HF2D11 and NN15, the value of ${\sigma }_{max}^{\left(\mathrm{best}\right)}$ for $\left(A,B,C\right)$ is very close to the value of ${\sigma }_{max}\left(F_{*}\right)$ for $\left(A,B\right)$.

Regarding the comparison with the MC Algorithm, we conclude that the MC algorithm actually failed in finding any improvement of ${\sigma }_{max}^{\left(\mathrm{best}\right)}$ over ${\sigma }_{max}^{\left(0\right)}$ for the systems: AC11, HE4, ROC1, ROC2, TF1 and NN12. The suggested algorithm performs significantly better regarding this key of performance for the systems: AC6, AC11, AC15, HE3, HE4, TF1, ROC4, NN9 and NN12. The suggested algorithm performs slightly better for the systems: AC1, AC2, DIS4, DIS5, HE1, HF2D10, HF2D11, REA1, REA2, ROC1, TMD, NN1, NN5 and NN13. Finally, the MC algorithm performs slightly better only for the systems: AC5, AC12 and NN16. We conclude that the suggested algorithm outperforms the MC Algorithm regarding the above-mentioned key of performance, although the MC Algorithm outperforms the suggested algorithm in terms of CPU time.

The MC Algorithm seems to perform better locally, while the suggested algorithm seems to perform better globally. Thus, practically, the best approach would be to apply the suggested algorithm in order to find a close neighborhood of a global minimum and then to apply the MC Algorithm on the result, for the local optimization.

4. Concluding Remarks

For a discrete-time system

$$\left\{\begin{array}{c}{x}_{k+1}=A{x}_k+B{u}_k\hfill \\ y_k=C{x}_k\hfill \end{array}\right.$$

(9)

and cost functional

$$J\left(x_0,u\right):=\sum _{k=0}^{\infty }\left(x_k^{T}Q{x}_k+u_k^{T}R{x}_k\right),$$

(10)

let $u_k=-K{y}_k$ be the SOF, and let $A_{c\ell }\left(K\right):=A-BKC$ be the closed-loop matrix. Let $\mathbb{D}$ denote the open unit-disk, let $0<\alpha <1$ and let ${\mathbb{D}}_{\alpha }$ denote the set of all $z\in \mathbb{D}$ with $\left|z\right|\le 1-\alpha$ (where $\left|z\right|$ is the absolute value of z). Let ${\mathcal{S}}^{q\times r}$ denote the set of all matrices $K\in {\mathbb{R}}^{q\times r}$ such that $\sigma \left(A_{c\ell }\right)\subset \mathbb{D}$ (i.e., stable in the discrete-time sense), and let ${\mathcal{S}}_{\alpha }^{q\times r}$ denote the set of all matrices $K\in {\mathbb{R}}^{q\times r}$ such that $\sigma \left(A_{c\ell }\right)\subset {\mathbb{D}}_{\alpha }$. In this case we say that $A_{c\ell }$ is $\alpha$-stable. Let $K\in {\mathcal{S}}_{\alpha }^{q\times r}$ be given. Substitution of $u_k=-K{y}_k=-KC{x}_k$ into (10) yields

$$J\left(x_0,K\right):=\sum _{k=0}^{\infty }{x}_k^T\left(Q+C^T{K}^{T}RKC\right)x_k.$$

(11)

Since $Q+C^T{K}^{T}RKC>0$ and since $A_{c\ell }\left(K\right)$ is stable, it follows that the Stein equation

$$P-A_{c\ell }{\left(K\right)}^{T}P{A}_{c\ell }\left(K\right)=Q+C^T{K}^{T}RKC$$

(12)

has a unique solution $P>0$, given by

$$P\left(K\right)=mat\left({\left(I_p\otimes I_p-A_{c\ell }{\left(K\right)}^T\otimes A_{c\ell }{\left(K\right)}^T\right)}^{-1}·vec\left(Q+C^T{K}^{T}RKC\right)\right).$$

Substitution of (12) into (11) and noting that $x_k=A_{c\ell }{\left(K\right)}^k{x}_0$ with $A_{c\ell }\left(K\right)$ stable, leads to

$$\begin{array}{cc}\hfill J\left(x_0,K\right)& =\sum _{k=0}^{\infty }{x}_k^T\left(P-A_{c\ell }{\left(K\right)}^{T}P{A}_{c\ell }\left(K\right)\right)x_k\hfill \\ & =\sum _{k=0}^{\infty }{x}_0^T{A}_{c\ell }{\left(K\right)}^{Tk}\left(P-A_{c\ell }{\left(K\right)}^{T}P{A}_{c\ell }\left(K\right)\right)A_{c\ell }{\left(K\right)}^k{x}_0=x_0^{T}P\left(K\right)x_0.\hfill \end{array}$$

Thus, we look for $K\in {\mathcal{S}}_{\alpha }^{q\times r}$ that minimizes the functional $J\left(x_0,K\right)=x_0^{T}P\left(K\right)x_0$, and when $x_0$ is unknown, we seek $K\in {\mathcal{S}}_{\alpha }^{q\times r}$ for which ${\sigma }_{max}\left(K\right):=max\left(\sigma \left(P\left(K\right)\right)\right)$ is minimal. Similarly to the continuous-time case, we have $\frac{J\left(x_0,K\right)}{{∥x_0∥}^2}\le {\sigma }_{max}\left(K\right)$ with equality in the worst case. Finally, if $\lambda$ is an eigenvalue of $A_{c\ell }\left(K\right)$ and v is a corresponding eigenvector then (12) yields $1-{\left|\lambda \right|}^2=\frac{v^{*}\left(Q+C^T{K}^{T}RKC\right)v}{v^{*}P\left(K\right)v}\ge \frac{v^{*}Qv}{v^{*}P\left(K\right)v}$. Therefore ${\left|\lambda \right|}^2\le 1-\frac{v^{*}Qv}{v^{*}P\left(K\right)v}$, and thus, minimizing ${\sigma }_{max}\left(K\right)$ results in larger abscissa. Now, the suggested algorithm can be readily applied to discrete-time systems. As to the MC Algorithm, we are not aware of any discrete-time analogue of it.

We conclude that the Ray-Shooting Method is a powerful tool, since it practically solves the problem of LQR via SOF, for real-life systems. The suggested algorithm has good performance, and is proved to converge in probability. The suggested method can similarly handle the problem of LQR via SOF for discrete-time systems. Obviously, this enlarges the scope and usability of the Ray-Shooting Method and we expect to receive more results in this direction.