Kaczmarz-Type Methods for Solving Matrix Equation AXB = C

Abstract

This paper proposes a class of randomized Kaczmarz and Gauss–Seidel-type methods for solving the matrix equation $AXB=C$, where the matrices A and B may be either full-rank or rank deficient and the system may be consistent or inconsistent. These iterative methods offer high computational efficiency and low memory requirements, as they avoid costly matrix–matrix multiplications. We rigorously establish theoretical convergence guarantees, proving that the generated sequences converge to the minimal Frobenius-norm solution (for consistent systems) or the minimal Frobenius-norm least squares solution (for inconsistent systems). Numerical experiments demonstrate the superiority of these methods over conventional matrix multiplication-based iterative approaches, particularly for high-dimensional problems.

1. Introduction

The study of matrix equations has emerged as a prominent research topic in numerical linear algebra. In this paper, we investigate the solution of the matrix equation of the form

$$AXB=C,$$

(1)

where $A\in {\mathbb{R}}^{m\times p}$, $B\in {\mathbb{R}}^{q\times n}$, and $C\in {\mathbb{R}}^{m\times n}$. This typical linear matrix equation has attracted considerable attention in both matrix theory and numerical analysis due to its important applications in fields such as systems theory, control theory, image restoration, and more [1,2].

There have been many literature studies and achievements on the matrix equation $AXB=C$. For instance, Chu provided sufficient and necessary conditions for the existence of symmetric solutions through singular value decomposition and generalized eigenvalue decomposition [3]. Most of these methods use direct methods, applying generalized inverse, singular value decomposition, QR factorization, and standard correlation decomposition to obtain the necessary and sufficient conditions and analytical expressions for the solution and least squares solution of matrix equations. Due to limitations in computer storage space and computing speed, the direct method may be less efficient in solving large matrices. Therefore, iterative methods for solving linear matrix equations have attracted great interest, such as in [4,5,6,7]. Many methods among these frequently use the matrix multiplication, which consumes a great deal of computing time.

By using the Kronecker products symbol, the matrix equation $AXB=C$ can be written in the following equivalent matrix vector form:

$$(B^T\otimes A)\mathrm{vec}\left(X\right)=\mathrm{vec}\left(C\right)$$

(2)

where the Kronecker product $B^T\otimes A\in {\mathbb{R}}^{mn\times pq}$, the right-side vector vec$\left(C\right)\in {\mathbb{R}}^{mn\times 1}$, and the unknown vector vec$\left(X\right)\in {\mathbb{R}}^{pq\times 1}$. With the application of Kronecker products, many algorithms are proposed to solve the matrix Equation (1) (see, e.g., [8,9,10,11,12]). However, when the dimensions of matrices A and B are large, the dimensions of the linear system in Equation (2) increase dramatically, which increases the memory usage and computational cost of numerical algorithms for finding an approximate solution.

Recently, the Kaczmarz method for solving the linear equation $Ax=b$ has been extended to solve matrix equations. In [13], Shafiei and Hajarian proposed a hierarchical Kaczmarz method for solving the Sylvester matrix equation $AX+XB=F$. Du et al. proposed the randomized block coordinate descent (RBCD) method for solving the matrix least squares problem $\underset{X\in {\mathbb{R}}^{p\times q}}{min}{\parallel C-AXB\parallel }_F^2$ in [14]. For the consistent matrix equation $AXB=C$, Wu et al. proposed the relaxed greedy randomized Kaczmarz (ME-RGRK) and maximal weighted residual Kaczmarz (ME-MWRK) methods [15]. Niu and Zheng proposed two randomized block Kaczmarz algorithms and studied block partitioning techniques [16]. Xing et al. proposed a randomized block Kaczmarz method along with an extended version to solve the matrix equation $AXB=C$ (consistent or inconsistent) [17]. In [18], Li et al. systematically derived Kaczmarz and coordinate descent algorithms for solving the matrix equations $AX=B$ and $XA=C$. Based on this work, we consider the use of Kaczmarz-type methods (row iteration) and Gauss–Seidel methods (column iteration) to solve (1) (which may be consistent or inconsistent) without matrix–matrix production.

The notations used throughout this paper are explained as follows. For a matrix A, its transpose, Moore–Penrose generalized inverse, rank, column space, and Frobenius norm are represented as $A^T$, $A^{+}$, $r\left(A\right)$, $R\left(A\right)$, and ${\parallel A\parallel }_F=\sqrt{\mathrm{trace}\left(A^{T}A\right)}$, respectively. We use ${\langle A,B\rangle }_F=\mathrm{trace}\left(A^{T}B\right)$ to denote the inner product of two matrices A and B. Let I denote the identity matrix, where the order is clear from the context. In addition, for a given matrix $G=\left(g_{ij}\right)\in {\mathbb{R}}^{m\times n}$, we use $G_{i,:}$, $G_{:,j}$, and ${\sigma }_{min}\left(G\right)$ to denote its ith row, jth column, and smallest nonzero singular value of G, respectively. In addition, we use $\mathbb{E}$ to denote the expectation and ${\mathbb{E}}_k$ to denote the conditional expectation of the first k iterations, that is,

$${\mathbb{E}}_k[·]=\mathbb{E}[·|i_0,j_0,i_1,j_1,\dots ,i_{k-1},j_{k-1}],$$

where $i_t$ and $j_t(t=0,1,\dots ,k-1)$ are the row and column indicators selected in the tth step. Let ${\mathbb{E}}_k^i[·]=\mathbb{E}[·|i_0,j_0,i_1,j_1,\dots ,i_{k-1},j_{k-1},j_k]$ and ${\mathbb{E}}_k^j[·]=\mathbb{E}[·|i_0,j_0,i_1,j_1,\dots ,i_{k-1},j_{k-1},i_k]$ represent the conditional expectations with respect to the random row index and random column index. Then, it is obvious that ${\mathbb{E}}_k[·]={\mathbb{E}}_k^i\left[{\mathbb{E}}_k^j[·]\right]$.

The rest of this paper is organized as follows. In the next two sections, we discuss the randomized Kaczmarz method for solving the consistent matrix equation $AXB=C$ and the randomized Gauss–Seidel (coordinate descent) method for solving the inconsistent matrix equation $AXB=C$. In Section 4, the extended Kaczmarz method and extended Gauss–Seidel method are derived for finding the minimal F-norm least squares solution of the matrix equation in (1). Section 5 presents some numerical experiments of the proposed methods, then conclusions are provided in the final section. This paper makes several significant advances in solving the matrix equation $AXB=C$:

• Algorithm Development: We propose novel randomized Kaczmarz (RK) and Gauss–Seidel-type methods that efficiently handle both full-rank and rank-deficient cases for matrices A and B, addressing both consistent and inconsistent systems.
• Theoretical Guarantees: We rigorously prove that all proposed algorithms converge linearly to either the minimal Frobenius-norm solution (for consistent systems) or the minimal Frobenius-norm least squares solution $A^{+}C{B}^{+}$ (for inconsistent systems). We summarize the convergence of the proposed methods in expectation to the minimal F-norm solution $X^{\ast }=A^{+}C{B}^{+}$ for all types of matrix equations in

  Table 1. Summary of the convergence of ME-RGRK [15], ME-MWRK [15], RBCD [14], CME-RK (Theorem 1), IME-RGS (Theorem 2), IME-REKRK (Theorem 3), IME-REKRGS (Theorem 4), DREK, and DREGS in expectation to the minimal F-norm solution $X^{\ast }=A^{+}C{B}^{+}$ for all types of matrix equations. (Note: Y means that the algorithm is convergent and N means that it is not.)

  Matrix Equation	r(A)	r(B)	ME-RGRK	ME-MWRK	RBCD	CME-RK	IME-RGS	IME-REKRK	IME-REKRGS	DREK	DREGS
  consistent	=p	=q	Y	Y	Y	Y	Y	Y	Y	Y	Y
  	=p	<q	Y	Y	N	Y	N	Y	Y (r(B) = n)	Y	Y
  	<P	=q	Y	Y	Y	Y	N	N	Y	Y	Y
  	<p	<q	Y	Y	N	Y	N	Y(r(B) = n)	N	Y	Y
  inconsistent	=p	=q	N	N	Y	N	Y	Y	Y	Y	Y
  	=p	<q	N	N	N	N	N	Y(r(B) = n)	N	Y	Y
  	<P	=q	N	N	Y	N	N	N	Y	Y	Y
  	<p	<q	N	N	N	N	N	Y(r(B) = n)	N	Y	Y

  .
• Computational Efficiency: All proposed methods avoid matrix–matrix multiplications, allowing them to achieve superior performance with low per-iteration cost and minimal storage requirements. Extensive experiments demonstrate the advantages of our approaches over conventional methods, particularly in high-dimensional scenarios.

2. Kaczmarz Method for Consistent Case

If the matrix equation in (1) is consistent, i.e., $A{A}^{+}C{B}^{+}B=C$ (necessary and sufficient conditions for consistency; hence, $A^{+}C{B}^{+}$ is a solution of the consistent matrix equation in (1)). In particular, if A is full row rank ($m\le p$) and B is full column rank ($q\ge n$), then the matrix equation in (1) is consistent because $X^{\ast }=A^T{\left(A{A}^T\right)}^{-1}C{\left(B^{T}B\right)}^{-1}{B}^T$ is one solution of this equation. In general, the matrix equation in (1) has multiple solutions. Next, we try to find its minimal F-norm solution $X^{\ast }=A^{+}C{B}^{+}$ via the Kaczmarz method.

Assume that A has no row that is all zeros and B has no column that is all zeros. Then, the matrix equation in (1) can be rewritten as the following system of matrix equations:

$$\left\{\begin{array}{c}AY=C,\\ B^T{X}^T=Y^T\end{array}\right.$$

(3)

where $Y\in {\mathbb{R}}^{p\times n}$. The classical Kaczmarz method, introduced in 1937 [19], is an iterative row projection algorithm for solving a consistent system $Ax=b$ in which $A\in {\mathbb{R}}^{m\times p}$, $b\in R^m$, and $x\in R^p$. This method involves only a single equation per iteration, which converges to the least norm solution $A^{+}b$ of $Ax=b$ with an initial iteration $x^{\left(0\right)}\in R\left(A^T\right)$, as follows:

$$x^{(k+1)}=x^{\left(k\right)}+{\displaystyle \frac{b_i-A_{i,:}{x}^{\left(k\right)}}{\parallel A_{i,:}{\parallel }_2^2}}{A}_{i,:}^T,k\ge 0$$

(4)

where $i=\left(kmodm\right)+1$. If we iterate the system of linear equations $A{Y}_{:,j}=C_{:,j}$, $j=1,\cdots ,n$ simultaneously and denote $Y^{\left(k\right)}=[Y_{:,1}^{\left(k\right)},Y_{:,2}^{\left(k\right)},\cdots ,Y_{:,n}^{\left(k\right)}]$, we obtain

$$Y^{(k+1)}=Y^{\left(k\right)}+{\displaystyle \frac{A_{i,:}^T}{\parallel A_{i,:}{\parallel }_2^2}}(C_{i,:}-A_{i,:}{Y}^{\left(k\right)}),k\ge 0,$$

(5)

where $i=\left(kmodm\right)+1$. Then, $A_{i,:}{Y}^{(k+1)}=C_{i,:}$ holds; that is, $Y^{(k+1)}$ is the projection of $Y^{\left(k\right)}$ onto the subspace $H_i=\left\{Y\in {\mathbb{R}}^{p\times n}:A_{i,:}Y=C_{i,:}\right\}$. Thus, we obtain an orthogonal projection method that can be used to solve the matrix equation $AY=C$.

Similarly, we can obtain the following orthogonal column projection method to solve the equation $B^T{X}^T={\left(Y^{(k+1)}\right)}^T$:

$$X^{(k+1)}=X^{\left(k\right)}+{\displaystyle \frac{Y_{:,j}^{(k+1)}-X^{\left(k\right)}{B}_{:,j}}{\parallel B_{:,j}{\parallel }_2^2}}{B}_{:,j}^T,k\ge 0$$

(6)

where $j=\left(kmodn\right)+1$. Then, $X^{(k+1)}{B}_{:,j}=Y_{:,j}^{(k+1)}$ holds; that is, $X^{(k+1)}$ is the projection of $X^{\left(k\right)}$ onto the subspace ${\widehat{H}}_j=\left\{X\in {\mathbb{R}}^{p\times q}:X{B}_{:,j}=Y_{:,j}^{(k+1)}\right\}$.

If i in (5) and j in (6) are selected randomly with probability proportional to the row norm of A and column norm of B, we obtain a randomized Kaczmarz-type algorithm, which is called the CME-RK algorithm (see Algirithm 1).

Algorithm 1 RK method for consistent matrix equation $AXB=C$ (CME-RK)

Input:  $A\in {\mathbb{R}}^{m\times p}$, $B\in {\mathbb{R}}^{q\times n}$, $C\in {\mathbb{R}}^{m\times n}$, $X^{\left(0\right)}\in {\mathbb{R}}^{p\times q}$, ${\left(X_{i,:}^{\left(0\right)}\right)}^T\in R\left(B\right),i=1,\dots ,p$, $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$, $Y_{:,j}^{\left(0\right)}\in R\left(A^T\right),j=1,\dots ,n$, K 1: For $i=1:m$, $M\left(i\right)=\parallel A_{i,:}{\parallel }_2^2$ 2: For $j=1:n$, $N\left(j\right)=\parallel B_{:,j}{\parallel }_2^2$ 3: for  $k=0,1,2,\cdots ,K-1$  do 4:    Pick i with probability $p_i\left(A\right)={\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ and j with probability ${\widehat{p}}_j\left(B\right)={\displaystyle \frac{\parallel B_{:,j}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$ 5:    Compute $Y^{(k+1)}=Y^{\left(k\right)}+{\displaystyle \frac{A_{i,:}^T}{M\left(i\right)}}(C_{i,:}-A_{i,:}{Y}^{\left(k\right)})$ 6:    Compute $X^{(k+1)}=X^{\left(k\right)}+{\displaystyle \frac{Y_{:,j}^{(k+1)}-X^{\left(k\right)}{B}_{:,j}}{N\left(j\right)}}{B}_{:,j}^T$ 7: end for 8: Output $X^{\left(K\right)}$

If the squared row norms of A and squared column norms of B are precomputed in advance, then the cost of each iteration of this method is $4p(n+q)+2p$ flops ($4np+p$ for step 5 and $4pq+p$ for step 6). In the following theorem, we use the idea of the RK method [20] to prove that $X^{\left(k\right)}$ generated by Algorithm 1 converges to the the minimal F-norm solution of $AXB=C$ when i and j are selected randomly at each iteration.

Before proving the convergence result of Algorithm 1, we analyze the convergence of $Y^{\left(k\right)}$ and provide the following lemmas, which can be considered as an extension of the theoretical results for solving the linear system of equations in [20]. Let $Y^{\ast }=A^{+}C$. The sequence $\left\{Y^{\left(k\right)}\right\}$ is generated by (5) starting from the initial matrix $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$. The following Lemmas 1–3 and their proofs can be found in [18].

Lemma 1

([18]). If the sequence $\left\{Y^{\left(k\right)}\right\}$ is convergent, it must converge to $Y^{\ast }=A^{+}C$, provided that $Y_{:,j}^{\left(0\right)}\in R\left(A^T\right),j=1,\dots ,n$.

Lemma 2

([18]). Let $A\in {\mathbb{R}}^{m\times p}$ be any nonzero matrix. For $Y_{:,j}\in R\left(A^T\right),j=1,\dots ,n$, it holds that

$${\parallel AY\parallel }_F^2\ge {\sigma }_{min}^2\left(A\right){\parallel Y\parallel }_F^2.$$

Lemma 3

([18]). The sequence $\left\{Y^{\left(k\right)}\right\}$ generated by (5) starting from the initial matrix $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$ in which $Y_{:,j}^{\left(0\right)}\in R\left(A^T\right),j=1,\dots ,n$ converges linearly to $A^{+}C$ in mean square form. Moreover, the solution error in the expectation for the iteration sequence $Y^{\left(k\right)}$ obeys

$$\mathbb{E}\left[{∥Y^{\left(k\right)}-A^{+}C∥}_F^2\right]\le {\rho }_1^k{∥Y^{\left(0\right)}-A^{+}C∥}_F^2,$$

(7)

where the ith row of A is selected with probability $p_i\left(A\right)={\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ and where ${\rho }_1=1-{\displaystyle \frac{{\sigma }_{min}^2\left(A\right)}{{\parallel A\parallel }_F^2}}$.

Similarly, we can obtain the following convergence result of the RK method for the matrix equation $B^T{X}^T={\left(Y^{\ast }\right)}^T$.

Lemma 4.

Let $X^{\ast }=A^{+}C{B}^{+}$. $\tilde{X}$ be generated by running a one-step RK update for solving the matrix equation $B^T{X}^T={\left(Y^{\ast }\right)}^T$ starting from any matrix $\widehat{X}\in {\mathbb{R}}^{p\times q}$ in which ${\left({\widehat{X}}_{i,:}\right)}^T\in R\left(B\right),$$i=1,\dots ,p$. Then, it holds that

$$\mathbb{E}\left[{∥\tilde{X}-A^{+}C{B}^{+}∥}_F^2\right]\le {\rho }_2{∥\widehat{X}-A^{+}C{B}^{+}∥}_F^2,$$

(8)

where the jth column of B is selected with probability ${\widehat{p}}_j\left(B\right)={\displaystyle \frac{\parallel B_{:,j}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$ and where ${\rho }_2=1-{\displaystyle \frac{{\sigma }_{min}^2\left(B\right)}{{\parallel B\parallel }_F^2}}$.

Lemma 5.

Let ${\tilde{H}}_j=\left\{X\in {\mathbb{R}}^{p\times q}:X{B}_{:,j}=Y_{:,j}^{\ast }\right\}$ be the subspaces consisting of the solutions to the unperturbed equations and let ${\widehat{H}}_j=\left\{X\in {\mathbb{R}}^{p\times q}:X{B}_{:,j}=Y_{:,j}^{(k+1)}\right\}$ be the solution spaces of the noisy equations. Then, ${\widehat{H}}_j=\left\{W+{\alpha }_j^{(k+1)}{B}_{:,j}^T,W\in {\tilde{H}}_j\right\}$, where ${\alpha }_j^{(k+1)}={\displaystyle \frac{Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }}{\parallel B_{:,j}{\parallel }_2^2}}$.

Proof. 

First, if $W\in {\tilde{H}}_j$, then

$$(W+{\alpha }_j^{(k+1)}{B}_{:,j}^T)B_{:,j}=W{B}_{:,j}+{\alpha }_j^{(k+1)}{B}_{:,j}^T{B}_{:,j}=Y_{:,j}^{\ast }+Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }=Y_{:,j}^{(k+1)},$$

so $W+{\alpha }_j^{(k+1)}{B}_{:,j}^T\in {\widehat{H}}_j$.

Next, let $V\in {\widehat{H}}_j$. Set $W=V-{\alpha }_j^{(k+1)}{B}_{:,j}^T$; then,

$$W{B}_{:,j}=(V-{\alpha }_j^{(k+1)}{B}_{:,j}^T)B_{:,j}=V{B}_{:,j}-{\alpha }_j^{(k+1)}{\parallel B_{:,j}\parallel }_2^2=Y_{:,j}^{(k+1)}-(Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast })=Y_{:,j}^{\ast },$$

meaning that $W\in {\tilde{H}}_j$. This completes the proof.    □

We present the convergence result of Algorithm 1 in the following theorem.

Theorem 1.

The sequence $\left\{X^{\left(k\right)}\right\}$ generated by Algorithm 1 starting from the initial matrix $X^{\left(0\right)}\in {\mathbb{R}}^{p\times q}$ converges linearly to the solution $X^{\ast }=A^{+}C{B}^{+}$ of the consistent matrix equation in (1) in mean square form if ${\left(X_{i,:}^{\left(0\right)}\right)}^T\in R\left(B\right),i=1,\dots ,p$ and $Y_{:,j}^{\left(0\right)}\in R\left(A^T\right),j=1,\dots ,n$. Moreover, the following relationship holds:

$$\mathbb{E}\left[{∥X^{\left(k\right)}-A^{+}C{B}^{+}∥}_F^2\right]\le \left({∥X^{\left(0\right)}-A^{+}C{B}^{+}∥}_F^2+{\displaystyle \frac{\eta }{{\parallel B\parallel }_F^2}}{∥Y^{\left(0\right)}-A^{+}C∥}_F^2\right){\rho }^k$$

where $\rho =max\{{\rho }_1,{\rho }_2\}$ and $\eta =\left\{\begin{array}{cc}{\displaystyle \frac{{\rho }_1}{|{\rho }_1-{\rho }_2|}},\hfill & if{\rho }_1\ne {\rho }_2,\hfill \\ k,\hfill & if{\rho }_1={\rho }_2.\hfill \end{array}\right.$

Proof. 

Let $X^{\left(k\right)}$ denote the kth iteration of the randomized Kaczmarz method (6) and let ${\widehat{H}}_j$ be the solution space chosen in the $(k+1)$th iteration. Then, $X^{(k+1)}$ is the orthogonal projection of $X^{\left(k\right)}$ onto ${\widehat{H}}_j$. Let ${\tilde{X}}^{(k+1)}$ denote the orthogonal projection of $X^{\left(k\right)}$ onto ${\tilde{H}}_j$. Using (6) and Lemma 5, we have

$$\begin{array}{cc}\hfill X^{(k+1)}& =X^{\left(k\right)}+{\displaystyle \frac{Y_{:,j}^{(k+1)}-X^{\left(k\right)}{B}_{:,j}}{\parallel B_{:,j}{\parallel }_2^2}}{B}_{:,j}^T\hfill \\ \hfill & =X^{\left(k\right)}+{\displaystyle \frac{Y_{:,j}^{\ast }-X^{\left(k\right)}{B}_{:,j}}{\parallel B_{:,j}{\parallel }_2^2}}{B}_{:,j}^T+{\displaystyle \frac{Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }}{\parallel B_{:,j}{\parallel }_2^2}}{B}_{:,j}^T\hfill \\ \hfill & ={\tilde{X}}^{(k+1)}+{\displaystyle \frac{Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }}{\parallel B_{:,j}{\parallel }_2^2}}{B}_{:,j}^T.\hfill \end{array}$$

Then,

$$\begin{array}{cc}\hfill {\langle X^{(k+1)}-{\tilde{X}}^{(k+1)},{\tilde{X}}^{(k+1)}-X^{\ast }\rangle }_F& ={〈{\displaystyle \frac{Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }}{\parallel B_{:,j}{\parallel }_2^2}}{B}_{:,j}^T,{\tilde{X}}^{(k+1)}-X^{\ast }〉}_F\hfill \\ \hfill & =\mathrm{trace}\left(B_{:,j}{\displaystyle \frac{{(Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast })}^T}{\parallel B_{:,j}{\parallel }_2^2}}({\tilde{X}}^{(k+1)}-X^{\ast })\right)\hfill \\ \hfill & =\mathrm{trace}\left(({\tilde{X}}^{(k+1)}-X^{\ast })B_{:,j}{\displaystyle \frac{{(Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast })}^T}{\parallel B_{:,j}{\parallel }_2^2}}\right)\hfill \\ \hfill & \left(\mathrm{by}\mathrm{trace}\right(MN)=\mathrm{trace}(NM)\mathrm{for}\mathrm{any}\mathrm{matrices}M,N)\hfill \\ \hfill & =0(\mathrm{by}{\tilde{X}}^{(k+1)}{B}_{:,j}=Y_{:,j}^{\ast },X^{\ast }{B}_{:,j}=Y_{:,j}^{\ast })\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill {∥X^{(k+1)}-{\tilde{X}}^{(k+1)}∥}_F^2& ={∥{\displaystyle \frac{Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }}{\parallel B_{:,j}{\parallel }_2^2}}{B}_{:,j}^T∥}_F^2\hfill \\ & ={\displaystyle \frac{1}{\parallel B_{:,j}{\parallel }_2^4}}\mathrm{trace}\left(B_{:,j}{(Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast })}^T(Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast })B_{:,j}^T\right)\hfill \\ \hfill & ={\displaystyle \frac{\parallel Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }{\parallel }_2^2}{\parallel B_{:,j}{\parallel }_2^4}}\mathrm{trace}\left(B_{:,j}{B}_{:,j}^T\right)={\displaystyle \frac{\parallel Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }{\parallel }_2^2}{\parallel B_{:,j}{\parallel }_2^2}}.\hfill \end{array}$$

Therefore,

$$\begin{array}{cc}\hfill {∥X^{(k+1)}-X^{\ast }∥}_F^2& ={∥X^{(k+1)}-{\tilde{X}}^{(k+1)}∥}_F^2+{∥{\tilde{X}}^{(k+1)}-X^{\ast }∥}_F^2\hfill \\ & ={∥{\tilde{X}}^{(k+1)}-X^{\ast }∥}_F^2+{\displaystyle \frac{\parallel Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }{\parallel }_2^2}{\parallel B_{:,j}{\parallel }_2^2}}.\hfill \end{array}$$

By taking the conditional expectation on both side of this equality, we can obtain

$$\begin{array}{cc}\hfill {\mathbb{E}}_k\left[{∥X^{(k+1)}-X^{\ast }∥}_F^2\right]& ={\mathbb{E}}_k\left[{∥{\tilde{X}}^{(k+1)}-X^{\ast }∥}_F^2\right]+{\mathbb{E}}_k\left[{\displaystyle \frac{\parallel Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }{\parallel }_2^2}{\parallel B_{:,j}{\parallel }_2^2}}\right].\hfill \end{array}$$

(9)

Next, we provide the estimates for the first and second parts of the right-hand side of the equality in (9). If ${\left(X_{i,:}^{\left(0\right)}\right)}^T\in R\left(B\right),i=1,\dots ,p$, then ${(X^{\left(0\right)}-A^{+}C{B}^{+})}_{i,:}^T\in R\left(B\right),$ $i=1,\dots ,p$. It is easy to show that ${(X^{\left(k\right)}-A^{+}C{B}^{+})}_{i,:}^T\in R\left(B\right),$ $i=1,\dots ,p$ by induction of (6). Then, per Lemma 4, we

$$\begin{array}{c}\hfill {\mathbb{E}}_k\left[\parallel {\tilde{X}}^{(k+1)}-A^{+}C{B}^{+}{\parallel }_F^2\right]\le {\rho }_2{∥X^{\left(k\right)}-A^{+}C{B}^{+}∥}_F^2.\end{array}$$

(10)

For the second part of the right-hand side of (9), we have

$$\begin{array}{cc}\hfill {\mathbb{E}}_k\left[{\displaystyle \frac{\parallel Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }{\parallel }_2^2}{\parallel B_{:,j}{\parallel }_2^2}}\right]& ={\mathbb{E}}_k^i{\mathbb{E}}_k^j\left[{\displaystyle \frac{\parallel Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }{\parallel }_2^2}{\parallel B_{:,j}{\parallel }_2^2}}\right]\hfill \\ \hfill & ={\mathbb{E}}_k^i\left[\sum _{j=1}^n{\displaystyle \frac{1}{{\parallel B\parallel }_F^2}}{\parallel Y_{:,j}^{(k+1)}-Y_{:,j}^{\ast }\parallel }_2^2\right]\hfill \\ \hfill & ={\displaystyle \frac{1}{{\parallel B\parallel }_F^2}}{\mathbb{E}}_k^i\left[{∥Y^{(k+1)}-Y^{\ast }∥}_F^2\right]\hfill \\ \hfill & ={\displaystyle \frac{1}{{\parallel B\parallel }_F^2}}{\mathbb{E}}_k\left[{∥Y^{(k+1)}-Y^{\ast }∥}_F^2\right].\hfill \end{array}$$

(11)

Substituting (10) and (11) into (9), we can obtain

$$\begin{array}{cc}\hfill {\mathbb{E}}_k\left[{∥X^{(k+1)}-A^{+}C{B}^{+}∥}_F^2\right]& \le {\rho }_2{∥X^{\left(k\right)}-A^{+}C{B}^{+}∥}_F^2+{\displaystyle \frac{1}{{\parallel B\parallel }_F^2}}{\mathbb{E}}_k\left[{∥Y^{(k+1)}-A^{+}C∥}_F^2\right],\hfill \end{array}$$

Then, applying this recursive relation iteratively and taking the full expectation, we have

$$\begin{array}{cc}\hfill \mathbb{E}\left[{∥X^{(k+1)}-A^{+}C{B}^{+}∥}_F^2\right]& \le {\rho }_2\mathbb{E}\left[{∥X^{\left(k\right)}-A^{+}C{B}^{+}∥}_F^2\right]+{\displaystyle \frac{1}{{\parallel B\parallel }_F^2}}\mathbb{E}\left[{∥Y^{(k+1)}-A^{+}C∥}_F^2\right]\hfill \\ \hfill & \le {\rho }_2\left({\rho }_{2}E\left[{∥X^{(k-1)}-A^{+}C{B}^{+}∥}_F^2\right]+{\displaystyle \frac{1}{{\parallel B\parallel }_F^2}}{\rho }_1^k{∥Y^{\left(0\right)}-A^{+}C∥}_F^2\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{{\parallel B\parallel }_F^2}}{\rho }_1^{k+1}{∥Y^{\left(0\right)}-A^{+}C∥}_F^2\left(\mathrm{by}\mathrm{Lemma} 3\right)\hfill \\ & \le \cdots \hfill \\ \hfill & \le {\rho }_2^{k+1}{∥X^{\left(0\right)}-A^{+}C{B}^{+}∥}_F^2+{\displaystyle \frac{\sum _{j=0}^k{\rho }_1^{j+1}{\rho }_2^{k-j}}{{\parallel B\parallel }_F^2}}{∥Y^{\left(0\right)}-A^{+}C∥}_F^2.\hfill \end{array}$$

(12)

If ${\rho }_1<{\rho }_2$, then

$$\sum _{j=0}^k{\rho }_1^{j+1}{\rho }_2^{k-j}={\rho }_2^{k+1}\sum _{j=0}^k{\left({\displaystyle \frac{{\rho }_1}{{\rho }_2}}\right)}^{j+1}\le {\displaystyle \frac{{\rho }_1}{{\rho }_2-{\rho }_1}}{\rho }_2^{k+1}.$$

If ${\rho }_1>{\rho }_2$, then

$$\sum _{j=0}^k{\rho }_1^{j+1}{\rho }_2^{k-j}={\rho }_1^{k+1}\sum _{j=0}^k{\left({\displaystyle \frac{{\rho }_2}{{\rho }_1}}\right)}^{j+1}\le {\displaystyle \frac{{\rho }_1}{{\rho }_1-{\rho }_2}}{\rho }_1^{k+1}.$$

Setting $\rho =max\{{\rho }_1,{\rho }_2\}$, (12) then becomes

$$\begin{array}{c}\hfill E\left[{∥X^{(k+1)}-A^{+}C{B}^{+}∥}_F^2\right]\le \left({∥X^{\left(0\right)}-A^{+}C{B}^{+}∥}_F^2+{\displaystyle \frac{{\rho }_1}{|{\rho }_1-{\rho }_2|}}{\displaystyle \frac{{∥Y^{\left(0\right)}-A^{+}C∥}_F^2}{{\parallel B\parallel }_F^2}}\right){\rho }^{k+1}.\end{array}$$

If ${\rho }_1={\rho }_2$, then ${\displaystyle \sum _{j=0}^k}{\rho }_1^{j+1}{\rho }_2^{k-j}=(k+1){\rho }^{k+1}$. Therefore, (12) becomes

$$\begin{array}{c}\hfill E\left[{∥X^{(k+1)}-A^{+}C{B}^{+}∥}_F^2\right]\le \left({∥X^{\left(0\right)}-A^{+}C{B}^{+}∥}_F^2+(k+1){\displaystyle \frac{{∥Y^{\left(0\right)}-A^{+}C∥}_F^2}{{\parallel B\parallel }_F^2}}\right){\rho }^{k+1}.\end{array}$$

This completes the proof.    □

Remark 1.

Algorithm 1 has the advantage that $X^{\left(k\right)}$ and $Y^{\left(k\right)}$ can be iteratively solved alternately at each step; alternatively, the approximate value of $Y^{\ast }=A^{+}C$ can first be obtained iteratively, then the approximate value of $X^{\ast }=A^{+}C{B}^{+}$ can be solved iteratively.

Generally, if we take $X^{\left(0\right)}=0\in {\mathbb{R}}^{p\times q}$ and $Y^{\left(0\right)}=0\in {\mathbb{R}}^{p\times n}$, then the initial conditions are all satisfied ($0\in R\left(A^T\right)$, $0\in R\left(B\right)$).

3. Gauss–Seidel Method for Inconsistent Case

If the matrix equation in (1) is inconsistent, then there is no solution to the equation. Considering the least squares solution of the matrix equation in (1), it is obvious that $X^{\ast }=A^{+}C{B}^{+}$ is the unique minimal F-norm least squares solution of the matrix equation in (1), that is,

$$X^{\ast }=A^{+}C{B}^{+}=argmin\left\{{\parallel X\parallel }_F:X\in arg\underset{X\in {\mathbb{R}}^{p\times q}}{min}{\parallel AXB-C\parallel }_F\right\}.$$

If A possesses full column rank and B possesses full row rank, then the matrix equation in (1) has a unique least squares solution $X^{\ast }={\left(A^{T}A\right)}^{-1}{A}^{T}C{B}^T{\left(B{B}^T\right)}^{-1}$. In general, the matrix equation in (1) has multiple least squares solutions. Here, we assume that A has no column that is all zeros and B has no row that is all zeros. Next, we find $X^{\ast }$ with the Gauss–Seidel (or coordinate descent) method.

When a linear system of equations $Ax=b$ is inconsistent, i.e., where $A\in {\mathbb{R}}^{m\times p}$ and $r\left(A\right)=p$ ($p\le m$), the RGS (RCD) method [21] below is a very effective method for solving its least squares solution:

$${\alpha }_k={\displaystyle \frac{A_{:,j}^T{\mathbb{R}}^{\left(k\right)}}{\parallel A_{:,j}{\parallel }_2^2}},x_j^{(k+1)}=x_j^{\left(k\right)}+{\alpha }_k,{\mathbb{R}}^{(k+1)}=r^{\left(k\right)}-{\alpha }_k{A}_{:,j},{\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$$

(13)

where $x^{\left(0\right)}\in R^p$ is arbitrary and $r^{\left(0\right)}=b-A{x}^{\left(0\right)}\in R^m$. Using simultaneous n-iterative formulae for solving $A{Y}_{:,l}=C_{:,l},l=1,\cdots ,n$, we obtain

$$W^{\left(k\right)}={\displaystyle \frac{A_{:,j}^T{R}^{\left(k\right)}}{\parallel A_{:,j}{\parallel }_2^2}},Y_{j,:}^{(k+1)}=Y_{j,:}^{\left(k\right)}+W^{\left(k\right)},{\mathbb{R}}^{(k+1)}=R^{\left(k\right)}-A_{:,j}{W}^{\left(k\right)},{\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}},$$

(14)

where $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$, $R^{\left(0\right)}=C-A{Y}^{\left(0\right)}$. This is a column projection method for solving the least squares solution of $AY=C$; the cost of each iteration in the method is $4mn+n$ flops if the squared the column norms of A are precomputed in advance.

Similarly, we can solve the least squares solution of $B^T{X}^T={\left(Y^{(k+1)}\right)}^T$ using the RGS method:

$$U^{\left(k\right)}={\displaystyle \frac{E^{\left(k\right)}{B}_{i,:}^T}{\parallel B_{i,:}{\parallel }_2^2}},X_{:,i}^{(k+1)}=X_{:,i}^{\left(k\right)}+U^{\left(k\right)},E^{(k+1)}=E^{\left(k\right)}-U^{\left(k\right)}{B}_{i,:}+I_{:,i}{W}^{\left(k\right)},p_i\left(B\right)={\displaystyle \frac{\parallel B_{i,:}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$$

(15)

where $X^{\left(0\right)}\in {\mathbb{R}}^{p\times q}$, $E^{\left(0\right)}=Y^{\left(1\right)}-X^{\left(0\right)}B$. This is a row projection method; the cost of each iteration of the method is $4np+n+p$ flops if the squared row norms of B are precomputed in advance.

With (14) and (15), we can obtain an RGS method for solving (1) as follows, which is called the IME-RGS algorithm (see Algorithm 2).

Algorithm 2 RGS method for inconsistent matrix equation $AXB=C$ (IME-RGS)

Input:  $A\in {\mathbb{R}}^{m\times p}$, $B\in {\mathbb{R}}^{q\times n}$, $C\in {\mathbb{R}}^{m\times n}$, $X^{\left(0\right)}\in {\mathbb{R}}^{p\times q}$, $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$, $R^{\left(0\right)}=C-A{Y}^{\left(0\right)}$, $E^{(-1)}=0\in {\mathbb{R}}^{p\times n}$, K 1: For $j=1:p$, $M\left(j\right)=\parallel A_{:,j}{\parallel }_2^2$ 2: For $i=1:q$, $N\left(i\right)=\parallel B_{i,:}{\parallel }_2^2$ 3: for  $k=0,1,2,\cdots ,K-1$  do 4:     Pick j with probability ${\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ and i with probability $p_i\left(B\right)={\displaystyle \frac{\parallel B_{i,:}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$ 5:     Compute $W^{\left(k\right)}={\displaystyle \frac{A_{:,j}^T{R}^{\left(k\right)}}{M\left(j\right)}}$, $Y_{j,:}^{(k+1)}=Y_{j,:}^{\left(k\right)}+W^{\left(k\right)}$, $R^{(k+1)}=R^{\left(k\right)}-A_{:,j}{W}^{\left(k\right)}$, $E_{j,:}^{\left(k\right)}=E_{j,:}^{(k-1)}+W^{\left(k\right)}$ 6:     Compute $U^{\left(k\right)}={\displaystyle \frac{E^{\left(k\right)}{B}_{i,:}^T}{N\left(i\right)}}$, $X_{:,i}^{(k+1)}=X_{:,i}^{\left(k\right)}+U^{\left(k\right)}$, $E^{(k+1)}=E^{\left(k\right)}-U^{\left(k\right)}{B}_{i,:}$ 7: end for 8: Output $X^{\left(K\right)}$

In order to prove the convergence of Algorithm 2, we provide the following Lemma 6, the proof of which can be found in [18].

Lemma 6

([18]). Let $Y^{\ast }=A^{+}C$. The sequence $\left\{Y^{\left(k\right)}\right\}$ is generated by (14) starting from the initial matrix $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$; then, the following holds:

$$\mathbb{E}\left[\parallel A{Y}^{\left(k\right)}-A{A}^{+}{C\parallel }_F^2\right]\le {\rho }_1^k{\parallel A{Y}^{\left(0\right)}-A{A}^{+}C\parallel }_F^2$$

(16)

where the jth column of A is selected with probability ${\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$.

Lemma 7.

Let $X^{\ast }=A^{+}C{B}^{+}$. $\tilde{X}$ be generated by running a one-step RGS update for solving the matrix equation $B^T{X}^T={\left(Y^{\ast }\right)}^T$ starting from any matrix $\widehat{X}\in {\mathbb{R}}^{p\times q}$. Then, it holds that

$$\mathbb{E}[\parallel A(\tilde{X}-X^{\ast }){B\parallel }_F^2]\le {\rho }_2{\parallel A(\widehat{X}-X^{\ast })B\parallel }_F^2,$$

(17)

where the ith row of B is selected with probability $p_i\left(B\right)={\displaystyle \frac{\parallel B_{i,:}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$.

Proof. 

From the definition of coordinate descent updates for $B^T{X}^T={\left(Y^{\ast }\right)}^T$, we have

$$\tilde{X}=\widehat{X}+{\displaystyle \frac{(Y^{\ast }-\widehat{X}B)B_{i,:}^T}{\parallel B_{i,:}{\parallel }_2^2}}{I}_{i,:},$$

which yields $A\tilde{X}B=A\widehat{X}B+{\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^2}}A(Y^{\ast }-\widehat{X}B)B_{i,:}^T{B}_{i,:}$. Using the projection formula satisfied by coordinate descent $B_{i,:}{B}^T{\left(\tilde{X}\right)}^T=B_{i,:}{\left(Y^{\ast }\right)}^T$ and the properties of MP generalized inversion, we have

$$\tilde{X}B{B}_{i,:}^T=Y^{\ast }{B}_{i,:}^T,X^{\ast }B{B}^T=Y^{\ast }{B}^T,A^{T}A{X}^{\ast }B{B}^T=A^{T}C{B}^T.$$

Then,

$$\begin{array}{cc}\hfill {\langle A(\tilde{X}-\widehat{X})B,A(\tilde{X}-X^{\ast })B\rangle }_F& ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^2}}{\langle A(Y^{\ast }-\widehat{X}B)B_{i,:}^T{B}_{i,:},A(\tilde{X}-X^{\ast })B\rangle }_F\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^2}}\mathrm{trace}\left(B_{i,:}^T{B}_{i,:}{(Y^{\ast }-\widehat{X}B)}^T{A}^{T}A(\tilde{X}-X^{\ast })B\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^2}}\mathrm{trace}\left(A(\tilde{X}-X^{\ast })B{B}_{i,:}^T{B}_{i,:}{(Y^{\ast }-\widehat{X}B)}^T{A}^T\right)\hfill \\ \hfill & =0(\mathrm{by}\tilde{X}B{B}_{i,:}^T=X^{\ast }B{B}_{i,:}^T)\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill \parallel A(\tilde{X}-\widehat{X}){B\parallel }_F^2& ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^4}}{\parallel A(Y^{\ast }-\widehat{X}B)B_{i,:}^T{B}_{i,:}\parallel }_F^2\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^4}}\mathrm{trace}\left(B_{i,:}^T{B}_{i,:}{(Y^{\ast }-\widehat{X}B)}^T{A}^{T}A(Y^{\ast }-\widehat{X}B)B_{i,:}^T{B}_{i,:}\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^4}}\mathrm{trace}\left(A(Y^{\ast }-\widehat{X}B)B_{i,:}^T{B}_{i,:}{B}_{i,:}^T{B}_{i,:}{(Y^{\ast }-\widehat{X}B)}^T{A}^T\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^2}}\mathrm{trace}\left(A(Y^{\ast }-\widehat{X}B)B_{i,:}^T{B}_{i,:}{(Y^{\ast }-\widehat{X}B)}^T{A}^T\right)\hfill \\ \hfill & ={\displaystyle \frac{\parallel A(Y^{\ast }-\widehat{X}B)B_{i,:}^T{\parallel }_2^2}{\parallel B_{i,:}{\parallel }_2^2}}={\displaystyle \frac{\parallel A(X^{\ast }-\widehat{X})B{B}_{i,:}^T{\parallel }_2^2}{\parallel B_{i,:}{\parallel }_2^2}}.\hfill \end{array}$$

Therefore,

$$\begin{array}{cc}\hfill \parallel A(\tilde{X}-X^{\ast }){B\parallel }_F^2& =\parallel A(\widehat{X}-X^{\ast }){B\parallel }_F^2-{\parallel A(\tilde{X}-\widehat{X})B\parallel }_F^2\hfill \\ \hfill & =\parallel A(\widehat{X}-X^{\ast }){B\parallel }_F^2-{\displaystyle \frac{\parallel A(X^{\ast }-\widehat{X})B{B}_{i,:}^T{\parallel }_2^2}{\parallel B_{i,:}{\parallel }_2^2}}.\hfill \end{array}$$

(18)

By taking the expectation on both sides of (18), we can obtain

$$\begin{array}{cc}\hfill \mathbb{E}[\parallel A(\tilde{X}-X^{\ast })B{\parallel }_F^2]& =\mathbb{E}\left[\parallel A(\widehat{X}-X^{\ast }){B\parallel }_F^2-{\displaystyle \frac{\parallel A(X^{\ast }-\widehat{X})B{B}_{i,:}^T{\parallel }_2^2}{\parallel B_{i,:}{\parallel }_2^2}}\right]\hfill \\ \hfill & =\parallel A(\widehat{X}-X^{\ast }){B\parallel }_F^2-\sum _{i=1}^q{\displaystyle \frac{\parallel B_{i,:}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}{\displaystyle \frac{\parallel A(X^{\ast }-\widehat{X})B{B}_{i,:}^T{\parallel }_2^2}{\parallel B_{i,:}{\parallel }_2^2}}\hfill \\ \hfill & =\parallel A(\widehat{X}-X^{\ast }){B\parallel }_F^2-{\displaystyle \frac{\parallel A(X^{\ast }-\widehat{X})B{B}^T{\parallel }_F^2}{{\parallel B\parallel }_F^2}}\hfill \\ \hfill & =\parallel A(\widehat{X}-X^{\ast }){B\parallel }_F^2-{\displaystyle \frac{\parallel B{\left(A(X^{\ast }-\widehat{X})B\right)}^T{\parallel }_F^2}{{\parallel B\parallel }_F^2}}\hfill \\ \hfill & \le \parallel A(\widehat{X}-X^{\ast }){B\parallel }_F^2-{\displaystyle \frac{{\sigma }_{min}^2\left(B\right)}{{\parallel B\parallel }_F^2}}{\parallel {\left(A(X^{\ast }-\widehat{X})B\right)}^T\parallel }_F^2\hfill \\ \hfill & =\left(1-{\displaystyle \frac{{\sigma }_{min}^2\left(B\right)}{{\parallel B\parallel }_F^2}}\right){\parallel A(X^{\ast }-\widehat{X})B\parallel }_F^2.\hfill \end{array}$$

The inequality is obtained by Lemma 2 because all columns of ${\left(A(X^{\ast }-\widehat{X})B\right)}^T$ are in the range of $B^T$. This completes the proof.    □

Theorem 2.

Let $\left\{X^{\left(k\right)}\right\}$ denote the sequence generated by Algorithm 2 for the inconsistent matrix equation in (1) starting from any initial matrix $X^{\left(0\right)}\in {\mathbb{R}}^{p\times q}$ and $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$. In exact arithmetic, it holds that

$$\mathbb{E}[\parallel A{X}^{\left(k\right)}B-A{A}^{+}C{B}^{+}B{\parallel }_F^2]\le \left[\parallel A{X}^{\left(0\right)}B-A{A}^{+}C{B}^{+}{B\parallel }_F^2+\eta {\parallel A{Y}^{\left(0\right)}-A{A}^{+}C\parallel }_F^2\right]{\rho }^k.$$

Proof. 

Let $X^{\left(k\right)}$ denote the kth iteration of the RGS method in (15) that solves $B^T{X}^T={\left(Y^{(k+1)}\right)}^T$ and let ${\tilde{X}}^{(k+1)}$ be the one-step RGS iteration that solves $B^T{X}^T={\left(Y^{\ast }\right)}^T$ from $X^{\left(k\right)}$; then, we have

$$X^{(k+1)}=X^{\left(k\right)}+{\displaystyle \frac{(Y^{(k+1)}-X^{\left(k\right)}B)B_{i,:}^T}{\parallel B_{i,:}{\parallel }_2^2}}{I}_{i,:},{\tilde{X}}^{(k+1)}=X^{\left(k\right)}+{\displaystyle \frac{(Y^{\ast }-X^{\left(k\right)}B)B_{i,:}^T}{\parallel B_{i,:}{\parallel }_2^2}}{I}_{i,:}.$$

Then,

$$\begin{array}{cc}\hfill & {\langle A(X^{(k+1)}-{\tilde{X}}^{(k+1)})B,A({\tilde{X}}^{(k+1)}-X^{\ast })B\rangle }_F\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^2}}{\langle A(Y^{(k+1)}-Y^{\ast })B_{i,:}^T{B}_{i,:},A({\tilde{X}}^{(k+1)}-X^{\ast })B\rangle }_F\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^2}}\mathrm{trace}\left(B_{i,:}^T{B}_{i,:}{(Y^{(k+1)}-Y^{\ast })}^T{A}^{T}A({\tilde{X}}^{(k+1)}-X^{\ast })B\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^2}}\mathrm{trace}\left(A({\tilde{X}}^{(k+1)}-X^{\ast })B{B}_{i,:}^T{B}_{i,:}{(Y^{(k+1)}-Y^{\ast })}^T{A}^T\right)\hfill \\ \hfill & =0(\mathrm{by}{\tilde{X}}^{(k+1)}B{B}_{i,:}^T=Y^{\ast }{B}_{i,:}^T,X^{\ast }B{B}_{i,:}^T=Y^{\ast }{B}_{i,:}^T)\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill \parallel A(X^{(k+1)}-{\tilde{X}}^{(k+1)}){B\parallel }_F^2& ={∥{\displaystyle \frac{A(Y^{(k+1)}-Y^{\ast })B_{i,:}^T{B}_{i,:}}{\parallel B_{i,:}{\parallel }_2^2}}∥}_F^2\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^4}}\mathrm{trace}\left(B_{i,:}^T{B}_{i,:}{(Y^{(k+1)}-Y^{\ast })}^T{A}^{T}A(Y^{(k+1)}-Y^{\ast })B_{i,:}^T{B}_{i,:}\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{\parallel B_{i,:}{\parallel }_2^4}}\mathrm{trace}\left(A(Y^{(k+1)}-Y^{\ast })B_{i,:}^T{B}_{i,:}{B}_{i,:}^T{B}_{i,:}{(Y^{(k+1)}-Y^{\ast })}^T{A}^T\right)\hfill \\ \hfill & ={\displaystyle \frac{\parallel A(Y^{(k+1)}-Y^{\ast })B_{i,:}^T{\parallel }_2^2}{\parallel B_{i,:}{\parallel }_2^2}}\le {\parallel A(Y^{(k+1)}-Y^{\ast })\parallel }_F^2.\hfill \end{array}$$

Therefore,

$$\begin{array}{cc}\hfill \parallel A(X^{(k+1)}-X^{\ast }){B\parallel }_F^2& =\parallel A(X^{(k+1)}-{\tilde{X}}^{(k+1)}){B\parallel }_F^2+{\parallel A({\tilde{X}}^{(k+1)}-X^{\ast })B\parallel }_F^2\hfill \\ \hfill & \le \parallel A(Y^{(k+1)}-Y^{\ast }){\parallel }_F^2+{\parallel A({\tilde{X}}^{(k+1)}-X^{\ast })B\parallel }_F^2.\hfill \end{array}$$

(19)

By taking the conditional expectation on both sides of (19), we can obtain

$$\begin{array}{cc}\hfill {\mathbb{E}}_k\left[\parallel A(X^{(k+1)}-X^{\ast }){B\parallel }_F^2\right]& \le {\mathbb{E}}_k\left[\parallel A({\tilde{X}}^{k+1}-X^{\ast }){B\parallel }_F^2\right]+{\mathbb{E}}_k\left[\parallel A(Y^{(k+1)}-Y^{\ast }){\parallel }_F^2\right]\hfill \\ \hfill & \le {\rho }_2\parallel A(X^{\left(k\right)}-X^{\ast }){B\parallel }_F^2+{\rho }_1{\parallel A(Y^{\left(k\right)}-Y^{\ast })\parallel }_F^2.\hfill \end{array}$$

The last inequality is obtained by Lemmas 6 and 7. Applying this recursive relation iteratively, we have

$$\begin{array}{cc}\hfill \mathbb{E}\left[\parallel A(X^{(k+1)}-X^{\ast }){B\parallel }_F^2\right]& \le {\rho }_2\mathbb{E}\left[\parallel A(X^{\left(k\right)}-X^{\ast }){B\parallel }_F^2\right]+{\rho }_1\mathbb{E}\left[\parallel A{Y}^{\left(k\right)}-A{Y}^{\ast }{\parallel }_F^2\right]\hfill \\ \hfill & \le {\rho }_2^2\mathbb{E}\left[\parallel A(X^{(k-1)}-X^{\ast }){B\parallel }_F^2\right]+{\rho }_1({\rho }_1^{k-1}{\rho }_2+{\rho }_1^k){\parallel A{Y}^{\left(0\right)}-A{Y}^{\ast }\parallel }_F^2\hfill \\ \hfill & \le \cdots \hfill \\ \hfill & \le {\rho }_2^{k+1}\parallel A(X^{\left(0\right)}-X^{\ast }){B\parallel }_F^2+\sum _{i=0}^k{\rho }_1^{i+1}{\rho }_2^{k-i}{\parallel A{Y}^{\left(0\right)}-A{Y}^{\ast }\parallel }_F^2\hfill \\ \hfill & \le (\parallel A(X^{\left(0\right)}-X^{\ast }){B\parallel }_F^2+\eta \parallel A{Y}^{\left(0\right)}-A{Y}^{\ast }{\parallel }_F^2){\rho }^{k+1},\hfill \end{array}$$

(20)

where $\eta$ and $\rho$ are defined in Theorem 1. This completes the proof.    □

Remark 2.

If A has full column rank and B has full row rank, Theorem 2 implies that $X^{\left(k\right)}$ converges linearly in expectation to $A^{+}C{B}^{+}$. If A does not have full column rank or B does not have full row rank, Algorithm 2 fails to converge (see Section 3.3 of Ma et al. [22]).

Remark 3.

If $Y^{\left(0\right)}=X^{\left(0\right)}B$, then

$$\begin{array}{cc}\hfill \parallel A{Y}^{\left(0\right)}-A{Y}^{\ast }{\parallel }_F^2& =\parallel A{X}^{\left(0\right)}B-A{A}^{+}C{B}^{+}B-A{A}^{+}C(I-B^{+}B){\parallel }_F^2\hfill \\ \hfill & =\parallel A(X^{\left(0\right)}-A^{+}C{B}^{+}){B\parallel }_F^2+{\parallel A{A}^{+}C-A{A}^{+}C{B}^{+}B\parallel }_F^2\hfill \\ \hfill & -2{\langle A(X^{\left(0\right)}-A^{+}C{B}^{+})B,A{A}^{+}C-A{A}^{+}C{B}^{+}B\rangle }_F.\hfill \end{array}$$

It follows from

$$\begin{array}{cc}\hfill & {\langle A(X^{\left(0\right)}-A^{+}C{B}^{+})B,A{A}^{+}C-A{A}^{+}C{B}^{+}B\rangle }_F\hfill \\ \hfill & =\mathrm{trace}\left(B^T{(X^{\left(0\right)}-A^{+}C{B}^{+})}^T{A}^T(A{A}^{+}C-A{A}^{+}C{B}^{+}B)\right)\hfill \\ \hfill & =\mathrm{trace}\left(A^T(A{A}^{+}C-A{A}^{+}C{B}^{+}B)B^T{(X^{\left(0\right)}-A^{+}C{B}^{+})}^T\right)\hfill \\ \hfill & =\mathrm{trace}(A^{T}A{A}^{+}C{B}^T-A^{T}A{A}^{+}C{B}^{+}B{B}^T){(X^{\left(0\right)}-A^{+}C{B}^{+})}^T)\hfill \\ \hfill & =\mathrm{trace}(A^{T}C{B}^T-A^{T}C{B}^T){(X^{\left(0\right)}-A^{+}C{B}^{+})}^T)=0\hfill \end{array}$$

that

$$\begin{array}{c}\hfill \parallel A{Y}^{\left(0\right)}-A{Y}^{\ast }{\parallel }_F^2=\parallel A(X^{\left(0\right)}-A^{+}C{B}^{+}){B\parallel }_F^2+{\parallel A{A}^{+}C-A{A}^{+}C{B}^{+}B\parallel }_F^2.\end{array}$$

Substituting this equality into (20), we have

$$\begin{array}{cc}\hfill \mathbb{E}\left[\parallel A(X^{(k+1)}-X^{\ast }){B\parallel }_F^2\right]& \le \left[(1+\eta )\parallel A{X}^{\left(0\right)}B-A{A}^{+}C{B}^{+}{B\parallel }_F^2\right.\hfill \\ \hfill & \left.+\eta \parallel A{A}^{+}C-A{A}^{+}C{B}^{+}{B\parallel }_F^2\right]{\rho }^{k+1}.\hfill \end{array}$$

(21)

Remark 4.

If the matrix equation in (1) is consistent, then $A{A}^{+}C=A{A}^{+}C{B}^{+}B=C$; therefore, (21) becomes

$$\mathbb{E}[\parallel A{X}^{\left(k\right)}{B-C\parallel }_F^2]\le (1+\eta ){\rho }^k{\parallel A{X}^{\left(0\right)}B-C\parallel }_F^2,$$

that is, $A{X}^{\left(k\right)}B$ converges to C in expectation (although $X^{\left(k\right)}$ does not necessarily converge).

Remark 5.

In Algorithm 2 $X^{\left(k\right)}$ and $Y^{\left(k\right)}$ can be iteratively solved alternately at each step, or the approximate value of $Y^{\ast }=A^{+}C$ can be iteratively obtained first and then the approximate value of $X^{\ast }=A^{+}C{B}^{+}$ can be iteratively solved. By using Lemmas 6 and 7, we can obtain similar convergence results. We omit the proof for the sake of conciseness.

4. Extended Kaczmarz Method and Extended Gauss–Seidel Method for $\mathbf{AXB}=\mathbf{C}$

When the matrix equation in (1) is inconsistent and either matrix A or matrix B is not full of rank, we can consider the REK method or REGS method to solve the matrix equation in (1) using the ideas of [23,24,25].

4.1. $AXB=C$ Inconsistent, A Not Full Rank, and B Full Column Rank ($q\ge n$)

The matrix equation $AY=C$ is solved by the REK method [23,24], while the matrix equation $XB=Y$ is solved by the RK method [20], because $XB=Y$ always has a solution ($B^T$ is full row rank). For this case, we use the REK-RK method to solve $AXB=C$, which is called the IME-REKRK algorithm (see Algorithm 3). The cost of each iteration of this method is $4mn+4np+4pq+m+2p$ flops ($4mn-n+m$ for step 6, $4np+p+n$ for step 7, and $4pq+p$ for step 8) if the squared row and column norms of A and the squared column norms of B are precomputed in advance.

Algorithm 3 REK-RK method for inconsistent matrix equation $AXB=C$ (IME-REKRK)

Input:  $A\in {\mathbb{R}}^{m\times p}$, $B\in {\mathbb{R}}^{q\times n}$, $C\in {\mathbb{R}}^{m\times n}$, $X^{\left(0\right)}=0\in {\mathbb{R}}^{p\times q}$, $Y^{\left(0\right)}=0\in {\mathbb{R}}^{p\times n}$, $Z^{\left(0\right)}=C$, K 1: For $i=1:m$, $M\left(i\right)=\parallel A_{i,:}{\parallel }_2^2$ 2: For $j=1:p$, $N\left(j\right)=\parallel A_{:,j}{\parallel }_2^2$ 3: For $l=1:n$, $T\left(l\right)=\parallel B_{:,l}{\parallel }_2^2$ 4: for  $k=0,1,\cdots ,K-1$  do 5:    Pick i with probability $p_i\left(A\right)={\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$, j with probability ${\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ and l with probability ${\widehat{p}}_l\left(B\right)={\displaystyle \frac{\parallel B_{:,l}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$ 6:    Compute $Z^{(k+1)}=Z^{\left(k\right)}-{\displaystyle \frac{A_{:,j}}{N\left(j\right)}}{A}_{:,j}^T{Z}^{\left(k\right)}$ 7:    Compute $Y^{(k+1)}=Y^{\left(k\right)}+{\displaystyle \frac{A_{i,:}^T}{M\left(i\right)}}(C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{Y}^{\left(k\right)})$ 8:    Compute $X^{(k+1)}=X^{\left(k\right)}+{\displaystyle \frac{Y_{:,l}^{(k+1)}-X^{\left(k\right)}{B}_{:,l}}{T\left(l\right)}}{B}_{:,l}^T$ 9: end for 10: Output $X^{\left(K\right)}$

Lemma 8.

Let $A\in {\mathbb{R}}^{m\times p}$ and $Y\in {\mathbb{R}}^{p\times n}$; then, it holds that

$$\begin{array}{c}\hfill \sum _{i=1}^m{\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}{∥\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)Y∥}_F^2={\parallel Y\parallel }_F^2-{\displaystyle \frac{{\parallel AY\parallel }_F^2}{{\parallel A\parallel }_F^2}}.\end{array}$$

Proof. 

Because ${\parallel A\parallel }_F^2={\sum }_{i=1}^m\parallel A_{i,:}{\parallel }_2^2={\sum }_{j=1}^p{\parallel A_{:,j}\parallel }_2^2$ and ${\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)}^2=I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}$, we have   

$$\begin{array}{cc}\hfill \sum _{i=1}^m{\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}{∥\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)Y∥}_F^2& =\sum _{i=1}^m{\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}\sum _{j=1}^n{∥\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)Y_{:,j}∥}_2^2\hfill \\ \hfill & =\sum _{i=1}^m{\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}\sum _{j=1}^n{Y}_{:,j}^T{\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)}^2{Y}_{:,j}\hfill \\ \hfill & =\sum _{j=1}^n{Y}_{:,j}^T\left[\sum _{i=1}^m{\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)\right]Y_{:,j}\hfill \\ \hfill & =\sum _{j=1}^n{Y}_{:,j}^T\left(I-{\displaystyle \frac{A^{T}A}{{\parallel A\parallel }_F^2}}\right)Y_{:,j}\hfill \\ \hfill & ={\parallel Y\parallel }_F^2-{\displaystyle \frac{{\parallel AY\parallel }_F^2}{{\parallel A\parallel }_F^2}}.\hfill \end{array}$$

This completes the proof.    □

Similar to the proof of Lemma 3, we can prove the following Lemma 9.

Lemma 9.

Let $Z^{\ast }=(I-A{A}^{+})C$, and let $\left\{Z^{\left(k\right)}\right\}$ denote the kth iterate of RK applied to $A^{T}Z=0$ with the initial guess $Z^{\left(0\right)}\in {\mathbb{R}}^{m\times n}$. If $Z_{:,j}^{\left(0\right)}\in C_{:,j}+R\left(A\right),j=1,\dots ,n$, then $Z^{\left(k\right)}$ converges linearly to $(I-A{A}^{+})C$ in mean square form. Moreover, the solution error in expectation for the iteration sequence $Z^{\left(k\right)}$ obeys

$$\mathbb{E}[\parallel Z^{\left(k\right)}-(I-A{A}^{+}){C\parallel }_F^2]\le {\rho }_1^k{\parallel Z^{\left(0\right)}-(I-A{A}^{+})C\parallel }_F^2,$$

(22)

where the jth column of A is selected with probability ${\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$.

Lemma 10.

The sequence $\left\{Y^{\left(k\right)}\right\}$ is generated by the REK method for $AY=C$ starting from the initial matrix $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$, in which $Y_{:,j}^{\left(0\right)}\in R\left(A^T\right),j=1,\dots ,n$ and the initial guess is $Z^{\left(0\right)}\in {\mathbb{R}}^{m\times n}$ with $Z_{:,j}^{\left(0\right)}\in C_{:,j}+R\left(A\right),j=1,\dots ,n$. In exact arithmetic, it holds that

$$\mathbb{E}[\parallel Y^{\left(k\right)}-A^{+}C{\parallel }_F^2]\le {\displaystyle \frac{k{\rho }_1^k}{{\parallel A\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2+{\rho }_1^k{∥Y^{\left(0\right)}-A^{+}C∥}_F^2,$$

(23)

where the ith row of A is selected with probability $p_i\left(A\right)={\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ and the jth column of A is selected with probability ${\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$.

Proof. 

Let $Y^{\left(k\right)}$ denote the kth iteration of the REK method for $AY=C$ and let ${\tilde{Y}}^{(k+1)}$ be the one-step Kaczmarz update for the matrix equation $AY=A{A}^{+}C$ from $Y^{\left(k\right)}$, i.e.,

$${\tilde{Y}}^{(k+1)}=Y^{\left(k\right)}+{\displaystyle \frac{A_{i,:}^T}{\parallel A_{i,:}{\parallel }_2^2}}(A_{i,:}{A}^{+}C-A_{i,:}{Y}^{\left(k\right)}).$$

We have

$$\begin{array}{cc}\hfill {\tilde{Y}}^{(k+1)}-A^{+}C& =Y^{\left(k\right)}-A^{+}C+{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}(A^{+}C-Y^{\left(k\right)})=\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)(Y^{\left(k\right)}-A^{+}C)\hfill \end{array}$$

and

$$Y^{(k+1)}-{\tilde{Y}}^{(k+1)}={\displaystyle \frac{A_{i,:}^T}{\parallel A_{i,:}{\parallel }_2^2}}(C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C).$$

It follows from

$$\begin{array}{cc}\hfill & {\langle {\tilde{Y}}^{(k+1)}-A^{+}C,Y^{(k+1)}-{\tilde{Y}}^{(k+1)}\rangle }_F\hfill \\ \hfill & ={〈\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)(Y^{\left(k\right)}-A^{+}C),{\displaystyle \frac{A_{i,:}^T}{\parallel A_{i,:}{\parallel }_2^2}}(C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C)〉}_F\hfill \\ \hfill & =\mathrm{trace}\left({(Y^{\left(k\right)}-A^{+}C)}^T\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right){\displaystyle \frac{A_{i,:}^T}{\parallel A_{i,:}{\parallel }_2^2}}(C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C)\right)\hfill \\ \hfill & =0(\mathrm{by}\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right){\displaystyle \frac{A_{i,:}^T}{\parallel A_{i,:}{\parallel }_2^2}}=0)\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill & {∥Y^{(k+1)}-{\tilde{Y}}^{(k+1)}∥}_F^2={∥{\displaystyle \frac{A_{i,:}^T}{\parallel A_{i,:}{\parallel }_2^2}}(C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C)∥}_F^2\hfill \\ \hfill & =\mathrm{trace}\left({(C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C)}^T{\displaystyle \frac{A_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}{\displaystyle \frac{A_{i,:}^T}{\parallel A_{i,:}{\parallel }_2^2}}(C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C)\right)\hfill \\ \hfill & ={\displaystyle \frac{{∥C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C∥}_2^2}{\parallel A_{i,:}{\parallel }_2^2}}\hfill \end{array}$$

that

$$\begin{array}{cc}\hfill {∥Y^{(k+1)}-A^{+}C∥}_F^2& ={∥Y^{(k+1)}-{\tilde{Y}}^{(k+1)}∥}_F^2+{\parallel {\tilde{Y}}^{(k+1)}-A^{+}C\parallel }_F^2\hfill \\ \hfill & ={\displaystyle \frac{{∥C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C∥}_2^2}{\parallel A_{i,:}{\parallel }_2^2}}+{\parallel {\tilde{Y}}^{(k+1)}-A^{+}C\parallel }_F^2.\hfill \end{array}$$

By taking the conditional expectation on the both sides of this equality, we have

$$\begin{array}{cc}\hfill {\mathbb{E}}_k\left[{∥Y^{(k+1)}-A^{+}C∥}_F^2\right]& ={\mathbb{E}}_k\left[{\displaystyle \frac{{∥C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C∥}_2^2}{\parallel A_{i,:}{\parallel }_2^2}}\right]+{\mathbb{E}}_k\left[\parallel {\tilde{Y}}^{(k+1)}-A^{+}{C\parallel }_F^2\right].\hfill \end{array}$$

(24)

Next, we provide the estimates for the two parts of the right-hand side of (24). It follows from

$$\begin{array}{cc}\hfill {\mathbb{E}}_k\left[{\displaystyle \frac{{∥C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C∥}_2^2}{\parallel A_{i,:}{\parallel }_2^2}}\right]& ={\mathbb{E}}_k^j{\mathbb{E}}_k^i\left[{\displaystyle \frac{{∥C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C∥}_2^2}{\parallel A_{i,:}{\parallel }_2^2}}\right]\hfill \\ \hfill & ={\mathbb{E}}_k^j\left[{\displaystyle \frac{1}{{\parallel A\parallel }_F^2}}\sum _{i=1}^m{∥C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C∥}_2^2\right]\hfill \\ \hfill & ={\displaystyle \frac{1}{{\parallel A\parallel }_F^2}}{\mathbb{E}}_k^j\left[{∥C-Z^{(k+1)}-A{A}^{+}C∥}_F^2\right]\hfill \\ \hfill & ={\displaystyle \frac{1}{{\parallel A\parallel }_F^2}}{\mathbb{E}}_k\left[{∥Z^{(k+1)}-(I-A{A}^{+})C∥}_F^2\right]\hfill \end{array}$$

that

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\displaystyle \frac{{∥C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C∥}_2^2}{\parallel A_{i,:}{\parallel }_2^2}}\right]& ={\displaystyle \frac{1}{{\parallel A\parallel }_F^2}}\mathbb{E}\left[{∥Z^{(k+1)}-(I-A{A}^{+})C∥}_F^2\right]\hfill \\ \hfill & \le {\displaystyle \frac{{\rho }_1^{k+1}}{{\parallel A\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2\left(\mathrm{by}\mathrm{Lemma} 9\right).\hfill \end{array}$$

(25)

By $Y_{:,j}^{\left(0\right)}\in R\left(A^T\right)$ and ${\left(A^{+}C\right)}_{:,j}\in R\left(A^T\right)$, $j=1,\dots ,n$, we have ${(Y^{\left(0\right)}-A^{+}C)}_{:,j}\in R\left(A^T\right),j=1,\dots ,n$. Then, by $Z_{:,j}^{\left(0\right)}\in C_{:,j}+R\left(A\right)$, it is easy to show that $Z_{:,j}^{\left(k\right)}\in C_{:,j}+R\left(A\right)$ and ${(Y^{\left(k\right)}-A^{+}C)}_{:,j}\in R\left(A^T\right),j=1,\dots ,n$ by induction. It follows from

$$\begin{array}{cc}\hfill {\mathbb{E}}_k[\parallel {\tilde{Y}}^{(k+1)}-A^{+}C{\parallel }_F^2]& ={\mathbb{E}}_k^i\left[{∥\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)(Y^{\left(k\right)}-A^{+}C)∥}_F^2\right]\hfill \\ \hfill & =\sum _{i=1}^m{\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}{∥\left(I-{\displaystyle \frac{A_{i,:}^T{A}_{i,:}}{\parallel A_{i,:}{\parallel }_2^2}}\right)(Y^{\left(k\right)}-A^{+}C)∥}_F^2\hfill \\ \hfill & ={∥Y^{\left(k\right)}-A^{+}C∥}_F^2-{\displaystyle \frac{\parallel A(Y^{\left(k\right)}-A^{+}C){\parallel }_F^2}{{\parallel A\parallel }_F^2}}\left(\mathrm{by}\mathrm{Lemma} 8\right)\hfill \\ \hfill & \le {∥Y^{\left(k\right)}-A^{+}C∥}_F^2-{\displaystyle \frac{{\sigma }_{min}^2\left(A\right)}{{\parallel A\parallel }_F^2}}{∥Y^{\left(k\right)}-A^{+}C∥}_F^2\left(\mathrm{by}\mathrm{Lemma} 2\right)\hfill \\ \hfill & ={\rho }_1{∥Y^{\left(k\right)}-A^{+}C∥}_F^2\hfill \end{array}$$

that

$$\begin{array}{c}\hfill \mathbb{E}\left[{∥{\tilde{Y}}^{(k+1)}-A^{+}C∥}_F^2\right]\le {\rho }_1\mathbb{E}\left[{∥Y^{\left(k\right)}-A^{+}C∥}_F^2\right].\end{array}$$

(26)

Combining (24), (25), and (26) yields

$$\begin{array}{cc}\hfill \mathbb{E}\left[\parallel Y^{(k+1)}-A^{+}{C\parallel }_F^2\right]& =\mathbb{E}\left[{\displaystyle \frac{{∥C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{A}^{+}C∥}_2^2}{\parallel A_{i,:}{\parallel }_2^2}}\right]+\mathbb{E}[\parallel {\tilde{Y}}^{(k+1)}-A^{+}C{\parallel }_F^2]\hfill \\ \hfill & \le {\displaystyle \frac{{\rho }_1^{k+1}}{{\parallel A\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2+{\rho }_1\mathbb{E}\left[{∥Y^{\left(k\right)}-A^{+}C∥}_F^2\right]\hfill \\ \hfill & \le {\displaystyle \frac{2{\rho }_1^{k+1}}{{\parallel A\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2+{\rho }_1^2\mathbb{E}\left[{∥Y^{(k-1)}-A^{+}C∥}_F^2\right]\hfill \\ \hfill & \le \cdots \le {\displaystyle \frac{(k+1){\rho }_1^{k+1}}{{\parallel A\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2+{\rho }_1^{k+1}{∥Y^{\left(0\right)}-A^{+}C∥}_F^2.\hfill \end{array}$$

This completes the proof.    □

With these preparations, the convergence proof of Algorithm 3 is provided below.

Theorem 3.

Let $\left\{X^{\left(k\right)}\right\}$ denote the sequence generated by Algorithm 3 (B is full column rank) with the initial guess $X^{\left(0\right)}\in {\mathbb{R}}^{p\times q}$ in which ${\left(X_{i,:}^{\left(0\right)}\right)}^T\in R\left(B\right),i=1,\dots ,p$. The sequence $\left\{Y^{\left(k\right)}\right\}$ is generated by the REK method for $AY=C$ starting from the initial matrix $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$, in which $Y_{:,j}^{\left(0\right)}\in R\left(A^T\right)$ and $Z^{\left(0\right)}\in {\mathbb{R}}^{m\times n}$ with $Z_{:,j}^{\left(0\right)}\in C_{:,j}+R\left(A\right),j=1,\dots ,n$. In exact arithmetic, it holds that

$$\begin{array}{cc}\hfill \mathbb{E}\left[\parallel X^{\left(k\right)}-A^{+}C{B}^{+}{\parallel }_F^2\right]\le & \left({∥X^{\left(0\right)}-A^{+}C{B}^{+}∥}_F^2+{\displaystyle \frac{\eta }{{\parallel B\parallel }_F^2}}{∥Y^{\left(0\right)}-A^{+}C∥}_F^2\right.\hfill \\ \hfill & \left.+{\displaystyle \frac{\gamma }{{\parallel A\parallel }_F^2{\parallel B\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2\right){\rho }^k\hfill \end{array}$$

where

$$\gamma =\left\{\begin{array}{cc}{\displaystyle \frac{{\rho }_1{\rho }_2}{{({\rho }_2-{\rho }_1)}^2}},\hfill & if{\rho }_1<{\rho }_2\hfill \\ {\displaystyle \frac{k{\rho }_1}{{\rho }_1-{\rho }_2}},\hfill & if{\rho }_1>{\rho }_2\hfill \\ {\displaystyle \frac{k(k+1)}{2}},\hfill & if{\rho }_1={\rho }_2.\hfill \end{array}\right.$$

Proof. 

Similar to the proof of Theorem 1 using Lemma 10 and Lemma 4, we can obtain

$$\begin{array}{cc}\hfill & \mathbb{E}\left[{∥X^{(k+1)}-A^{+}C{B}^{+}∥}_F^2\right]\le {\rho }_2\mathbb{E}\left[{∥X^{\left(k\right)}-A^{+}C{B}^{+}∥}_F^2\right]+{\displaystyle \frac{1}{{\parallel B\parallel }_F^2}}\mathbb{E}\left[\parallel Y^{(k+1)}-A^{+}{C\parallel }_F^2\right]\hfill \\ \hfill & \le {\rho }_2\mathbb{E}\left[{∥X^{\left(k\right)}-A^{+}C{B}^{+}∥}_F^2\right]+{\displaystyle \frac{(k+1){\rho }_1^{k+1}}{{\parallel A\parallel }_F^2{\parallel B\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2+{\displaystyle \frac{{\rho }_1^{k+1}}{{\parallel B\parallel }_F^2}}{∥Y^{\left(0\right)}-A^{+}C∥}_F^2\hfill \\ \hfill & \le {\rho }_2^2\mathbb{E}\left[\parallel X^{(k-1)}-A^{+}C{B}^{+}{\parallel }_F^2\right]+{\displaystyle \frac{{\rho }_1^{k+1}+{\rho }_1^k{\rho }_2}{{\parallel B\parallel }_F^2}}{∥Y^{\left(0\right)}-A^{+}C∥}_F^2\hfill \\ \hfill & +{\displaystyle \frac{(k+1){\rho }_1^{k+1}+k{\rho }_1^k{\rho }_2}{{\parallel A\parallel }_F^2{\parallel B\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2\le \cdots \hfill \\ \hfill & \le {\rho }_2^{k+1}{∥X^{\left(0\right)}-A^{+}C{B}^{+}∥}_F^2+{\displaystyle \frac{{\displaystyle \sum _{j=0}^k}{\rho }_1^{j+1}{\rho }_2^{k-j}}{{\parallel B\parallel }_F^2}}{∥Y^{\left(0\right)}-A^{+}C∥}_F^2\hfill \\ \hfill & +{\displaystyle \frac{{\displaystyle \sum _{j=0}^k}(j+1){\rho }_1^{j+1}{\rho }_2^{k-j}}{{\parallel A\parallel }_F^2{\parallel B\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2.\hfill \end{array}$$

(27)

If ${\rho }_1<{\rho }_2$, then from the analytical properties of geometric series we can obtain

$$\sum _{j=0}^k(j+1){\rho }_1^{j+1}{\rho }_2^{k-j}={\rho }_2^{k+1}\sum _{j=0}^k(j+1){\left({\displaystyle \frac{{\rho }_1}{{\rho }_2}}\right)}^{j+1}\le {\displaystyle \frac{{\rho }_1{\rho }_2}{{({\rho }_2-{\rho }_1)}^2}}{\rho }_2^{k+1}.$$

If ${\rho }_1>{\rho }_2$, then

$$\sum _{j=0}^k(j+1){\rho }_1^{j+1}{\rho }_2^{k-j}={\rho }_1^{k+1}\sum _{i=0}^k(k+1-i){\left({\displaystyle \frac{{\rho }_2}{{\rho }_1}}\right)}^i\le {\displaystyle \frac{(k+1){\rho }_1}{{\rho }_1-{\rho }_2}}{{\rho }_1}^{k+1}.$$

If ${\rho }_1={\rho }_2$, then

$$\sum _{j=0}^k(j+1){\rho }_1^{j+1}{\rho }_2^{k-j}={\displaystyle \frac{(k+1)(k+2)}{2}}{\rho }^{k+1}.$$

Substituting the above inequalities into (27) completes the proof.    □

4.2. $AXB=C$ Inconsistent, A Not Full Rank, and B Full Row Rank ($q\le n$)

The matrix equation $AY=C$ is solved by the REK method, while the matrix equation $XB=Y$ is solved by the RGS method, because $XB=Y$ has a unique least-squares solution ($B^T$ is full column rank). For this case, we refer the algorithm for solving $AXB=C$ as IME-REKRGS (see Algorithm 4). The cost of each iteration of this method is $4mn+8np+m+n+2p$ flops ($4mn-n+m$ for step 6, $4np+p+n$ for step 7, and $4np+n+p$ for step 8) if the squared row and column norms of A and the squared row norms of B are precomputed in advance.

Algorithm 4 REK-RGS method for inconsistent matrix equation $AXB=C$ (IME-REKRGS)

Input:  $A\in {\mathbb{R}}^{m\times p}$, $B\in {\mathbb{R}}^{q\times n}$, $C\in {\mathbb{R}}^{m\times n}$, $X^{\left(0\right)}=0\in {\mathbb{R}}^{p\times q}$, $Y^{\left(0\right)}=0\in {\mathbb{R}}^{p\times n}$, $Z^{\left(0\right)}=C$, $E^{\left(0\right)}=0\in {\mathbb{R}}^{p\times n}$, K 1: For $i=1:m$, $M\left(i\right)=\parallel A_{i,:}{\parallel }_2^2$ 2: For $j=1:p$, $N\left(j\right)=\parallel A_{:,j}{\parallel }_2^2$ 3: For $l=1:q$, $T\left(l\right)=\parallel B_{l,:}{\parallel }_2^2$ 4: for$k=0,1,\cdots ,K-1$  do 5:    Pick i with probability $p_i\left(A\right)={\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$, j with probability ${\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ and l with probability $p_l\left(B\right)={\displaystyle \frac{\parallel B_{l,:}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$ 6:    Compute $Z^{(k+1)}=Z^{\left(k\right)}-{\displaystyle \frac{A_{:,j}}{N\left(j\right)}}{A}_{:,j}^T{Z}^{\left(k\right)}$ 7:    Compute $Y^{(k+1)}=Y^{\left(k\right)}+{\displaystyle \frac{A_{i,:}^T}{M\left(i\right)}}(C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{Y}^{\left(k\right)})$ 8:    Compute $U^{\left(k\right)}={\displaystyle \frac{E^{\left(k\right)}{B}_{l,:}^T}{T\left(l\right)}}$, $X_{:,l}^{(k+1)}=X_{:,l}^{\left(k\right)}+U^{\left(k\right)}$, $E^{(k+1)}=E^{\left(k\right)}-U^{\left(k\right)}{B}_{l,:}$ 9: end for 10: Output $X^{\left(K\right)}$

Similarly, letting $Y=AX$, we can transform the equation $AXB=C$ into the system of equations composed of

$$\left\{\begin{array}{c}{B}^T{Y}^T=C^T,\\ AX=Y.\end{array}\right.$$

(28)

The matrix equation $B^T{Y}^T=C^T$ is solved by the RGS method because it has a unique least squares solution ($B^T$ is full column rank), while the matrix equation $AX=Y$ is solved by the REK method. For this case, we refer the algorithm for solving $AXB=C$ as IME-RGSREK.

The above two methods can be seen as the combination of two separate algorithms; thus, we do not discuss the algorithms in detail. We only provide the convergence results of the IME-REKRGS method, and omit the proof.

Theorem 4.

Let $\left\{X^{\left(k\right)}\right\}$ denote the sequence generated by IME-REKRGS method (B is full row rank) with initial guess $X^{\left(0\right)}\in {\mathbb{R}}^{p\times q}$. The sequence $\left\{Y^{\left(k\right)}\right\}$ is generated by the REK method for $AY=C$ starting from the initial matrix $Y^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$, in which $Y_{:,j}^{\left(0\right)}\in R\left(A^T\right)$ and the initial guess is $Z^{\left(0\right)}\in {\mathbb{R}}^{m\times n}$ with $Z_{:,j}^{\left(0\right)}\in C_{:,j}+R\left(A\right),j=1,\dots ,n$. In exact arithmetic, it holds that

$$\begin{array}{cc}\hfill \mathbb{E}[\parallel (X^{\left(k\right)}-A^{+}C{B}^{+})B{\parallel }_F^2]\le & \left({∥(X^{\left(0\right)}-A^{+}C{B}^{+})B∥}_F^2+\eta {∥Y^{\left(0\right)}-A^{+}C∥}_F^2\right.\hfill \\ \hfill & \left.+{\displaystyle \frac{\gamma }{{\parallel A\parallel }_F^2}}{∥Z^{\left(0\right)}-(I-A{A}^{+})C∥}_F^2\right){\rho }^k.\hfill \end{array}$$

4.3. Double Extended Kaczmarz Method for Solving General Matrix Equation $AXB=C$

In general, the matrix equation in (1) may be inconsistent, and A and B may not be full rank; thus, we consider solving both matrix equations $AY=C$ and $XB=Y$ by the REK method. The algorithm is described below (see Algorithm 5).

Algorithm 5 Double REK method for general $AXB=C$ (DREK)

Input:  $A\in {\mathbb{R}}^{m\times p}$, $B\in {\mathbb{R}}^{q\times n}$, $C\in {\mathbb{R}}^{m\times n}$, $X^{\left(0\right)}=0\in {\mathbb{R}}^{p\times q}$, $Y^{\left(0\right)}=0\in {\mathbb{R}}^{p\times n}$, $Z^{\left(0\right)}=C$, $K_1,K_2\in R$ 1: For $i=1:m$, $Mi\left(i\right)=\parallel A_{i,:}{\parallel }_2^2$; For $j=1:p$, $Mj\left(j\right)=\parallel A_{:,j}{\parallel }_2^2$ 2: For $i=1:q$, $Ni\left(i\right)=\parallel B_{i,:}{\parallel }_2^2$; For $j=1:n$, $Nj\left(j\right)=\parallel B_{:,j}{\parallel }_2^2$ 3: for  $k=0,1,2,\cdots ,K_1-1$  do 4:    Set $p_i\left(A\right)={\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ and ${\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ 5:    Compute $Z^{(k+1)}=Z^{\left(k\right)}-{\displaystyle \frac{A_{:,j}}{N\left(j\right)}}\left(A_{:,j}^T{Z}^{\left(k\right)}\right)$ 6:    Compute $Y^{(k+1)}=Y^{\left(k\right)}+{\displaystyle \frac{A_{i,:}^T}{M\left(i\right)}}(C_{i,:}-Z_{i,:}^{(k+1)}-A_{i,:}{Y}^{\left(k\right)})$ 7: end for 8: Set $W^{\left(0\right)}={\left(Y^{\left(K_1\right)}\right)}^T$ 9: for  $k=0,1,2,\cdots ,K_2-1$  do 10:    Set $p_s\left(B\right)={\displaystyle \frac{\parallel B_{s,:}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$ and ${\widehat{p}}_t\left(B\right)={\displaystyle \frac{\parallel B_{:,t}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$ 11:    Compute $W^{(k+1)}=W^{\left(k\right)}-{\displaystyle \frac{B_{s,:}^T}{Ni\left(s\right)}}{B}_{s,:}{W}^{\left(k\right)}$ 12:    Compute $X^{(k+1)}=X^{\left(k\right)}+(Y_{:,t}^{\left(K_1\right)}-{\left(W_{t,:}^{(k+1)}\right)}^T-X^{\left(k\right)}{B}_{:,t}){\displaystyle \frac{B_{:,t}^T}{Nj\left(t\right)}}$ 13: end for 14: Output $X^{\left(K_2\right)}$

The convergence result is the superposition of the corresponding convergence results of the two REK methods, and the proof is omitted.

Similar to the DREK method, we can employ the double REGS (DREGS) method to solve the general matrix equation $AXB=C$ (see Algorithm 6). Because the convergence results and proof methods are very similar to the previous ones, we omit them here.

Algorithm 6 Double REGS method for general $AXB=C$ (DREGS)

Input:  $A\in {\mathbb{R}}^{m\times p}$, $B\in {\mathbb{R}}^{q\times n}$, $C\in {\mathbb{R}}^{m\times n}$, $X^{\left(0\right)}=0\in {\mathbb{R}}^{p\times q}$, $Y^{\left(0\right)}=0\in {\mathbb{R}}^{p\times n}$, $F^{\left(0\right)}\in {\mathbb{R}}^{p\times n}$, $U^{\left(0\right)}\in {\mathbb{R}}^{p\times q}$, $R^{\left(0\right)}=C$, $K_1,K_2\in R$ 1: For $i=1:m$, $Mi\left(i\right)=\parallel A_{i,:}{\parallel }_2^2$; For $j=1:p$, $Mj\left(j\right)=\parallel A_{:,j}{\parallel }_2^2$ 2: For $i=1:q$, $Ni\left(i\right)=\parallel B_{i,:}{\parallel }_2^2$; For $j=1:n$, $Nj\left(j\right)=\parallel B_{:,j}{\parallel }_2^2$ 3: for  $k=0,1,2,\cdots ,K_1-1$  do 4:    Set $p_i\left(A\right)={\displaystyle \frac{\parallel A_{i,:}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ and ${\widehat{p}}_j\left(A\right)={\displaystyle \frac{\parallel A_{:,j}{\parallel }_2^2}{{\parallel A\parallel }_F^2}}$ 5:    Compute $W^{\left(k\right)}={\displaystyle \frac{A_{:,j}^T{\mathbb{R}}^{\left(k\right)}}{Mj\left(j\right)}}$, $F_{j,:}^{(k+1)}=F_{j,:}^{\left(k\right)}+W^{\left(k\right)}$, $R^{(k+1)}=R^{\left(k\right)}-A_{:,j}{W}^{\left(k\right)}$ 6:    Compute $Y^{(k+1)}=Y^{\left(k\right)}-A_{i,:}^T{\displaystyle \frac{A_{i,:}(Y^{\left(k\right)}-F^{(k+1)})}{Mi\left(i\right)}}$ 7: end for 8: Set $E^{\left(0\right)}=Y^{\left(K_1\right)}$ 9: for  $k=0,1,2,\cdots ,K_2-1$  do 10:    Set $p_s\left(B\right)={\displaystyle \frac{\parallel B_{s,:}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$ and ${\widehat{p}}_t\left(B\right)={\displaystyle \frac{\parallel B_{:,t}{\parallel }_2^2}{{\parallel B\parallel }_F^2}}$ 11:    Compute $V^{\left(k\right)}={\displaystyle \frac{E^{\left(k\right)}{B}_{s,:}^T}{Ni\left(s\right)}}$, $U_{:,i}^{(k+1)}=U_{:,i}^{\left(k\right)}+V^{\left(k\right)}$, $E^{(k+1)}=E^{\left(k\right)}-V^{\left(k\right)}{B}_{s,:}$ 12:    Compute $X^{(k+1)}=X^{\left(k\right)}-{\displaystyle \frac{(X^{\left(k\right)}-U^{(k+1)})B_{:,t}}{Nj\left(t\right)}}{B}_{:,t}^T$ 13: end for 14: Output $X^{\left(K_2\right)}$

5. Numerical Experiments

In this section, we present some experimental results of the proposed algorithms for solving various matrix equations and compare them with ME-RGRK and ME-MWRK from [15] for consistent matrix equations and RBCD from [14] for inconsistent matrix equations. All experiments were carried out in MATLAB (version R2020a) on a DESKTOP-8CBRR86 with an Intel(R) Core(TM) i7-4712MQ CPU@2.30GHz, 8GB RAM, and Windows 10.

All computations were started from the initial guess $X^{\left(0\right)}=0,Y^{\left(0\right)}=0$ and terminated when the relative error (RE) of the solution, defined by

$$RE={\displaystyle \frac{\parallel X^{\left(k\right)}-X^{\ast }{\parallel }_F^2}{\parallel X^{\ast }{\parallel }_F^2}}$$

at the the current iteration $X^{\left(k\right)}$, satisfied $RE<{10}^{-6}$ or exceeded the maximum iteration K = 50,000, where $X^{\ast }=A^{+}C{B}^{+}$. We report the average number of iterations (denoted as “IT”) and average computing time in seconds (denoted as“CPU”) for 20 repeated trial runs of the corresponding method. We considered the following methods:

• CME-RK (Algorithm 1) compared with ME-RGRK and ME-MWRK in [15] for consistent matrix equations. We used $\theta =0.5$ in the ME-RGRK method, which is the same as in [15].
• IME-RGS (Algorithm 2), IME-REKRGS (Algorithm 4) compared with RBCD in [14] for inconsistent matrix equations. We used $\alpha ={\displaystyle \frac{1.5}{{\parallel A\parallel }_2^2}}$ in the RBCD method, which is the same as in [14].
• IME-REKRK (Algorithm 3), DREK (Algorithm 5), and DREGS (Algorithm 6) for inconsistent matrix equations. The last two methods have no requirements regarding whether matrix A and matrix B have full row rank or full column rank.

We tested the performance of various methods with synthetic dense data and real-world sparse data. Synthetic data were generated as follows:

• Type I: For given $m,p,q,n$, the entries of A and B were generated from a standard normal distribution, i.e., $A=randn(m,p),B=randn(q,n).$ We also constructed the rank-deficient matrix by $A=randn(m,p/2),A=[A,A]$ or $B=randn(q/2,n),B=[B;B]$, etc.
• Type II: As in [16], for given $m,p$, and $r_1=rank\left(A\right)$, we constructed a matrix A by $A=U_1{D}_1{V}_1^T$, where $U_1\in {\mathbb{R}}^{m\times r_1}$ and $V_1\in {\mathbb{R}}^{p\times r_1}$ are orthogonal columns matrices, $D\in {\mathbb{R}}^{r_1\times r_1}$ is a diagonal matrix with the first $r_1-2$ diagonal entries uniformly distributed numbers in $[{\sigma }_{min\left(A\right)},{\sigma }_{max\left(A\right)}]$, and the last two diagonal entries are ${\sigma }_{max\left(A\right)},{\sigma }_{min\left(A\right)}$. Similarly, for given $q,n$, and $r_2=rank\left(B\right)$, we constructed a matrix B by $B=U_2{D}_2{V}_2^T$, where $U_2\in {\mathbb{R}}^{q\times r_2}$ and $V_2\in {\mathbb{R}}^{n\times r_2}$ are orthogonal columns matrices, $D\in {\mathbb{R}}^{r_2\times r_2}$ is a diagonal matrix with the first $r_2-2$ diagonal entries uniformly distributed numbers in $[{\sigma }_{min\left(B\right)},{\sigma }_{max\left(B\right)}]$, and the last two diagonal entries are ${\sigma }_{max\left(B\right)},{\sigma }_{min\left(B\right)}$.

The real-world sparse data were taken from the Florida sparse matrix collection [26].

Table 2. Detailed features of the sparse matrices from [26].

Name	Size	Rank	Sparsity
ash219	$219\times 85$	85	$97.65\%$
ash958	$958\times 292$	292	$99.32\%$
divorce	$50\times 9$	9	$50\%$
Worldcities	$315\times 100$	100	$76.13\%$

lists the features of these sparse matrices.

5.1. Consistent Matrix Equation

First, we compared the performance of the ME-RGRK, ME-MWRK, and CME-RK methods for the consistent matrix equation $AXB=C$. To construct a consistent matrix equation, we set $C=A{X}^{\ast }B$, where $X^{\ast }$ is a random matrix generated by $X^{\ast }=randn(p,q)$.

Example 1.

The ME-RGRK, ME-MWRK, and CME-RK methods with synthetic dense data.

Table 3. IT and CPU of ME-RGRK, ME-MWRK, and CME-RK for consistent matrix equations with Type I.

m	p	${\mathit{r}}_1$	q	n	${\mathit{r}}_2$		ME-RGRK	ME-MWRK	CME-RK
100	40	40	40	100	40	IT	49,707	27,579	1600.9
						CPU	0.71	2.01	0.06
100	40	20	40	100	20	IT	2979.6	1064	454.2
						CPU	0.04	0.09	0.02
40	100	40	100	40	40	IT	>	49,332.7	1807.2
						CPU	>	2.15	0.13
40	100	20	100	40	20	IT	14,788	3484	441.1
						CPU	0.19	0.20	0.03
500	100	100	100	500	100	IT	>	32,109	2250.4
						CPU	>	57.61	0.33
500	100	50	100	500	50	IT	9193.9	3158.6	935.3
						CPU	6.07	5.66	0.13
1000	200	100	100	1000	50	IT	15,206.4	5076.7	1655.5
						CPU	58.43	52.45	1.23
1000	200	200	100	1000	100	IT	>	39,848	3906.7
						CPU	>	402.15	2.51

and

Table 4. IT and CPU of ME-RGRK, ME-MWRK and CME-RK for consistent matrix equations with Type II.

m	p	${\mathit{r}}_1$	$\frac{{\mathit{\sigma }}_{max}\left(\mathit{A}\right)}{{\mathit{\sigma }}_{min}\left(\mathit{A}\right)}$	q	n	${\mathit{r}}_2$	$\frac{{\mathit{\sigma }}_{max}\left(\mathit{B}\right)}{{\mathit{\sigma }}_{min}\left(\mathit{B}\right)}$		ME-RGRK	B	CME-RK
100	40	40	2	40	100	40	2	IT	10,865.1	5617	842.3
								CPU	0.15	0.50	0.03
100	40	20	2	40	100	20	2	IT	2409	836	422
								CPU	0.03	0.07	0.02
100	40	20	5	40	100	20	5	IT	22,423.2	6439.8	1145.2
								CPU	0.33	0.57	0.04
500	100	100	2	100	500	100	2	IT	40,768	20,507.2	1992.3
								CPU	21.03	34.02	0.29
500	100	50	5	500	100	50	5	IT	>	35,159	2893.6
								CPU	>	59.02	0.39
500	100	50	10	500	100	50	10	IT	>	>	10,693.4
								CPU	>	>	1.68
1000	200	100	2	100	1000	50	2	IT	19,679.2	6974.9	1722.8
								CPU	73.45	70.56	1.20
1000	200	100	5	100	1000	50	5	IT	>	>	6037.4
								CPU	>	>	4.23

report the average IT and CPU of ME-RGRK, ME-MWRK, and CME-RK for solving a consistent matrix with Type I and Type II matrices. In the following tables, the item “>” indicates that the number of iteration steps exceeded the maximum iterations (50,000), while the item “-” indicates that the method did not converge. From Table 3, it can be seen that the CME-RK method vastly outperforms the ME-RGRK and ME-MWRK methods in terms of both IT and CPU times. The CME-RK method has the least iteration steps and runs for the least time regardless of whether or not matrices A and B are full column/row rank. We observe that when the linear system is consistent, the speed-up is at least 2.00 and the largest speed-up reaches 3.75. As the matrix dimension increases, the CPU time of the CME-RK method increases only slowly, while the running times of ME-RGRK and ME-MWRK increase dramatically. The numerical advantages of CME-RK for large consistent matrix equations are more obvious in Table 4. Moreover, when ${\displaystyle \frac{{\sigma }_{max}\left(A\right)}{{\sigma }_{min}\left(A\right)}}$ and ${\displaystyle \frac{{\sigma }_{max}\left(B\right)}{{\sigma }_{min}\left(B\right)}}$ are large (e.g., ${\displaystyle \frac{{\sigma }_{max}}{{\sigma }_{min}}}=5$ ), the convergence speeds of ME-RGRK and ME-MWRK are very slow, which is because the convergence rate of these two methods depends on $1-{\displaystyle \frac{{\sigma }_{min}^2\left(A\right){\sigma }_{min}^2\left(B\right)}{{\parallel A\parallel }_F^2{\parallel B\parallel }_F^2}}$.

Figure 1 shows the plots of the relative error (RE) in base-10 logarithm versus the IT and CPU of different methods with Type I ($A=randn(500,50),A=[A,A]$, $B=randn(150,600)$) and Type II ($m=500,p=100,r_1=50,{\displaystyle \frac{{\sigma }_{max}\left(A\right)}{{\sigma }_{min}\left(A\right)}}=2,q=150$, $n=600,r_2=50,{\displaystyle \frac{{\sigma }_{max}\left(B\right)}{{\sigma }_{min}\left(B\right)}}=2$). Again, it can be seen that the relative error of CME-RK decreases rapidly with the increase in iteration steps and the computing time.

Example 2.

The ME-RGRK, ME-MWRK, and CME-RK methods with real-world sparse data.

For the sparse matrices from [26], the numbers of iteration steps and the computing times for the ME-RGRK, ME-MWRK, and CME-RK methods are listed in

Table 5. IT and CPU of ME-RGRK, ME-MWRK, and CME-RK for the consistent matrix equations with sparse matrices from [26], where T represents transposition.

A	B		ME-RGRK	ME-MWRK	CME-RK
ash219	divorceT	IT	49,871.1	15,423.5	3522.4
		CPU	0.78	1.26	0.17
divorce	ash219T	IT	43,927.8	14,164.4	3521.3
		CPU	1.15	1.35	0.17
divorce	ash219	IT	40,198.7	17,251.4	3238.9
		CPU	0.63	0.80	0.14
ash958	ash219T	IT	>	>	6706.2
		CPU	>	>	1.23
ash219	ash958T	IT	>	>	5762.6
		CPU	>	>	1.18
ash958	WorldcitiesT	IT	-	>	38,088.5
		CPU	-	>	7.94

. It can be observed that the CME-RK method successfully computes an approximate solution of the consistent matrix equation for various A and B. For the fist three cases in Table 5, the ME-RGRK, ME-MWRK, and CME-RK methods all converge to the solution; however, the CME-RK method is significantly better than the ME-RGRK and ME-MWRK methods in terms of both iteration steps and running time. For the last three cases, the ME-RGRK and ME-MWRK methods fail to converge to the solution because the iteration steps exceed the maximum of 50,000.

5.2. Inconsistent Matrix Equation

Next, we compare the performance of the RBCD, IME-RGS, and IME-REKRGS methods for the inconsistent matrix equation $AXB=C$, where B is full row rank. To construct an inconsistent matrix equation, we set $C=A{X}^{\ast }B+R$, where $X^{\ast }$ and R are random matrices generated by $X^{\ast }=randn(p,q)$ and $R=\delta \ast randn(p,q),\delta \in (0,1)$. In addition, we show the experiment results of the REKRK, DREK, and DREGS methods, which do not require B to have full row rank.

Example 3.

The RBCD, IME-RGS, and IME-REKRGS methods with synthetic dense data.

Table 6. IT and CPU of RBCD, IME-RGS, and IME-REKRGS for inconsistent matrix equations with Type I.

m	p	${\mathit{r}}_1$	q	n	${\mathit{r}}_2$		RBCD	IME-RGS	IME-REKRGS
100	40	40	40	100	40	IT	11,116	1883.7	2449.2
						CPU	0.43	0.09	0.18
100	40	20	40	100	40	IT	12,416	-	1725.8
						CPU	0.49	-	0.13
500	100	100	50	200	50	IT	2820.1	2011.5	2603.7
						CPU	0.56	0.31	0.62
500	100	100	100	500	100	IT	5067	2314.7	2782.2
						CPU	4.06	1.76	2.55
1000	100	100	200	1000	200	IT	5969.7	3833.4	3867.9
						CPU	27.02	18.18	19.47
1000	200	200	200	1000	200	IT	9738.4	4485.2	5442
						CPU	50.43	24.27	39.64

and

Table 7. IT and CPU of RBCD, IME-RGS, and IME-REKRGS for inconsistent matrix equations with Type II.

m	p	${\mathit{r}}_1$	$\frac{{\mathit{\sigma }}_{max}\left(\mathit{A}\right)}{{\mathit{\sigma }}_{min}\left(\mathit{A}\right)}$	q	n	${\mathit{r}}_2$	$\frac{{\mathit{\sigma }}_{max}\left(\mathit{B}\right)}{{\mathit{\sigma }}_{min}\left(\mathit{B}\right)}$		RBCD	IME-RGS	IME-REKRGS
100	40	40	2	40	100	40	2	IT	1176.5	716.2	949.6
								CPU	0.04	0.03	0.09
100	40	40	5	40	100	40	5	IT	27,631	2974.3	3773.5
								CPU	1.10	0.16	0.27
500	100	100	2	100	500	100	2	IT	2953.4	2101.2	2307.8
								CPU	2.52	1.78	2.25
500	100	100	5	100	500	100	5	IT	>	6432.5	8101.3
								CPU	>	5.29	7.96
1000	100	100	2	200	1000	200	2	IT	5577.3	3242.6	3380.7
								CPU	25.68	15.33	17.13
1000	100	100	5	200	1000	200	5	IT	>	10,672.5	11,006.4
								CPU	>	49.49	56.54

report the average IT and CPU of the RBCD, IME-RGS, and IME-REKRGS methods for solving an inconsistent matrix with Type I and Type II matrices. Figure 2 shows the plots of the relative error (RE) in base-10 logarithm versus the IT and CPU of different methods with Type I ($A=randn(500,100),B=randn(150,600)$) and Type II ($m=500,p=100,r_1=100,{\displaystyle \frac{{\sigma }_{max}\left(A\right)}{{\sigma }_{min}\left(A\right)}}=2,q=150,n=600,r_2=150,{\displaystyle \frac{{\sigma }_{max}\left(B\right)}{{\sigma }_{min}\left(B\right)}}=2$). From these tables, it can be seen that the IME-RGS and IME-REKRGS methods are better than the RBCD method in terms of IT and CPU time, especially when the matrix dimension is large (see the last two cases in Table 6) or when ${\displaystyle \frac{{\sigma }_{max}}{{\sigma }_{min}}}$ is large (see the last three cases in Table 7). From Figure 2, it can be seen that the IME-RGS and IME-REKRGS methods converge faster than the RBCD method, although the relative error of RBCD decreases faster in the initial iteration.

Example 4.

The RBCD, IME-RGS, and IME-REKRGS methods with real-world sparse data.

Table 8. IT and CPU of RBCD, IME-RGS, and IME-REKRGS for inconsistent matrix equations with sparse matrices from [26].

A	B		RBCD	IME-RGS	IME-REKRGS
ash219	divorceT	IT	13,115.2	3543.7	3653.6
		CPU	1.15	0.20	0.28
divorce	ash219T	IT	>	3371.8	4150.9
		CPU	>	0.26	0.48
ash958	ash219T	IT	7200	7536.4	7808.6
		CPU	10.44	5.06	6.04
ash219	ash958T	IT	21,118.5	6749.2	5675.7
		CPU	14.93	5.37	6.36
ash958	WorldcitiesT	IT	>	39,183.6	38,538.1
		CPU	>	42.34	53.65

lists the average IT and CPU of the RBCD, IME-RGS, and IME-REKRGS methods for solving an inconsistent matrix with sparse matrices. It can be observed that the IME-RGS and IME-REKRGS methods require less CPU time than the RBCD method in all case and less IT in all cases except for $A=ash958,B=ash{219}^T$.

Example 5.

The IME-REKRK, DREK, and DREGS methods.

Finally, we tested the effectiveness of the IME-REKRK, DREK, and DREGS methods for inconsistent matrix equations, including both synthetic dense data and real-world sparse data. The features of A and B are provided in

Table 9. Detailed features of A and B for Example 5.

Type	A	$\mathit{r}\left(\mathit{A}\right)$	B	$\mathit{r}\left(\mathit{B}\right)$
Type a	$A=randn(500,100),A=[A,A;A,A]$	100	$B=randn(1000,100)$	100
Type b	$A=randn(500,100),A=[A,A;A,A]$	100	$B=randn(50,500),B=[B,B;B,B]$	50
Type c	$A=U_1{D}_1{V}_1^T,$	50	$B=U_2{D}_2{V}_2^T$	40
	$m=1000,p=100,{\displaystyle \frac{{\sigma }_{max}\left(A\right)}{{\sigma }_{min}\left(A\right)}}=2$		$q=200,n=1000,{\displaystyle \frac{{\sigma }_{max}\left(B\right)}{{\sigma }_{min}\left(B\right)}}=2$
Type d	$A=U_1{D}_1{V}_1^T,$	100	$B=U_2{D}_2{V}_2^T,$	100
	$m=1000,p=100,{\displaystyle \frac{{\sigma }_{max}\left(A\right)}{{\sigma }_{min}\left(A\right)}}=5$		$q=1000,n=100,{\displaystyle \frac{{\sigma }_{max}\left(B\right)}{{\sigma }_{min}\left(B\right)}}=5$
Type e	A = ash219	85	B = Worldcities	100
Type f	A = ash958	292	B = ash219	85

, while the experimental results are listed in

Table 10. IT and CPU of the IME-REKRK, DREK, and DREGS methods for the inconsistent matrix equations.

Method		Type a	Type b	Type c	Type d	Type e	Type f
IME-REKRK	IT	2692.3	2569.4	-	8087.4	41,373.1	8733.5
	CPU	3.81	14.52	-	2.76	8.36	2.60
DREK	IT	1785.2 + 1981.7	2110.5 + 1067.2	1171.6 + 942.1	8023.7.2 +6153.4	1831.3 + 596.6	7780.7 + 3236.6
	CPU	0.60 + 1.98	10.87 + 1.51	5.88 + 0.97	1.98 + 1.32	0.33 + 0.16	1.56 + 0.62
DREGS	IT	2087.3 + 1792.6	2358.2 + 1199.5	1327.6 + 1194.8	9069.2 + 7623.4	2498.2 + 703.9	7998.1 + 3568.5
	CPU	0.74 + 3.26	12.36 + 1.54	6.97 + 1.22	2.45 + 1.47	0.30 + 0.20	1.69 + 0.71

. For the DREK and DREGS methods, the iteration steps and running time for calculating $Y^{\left(k\right)}$ and $X^{\left(k\right)}$ are represented by “+”. From Table 10, it can be observed that the IME-REKRK method ia able to compute an approximate solution to the linear least squares problems when B has full column rank. The DREK and DREGS methods can successfully solve the linear least squares solution for all cases.

6. Conclusions

In this paper, we propose a Kaczmarz-type algorithm for the consistent matrix equation $AXB=C$. We develop a Gauss–Seidel-type algorithm to address the inconsistent case when A is full column rank and B is full row rank. In addition, we introduce extended Kaczmarz and extended Gauss–Seidel algorithms for inconsistent systems where either A or B lacks full rank. Theoretical analyses establish the linear convergence of these methods to the unique minimal Frobenius-norm solution or the least squares solution (i.e., $A^{+}C{B}^{+}$). Numerical experiments demonstrate the efficiency and robustness of the proposed algorithms.

In future work, we aim to extend the proposed algorithms to tensor equations and more general multilinear systems as well as to further explore their use in large-scale optimization and machine learning applications. Additionally, we will investigate accelerated variants of the proposed algorithms to enhance their convergence rates by incorporating strategies such as greedy selection rules, block iterative processing, and adaptive step size techniques. These improvements are expected to significantly boost computational efficiency, making the proposed methods more suitable for high-dimensional and real-world problems.