DIAS: A Data-Informed Active Subspace Regularization Framework for Inverse Problems

Abstract

This paper presents a regularization framework that aims to improve the fidelity of Tikhonov inverse solutions. At the heart of the framework is the data-informed regularization idea that only data-uninformed parameters need to be regularized, while the data-informed parameters, on which data and forward model are integrated, should remain untouched. We propose to employ the active subspace method to determine the data-informativeness of a parameter. The resulting framework is thus called a data-informed (DI) active subspace (DIAS) regularization. Four proposed DIAS variants are rigorously analyzed, shown to be robust with the regularization parameter and capable of avoiding polluting solution features informed by the data. They are thus well suited for problems with small or reasonably small noise corruptions in the data. Furthermore, the DIAS approaches can effectively reuse any Tikhonov regularization codes/libraries. Though they are readily applicable for nonlinear inverse problems, we focus on linear problems in this paper in order to gain insights into the framework. Various numerical results for linear inverse problems are presented to verify theoretical findings and to demonstrate advantages of the DIAS framework over the Tikhonov, truncated SVD, and the TSVD-based DI approaches.

1. Introduction with Related Work and Novelties

Over the past few decades, the development of new regularization techniques has played an important role in addressing the ill-posedness of inverse problems. A few example applications include image reconstruction in X-ray tomography [1,2] and various partial differential equation constrained inversion problems, for instance, inverse scattering and parameter identification [3,4,5,6,7]. Popular techniques, such as Tikhonov regularization and truncated singular value decomposition (truncated SVD), are ubiquitous in practical inverse problems [8,9,10]. However, one particular challenge of Tikhonov-based regularization strategies is the need to specify a suitable choice of the regularization parameter. Suboptimal choices can lead to excessively smooth or unstable reconstructions. While methods such as Morozov’s discrepancy principle, the L-curve criterion, and cross validation are popular for choosing a satisfactory regularization parameter, these are often computationally expensive and may require the computation of multiple inverse solutions. Even when the “right” regularization parameter is chosen, the smoothing effect leading to smeared reconstructions is unavoidable. On the other hand, spectral-decomposition-based methods, such as truncated SVD, aim to avoid regularizing data-informative features while applying infinite regularization on the rest. It is, however, not trivial to determine how many dominant modes should be retained. Furthermore, these SVD-typed approaches are typically suitable only for linear inverse problems.

Other regularization techniques have been proposed to combat the smoothing effect of Tikhonov regularization. Total variation (TV) regularization is designed with an anisotropic diffusion mechanism to ensure discontinuities and sharp interfaces in the inverse solution to be maximally preserved [9,10,11]. One problem with TV regularization is that, due to the non-differentiability of the TV functional, it could produce a staircasing effect [12]. To overcome this issue, smooth approximations and sophisticated optimization methods have been developed [9,13]. One of the reasons why Tikhonov and TV regularizations are popular is their convexity. This is a particularly appealing feature for inversion techniques using optimization. Though less popular, non-convex regularization strategies [14,15,16] are viable alternatives. However, inverse solutions with non-convex regularizations also require advanced optimization methods, such as alternating direction method of multipliers (ADMM) [17,18,19] or iteratively reweighted least squares (IRLS) [20].

In our previous work [21], inspired by the truncated SVD method, we put forward a new regularization approach, called data-informed (DI) regularization. DI was designed to overcome the ill-posed nature of inverse problems by placing regularization only where it is necessary in order to preserve the fidelity of the reconstructions. This is accomplished by a two-step process: first, find data-informed directions in the solution space, and then apply Tikhonov regularization only in the data-uninformed directions. We theoretically showed that DI is a valid regularization strategy. Numerically, we demonstrated that the reconstruction accuracy is robust for a wide range of regularization parameter values and DI outperforms both Tikhonov and truncated SVD (TSVD) for various inverse and imaging problems. Since DI, as well as TSVD, exploits the SVD of the forward operator, extension is necessary for nonlinear inverse problems. One straightforward extension for the Newton-based optimizer is to apply the DI approach at each Newton iteration by linearizing the forward operator around the current parameter estimate. However, this approach can require significant additional computation, especially in high dimensions.

Meanwhile, one recent tool for studying sensitivity is the active subspace (AS) method introduced in [22]. The active subspace is designed to study the sensitivity of a given nonlinear function with respect to the input parameters. The key idea is to identify a set of the directions in the input space that, on average, contribute most to the variation of the function. The beauty of the approach is that it requires the computation of the active subspace only once for any nonlinear input–output map. An application where AS is particularly useful is dimension reduction [23,24,25,26,27]. Due to the computational cost involved in dealing with high-dimensional parameter spaces in traditional Markov chain Monte Carlo simulations (MCMC), the active subspace method was used to accelerate MCMC [28]. The work in [29] combined the active subspace method and proper orthogonal decomposition with interpolation to obtain a better reconstruction of the modal coefficients when a small number of snapshots are used. More recently, the active subspace method was adopted to develop neural network based surrogate models [30].

In this work, we equip the DI idea with the active space approach and we call this combination the data-informed active subspace (DIAS) regularization strategy. The DIAS method retains all of the DI advantages, including being robust with respect to the regularization parameter, and avoids polluting solution features informed by the data while estimating the data-informed subspace better than the original DI approach. Unlike DI or Tikhonov, DIAS takes into account the uncertain nature of the inverse problem via the active subspace using the prior distribution of the unknown parameter. More importantly, it is applicable not only for linear, but also seamlessly for nonlinear inverse problems. Indeed, the active subspace, thanks to its global nature, is computed once at the beginning regardless of the linear or nonlinear nature of the inverse problem under consideration. Furthermore, the DIAS approach can effectively reuse any Tikhonov regularization codes/libraries. In fact, DIAS regularization can be considered a special Tikhonov regularization once the DIAS regularization operator is constructed.

Table 1. Comparison of features between different approaches (✓ = possess, ✗ = not possess).

	TSVD	Tikhonov	DI [21]	DIAS
Solving ill-posed inverse problem	✓	✓	✓	✓
Parameter robustness/Avoiding regularizing data-informed modes	✗	✗	✓	✓
Ordering data-informed modes based on observational data	✗	✗	✗	✓
Readily aplicable to nonlinear inverse problems	✗	✓	✗	✓
Taking the uncertain nature of the inverse problem into account	✗	✗	✗	✓

summarizes the main advantageous features of DIAS approaches over the classical Tikhonov regularization method and the DI method [21].

The paper is structured as follows. Section 2 briefly reviews the standard (uncentered) active subspace method and discusses its centered variation. The relationship between uncentered and centered active subspaces is then rigorously investigated. Each of these active subspace approaches can be incorporated into the DI framework with either the original data misfit or an approximate one. For linear inverse problems, Section 3 proposes two DIAS regularization methods with approximate data misfit: one using centered active subspaces (cDIAS-A) and the other using uncentered active subspaces (DIAS-A). An important result is that the truncated SVD approach is a special case of cDIAS-A. Similarly, two DIAS approaches with the original (full) data misfit—one using centered active subspaces (cDIAS-F) and the other using uncentered active subspaces (DIAS-F)—are presented for linear inverse problems in Section 4. It is in this section that the practical aspect of the four DIAS variants is discussed. The full data misfit variants, DIAS-F and cDIAS-F, are non-intrusive, while the approximate misfit counterparts are intrusive. We also show that the DI approach in [21] is a special case of cDIAS-F. Various numerical results for linear inverse problems are presented in Section 5 to expose the pros and cons of each of the DIAS variants and to compare them with Tikhonov regularization. Section 6 concludes the paper with future work.

2. Centered versus Uncentered Active Subspace Methods for Linear Inverse Problems

In this section, we first recall some results on the active subspaces (AS) method that are useful for our developments. A detailed derivation of the active subspace can be consulted in [28]. The main issue that we investigate is the difference between the uncentered AS and the centered AS. To begin, let $f\left(\mathit{x}\right)$ be a differentiable function from ${\mathbb{R}}^p$ to $\mathbb{R}$ and its gradient vector be denoted as $\nabla f\left(\mathit{x}\right)\in {\mathbb{R}}^p$. The key object in the AS method is the uncentered covariance matrix $\mathit{C}\in {\mathbb{R}}^{p\times p}$ defined by

$$\mathit{C}=\int \nabla f\left(\mathit{x}\right)\nabla f{\left(\mathit{x}\right)}^T\rho \left(\mathit{x}\right)d\mathit{x}=\mathit{W}\mathit{D}{\mathit{W}}^T,$$

(1)

where $\rho \left(\mathit{x}\right)$ is a probability density in ${\mathbb{R}}^p$ (we assume it to be a centered Gaussian $\mathcal{N}\left(0,\Gamma \right)$ in this paper, where $\Gamma$ is the covariance matrix), and $\left(\mathit{W},\mathit{D}\right)$ are matrices containing eigenpairs of $\mathit{C}$. The eigenvalue and eigenvector matrices can be partitioned as

$$\mathit{D}=\left[\begin{array}{cc}{\mathit{D}}_1& \\ & {\mathit{D}}_2\end{array}\right]\mathit{W}=\left[\begin{array}{cc}{\mathit{W}}_1& {\mathit{W}}_2\end{array}\right],$$

where ${\mathit{D}}_1$ is the diagonal matrix containing the r largest eigenvalues of $\mathit{C}$, and ${\mathit{W}}_1$ the corresponding eigenvectors. The active subspace —the subspace on which $f\left(\mathit{x}\right)$ is most sensitive on average—is defined to be the span of the columns of ${\mathit{W}}_1$. This is in turn determined by the r largest eigenvalues. Likewise, the inactive subspace—the subspace on which $f\left(\mathit{x}\right)$ is almost invariant on average—is defined by the span of the columns of ${\mathit{W}}_2$. It is thus sensible to “ignore” the inactive subspace without compromising much of the accuracy in computing $f\left(\mathit{x}\right)$ (see [28] for a rigorous proof). If $r\ll p$, significant computational gain can be achieved by focusing on only the active variables. One way to eliminate inactive variables is to integrate them out, as we now describe. Any $\mathit{x}\in {\mathbb{R}}^p$ can be written as

$$\mathit{x}={\mathit{W}}_1{\mathit{W}}_1^T\mathit{x}+{\mathit{W}}_2{\mathit{W}}_2^T\mathit{x}={\mathit{W}}_1\mathit{y}+{\mathit{W}}_2\mathit{z},$$

where $\mathit{y}={\mathit{W}}_1^T\mathit{x}$ are called the active variables, and $\mathit{z}={\mathit{W}}_2^T\mathit{x}$ the inactive variables. The density function $\rho \left(\mathit{x}\right)$ can be therefore considered as the joint density between the active and inactive variables:

$$\rho \left(\mathit{x}\right)=\rho \left({\mathit{W}}_1\mathit{y}+{\mathit{W}}_2\mathit{z}\right)=\rho \left(\mathit{y},\mathit{z}\right).$$

In the AS approach, one typically approximates $f\left(\mathit{x}\right)$ using the active variables by integrating out the inactive ones

$$f\left(\mathit{x}\right)\approx g\left(\mathit{y}\right)=\int f\left({\mathit{W}}_1\mathit{y}+{\mathit{W}}_2\mathit{z}\right)\rho \left(\mathit{z}|\mathit{y}\right)d\mathit{z},$$

(2)

where $\rho \left(\mathit{z}\right|\mathit{y})$ is the conditional density of $\mathit{z}$ given $\mathit{y}$. Note that if $\rho \left(\mathit{x}\right)=\mathcal{N}\left(0,\Gamma \right)$, which is the case in this paper, $\rho \left(\mathit{z}|\mathit{y}\right)$ is trivial (see Section 3.1). The integral evaluation is less straightforward, in fact, computationally prohibited, for nonlinear $f\left(\mathit{x}\right)$, and it is typically approximated using Monte Carlo sampling.

From a statistical perspective, the uncentered covariance matrix of the form (1) is not common, though it has been investigated. Comparisons between centered and uncentered covariance matrices have been carried out for general covariance matrices in the context of principal component analysis [31,32,33,34]. We now address the difference between uncentered and centered covariance matrices in the context of active subspaces. As we will show, more can be said in this context by exploiting the structure of the inverse problems. To that end, let us introduce the centered version of the AS covariance matrix

$$\tilde{\mathit{C}}=\int \left[\nabla f\left(\mathit{x}\right)-\overline{\nabla f}\left(\mathit{x}\right)\right]{\left[\nabla f\left(\mathit{x}\right)-\overline{\nabla f}\left(\mathit{x}\right)\right]}^T\rho \left(\mathit{x}\right)d\mathit{x},$$

(3)

where the gradient mean is given by

$$\overline{\nabla f}\left(\mathit{x}\right)=\int \nabla f\left(\mathit{x}\right)\rho \left(\mathit{x}\right)d\mathit{x}.$$

For notational convenience, we will refer to the active subspace based on (1) as the uncentered active subspace, and the one based on (3) as the centered active subspace.

To understand the advantages/disadvantages of uncentered AS and centered AS, we restrict ourselves in the linear inversion setting. Consider the following additive noise observational model

$$\mathit{d}=\mathit{A}\mathit{x}+\mathit{e},$$

with $\mathit{A}\in {\mathbb{R}}^{n\times p}$, $\mathit{e}\sim \mathcal{N}\left(0,\Lambda \right)$, and the data $\mathit{d}\in {\mathbb{R}}^n$. The inverse problem is to determine $\mathit{x}\in {\mathbb{R}}^p$ given $\mathit{d}$. Posing this problem as a least squares problem in a weighted Euclidean norm, we minimize the data misfit:

$$\underset{\mathit{x}}{min}\frac{1}{2}{∥\mathit{A}\mathit{x}-\mathit{d}∥}_{{\Lambda }^{-1}}^2.$$

To overcome an ill-conditioning issue due to the ill-posed nature of the inverse problem, we can employ the classical Tikhonov regularization approach:

$$\underset{\mathit{x}}{min}\frac{1}{2}{∥\mathit{A}\mathit{x}-\mathit{d}∥}_{{\Lambda }^{-1}}^2+\frac{1}{2}{∥\mathit{x}∥}_{{\Gamma }^{-1}}^2,$$

(4)

where ${\Gamma }^{-1}$ is a given weight matrix. In the context of Bayesian inverse problems, $\Gamma$ is typically chosen as the covariance of the prior distribution of $\mathit{x}$. Following [22], we determine the active subspaces based on the data misfit, i.e.,

$$f\left(\mathit{x}\right):=\frac{1}{2}{∥\mathit{A}\mathit{x}-\mathit{d}∥}_{{\Lambda }^{-1}}^2,$$

(5)

whose gradient is

$$\nabla f\left(\mathit{x}\right)={\mathit{A}}^T{\Lambda }^{-1}\left(\mathit{A}\mathit{x}-\mathit{d}\right).$$

Since we choose $\rho \left(\mathit{x}\right)=\mathcal{N}\left(0,\Gamma \right)$, it is easy to see that the uncentered covariance matrix $\mathit{C}$ in (1) can be written as [22]

$$\mathit{C}=\int \nabla f\left(\mathit{x}\right){\left(\nabla f\left(\mathit{x}\right)\right)}^T\rho \left(\mathit{x}\right)d\mathit{x}={\mathit{A}}^T{\Lambda }^{-1}\left(\mathit{A}\Gamma {\mathit{A}}^T+\mathit{d}{\mathit{d}}^T\right){\Lambda }^{-1}\mathit{A}.$$

Similarly, the mean of the gradient with respect to $\rho \left(\mathit{x}\right)$ is

$$\overline{\nabla f}\left(\mathit{x}\right)=\int {\mathit{A}}^T{\Lambda }^{-1}\left(\mathit{A}\mathit{x}-\mathit{d}\right)\rho \left(\mathit{x}\right)d\mathit{x}=-{\mathit{A}}^T{\Lambda }^{-1}\mathit{d},$$

and thus the centered covariance matrix $\tilde{\mathit{C}}$ in (3) becomes

$$\begin{array}{c}\tilde{\mathit{C}}=\int \left[\nabla f\left(\mathit{x}\right)-\overline{\nabla f}\left(\mathit{x}\right)\right]{\left[\nabla f\left(\mathit{x}\right)-\overline{\nabla f}\left(\mathit{x}\right)\right]}^T\rho \left(\mathit{x}\right)d\mathit{x}\hfill \\ \hfill =\mathit{C}-\overline{\nabla f}\left(\mathit{x}\right){\left(\overline{\nabla f}\left(\mathit{x}\right)\right)}^T={\mathit{A}}^T{\Lambda }^{-1}\mathit{A}\Gamma {\mathit{A}}^T{\Lambda }^{-1}\mathit{A}.\end{array}$$

In order to gain insights into the difference between centered and uncentered active subspaces, for the rest of Section 2, we assume $\Gamma =\mathit{I}$. Let the full SVD of the noise covariance whitened forward operator be ${\Lambda }^{-\frac{1}{2}}\mathit{A}=\mathit{U}\Sigma {\mathit{V}}^T$. Since ${\Lambda }^{-\frac{1}{2}}\mathit{d}$ resides in the column space of $\mathit{U}$, there exists $\mathit{\beta }\in {\mathbb{R}}^n$ such that

$${\Lambda }^{-\frac{1}{2}}\mathit{d}=\mathit{U}\mathit{\beta }.$$

Lemma 1.

Let $\mathit{D}:=\mathit{diag}\left({\lambda }_1,\dots ,{\lambda }_p\right)$ and $\tilde{\mathit{D}}:=\mathit{diag}\left({\tilde{\lambda }}_1,\dots ,{\tilde{\lambda }}_p\right)$ be eigenvalue matrices of $\mathit{C}$ and $\tilde{\mathit{C}}$, respectively, such that ${\lambda }_i\ge {\lambda }_{i+1}$ and ${\tilde{\lambda }}_i\ge {\tilde{\lambda }}_{i+1}$.

• For $k\le min\left\{n,p\right\}$${\tilde{\lambda }}_k={\sigma }_k^4$, and ${\tilde{\lambda }}_k=0$ for $n<k\le p$.
• Define $\gamma ={\Sigma }^T\mathit{\beta }$. For $2\le k\le p$, ${\tilde{\lambda }}_k\le {\lambda }_k\le {\tilde{\lambda }}_{k-1}$, and ${\tilde{\lambda }}_1\le {\lambda }_1\le {\tilde{\lambda }}_1+{\gamma }^T\gamma$.

Proof. 

The first assertion is obvious as

$$\tilde{\mathit{C}}=\mathit{V}{\Sigma }^T\Sigma {\Sigma }^T\Sigma {\mathit{V}}^T$$

using the SVD of ${\Lambda }^{-\frac{1}{2}}\mathit{A}$. For the second assertion, we have

$$\begin{array}{cc}\hfill \mathit{C}& ={\mathit{A}}^T{\Lambda }^{-1}\left(\mathit{A}{\mathit{A}}^T+\mathit{d}{\mathit{d}}^T\right){\Lambda }^{-1}\mathit{A}\hfill \\ \hfill & =\mathit{V}{\Sigma }^T{\mathit{U}}^T\left(\mathit{U}\Sigma {\Sigma }^T{\mathit{U}}^T+\mathit{U}\mathit{\beta }{\mathit{\beta }}^T{\mathit{U}}^T\right)\mathit{U}\Sigma {\mathit{V}}^T\hfill \\ \hfill & =\mathit{V}\underset{\Theta }{\underbrace{\left(\tilde{\mathit{D}}+\gamma {\gamma }^T\right)}}{\mathit{V}}^T\hfill \\ \hfill & =\mathit{V}\Theta {\mathit{V}}^T.\hfill \end{array}$$

Since $\mathit{C}$ is a similarity transformation of $\Theta$, we seek the relationship between the spectra of $\Theta$ and $\tilde{\mathit{C}}$. Now, since $\Theta$ is a rank-one perturbation of the diagonal matrix $\tilde{\mathit{D}}$, using a standard interlacing eigenvalue perturbation result (see, for example, ([35], Section 5) and [36]) concludes the proof. □

Lemma 1 shows that the eigenvalues of $\mathit{C}$ are not smaller than those of $\tilde{\mathit{C}}$, but this does not reveal insights on how $\mathit{d}$ could make a difference going from $\tilde{\mathit{C}}$ to $\mathit{C}$. To wit, let us consider a special case of $\mathit{d}=\beta {\sigma }_k{\mathit{u}}_k$, where ${\mathit{u}}_k$ is the k-th column of $\mathit{U}$ and $\beta$ is some number, then it is straightforward to see that

$$\begin{array}{c}\hfill \mathit{C}=\mathit{V}\mathrm{diag}\left\{{\tilde{\lambda }}_1,\dots ,{\tilde{\lambda }}_i,\dots ,{\tilde{\lambda }}_k\left(1+{\beta }^2\right),\dots ,{\tilde{\lambda }}_p\right\}{\mathit{V}}^T.\end{array}$$

Now if ${\beta }^2$ is sufficiently large such that ${\tilde{\lambda }}_k\left(1+{\beta }^2\right)>{\tilde{\lambda }}_i$ where $i\le r<k$, then ${\lambda }_i={\tilde{\lambda }}_k\left(1+{\beta }^2\right)$. As a direct consequence, while ${\mathit{v}}_k$ is not part of the centered active subspace, it is for the uncentered one. In other words, it is important to see that the uncentered AS approach takes both the forward operator $\mathit{A}$ and the data into account when constructing the active subspace, while the centered AS approach is purely determined by the spectrum of the forward operator. When $k\le r$, both approaches are similar. However, the uncentered AS is expected to outperform the centered counterpart when $k>r$ since eigendirections, classified as inactive in the centered approach, are in fact active when taking the data into account; we verify this fact later in Section 5. The proof of Lemma 1 also implies that, due to the symmetry of $\Theta$, the eigenvectors of $\mathit{C}$ are not only a reordering, but also a rotation of the eigenvectors of $\tilde{\mathit{C}}$ in general since $\gamma {\gamma }^T$ is not necessarily diagonal.

Remark 1.

Note that we can alternatively first perform the whitening to transform the inverse problem to the standard setting with $\Lambda =\mathit{I}$ and $\Gamma =\mathit{I}$, and then compute the active subspaces. This simplifies the exposition. However, as shown in Appendix A, the active subspaces for the whitened problem change, and the corresponding DIAS solutions are less accurate.

3. Data-Informed Active Subspace Regularization for Linear Inverse Problems: Approximate Data Misfit

In our previous work [21], we proposed a data-informed (DI) regularization approach for linear inverse problems. The workhorse behind the DI approach is the generalized singular value decomposition taking advantage of the linear nature of the forward operator. In Section 2, we compare and contrast the centered and uncentered active subspace approaches. In this section, inspired by the DI idea, we construct a data-informed active subspace regularization approach for both centered and uncentered cases. We take advantage of the linear structure to approximate the data misfit. We consider the full data misfit in Section 4.

3.1. DIAS with Uncentered Active Subspace (DIAS-A)

The goal of this section is to explicitly derive the DIAS regularization for linear inverse problems using the uncentered active subspace method. Suppose that the active subspace ${\mathit{W}}_1$ and the inactive subspace ${\mathit{W}}_2$ are identified. The joint prior probability density $\rho \left(\mathit{x}\right)$ can be written as

$$\begin{array}{cc}\hfill \rho \left(\mathit{x}\right)& =\rho (\mathit{y},\mathit{z})=c_{x}exp\left(-\frac{1}{2}{∥\mathit{x}∥}_{{\Gamma }^{-1}}^2\right)\hfill \\ \hfill & =c_{x}exp\left(-\frac{1}{2}\left[\begin{array}{cc}{\mathit{y}}^T& {\mathit{z}}^T\end{array}\right]\left[\begin{array}{cc}{\mathit{W}}_1^T{\Gamma }^{-1}{\mathit{W}}_1& {\mathit{W}}_1^T{\Gamma }^{-1}{\mathit{W}}_2\\ {\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_1& {\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2\end{array}\right]\left[\begin{array}{c}\mathit{y}\\ \mathit{z}\end{array}\right]\right),\hfill \end{array}$$

where $c_x$ is a normalization constant. It follows that the conditional probability $\rho \left(\mathit{z}\right|\mathit{y})$ is

$$\rho \left(\mathit{z}\right|\mathit{y})=c_{z,y}exp\left(-\frac{1}{2}{∥\mathit{z}-\overline{\mathit{z}}∥}_{{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2}^2\right),$$

(6)

where $\overline{\mathit{z}}=-{\left({\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2\right)}^{-1}{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_1\mathit{y}$, and $c_{z,y}$ is a normalization constant. From (2), the AS approximation $g\left(\mathit{y}\right)$ of the data misfit term $f\left(\mathit{x}\right)$ in (5) is now simplified using (6)

$$\begin{array}{cc}\hfill g\left(\mathit{y}\right)& =\frac{1}{2}\int {∥\mathit{A}\left({\mathit{W}}_1\mathit{y}+{\mathit{W}}_2\mathit{z}\right)-\mathit{d}∥}_{{\Lambda }^{-1}}^2{c}_{z,y}exp\left(-\frac{1}{2}{∥\mathit{z}-\overline{\mathit{z}}∥}_{{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2}^2\right)d\mathit{z},\hfill \\ \hfill & =\frac{1}{2}{∥\mathit{A}\left(\mathit{I}-{\mathit{W}}_2{\left[{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2\right]}^{-1}{\mathit{W}}_2^T{\Gamma }^{-1}\right){\mathit{W}}_1\mathit{y}-\mathit{d}∥}_{{\Lambda }^{-1}}^2+c^2\hfill \end{array}$$

where

$$c^2=\frac{c_{z,y}}{2}\int {∥\mathit{A}{\mathit{W}}_2\left(\mathit{z}-\overline{\mathit{z}}\right)∥}_{{\Lambda }^{-1}}^{2}exp\left(-\frac{1}{2}{∥\mathit{z}-\overline{\mathit{z}}∥}_{{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2}^2\right)d\mathit{z},$$

which is further simplified as

$$\begin{array}{cc}\hfill c^2& =\frac{c_{z,y}}{2}\mathbf{Trace}\left(\int {∥\mathit{A}{\mathit{W}}_2\left(\mathit{z}-\overline{\mathit{z}}\right)∥}_{{\Lambda }^{-1}}^{2}exp\left(-\frac{1}{2}{∥\mathit{z}-\overline{\mathit{z}}∥}_{{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2}^2\right)d\mathit{z}\right)\hfill \\ \hfill & =\frac{c_{z,y}}{2}\int \mathbf{Trace}\left({∥\mathit{A}{\mathit{W}}_2\left(\mathit{z}-\overline{\mathit{z}}\right)∥}_{{\Lambda }^{-1}}^2\right)exp\left(-\frac{1}{2}{∥\mathit{z}-\overline{\mathit{z}}∥}_{{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2}^2\right)d\mathit{z}\hfill \\ \hfill & =\frac{c_{z,y}}{2}\int \mathbf{Trace}\left({\left(\mathit{z}-\overline{\mathit{z}}\right)}^T{\mathit{W}}_2^T{\mathit{A}}^T{\Lambda }^{-1}\mathit{A}{\mathit{W}}_2\left(\mathit{z}-\overline{\mathit{z}}\right)\right)\hfill \\ \hfill & exp\left(-\frac{1}{2}{∥\mathit{z}-\overline{\mathit{z}}∥}_{{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2}^2\right)d\mathit{z}\hfill \\ \hfill & =\frac{c_{z,y}}{2}\int \mathbf{Trace}\left({\mathit{W}}_2^T{\mathit{A}}^T{\Lambda }^{-1}\mathit{A}{\mathit{W}}_2\left(\mathit{z}-\overline{\mathit{z}}\right){\left(\mathit{z}-\overline{\mathit{z}}\right)}^T\right)\hfill \\ \hfill & exp\left(-\frac{1}{2}{∥\mathit{z}-\overline{\mathit{z}}∥}_{{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2}^2\right)d\mathit{z}\hfill \\ \hfill & =\frac{c_{z,y}}{2}\mathbf{Trace}\left(\int {\mathit{W}}_2^T{\mathit{A}}^T{\Lambda }^{-1}\mathit{A}{\mathit{W}}_2\left(\mathit{z}-\overline{\mathit{z}}\right){\left(\mathit{z}-\overline{\mathit{z}}\right)}^{T}exp\left(-\frac{1}{2}{∥\mathit{z}-\overline{\mathit{z}}∥}_{{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2}\right)d\mathit{z}\right)\hfill \\ \hfill & =\frac{{\tilde{c}}_{z,y}}{2}\mathbf{Trace}\left({\mathit{W}}_2^T{\mathit{A}}^T{\Lambda }^{-1}\mathit{A}{\mathit{W}}_2{\left({\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2\right)}^{-1}\right),\hfill \end{array}$$

where we have used the commutativity of matrix trace and integral, and the invariance of matrix trace under cyclic permutations. Here, ${\tilde{c}}_{z,y}$ is the product of $c_{z,y}$ and the constant from the integration.

Since $c^2$ is a constant, the inverse problem (4) can be equivalently written as

$$\underset{\mathit{x}}{min}\frac{1}{2}{∥\mathit{M}\mathit{x}-\mathit{d}∥}_{{\Lambda }^{-1}}^2+\frac{1}{2}{∥\mathit{x}∥}_{{\Gamma }^{-1}}^2,$$

(7)

where, for brevity, we have defined

$$\mathit{M}:=\mathit{A}\left(\mathit{I}-{\mathit{W}}_2{\left[{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2\right]}^{-1}{\mathit{W}}_2^T{\Gamma }^{-1}\right){\mathit{W}}_1{\mathit{W}}_1^T.$$

It should be pointed out that we have used the relationship $\mathit{y}={\mathit{W}}_1^T\mathit{x}$ to write the inverse problem in terms of the original parameter $\mathit{x}$.

Proposition 1.

The term $\mathit{M}$ can be simplified further as

$$\mathit{M}=\mathit{A}\left(\mathit{I}-{\mathit{W}}_2{\left[{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2\right]}^{-1}{\mathit{W}}_2^T{\Gamma }^{-1}\right){\mathit{W}}_1{\mathit{W}}_1^T=\mathit{A}{\mathit{W}}_1{\mathit{W}}_1^T.$$

Proof. 

Let the singular value decomposition of ${\Gamma }^{-\frac{1}{2}}$ be ${\Gamma }^{-\frac{1}{2}}=\mathit{R}\mathit{K}{\mathit{R}}^T$. We have

$$\begin{array}{cc}\hfill & {\Gamma }^{-\frac{1}{2}}{\mathit{W}}_1=\mathit{R}\mathit{K}{\mathit{R}}^T\mathit{W}\left[\begin{array}{c}\mathit{I}\\ 0\end{array}\right],\hfill \\ \hfill & {\Gamma }^{-\frac{1}{2}}{\mathit{W}}_2=\mathit{R}\mathit{K}{\mathit{R}}^T\mathit{W}\left[\begin{array}{c}0\\ \mathit{I}\end{array}\right].\hfill \end{array}$$

(8)

We can rewrite $\mathit{M}$ as

$$\begin{array}{cc}\hfill \mathit{M}& =\mathit{A}\left(\mathit{I}-{\mathit{W}}_2{\left[{\mathit{W}}_2^T{\Gamma }^{-1}{\mathit{W}}_2\right]}^{-1}{\mathit{W}}_2^T{\Gamma }^{-1}\right){\mathit{W}}_1{\mathit{W}}_1^T\hfill \\ \hfill & =\mathit{A}{\mathit{W}}_1{\mathit{W}}_1^T-\mathit{A}{\Gamma }^{\frac{1}{2}}\underset{T}{\underbrace{{\Gamma }^{-\frac{1}{2}}{\mathit{W}}_2{\left[{\mathit{W}}_2^T{\Gamma }^{-\frac{1}{2}}{\Gamma }^{-\frac{1}{2}}{\mathit{W}}_2\right]}^{-1}{\mathit{W}}_2^T{\Gamma }^{-\frac{1}{2}}{\Gamma }^{-\frac{1}{2}}{\mathit{W}}_1{\mathit{W}}_1^T{\Gamma }^{-\frac{1}{2}}}}{\Gamma }^{\frac{1}{2}}.\hfill \end{array}$$

(9)

Since ${\mathit{W}}_2$ is a full column rank matrix, so is ${\Gamma }^{-\frac{1}{2}}{\mathit{W}}_2$. Thus

$${\left[{\left({\Gamma }^{-\frac{1}{2}}{\mathit{W}}_2\right)}^T{\Gamma }^{-\frac{1}{2}}{\mathit{W}}_2\right]}^{-1}{\left({\Gamma }^{-\frac{1}{2}}{\mathit{W}}_2\right)}^T={\left\{{\mathit{W}}_2^T{\Gamma }^{-\frac{1}{2}}\right\}}^{†}=\left[\begin{array}{cc}0& \mathit{I}\end{array}\right]{\mathit{W}}^T\mathit{R}{\mathit{K}}^{-1}{\mathit{R}}^T.$$

The second equality is simply the pseudo-inverse using the singular value decomposition. Substituting (8) into the $\mathit{T}$ in the Equation (9) yields

$$\begin{array}{cc}\hfill \mathit{T}& =\mathit{R}\mathit{K}{\mathit{R}}^T\mathit{W}\left[\begin{array}{c}0\\ \mathit{I}\end{array}\right]\left[\begin{array}{cc}0& \mathit{I}\end{array}\right]{\mathit{W}}^T\mathit{R}{\mathit{K}}^{-1}{\mathit{R}}^T\mathit{R}\mathit{K}{\mathit{R}}^T\mathit{W}\left[\begin{array}{c}\mathit{I}\\ 0\end{array}\right]\left[\begin{array}{cc}\mathit{I}& 0\end{array}\right]{\mathit{W}}^T\mathit{R}\mathit{K}{\mathit{R}}^T=\mathbf{0}.\hfill \end{array}$$

□

It is well known that the Tikhonov approach (7) applies regularization on all parameters and this is the reason why a Tikhonov solution is typically smeared out. As in the DI approach [21], we argue that only data-uninformed parameters should be regularized while the data-informed ones should remain untouched to preserve the fidelity of inverse solutions.

In this paper, we treat the active parameters as the data-informed parameters and the inactive parameters as the data-uninformed parameters. Recall that $\rho \left(\mathit{x}\right)=\mathcal{N}\left(0,\Gamma \right)$, thus $\mathit{z}={\mathit{W}}_2^T\mathit{x}\sim \mathcal{N}\left(0,{\mathit{W}}_2^T\Gamma {\mathit{W}}_2\right)$. In particular, we replace the Tikhonov regularization in the inverse problem (7) with the DIAS regularization, in which we regularize only the inactive modes,

$$\underset{\mathit{x}}{min}\frac{1}{2}{∥\mathit{M}\mathit{x}-\mathit{d}∥}_{{\Lambda }^{-1}}^2+\frac{1}{2}{∥{\mathit{W}}_2^T\mathit{x}∥}_{{\left({\mathit{W}}_2^T\Gamma {\mathit{W}}_2\right)}^{-1}}^2,$$

(10)

which yields the following inverse solution:

$${\mathit{x}}_{\mathrm{DIAS}-\mathrm{A}}={\left[{\mathit{M}}^T{\Lambda }^{-1}\mathit{M}+{\mathit{W}}_2{\left({\mathit{W}}_2^T\Gamma {\mathit{W}}_2\right)}^{-1}{\mathit{W}}_2^T\right]}^{-1}\left({\mathit{M}}^T{\Lambda }^{-1}\mathit{d}\right).$$

(11)

3.2. Dias with Centered Active Subspace (cDIAS-A)

Let the eigenvalue decomposition of $\tilde{\mathit{C}}$ be

$$\tilde{\mathit{C}}=\mathit{V}\mathit{E}{\mathit{V}}^T,$$

(12)

and similar to the uncentered AS, we partition the eigenvalue and eigenvector matrices as

$$\mathit{E}=\left[\begin{array}{cc}{\mathit{E}}_1& \\ & {\mathit{E}}_2\end{array}\right],\mathit{V}=\left[\begin{array}{cc}{\mathit{V}}_1& {\mathit{V}}_2\end{array}\right]$$

in which ${\mathit{E}}_1$ is the diagonal matrix containing the r largest eigenvalues and ${\mathit{V}}_1$ consists of the corresponding eigenvectors. For notational convenience, we still denote the active and inactive parameters as $\mathit{y}$ and $\mathit{z}$, respectively. Whether they correspond to the uncentered or the centered AS should be clear from the context.

It is easy to see that any parameter $\mathit{x}\in {\mathbb{R}}^p$ can be uniquely expressed as

$$\mathit{x}=\overline{\nabla f}\left(\mathit{x}\right)+{\mathit{V}}_1\mathit{y}+{\mathit{V}}_2\mathit{z}={\mathit{V}}_1\left(\mathit{y}-{\mathit{y}}_0\right)+{\mathit{V}}_2\left(\mathit{z}-{\mathit{z}}_0\right),$$

where ${\mathit{y}}_0:={\mathit{V}}_1^T\overline{\nabla f}$ and ${\mathit{z}}_0:={\mathit{V}}_2^T\overline{\nabla f}$. Note that the relationship between the active and original parameters is given by $\mathit{y}={\mathit{V}}_1^T\mathit{x}+{\mathit{y}}_0$, and similarly $\mathit{z}={\mathit{V}}_2^T\mathit{x}+{\mathit{z}}_0$ for the relationship between the inactive and original parameters.

Now following the same procedure as in Section 3.1, we can obtain the DIAS inverse solution with centered AS:

$${\mathit{x}}_{\mathrm{cDIAS}-\mathrm{A}}={\left[{\mathit{G}}^T{\Lambda }^{-1}\mathit{G}+{\mathit{V}}_2{\left({\mathit{V}}_2^T\Gamma {\mathit{V}}_2\right)}^{-1}{\mathit{V}}_2^T\right]}^{-1}\left({\mathit{G}}^T{\Lambda }^{-1}\mathit{d}\right),$$

(13)

where

$$\mathit{G}:=\mathit{A}\left(\mathit{I}-{\mathit{V}}_2{\left[{\mathit{V}}_2^T{\Gamma }^{-1}{\mathit{V}}_2\right]}^{-1}{\mathit{V}}_2^T{\Gamma }^{-1}\right){\mathit{V}}_1{\mathit{V}}_1^T=\mathit{A}{\mathit{V}}_1{\mathit{V}}_1^T.$$

The second equality can be achieved in the same manner as Proposition 1.

Proposition 2.

If $\Gamma ={\beta }^2\mathit{I}$, then the cDIAS-A solution ${\mathit{x}}_{\mathrm{cDIAS}-\mathrm{A}}$ reduces to the truncated SVD solution.

Proof. 

Given $\Gamma ={\beta }^2\mathit{I}$, then $\mathit{V}$ in (12) is the right eigenvectors of ${\Lambda }^{-\frac{1}{2}}\mathit{A}$. In this case, the solution (13) reads

$$\begin{array}{cc}\hfill {\mathit{x}}_{\mathrm{cDIAS}-\mathrm{A}}& ={\left({\mathit{V}}_1{\mathit{V}}_1^T{\mathit{A}}^T{\Lambda }^{-1}\mathit{A}{\mathit{V}}_1{\mathit{V}}_1^T+\frac{1}{{\beta }^2}\left(\mathit{I}-{\mathit{V}}_1{\mathit{V}}_1^T\right)\right)}^{†}{\mathit{V}}_1{\mathit{V}}_1^T{\mathit{A}}^T{\Lambda }^{-1}\mathit{d}\hfill \\ & =\mathit{V}\left\{\left[\begin{array}{cc}diag{\left(\frac{1}{{\sigma }_i^2}\right)}_{i=1}^r& 0\\ 0& 0\end{array}\right]+\left[\begin{array}{cc}0& 0\\ 0& diag{\left({\beta }^2\right)}_{r+1}^p\end{array}\right]\right\}{\mathit{V}}^T{\mathit{V}}_1{\mathit{V}}_1^T{\mathit{A}}^T{\Lambda }^{-1}\mathit{d}\hfill \\ & =\mathit{V}\left[\begin{array}{cc}diag{\left(\frac{1}{{\sigma }_i^2}\right)}_{i=1}^r& 0\\ 0& diag{\left({\beta }^2\right)}_{r+1}^p\end{array}\right]\left[\begin{array}{cc}diag{\left({\sigma }_i\right)}_{i=1}^r& 0\\ 0& 0\end{array}\right]{\mathit{U}}^T{\Lambda }^{-\frac{1}{2}}\mathit{d}\hfill \\ & =\mathit{V}\left[\begin{array}{cc}diag{\left(\frac{1}{{\sigma }_i}\right)}_{i=1}^r& 0\\ 0& 0\end{array}\right]{\mathit{U}}^T{\Lambda }^{-\frac{1}{2}}\mathit{d}=\sum _{i=1}^r\frac{{\mathit{u}}_i^T{\Lambda }^{-\frac{1}{2}}\mathit{d}}{{\sigma }_i}{\mathit{v}}_i,\hfill \end{array}$$

where r is the dimension of the active subspace. This is exactly the solution of the truncated SVD method of (10). □

4. Data-Informed Active Subspace Regularization for Linear Inverse Problems: Non-Approximate Data Misfit (DIAS-F)

Following the standard practice in the AS method, we intrusively replaced the original data misfit with an approximate one in Section 3. Though this approach is clean and significantly reduces the number of optimization variables to manage for linear inverse problems, its intrusive nature requires a complete overhaul of the original inverse codes. This section avoids this issue by considering a DIAS formulation without changing the data misfit at the expense of working with the original high-dimensional parameter $\mathit{x}$. As this approach does not approximate the data misfit, it could provide more robust solutions as demonstrated numerically in Section 5.

The uncentered and centered DIAS formulations in this case simply read

$$\begin{array}{cc}\hfill & \underset{\mathit{x}}{min}\frac{1}{2}{∥\mathit{A}\mathit{x}-\mathit{d}∥}_{{\Lambda }^{-1}}^2+\frac{1}{2}{∥{\mathit{W}}_2^T\mathit{x}∥}_{{\left({\mathit{W}}_2^T\Gamma {\mathit{W}}_2\right)}^{-1}}^2,\mathrm{and}\hfill \\ \hfill & \underset{\mathit{x}}{min}\frac{1}{2}{∥\mathit{A}\mathit{x}-\mathit{d}∥}_{{\Lambda }^{-1}}^2+\frac{1}{2}{∥{\mathit{V}}_2^T\mathit{x}∥}_{{\left({\mathit{V}}_2^T\Gamma {\mathit{V}}_2\right)}^{-1}}^2,\hfill \end{array}$$

respectively. Thus, the inverse solution using the uncentered AS, analogous to (11), is

$${\mathit{x}}_{\mathrm{DIAS}-\mathrm{F}}={\left({\mathit{A}}^T{\Lambda }^{-1}\mathit{A}+{\mathit{W}}_2{\left({\mathit{W}}_2^T\Gamma {\mathit{W}}_2\right)}^{-1}{\mathit{W}}_2^T\right)}^{-1}{\mathit{A}}^T{\Lambda }^{-1}\mathit{d},$$

(14)

and the centered AS solution, similar to (13), reads

$${\mathit{x}}_{\mathrm{cDIAS}-\mathrm{F}}={\left({\mathit{A}}^T{\Lambda }^{-1}\mathit{A}+{\mathit{V}}_2{\left({\mathit{V}}_2^T\Gamma {\mathit{V}}_2\right)}^{-1}{\mathit{V}}_2^T\right)}^{-1}{\mathit{A}}^T{\Lambda }^{-1}\mathit{d}.$$

(15)

Proposition 3.

If $\Gamma ={\beta }^2\mathit{I}$, then the cDIAS-F solution (15) reduces to the DI solution proposed in [21].

Proof. 

Let $\overline{\mathit{A}}={\Lambda }^{-\frac{1}{2}}\mathit{A}$ and $\overline{\mathit{d}}={\Lambda }^{-\frac{1}{2}}\mathit{d}$, then the cDIAS-F solution (15) becomes

$${\mathit{x}}_{\mathrm{cDIAS}-\mathrm{F}}={\left({\overline{\mathit{A}}}^T\overline{\mathit{A}}+\frac{1}{{\beta }^2}\left[\mathit{I}-{\mathit{V}}_1{\mathit{V}}_1^T\right]\right)}^{-1}{\overline{\mathit{A}}}^T\overline{\mathit{d}}.$$

which is exactly the DI solution in [21]. □

Theorem 1.

Let ${\lambda }_i$ be the i-th eigenvalue of the active subspace matrix $\mathit{C}$ or $\tilde{\mathit{C}}$, and

$$r_{\epsilon }:=max\left\{i:1\le i\le p\mathit{and}{\lambda }_i\ge \epsilon \right\},\epsilon >0.$$

Set ${\mathit{L}}^T\mathit{L}={\mathit{W}}_2{\left({\mathit{W}}_2^T\Gamma {\mathit{W}}_2\right)}^{-1}{\mathit{W}}_2^T$ and ${\mathit{L}}^T\mathit{L}={\mathit{V}}_2{\left({\mathit{V}}_2^T\Gamma {\mathit{V}}_2\right)}^{-1}{\mathit{V}}_2^T$ for the uncentered active subspace and centered active subspace, respectively. Suppose that the nullspace of $\mathit{A}$ is trivial, i.e., $\mathcal{N}\left(\mathit{A}\right)=\left\{0\right\}$. Consider the inverse problem

$$\underset{\mathit{x}}{min}\frac{1}{2}{∥{\Lambda }^{-\frac{1}{2}}\mathit{B}\mathit{x}-{\Lambda }^{-\frac{1}{2}}\mathit{d}∥}_2^2+\frac{1}{2}{∥\mathit{x}∥}_2^2,$$

(16)

using the DIAS regularization approaches with rank-$r_{\epsilon }$ active subspace, where $\mathit{B}=\mathit{M},$ or $\mathit{G},$ or $\mathit{A},$ which correspond to the DIAS-A, cDIAS-A, or (c)DIAS-F approaches, respectively. Define the “reconstruction operator” [7,37], approximating the map from observations $\mathit{d}$ to parameter $\mathit{x}$, as

$${\mathcal{R}}_{\epsilon }:={\left({\mathit{H}}^{r_{\epsilon }}\right)}^{-1}{\mathit{B}}^T{\Lambda }^{-\frac{1}{2}},$$

where

$$\begin{array}{cc}\hfill & {\mathit{H}}^{r_{\epsilon }}={\mathit{B}}^T{\Lambda }^{-1}\mathit{B}+{\mathit{L}}^T\mathit{L}.\hfill \end{array}$$

The following hold:

(i) 

The inverse problem with rank-$r_{\epsilon }$ DIAS approaches, i.e., the optimization problem (16), is well-posed in the Hadamard sense.

(ii) 

DIAS techniques are regularization strategies [7,37] in the following sense:

$$\underset{\epsilon \to 0}{lim}{\mathcal{R}}_{\epsilon }{\Lambda }^{-\frac{1}{2}}\mathit{A}\mathit{x}=\mathit{x}.$$

(iii) 

The rank-$r_{\epsilon }$ DIAS technique is an admissible regularization method.

Proof. 

The fact that ${\mathit{H}}^{r_{\epsilon }}$ is invertible for all approaches is clear.

(i)

It is sufficient to show that the solution is continuous with respect to the observational data. Indeed, we have

$${∥{\mathit{x}}_{\mathrm{DIAS}}∥}_2\le {∥{\left({\mathit{H}}^{r_{\epsilon }}\right)}^{-1}∥}_2{∥\mathit{A}∥}_2{∥{\Lambda }^{-1}∥}_2{∥\mathit{d}∥}_2\le \frac{1}{\varphi }{∥\mathit{A}∥}_2{∥{\Lambda }^{-1}∥}_2{∥\mathit{d}∥}_2$$

where $\varphi$ is the smallest singular value of ${\mathit{H}}^{r_{\epsilon }}$. Note that, owing to the invertibility of ${\mathit{H}}^{r_{\epsilon }}$, $\varphi$ is bounded away from 0 and approaches the smallest singular value of ${\mathit{A}}^T{\Lambda }^{-1}\mathit{A}$, which is larger than 0 due to the triviality of the nullspace of $\mathit{A}$.

(ii)

The matrix inversion is a continuous mapping (which is the composition of continuous maps, i.e., the matrix determinant and the matrix adjoint), thus

$$\begin{array}{c}\underset{\epsilon \to 0}{lim}{\mathcal{R}}_{\epsilon }{\Lambda }^{-\frac{1}{2}}\mathit{A}\mathit{x}=\left(\underset{\epsilon \to 0}{lim}{\left({\mathit{H}}^{r_{\epsilon }}\right)}^{-1}\right){\mathit{A}}^T{\Lambda }^{-1}\mathit{A}\mathit{x}={\left(\underset{\epsilon \to 0}{lim}{\mathit{H}}^{r_{\epsilon }}\right)}^{-1}{\mathit{A}}^T{\Lambda }^{-1}\mathit{A}\mathit{x}\hfill \\ \hfill ={\left({\mathit{A}}^T{\Lambda }^{-1}\mathit{A}\right)}^{-1}{\mathit{A}}^T{\Lambda }^{-1}{\Lambda }^{-\frac{1}{2}}\mathit{A}\mathit{x}=\mathit{x},\end{array}$$

where we have used the following facts

• The rank of active subspace $r_{\epsilon }\to p$ as $\epsilon \to 0$, and thus ${\mathit{L}}^T\mathit{L}=\mathbf{0}$.
• As $r_{\epsilon }\to p$, we have $\mathit{B}=\mathit{A}$.

(iii)

It is sufficient to show that

$$\underset{\mathit{y}}{sup}\left\{{∥{\mathcal{R}}_{\epsilon }{\Lambda }^{-\frac{1}{2}}\mathit{y}-\mathit{x}∥}_2:{∥{\Lambda }^{-\frac{1}{2}}\left(\mathit{A}\mathit{x}-\mathit{y}\right)∥}_2\le \epsilon \right\}\to 0\mathrm{as}\epsilon \to 0,$$

for any $\mathit{x}$. We have

$$\begin{array}{cc}\hfill {∥{\mathcal{R}}_{\epsilon }{\Lambda }^{-\frac{1}{2}}\mathit{y}-\mathit{x}∥}_2& \le {∥{\mathcal{R}}_{\epsilon }{\Lambda }^{-\frac{1}{2}}\mathit{A}\mathit{x}-\mathit{x}∥}_2+{∥{\mathcal{R}}_{\epsilon }{\Lambda }^{-\frac{1}{2}}\left(\mathit{A}\mathit{x}-\mathit{y}\right)∥}_2\hfill \\ \hfill & \le {∥{\left({\mathit{H}}^{r_{\epsilon }}\right)}^{-1}{\mathit{B}}^T{\Lambda }^{-1}\mathit{A}-\mathit{I}∥}_2{∥\mathit{x}∥}_2+{∥{\mathcal{R}}_{\epsilon }∥}_2\epsilon \hfill \\ \hfill & \le {∥{\left({\mathit{H}}^{r_{\epsilon }}\right)}^{-1}∥}_2\left({∥{\mathit{B}}^T{\Lambda }^{-1}\mathit{A}-{\mathit{B}}^T{\Lambda }^{-1}\mathit{B}∥}_2+{∥{\mathit{L}}^T\mathit{L}∥}_2\right){∥\mathit{x}∥}_2\hfill \\ \hfill & +\frac{\epsilon }{\varphi }{∥\mathit{B}∥}_2{∥{\Lambda }^{-\frac{1}{2}}∥}_2\hfill \\ \hfill & \le \frac{1}{\varphi }\left(\sigma \left(\epsilon \right)+\gamma \left(\epsilon \right)\right){∥\mathit{x}∥}_2+\frac{\epsilon }{\varphi }{∥\mathit{B}∥}_2{∥{\Lambda }^{-\frac{1}{2}}∥}_2,\hfill \end{array}$$

where $\sigma \left(\epsilon \right)$ is the maximum singular value of ${\mathit{B}}^T{\Lambda }^{-1}\mathit{A}-{\mathit{B}}^T{\Lambda }^{-1}\mathit{B}$, $\gamma \left(\epsilon \right)$ is the maximum eigenvalue of ${\mathit{L}}^T\mathit{L}$. We note that, as $\epsilon \to 0$, $\sigma \left(\epsilon \right)\to 0$ and $\gamma \left(\epsilon \right)\to 0$.

□

5. Numerical Results

We now test the proposed DIAS approaches against the Tikhonov method on a variety of popular linear inverse problems. In particular, we consider one-dimensional (1D) deconvolution, various benchmark problems from [38], and X-ray tomography. In all linear inverse problems, we assume $\Lambda ={\delta }^2\mathit{I}$ and $\Gamma ={\beta }^2\mathit{I}$. Under this assumption, we have $\alpha =\frac{{\beta }^2}{{\delta }^2}$. Furthermore, by Propositions 2 and 3, the cDIAS-A and cDIAS-F methods become the truncated SVD (TSVD) and the DI approaches, respectively. Thus, for clarity we use TSVD and DI in the places of cDIAS-A and cDIAS-F for all examples. Additionally, by $a\%$ additive white noise we mean additive Gaussian noise with the standard deviation of $a\%$ of the maximum value of synthetic data. In the 1D deconvolution problem, we numerically investigate the difference in the inverse solutions, using the uncentered AS approaches and the centered AS approaches. In particular, we highlight the reordering and rotation effect of the uncentered AS induced by the data $\mathit{d}$ (see Section 2 for a theoretical discussion).

5.1. 1D Deconvolution Problem

We consider the one-dimensional deconvolution problem (see, for example, [9,39])

$$d\left(s_j\right)={\int }_0^{1}a(s_j,t)x\left(t\right)dt+e\left(s_j\right),j=1,\dots ,n,$$

where $a(s,t)=\frac{1}{\sqrt{2\pi {\mu }^2}}exp\left(-\frac{1}{2{\mu }^2}{\left(t-s\right)}^2\right)$ is a Gaussian convolution kernel with $\mu =0.05$, $s_j=\frac{j}{n}$ ($0\le j<n$), $n=1000$, and e is 5% additive white noise. The exact solution, which we aim to reconstruct from the noisy data $\mathit{d}$, is given as

$$x\left(t\right)=10\left(t-0.5\right)exp\left[-50{\left(t-0.5\right)}^2\right]-0.8+1.6t,t\in \left[0,1\right].$$

Figure 1a shows the projected components of the true solution ${\mathit{x}}_t$ in the uncentered active eigenspace ${\mathit{W}}_1$ and the centered active eigenspace ${\mathit{V}}_1$. In the uncentered eigenspace, the true solution lies almost entirely in the first eigenmode, while it predominantly lies in the second and sixth modes of the centered eigenspace. The relative error between projection ${\mathit{x}}_r$ with $r—$dimensional AS and the true solution ${\mathit{x}}_t$ is shown in Figure 1b. It can be seen that the uncentered AS provides more accurate projection, even with one active direction ($r=1$). The result also shows that the centered AS needs at least 10 active directions to start being comparable to the uncentered counterpart in terms of accuracy. These numerical observations verify the reordering and rotating results of Lemma 1.

The uncentered AS eigenvector reordering and rotating effects induced by the data $\mathit{d}$ are presented in

Table 2. $cos\left({\mathit{w}}_i,{\mathit{v}}_j\right)$ between first ten eigenvectors in ${\mathit{W}}_1$ and ${\mathit{V}}_1$.

	${\mathit{v}}_1$	${\mathit{v}}_2$	${\mathit{v}}_3$	${\mathit{v}}_4$	${\mathit{v}}_5$	${\mathit{v}}_6$	${\mathit{v}}_7$	${\mathit{v}}_8$	${\mathit{v}}_9$	${\mathit{v}}_{10}$
${\mathit{w}}_1$	−0.01	0.97	0	−0.02	0	0.22	0	−0.01	0	−0.02
${\mathit{w}}_2$	1	0.01	0	0	0	0	0	0	0	0
${\mathit{w}}_3$	0	0	1	0	0	0	0	0	0	0
${\mathit{w}}_4$	0	0.02	0	1	0	−0.01	0	0	0	0
${\mathit{w}}_5$	0	0	0	0	1	0	0	0	0	0
${\mathit{w}}_6$	0	−0.22	0	0.01	0	0.97	0	−0.01	0	−0.01
${\mathit{w}}_7$	0	0	0	0	0	0	−1	0	0	0
${\mathit{w}}_8$	0	0.01	0	0	0	0.01	0	1	0	0
${\mathit{w}}_9$	0	0	0	0	0	0	0	0	1	0
${\mathit{w}}_{10}$	0	0.02	0	0	0	0.01	0	0	0	1

, where we compute the cosine of the angle between the centered and uncentered active modes. As can be seen, ${\mathit{w}}_1$ is slightly rotated from ${\mathit{v}}_2$. That is, compared to the centered AS method, the uncentered AS, under the influence of the data, reorders the eigenvectors of the forward operator so that ${\mathit{v}}_2$ is in the first position and slightly rotates it to align better with the directions in which the data are informative. The most significant shortcoming of the DI method and others using the basis $\mathit{V}$ is that they misclassify ${\mathit{v}}_1$ as the most data-informed direction, while for the data under consideration, ${\mathit{v}}_2$ is much more informative. Indeed, the relative error in Figure 1b shows that the true parameter is almost orthogonal to ${\mathit{v}}_1$.

Figure 2 presents solutions for the 1D deconvolution problem with various combinations of active subspace dimension r and regularization parameter $\alpha$. For the one-dimensional active subspace $r=1$, the TSVD method is not able to reconstruct the solution since the first mode of ${\mathit{V}}_1$ contributes very little to the true solution. On the contrary, DIAS-A yields a reasonable solution, as its first mode is the most data-informative one. For large dimensional active subspace, the DIAS-A and TSVD solutions are almost the same. These observations are consistent with the discussion following Lemma 1. Recall that the uncentered eigenvectors are a reordering and rotation of the centered ones. By considering a sufficiently large number of modes to be data informed, the subspaces spanned by ${\mathit{V}}_1$ and ${\mathit{W}}_1$ become more similar. This can also be clearly seen in Table 2 along the diagonal. Notice that ${\mathit{v}}_i\approx {\mathit{w}}_i$ except for the first two modes. This is because the first two eigenvectors are swapped and slightly rotated. Furthermore, inherited from the DI approach [21], DIAS-A, and TSVD (in fact cDIAS-A) solutions are almost invariant with respect to the regularization parameter. This is not surprising since the DIAS approach, by design, regularizes only the data-uninformed modes. Its solution should remain unaffected if sufficient data-informed modes are taken into account.

Figure 2 also shows that the DIAS-F, DI, and Tikhonov solutions are indistinguishable for small regularization parameter regardless of whether $r=1$ or $r=10$. This is due to two reasons: (1) the DIAS approaches and Tikhonov regularize inactive modes in the same way, and (2) Tikhonov regularization has little effect on the active modes when the regularization parameter is small, thus having little impact on the inverse solution. The situation changes for larger regularization parameters, especially for $r=10$. Both DIAS-F and DI, by leaving the active parameters untouched, are insensitive to the regularization while Tikhonov oversmooths the solution by heavily regularizing the active parameters.

Another important observation that Figure 2 conveys is the difference among DIAS approaches with centered and uncentered active subspaces. For $r=1$, the uncentered approaches DIAS-A and DIAS-F outperform the centered counterparts TSVD and DI, especially for large regularization parameters. In other words, uncentered approaches are more robust to the regularization parameter. The reason is that uncentered methods do not penalize the data-informed direction ${\mathit{w}}_1$ while the centered ones—without taking data into account—regularize ${\mathit{v}}_2$, the most data-informative direction in the basis $\mathit{V}$. As discussed in Section 2, for sufficiently large active subspace dimension $r=10$, all the DIAS solutions are similar since all methods end up spanning the same subspace. However, at the optimal regularization parameter $\alpha =1$, determined by the L-curve method [38], the DIAS-F solution is visibly closer to the ground truth than TSVD and Tikhonov.

5.2. Benchmark Problems

In this section, we apply the DIAS regularization approach with centered AS and uncentered AS methods to six benchmark problems from regularization tools [38]. We briefly describe the Shaw benchmark [40] here, and we refer the readers to [38] for the descriptions of other benchmark problems. It is in fact a deconvolution problem in which the kernel is given as

$$a(s,t)=\left(cos\left(s\right)+cos\left(t\right)\right){\left(\frac{sin\left(u\right)}{u}\right)}^2,$$

where $u=\pi \left(sin\left(s\right)+sin\left(t\right)\right)$, and the true solution is given by

$$x\left(t\right)=a_{1}exp\left(-c_1{\left(t-t_1\right)}^2\right)+a_{2}exp\left(-c_2{\left(t-t_2\right)}^2\right),$$

where $a_1=2,c_1=6,t_1=0.8,a_2=1,c_2=2,t_2=-0.5$. For all benchmark problems, the domain $\left[0,1\right]$ is divided into 1000 subdomains. These problems provide good examples to study the robustness and accuracy properties of the DIAS approaches. Observational data for each problem are corrupted with 1% additive white noise. Figure 3 measures the relative error between the ground truth ${\mathit{x}}_t$ and its projection ${\mathit{x}}_r$ on both centered and uncentered active subspaces with various values of the active subspace dimension r. For Shaw, heat, gravity, and Phillips problems, projecting the exact solution on the uncentered AS results in a lower error than on the centered AS. For the Deriv2 and Foxgood problems, the results are identical for both centered and uncentered AS. The reason is that the data for these two problems do not provide new information and thus the active subspaces are entirely determined by the forward operator $\mathit{A}$.

Figure 4, Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9 show the following: (1) top row—relative errors in the inverse solutions for different regularization methods for two values of regularization parameter (${\alpha }_{optimal}$ and $500\times {\alpha }_{optimal}$); (2) middle row—inverse solutions with Tikhonov regularization and DIAS regularizations with one dimensional active subspace ($r=1$) for two values of regularization parameter; and (3) bottom row—inverse solutions with Tikhonov regularization and DIAS regularizations with optimal active subspace dimension for two values of the regularization parameter. Here, the optimal regularization parameters are chosen based on the L-curve method with Tikhonov regularization [38], and the optimal AS dimension is found experimentally for each method. It turns out that the optimal AS dimension is the same for all methods.

Around the optimal AS dimension (top rows of all figures), regardless of what the regularization parameter is, all DIAS regularizations have similar accuracy and they are more accurate than Tikhonov, as expected. As can be seen in the middle rows, when the active subspace dimension is from $r=1$ to $r=6$, less than the optimal dimension $r_{\mathrm{optimal}}=7$ for the Shaw problem, the full data misfit methods outperform the approximated misfit counterparts. We provide the reason only for the Shaw problem, as it is the same for others. When taking $r=1$, the approximate misfit approaches completely remove six other active modes, which are misclassified as inactive, in addition to removing the truly inactive modes. The inverse solutions thus lack the important contribution from these modes, leading to inaccurate reconstructions. Even when the active subspace is chosen to be too small, the full misfit methods only regularize the misclassified modes rather than eliminating them from the solution entirely.

The results also show that the DIAS regularizations with full data misfit are at least as good as Tikhonov regularization with ${\alpha }_{optimum}$ and are more accurate with $500\times {\alpha }_{optimum}$. Again, the reason is that for a reasonably small regularization parameter, the smoothing effect from Tikhonov regularization does not change the solution significantly. On the bottom rows where the AS dimensions are optimal, DIAS solutions are similar and outperform the Tikhonov counterparts for both values of regularization parameters.

From middle to bottom rows, the DIAS solutions with approximate data misfit change from worse to comparable to the DIAS solutions with original data misfit, and thus from worse to more accurate than Tikhonov solutions. This is not surprising: Equation (2) shows that as the active subspace dimension increases, the error due to the data misfit approximation decreases. On the other hand, for the same reasons as in the deconvolution problem, the uncentered AS methods (DIAS-A and DIAS-F) are more accurate than the corresponding centered AS counterparts (TSVD and DI) for all problems with any active subspace dimension and with any regularization parameter.

Note that for these benchmarks, we take $500\times {\alpha }_{\mathrm{optimal}}$ as a large regularization parameter case to show that DIAS regularization is robust with respect to the choice of regularization parameter while Tikhonov is not. The DIAS solutions are much more accurate than the Tikhonov solutions, as the latter overregularizes all modes in this case. Additionally, we observe that the full data misfit approaches become similar to the approximate misfit ones as the regularization parameter increases. To see why, we recall from Equation (7) that the only difference between approximate and full misfit approaches is the removal of the inactive variables in the former. When the regularization parameter approaches infinity, it annihilates the contribution of the inactive subspace in the inverse solution. In fact, they behave like TSVD in the limit and we know that TSVD is equivalent to applying infinite regularization on the truncated modes [21]. Thus both approaches would yield identical solutions in the limit. For the case we consider here, the regularization is sufficiently large for us to already see this asymptotic behavior.

5.3. X-ray Tomography

X-ray tomographic reconstruction is another well-known linear inverse problem. A more detailed description of this problem can be found in [9]. Synthetic data are generated with 1%, 3% and 5% white noise added to be realistic. Tikhonov solutions using the optimal regularization parameter $\alpha =1.3\times {10}^5$, obtained by the L-curve method [9], are compared with DIAS approaches.

Figure 10 depicts the eigenvalues of $\mathit{C}$ and $\tilde{\mathit{C}}$. While the first eigenvalue of the uncentered active subspace matrix is significantly larger than the first eigenvalue of the centered active subspace matrix, this only hints that ${\mathit{w}}_1$ is more important than any of the vectors in $\mathit{V}$. Figure 11 shows that there is indeed a striking difference between the first eigenvector of the two active subspaces. Visual inspection of the eigenvectors ${\mathit{w}}_1$ and ${\mathit{v}}_1$ makes it obvious that ${\mathit{w}}_1$ contributes significantly to the solution, while the contribution of ${\mathit{v}}_1$ to the solution is much less pronounced.

The Tikhonov solutions with different regularization parameters and the original image are shown in the Figure 12 for the case of $1\%$ noise. Clearly, underregularization results in noisy solutions and overregularization yields overly smoothed and blurry solutions.

Table 3. Relative error (%) of inverse solutions by DIAS and Tikhonov regularization for underregularization case with $\alpha ={10}^2$, noise level 1%.

Method	Number of Dimensions of Active Subspace
	r = 1	r = 500	r = 2000	r = 5000	r = 10,000	r = 12,000
DIAS-A	79.47	49.31	36.13	27.39	22.74	35.36
TSVD	84.58	56.50	40.17	29.10	22.98	35.36
DIAS-F	389.15	389.16	389.15	389.16	389.13	389.01
DI	389.15	389.16	389.15	389.17	389.13	389.01
Tikhonov	389.15

and Figure 13 show the relative error and the reconstructed images for the underregularization case. The approximated misfit methods (DIAS-A and TSVD) remove the inactive variables so they are not prone to noise pollution amplified by inverting the small singular values corresponding to these data-uninformed modes. This has a regularization effect known as regularization by truncation. Since the full misfit approaches and Tikhonov capture all modes, their solutions are much more vulnerable to noise in the underregularization regime. For optimal regularization case in

Table 4. Relative error (%) of inverse solutions by DIAS and Tikhonov regularization for optimal regularization case with ${\alpha }_{\mathrm{optimal}}=1.3\times {10}^5$, noise level 1%.

Method	Number of Dimensions of Active Subspace
	r = 1	r = 500	r = 2000	r = 5000	r = 10,000	r = 12,000
DIAS-A	79.47	49.31	36.13	27.39	22.74	35.36
TSVD	84.58	56.50	40.17	29.10	22.98	35.36
DIAS-F	21.35	21.22	21.08	21.05	22.17	35.39
DI	21.36	21.33	21.24	21.19	22.21	35.39
Tikhonov	21.36

and Figure 14, the full misfit methods DIAS-F and DI are more accurate than their approximate misfit counterparts for small active subspace dimensions, while all methods give similar results when the active subspace dimension is sufficiently large. For the overregularization case in

Table 5. Relative error (%) of inverse solutions by DIAS and Tikhonov regularization for overregularization case with $\alpha ={10}^8$, noise level 1%.

Method	Number of Dimensions of Active Subspace
	r = 1	r = 500	r = 2000	r = 5000	r = 10,000	r = 12,000
DIAS-A	79.47	49.31	36.13	27.39	22.74	35.36
TSVD	84.58	56.50	40.17	29.10	22.98	35.36
DIAS-F	74.29	48.65	35.91	27.32	22.73	35.36
DI	78.08	55.6	39.89	29.01	22.97	35.36
Tikhonov	80.77

and Figure 15, the approximate and full misfit approaches provide comparable solutions, as they both behave like the TSVD method. Moreover, DIAS regularization solutions are robust to the regularization parameter as opposed to the Tikhonov solution. Another observation is that DIAS methods perform poorly when the active subspace dimension is taken to be too large ($r=$ 12,000). This is not surprising since as $r\to p$, all regularization is removed and the problem becomes ill-posed again.

To better understand the robustness of the DIAS approaches to various noise levels, we perform a sensitivity analysis, increasing the noise levels to 3% and 5%. One might expect that the DIAS method would be especially sensitive to noise since it is a data-driven approach. The following discussion shows, however, that the DIAS approaches maintain their advantages, even in the presence of significant noise. The results with higher noise levels are presented in

Table 6. Relative error (%) of inverse solutions by DIAS and Tikhonov regularization for underregularization case with $\alpha ={10}^2$, noise level 3%.

Method	Number of Dimensions of Active Subspace
	r = 1	r = 500	r = 2000	r = 5000	r = 10,000	r = 12,000
DIAS-A	79.47	49.42	37.32	33.95	48.46	97.10
TSVD	84.58	56.57	41.05	35.10	48.53	97.10
DIAS-F	371.83	371.82	371.81	371.80	371.81	372.87
DI	371.83	371.83	371.83	371.81	371.81	372.88
Tikhonov	371.83

,

Table 7. Relative error (%) of inverse solutions by DIAS and Tikhonov regularization for underregularization case with $\alpha =1.35\times {10}^5$, noise level 3%.

Method	Number of Dimensions of Active Subspace
	r = 1	r = 500	r = 2000	r = 5000	r = 10,000	r = 12,000
DIAS-A	79.47	49.42	37.32	33.95	48.46	97.10
TSVD	84.58	56.57	41.05	35.10	48.53	97.10
DIAS-F	33.99	32.24	30.89	32.77	48.42	97.10
DI	34.17	33.56	32.11	33.30	48.46	97.10
Tikhonov	34.17

and

Table 8. Relative error (%) of inverse solutions by DIAS and Tikhonov regularization for underregularization case with $\alpha ={10}^8$, noise level 3%.

Method	Number of Dimensions of Active Subspace
	r = 1	r = 500	r = 2000	r = 5000	r = 10,000	r = 12,000
DIAS-A	79.47	49.42	37.32	33.95	48.46	97.10
TSVD	84.58	56.57	41.05	35.10	48.52	97.10
DIAS-F	78.77	49.34	37.30	33.95	48.46	97.10
DI	83.68	56.47	41.02	35.09	48.52	97.10
Tikhonov	95.19

for 3%, and

Table 9. Relative error (%) of inverse solutions by DIAS and Tikhonov regularization for underregularization case with $\alpha ={10}^2$, noise level 5%.

Method	Number of Dimensions of Active Subspace
	r = 1	r = 500	r = 2000	r = 5000	r = 10,000	r = 12,000
DIAS-A	79.47	49.54	39.03	43.49	77.18	161.65
TSVD	84.58	56.69	42.62	44.39	77.22	161.62
DIAS-F	326.77	326.77	326.78	326.77	326.69	332.71
DI	326.77	326.77	326.78	326.78	326.69	332.71
Tikhonov	326.77

,

Table 10. Relative error (%) of inverse solutions by DIAS and Tikhonov regularization for underregularization case with $\alpha =1.35\times {10}^5$, noise level 5%.

Method	Number of Dimensions of Active Subspace
	r = 1	r = 500	r = 2000	r = 5000	r = 10,000	r = 12,000
DIAS-A	79.47	49.54	39.03	43.49	77.18	161.65
TSVD	84.58	56.69	42.62	44.39	77.22	161.62
DIAS-F	43.55	38.73	35.93	43.08	77.19	161.65
DI	44.05	41.73	38.07	43.73	77.22	161.62
Tikhonov	44.06

and

Table 11. Relative error (%) of inverse solutions by DIAS and Tikhonov regularization for underregularization case with $\alpha ={10}^8$, noise level 5%.

Method	Number of Dimensions of Active Subspace
	r = 1	r = 500	r = 2000	r = 5000	r = 10,000	r = 12,000
DIAS-A	79.47	49.54	39.03	43.49	77.18	161.65
TSVD	84.58	56.69	42.62	44.39	77.22	161.61
DIAS-F	79.21	49.52	39.02	43.49	77.18	161.64
DI	84.25	56.66	42.61	44.38	77.22	161.61
Tikhonov	98.06

for 5%. It can be observed that approximately the first 500 modes of the active subspace are not perturbed by noise (1%, 3% and 5%). For example, at $r=500$, the relative error of DIAS-A and TSVD are approximately 49% and 56%, respectively, for all noise levels. Meanwhile, when $r=5000$, the corresponding figures for both approaches in the cases of 1%, 3% and 5% are (27.39%, 29.10%), (33.95%, 35.10%) and (43.49 % and 44.39%), respectively. This is consistent with the well-known fact that higher modes of the active subspace (corresponding to smaller eigenvalues) contain more noise. In other words, as the noise level increases, the 5000-th mode moves to the inactive side of the space because it becomes dominated by noise rather than useful information.

6. Conclusions

We have presented four different data-informed regularization approaches for linear inverse problems using active subspaces (DIAS): two with approximate data misfit, and two with full data misfit. We rigorously showed the connection between the centered and uncentered active subspaces and the consequences on the performance of the corresponding DIAS approaches. For linear inverse problems, we showed that the TSVD and the DI regularization methods are members of DIAS regularization. Regularizing only the inactive directions is fundamental to the success of the DIAS approaches. All four DIAS regularization methods are robust to a wide range of regularization parameter values and outperform Tikhonov regularization for many choices of regularization parameter. The uncentered DIAS approaches are more robust and more accurate than their centered counterparts (the TSVD and DI approaches). Among DIAS regularizations methods, DIAS-F (uncentered active subspace with the original data misfit) has the best compromise: it is a data-informed non-intrusive approach.

By data-informed, we mean that the method balances the information encoded in the forward operator and information gained from the particular realization of the data. By non-intrusive, we mean that the method provides the ability to reuse existing inverse codes with minor modification only on the regularization. Various numerical results have demonstrated that the DIAS-F approach is the most effective method presented. In particular, excellent results can be obtained from DIAS-F with only a one-dimensional data informed subspace.

For problems with significant noise in the data, DIAS regularization methods could result in noisy reconstructions unless a small number of active directions are taken, since the less data-informed directions, reflecting the noise in the data, still need some regularization to smooth out the noise. Ongoing work is to equip the DIAS regularization approach with a mechanism to combat high noise scenarios while ensuring the fidelity of inverse solutions. Another appealing feature of the DIAS framework is that it is applicable not only for linear, but also seamlessly for nonlinear inverse problems. Detailed numerical investigations and rigorous analysis of the DIAS approach for nonlinear inverse problems are part of ongoing work.