Hybrid Deep Learning and Sensitivity Operator-Based Algorithm for Identification of Localized Emission Sources

Abstract

Hybrid approaches combining machine learning with traditional inverse problem solution methods represent a promising direction for the further development of inverse modeling algorithms. The paper proposes an approach to emission source identification from measurement data for advection–diffusion–reaction models. The approach combines general-type source identification and post-processing refinement: first, emission source identification by measurement data is carried out by a sensitivity operator-based algorithm, and then refinement is done by incorporating a priori information about unknown sources. A general-type distributed emission source identified at the first stage is transformed into a localized source consisting of multiple point-wise sources. The second, refinement stage consists of two steps: point-wise source localization and emission rate estimation. Emission source localization is carried out using deep learning with convolutional neural networks. Training samples are generated using a sensitivity operator obtained at the source identification stage. The algorithm was tested in regional remote sensing emission source identification scenarios for the Lake Baikal region and was able to refine the emission source reconstruction results. Hence, the aggregates used in traditional inverse problem solution algorithms can be successfully applied within machine learning frameworks to produce hybrid algorithms.

1. Introduction

Hybrid inverse modeling algorithms combining traditional approaches with machine learning techniques are promising to improve inverse modeling results. A review of the applications of machine learning methods to inverse problems can be found in [1,2,3,4,5].

Source identification problems are important in air quality applications since the available information about the distributions of emission sources (“right-hand side” of transport and transformation model equations) [6,7] is a key factor in air quality modeling and assessment. Emission sources can be localized or distributed over some area. Typical localized source classes are point-wise, linear, and piece-wise-linear sources. Examples of regional point-wise sources may be factories, power plants, accidental emission sources, forest fires, etc. Examples of linear sources are transportation routes. In the present paper, we consider multiple point-wise emission sources emitting specific substances.

The problem of using sparsity assumptions in general algorithms is well known in inverse problem theory and practice [1,8,9,10]. It can be approached, for example, by adding some special stabilizers to the Tikhonov functional. In this case, a solution is sought as a general distributed function, but the stabilizers make sparser solutions (or solutions with smaller supports) more preferable.

Another option is to explicitly include a detailed description of unknown sources (for example, the sources are assumed to be point-wise) into the inverse problem statement. In this case, we have the task of finding several coordinates in space and time and the emission rate [11,12,13,14,15,16]. A drawback of this approach is that the resulting algorithms, as a rule, are more specialized and their application to other types of sources may be impossible or require a redesign.

In this paper, we present a post-processing algorithm with deep learning that aims to improve the quality of emission source identification by applying a priori information about the sources. A review of machine learning approaches to air quality forecasting can be found in [17,18]. Machine learning approaches to source identification are analyzed in [17,19,20,21,22,23,24,25]. According to [19], deep learning can be used in source identification (data-driven source term estimation) to construct surrogate models for operators mapping unknown functions to measurement data to be used in inversion algorithms [20] or to construct surrogates of inverse operators [25,26] mapping measurements to source parameters. Hybrid schemes including physical models to obtain the simulation–observation difference and long short-term memory (LSTM) and visual geometry group (VGG) artificial neural networks (ANN) are developed in [21]. In [22], a U-net-based model is trained to predict NOx emissions using satellite and in situ data. In [23,27], a multi-layer perceptron (MLP) is trained using solutions of a direct problem to localize a single point-wise emission source by measurement data. In [24], an ANN-based inversion operator for concentration measurement data is trained using a number of independent scenarios describing possible hazardous gas leakages in the chemical industry. In [19], a federated learning framework is proposed to conduct data-driven source term estimation.

In our previous works, we developed a method of general-type source identification for advection–diffusion–reaction models based on sensitivity operators and adjoint ensembles [28,29], implemented in a software package called the Inverse Modeling and Data Assimilation Framework (IMDAF). The approach stems from works by G.I. Marchuk [30] and others [13,31,32,33]. In this approach, an inverse problem is transformed into a quasi-linear operator equation with a sensitivity operator constructed from an ensemble of adjoint equation solutions. An explicitly evaluated sensitivity operator can be used to both solve the inverse problem and predict the results of reconstruction provided by a “true” solution [29]. The latter feature is useful in monitoring network design and analysis and will be further used in the present work. The idea of the work is to refine the reconstruction results by cleaning it of the artifacts introduced into the “true” solution via the reconstruction procedure, and a sensitivity operator is used to describe these artifacts.

The objective of the present work is to test a deep learning approach in extracting point-wise emission sources from the results of a general sensitivity operator-based source identification algorithm. The paper is organized as follows. Section 2.1 defines direct, inverse, and refinement problems. Section 2.2 describes briefly a sensitivity operator-based source identification algorithm. In Section 2.3, an estimate of the reconstruction results is given as a function of a “true” solution. Section 2.4 describes a validation scenario for a regional emission source identification problem. A deep-learning-based refinement algorithm is presented in Section 2.5. Section 2.6 describes the neural networks used in the refinement algorithm. Section 3 demonstrates the numerical results obtained with the refinement algorithm. Section 4 contains a discussion of the algorithm and its numerical results. Section 5 summarizes the results of the paper.

2. Methods

2.1. Problem Statements

The chemical transport model for $N_c$ reacting substances is defined in a domain ${\Omega }_T=\Omega \times (0,T)$, where $\Omega$ is a sufficiently smooth approximation of a bounded rectangular domain $[0,X]\times [0,Y]$ in ${\mathbb{R}}^2$, $T>0$. ${\Omega }_T$ is bounded by $\partial {\Omega }_T=\partial \Omega \times [0,T]$.  

$$\begin{array}{c}\frac{\partial {\phi }_l}{\partial t}-\nabla ·\left(\mathrm{diag}\left({\mathit{\mu }}_l\right)\nabla {\phi }_l-\mathbf{u}{\phi }_l\right)+P_l(t,\mathit{\phi }){\phi }_l={\Pi }_l(t,\mathit{\phi })+f_l+r_l,(\mathbf{x},t)\in {\Omega }_T,\end{array}$$

(1)

$$\begin{array}{c}\mathbf{n}·\left(\mathrm{diag}\left({\mathit{\mu }}_l\right)\nabla {\phi }_l\right)+{\beta }_l{\phi }_l={\alpha }_l,(\mathbf{x},t)\in {\Gamma }^{\left(out\right)}\subset \partial \Omega \times [0,T],\end{array}$$

(2)

$$\begin{array}{c}{\phi }_l={\alpha }_l,(\mathbf{x},t)\in {\Gamma }^{\left(in\right)}\subset \partial \Omega \times [0,T],\end{array}$$

(3)

$$\begin{array}{c}{\phi }_l={\phi }_l^0,\mathbf{x}\in \Omega ,t=0,\end{array}$$

(4)

$$\begin{array}{c}l=1,\dots ,N_c,\end{array}$$

(5)

where t is time and $\mathbf{x}$ is a space coordinate, ${\phi }_l={\phi }_l(\mathbf{x},t)$ denotes the concentration of the $l^{th}$ substance at a point $(\mathbf{x},t)\in {\Omega }_T$, $\mathbf{\phi }$ is the vector of ${\phi }_l(\mathbf{x},t)$ for $l=1,\dots ,N_c$, which is called the state function, $L=\{1,\dots ,N_c\}$. The functions ${\mathit{\mu }}_l(\mathbf{x},t)\in {\mathbb{R}}^2$ correspond to the diffusion coefficients, $\mathrm{diag}\left(\mathbf{a}\right)$ is the diagonal matrix with the vector $\mathbf{a}$ on the diagonal, $\mathbf{u}(\mathbf{x},t)\in {\mathbb{R}}^2$ is the wind speed vector at the surface level. ${\Gamma }^{\left(in\right)}$ and ${\Gamma }^{\left(out\right)}$ are the parts of domain boundary $\partial {\Omega }_T$ in which the vector $\mathbf{u}(\mathbf{x},t)$ points inwards to the domain ${\Omega }_T$, and it is zero or points outwards to the domain ${\Omega }_T$, correspondingly, and $\mathbf{n}$ is the outer normal. The functions ${\alpha }_l(\mathbf{x},t)$, ${\phi }_l^0\left(\mathbf{x}\right)$ describe the boundary and initial conditions, correspondingly; ${\beta }_l$ is the boundary condition parameter, $f_l(\mathbf{x},t)$ is the a priori known source function, and $r_l(\mathbf{x},t)$ is the unknown source function. $P_l,{\Pi }_l:[0,T]\times {\mathbb{R}}_{+}^{N_c}\to {\mathbb{R}}_{+}$ are loss and production operator elements defined by the transformation model (see Section 2.4).

The set of admissible emission sources is denoted by Q. As the direct problem operator $\mathit{\phi }\left[\mathbf{q}\right]$, we consider the operator that maps the emission source distribution function $\mathbf{q}\in Q$ ($\mathbf{q}$ is the vector of $r_l(\mathbf{x},t)$ for $l=1,\dots ,N_c$) to the solution $\mathit{\phi }$ of (1)–(5). In our case, the emission sources Q are constant in time, and only a specific set of substances $L_{src}\subset L$ is emitted. Let there be an “exact” source function ${\mathbf{q}}^{(\ast )}\in Q^{(\ast )}\subset Q$ to be found, and let ${\mathit{\phi }}^{(\ast )}=\mathit{\phi }\left[{\mathbf{q}}^{(\ast )}\right]$ be the corresponding solution of the direct problem with the source function ${\mathbf{q}}^{(\ast )}$. The subset $Q^{(\ast )}$ is defined by additional a priori information about the unknown sources. In our case, $Q^{(\ast )}$ is a set of multiple point-wise sources:

$$Q^{(\ast )}=\left\{\sum _{s=1}^S\sum _{l\in L_{src}}{q}_l^{\left(s\right)}{\mathbf{e}}_l\delta (x-x^{\left(s\right)})\delta (y-y^{\left(s\right)})|S\in {\mathbb{N}}_{+},q_l^{\left(s\right)}>0,(x^{\left(s\right)},y^{\left(s\right)})\in \Omega \right\},$$

where $\delta$ is the delta function and ${\mathbf{e}}_l$ is the l-th element of the canonical basis in ${\mathbb{R}}^{N_c}$. In the inverse problem, the source function ${\mathit{q}}^{(\ast )}$ has to be identified from the “Snapshot”measurements of ${\mathit{\phi }}^{(\ast )}$ at the final time moment T:

$$\left\{{\phi }_l^{(\ast )}(\mathbf{x},T)|\mathbf{x}\in \Omega ,l\in L_{meas}\subset L\right\}.$$

The details can be found in [29]. Let ${\mathbf{q}}^{\left(\infty \right)}\in Q$ be the reconstruction result of the source identification algorithm.

Let $\Lambda :Q^{(\ast )}\to Q$ be the mapping from a “true” solution ${\overline{\mathbf{q}}}^{(\ast )}$ to the result of its reconstruction ${\overline{\mathbf{q}}}^{\left(\infty \right)}$ by the source identification algorithm. To refine the reconstruction result ${\mathbf{q}}^{\left(\infty \right)}$, we need to invert $\Lambda$ on ${\mathbf{q}}^{\left(\infty \right)}=$$\Lambda \left({\mathbf{q}}^{(\ast )}\right)$ provided that the result of inversion is from $Q^{(\ast )}$. Since we consider the refinement operation as post-processing, we are able to use both ${\mathbf{q}}^{\left(\infty \right)}$ and the corresponding sensitivity operator.

2.2. Source Identification Algorithm

The general source identification algorithm is based on a quasi-linear representation of the source identification problem. If ${\mathbf{q}}^{(\ast )}$ is the exact solution of the source identification problem, $\mathbf{I}$ is the measurement data aggregated in the state-function form (i.e., it is equal to ${\mathit{\phi }}^{(\ast )}$ in the parts of ${\Omega }_T$ where there are measurements and zero everywhere else), and $\delta \mathbf{I}$ is its perturbation (i.e., measurement noise), then, for any U and $\mathbf{q}$, the following relation holds [28]:

$$M_U\left[{\mathbf{q}}^{(\ast )},\mathbf{q}\right]\left({\mathbf{q}}^{(\ast )}-\mathbf{q}\right)=H_U\mathbf{I}+H_U\mathit{\delta }\mathbf{I}-H_U\mathit{\phi }\left[\mathbf{q}\right],$$

(6)

where

$$\begin{array}{c}{M}_U\left[{\mathbf{q}}^{\left(2\right)},{\mathbf{q}}^{\left(1\right)}\right]\mathbf{z}=\sum _{\xi =1}^{\Xi }{\mathbf{e}}^{\left(\xi \right)}{〈S[{\mathbf{q}}^{\left(2\right)},{\mathbf{q}}^{\left(1\right)};{\mathbf{h}}^{\left(\xi \right)}],\mathbf{z}〉}_Q,\end{array}$$

(7)

$$\begin{array}{c}{H}_U\mathit{\phi }=\sum _{\xi =1}^{\Xi }{\mathbf{e}}^{\left(\xi \right)}{〈{\mathbf{h}}^{\left(\xi \right)},\mathit{\phi }〉}_H.\end{array}$$

(8)

Here, ${\mathbf{e}}^{\left(\xi \right)}$ is the $\xi$-th element of the canonical basis in ${\mathbb{R}}^{\Xi }$, and the scalar products are

$$\begin{array}{c}{〈\mathbf{a},\mathbf{b}〉}_H=\sum _{l=1}^{N_c}{\int }_0^T{\int }_{\Omega }{a}_l(\mathbf{x},t)b_l(\mathbf{x},t)d\mathbf{x}dt,\end{array}$$

(9)

$$\begin{array}{c}{〈\mathbf{a},\mathbf{b}〉}_Q=\sum _{l=1}^{N_c}{\int }_{\Omega }{a}_l\left(\mathbf{x}\right)b_l\left(\mathbf{x}\right)d\mathbf{x},\end{array}$$

(10)

$U={\left\{{\mathbf{h}}^{\left(\xi \right)}\right\}}_{\xi =1}^{\Xi }$ is a set of projection functions, and $S[{\mathbf{q}}^{\left(2\right)},{\mathbf{q}}^{\left(1\right)};\mathbf{h}]$ is a sensitivity function such that for any ${\mathbf{q}}^{\left(2\right)},{\mathbf{q}}^{\left(1\right)}\in Q$

$${〈S[{\mathbf{q}}^{\left(2\right)},{\mathbf{q}}^{\left(1\right)};\mathbf{h}],{\mathbf{q}}^{\left(2\right)}-{\mathbf{q}}^{\left(1\right)}〉}_Q={〈\mathbf{h},\mathit{\phi }\left[{\mathbf{q}}^{\left(\mathbf{2}\right)}\right]-\mathit{\phi }\left[{\mathbf{q}}^{\left(\mathbf{1}\right)}\right]〉}_H$$

(11)

holds. The sensitivity function is calculated by solving the adjoint problem determined by its source function $\mathbf{h}$ [28]. The quasi-linear operator Equation (6) can be used to solve and analyze the inverse problem by analyzing the properties of the sensitivity operator.

To process the Snapshot data, we use the following projection system:

$$\begin{array}{c}{\mathbf{h}}^{\left(\xi \right)}=C(X,{\theta }_x^{\left(\xi \right)},x)C(Y,{\theta }_y^{\left(\xi \right)},y)\delta (t-t^{\left(\xi \right)})\delta (l-l^{\left(\xi \right)}),\\ \xi =1,\dots ,{\Xi }_{Snapshot},\end{array}$$

where $\delta$ are appropriate delta functions over time and chemical substances, $C(X,\theta ,x)$ are elements of the cosine basis on the interval $[0,X]$:

$$C(X,\theta ,x)=\frac{1}{\sqrt{X}}\left\{\genfrac{}{}{0.0pt}{}{\sqrt{2}cos\left(\frac{\pi \theta x}{X}\right),\theta >0}{1,\theta =0}\right..$$

The projection system has two parameters, ${\Theta }_{Snapshot}^{\left(x\right)}$ and ${\Theta }_{Snapshot}^{\left(y\right)}$, which define the spatial resolution of the considered data. For any image, ${\theta }_x$ and ${\theta }_y$ range within $0,\dots ,{\Theta }_{Snapshot}^{\left(x\right)}-1$ and $0,\dots ,{\Theta }_{Snapshot}^{\left(y\right)}-1$, correspondingly. Hence, ${\Xi }_{Snapshot}=N_{Snapshot}\times {\Theta }_{Snapshot}^{\left(x\right)}\times {\Theta }_{Snapshot}^{\left(y\right)}$. The computational cost of solving (6) is proportional to the number of projection functions in U (see Section 2.4); therefore, it makes sense to minimize U.

The source identification algorithm in [29] solves (6) using a Newton–Kantorovich-type iterative algorithm with a sensitivity operator matrix inversion regularized by truncated singular value decomposition. The details can be found in [29] and in Section 2.3, where we try to use the explicit form of the iterations to estimate the reconstruction result.

For the numerical implementation, the grid domain ${\omega }_T$ is introduced in ${\Omega }_T$. The grid domain has $N_x\times N_y\times N_t$ grid points, and the spatial grid domain $\omega$ has $N_x\times N_y$ grid points. The differential and integral aggregates involved in the source identification algorithm are substituted by their finite-dimensional analogues. To solve a multi-dimensional advection–diffusion–reaction problem, a splitting scheme as in [34] is implemented. We use consistent numerical schemes for direct and adjoint problem solution in the sense that a discrete analogue of the sensitivity relation (11) also holds for the numerical schemes. In the rest of the paper, we assume all the aggregates to be finite-dimensional.

2.3. Estimating Reconstruction Results

Given an exact source function ${\mathbf{q}}^{(\ast )}$ or its relevant approximation, we can estimate its reconstruction result by using the kernel of the sensitivity operator [35] or a “model resolution matrix” [33,36]. The kernel of a linear operator is the subspace that is mapped to the zero vector. Hence, it is impossible to reconstruct information on ${\mathbf{q}}^{(\ast )}-\mathbf{q}$ that is in the kernel of $M_U\left[{\mathbf{q}}^{(\ast )},\mathbf{q}\right]$ by solving (6). On the contrary, information preserved in the right-hand side of (6) on the way from a source to its reconstruction via measurements can be represented by the orthogonal projector on the orthogonal complement to the sensitivity operator kernel and can be evaluated as

$${\rm Y}[{\mathbf{q}}^{\left(2\right)},{\mathbf{q}}^{\left(1\right)}]:=M^{\ast }{\left(M{M}^{\ast }\right)}^{†}M,$$

where $M=M_U[{\mathbf{q}}^{\left(2\right)},{\mathbf{q}}^{\left(1\right)}]$, $M^{\ast }$ is the adjoint of M, and ${\left(M{M}^{\ast }\right)}^{†}$ is the generalized Moore–Penrose inverse of $M{M}^{\ast }$. In [29], we used the aggregates

$$\begin{array}{cc}\hfill {\mathbf{q}}^{\left(p\right)}& ={\mathbf{q}}^{\left(0\right)}+{\rm Y}[{\mathbf{q}}^{\left(0\right)},{\mathbf{q}}^{\left(0\right)}]\left({\mathbf{q}}^{(\ast )}-{\mathbf{q}}^{\left(0\right)}\right),\hfill \\ \hfill {\mathbf{q}}^{\left(e\right)}& ={\mathbf{q}}^{\left(0\right)}+{\rm Y}[{\mathbf{q}}^{(\ast )},{\mathbf{q}}^{\left(0\right)}]\left({\mathbf{q}}^{(\ast )}-{\mathbf{q}}^{\left(0\right)}\right),\hfill \end{array}$$

to estimate the reconstruction error $∥{\mathbf{q}}^{\left(\ast \right)}-{\mathbf{q}}^{\left(\infty \right)}∥$ for different types of measurement data without explicitly solving the inverse problem. Here, ${\mathbf{q}}^{\left(0\right)}$ is an initial guess, (which is zero). In [29], we studied the quality of these estimates experimentally and they showed similar performance.

To invert $\Lambda$ on ${\mathbf{q}}^{\left(\infty \right)}=\Lambda \left({\mathbf{q}}^{\left(\ast \right)}\right)$, we need an estimate at least in the vicinity of unknown ${\mathbf{q}}^{(\ast )}$ (as a function of ${\mathbf{q}}^{(\ast )}$). A direct method of evaluating $\Lambda$ is to apply the reconstruction algorithm, which is time-consuming (see Section 2.4); therefore we need to construct a computationally cheaper estimate. To do this, let us look at the final iterations of the source reconstruction algorithm. According to [29], at the n-th iteration ${\mathbf{q}}^{\left[n\right]}$ of the IMDAF algorithm, we have

$$\begin{array}{c}{\mathbf{q}}^{[n+1]}-{\mathbf{q}}^{\left[n\right]}=\gamma \Theta \left(M^{\left[n\right]},\Sigma \right)M_U\left[{\mathbf{q}}^{(\ast )},{\mathbf{q}}^{\left[n\right]}\right]\left({\mathbf{q}}^{(\ast )}-{\mathbf{q}}^{\left[n\right]}\right),\\ \Theta (m,\Sigma ):=m^T{\left[m{m}^T\right]}_{\Sigma }^{†},\\ M^{\left[n\right]}=M_U\left[{\mathbf{q}}^{\left[n\right]},{\mathbf{q}}^{\left[n\right]}\right].\end{array}$$

Here, ${\left[m{m}^T\right]}_{\Sigma }^{†}$ denotes a regularized matrix inversion procedure based on a truncated SVD. We use the following notation for the r-pseudoinverse matrix [37] for a matrix $m{m}^T\in {\mathbb{R}}^{K\times K}$:

$${\left[m{m}^T\right]}_{\Sigma }^{†}=\sum _{l=1}^p\frac{{\mathbf{U}}_l}{s_l^2}{〈.,{\mathbf{U}}_l〉}_{{\mathbb{R}}^K},s_1^2/s_p^2\le \Sigma <s_1^2/s_{p+1}^2,$$

(12)

where ${〈.,.〉}_{{\mathbb{R}}^K}$ is the Euclidean scalar product in ${\mathbb{R}}^K$, ${\left\{{\mathbf{U}}_l\right\}}_{l=1}^{rank\left(m\right)}$ is the orthonormal system of left singular vectors of m, and $s_l$ are the singular values. Hence,

$$\begin{array}{c}{\mathbf{q}}^{[n+1]}={\mathbf{q}}^{\left[n\right]}+{\gamma }^{\left[n\right]}\Theta \left(M^{\left[n\right]},\Sigma \right)M^{\left[n\right]}\left({\mathbf{q}}^{(\ast )}-{\mathbf{q}}^{\left[n\right]}\right)+{\mathbf{w}}^{\left[n\right]},\\ {\mathbf{w}}^{\left[n\right]}={\gamma }^{\left[n\right]}\Theta \left(M^{\left[n\right]},\Sigma \right)\left(M_U\left[{\mathbf{q}}^{(\ast )},{\mathbf{q}}^{\left[n\right]}\right]-M^{\left[n\right]}\right)\left({\mathbf{q}}^{(\ast )}-{\mathbf{q}}^{\left[n\right]}\right).\end{array}$$

At the final steps of the algorithm, the whole nonzero singular spectrum is involved in inversion: $\Theta \left(m,\Sigma \right)\approx$$m^T{\left[m{m}^T\right]}^{†}$. Moreover, let us skip the quadratic ${\mathbf{w}}^{\left[n\right]}$ and let ${\gamma }^{\left[n\right]}\approx 1$. With these assumptions, we can obtain an estimate of ${\mathbf{q}}^{[n+1]}$ as a function of ${\mathbf{q}}^{(\ast )}$:

$${\mathbf{q}}^{[n+1]}\approx P^{\left(n\right)}\left({\mathbf{q}}^{(\ast )}\right),$$

where

$$P^{\left(n\right)}\left(\mathbf{q}\right)={\mathbf{q}}^{\left[n\right]}+{\rm Y}[{\mathbf{q}}^{\left[n\right]},{\mathbf{q}}^{\left[n\right]}]\left(\mathbf{q}-{\mathbf{q}}^{\left[n\right]}\right).$$

Following this logic, we consider $P^{\left(\infty \right)}\left(\mathbf{q}\right)$ as an estimate of $\Lambda \left(\mathbf{q}\right)$:

$$P\left(\mathbf{q}\right)=P^{\left(\infty \right)}\left(\mathbf{q}\right)={\mathbf{q}}^{\left(\infty \right)}+{\rm Y}[{\mathbf{q}}^{\left(\infty \right)},{\mathbf{q}}^{\left(\infty \right)}]\left(\mathbf{q}-{\mathbf{q}}^{\left(\infty \right)}\right).$$

In Section 2.4, we present a numerical evaluation of the estimation quality.

2.4. Validation Basis

We use the same “realistic” inverse modeling scenario as in [29], where the domain of study comprises the Baikal Natural Territory (Figure 1). In the experiments, we compare three scenarios with different locations of the emission sources: Figure 1a shows sources located in all the cities in the region (marked as “All”), Figure 1b shows relatively weak sources in the northern part of the domain (marked as “Invis”), and Figure 1c shows relatively strong sources in the southern part (marked as “Vis”). For the sake of simplicity, we eliminate the geographical background layer from the figures in the rest of the paper.

In the reaction model, we consider $N_c=5$ chemical species including $O_3$ and $NO$. We suppose that only $NO$ is emitted ($L_{src}=\left\{NO\right\}$) with different constant emission rates in the sites presented in Figure 1. Ozone ($O_3$) concentrations are measured ($L_{meas}=\left\{O_3\right\}$) as a snapshot of the concentration field at the final time moment. Measurement noise $\delta I=0$. The calculations are carried out on a grid of $N_x=57$ by $N_y=60$ points in space, corresponding to the geographical domain presented in Figure 1, with $N_t=4401$ points in a time period starting at 2019-07-23 T12:00:00 with a time step of 54 s.

As a target for refinement by means of deep learning, we consider the reconstruction results for $N_{Snapshot}=1$ and

$${\Theta }_{Snapshot}^{\left(x\right)}\times {\Theta }_{Snapshot}^{\left(y\right)}\in \left\{15\times 15,25\times 25,35\times 35,45\times 45\right\}$$

(13)

projection functions.

Figure 2 demonstrates $q^{(\infty )}$ (the result of reconstruction) and its estimate $P\left(q^{(\ast )}\right)$ for the “Invis” source configuration with different numbers of projection functions. Analyzing Figure 2a,c, we can see that some of the “true” emission sources are located inside of the reconstructed distributed sources, (producing “excessively smeared point-wise source errors”). We can also see that some of the “true” sources have no explicit corresponding objects in the reconstruction (“false negative errors”) and some of the reconstruction artifacts do not correspond to any “true” sources (“false positive errors”). The ideal refinement algorithm must fix all three types of errors. Another quantitative observation that we have in Figure 2b,d is that the reconstruction estimates have a similar structure in space to the corresponding reconstructions (Figure 2a,c), but the values are different (see the scales).

In Figure 3, we present the estimate $P\left({\mathbf{q}}^{(\ast )}\right)$ error ${\epsilon }_{defect}$ with respect to the “true” solution ${\mathbf{q}}^{(\ast )}$ for all the considered numerical experiments:

$${\epsilon }_{defect}=\frac{{∥P\left({\mathbf{q}}^{(\ast )}\right)-{\mathbf{q}}^{(\infty )}∥}_Q}{{∥{\mathbf{q}}^{(\ast )}∥}_Q}.$$

(14)

A significant error is expected since we estimate the nonlinear mapping $\Lambda \left(\mathbf{q}\right)$ by affine-type estimation $P\left(\mathbf{q}\right)$. For example, the differences between Figure 2a,b and Figure 2c,d correspond to the minimal and maximum relative errors, correspondingly.

In Figure 4, we present the computation times of the inverse problem solutions for the three source configurations and consider the number of the projection functions ${\Theta }_{Snapshot}^{\left(x\right)}={\Theta }_{Snapshot}^{\left(y\right)}$ (13). We carried out the calculations at the Siberian Supercomputer Center on the NKS-1P hybrid cluster by RSC Group, Moscow, Russia using three Intel Xeon Gold 6248R nodes (each has 2 CPU × 24 cores × 2 threads, 3.00 GHz, 384 GB RAM). The total number of cores used is 144. The nodes are connected with Cluster Interconnect Omni-Path 100 Gbps.

Analyzing Figure 4, we can see that the computation time is proportional to the number of projection functions; therefore, it makes sense to use deep learning to obtain a solution corresponding to more projection functions from a solution with less functions. This problem resembles a computer tomography inversion problem with incomplete data (limited-angle computer tomography). This problem can be approached using deep learning [38,39,40,41]. For example, in [38], the authors used a U-net-type convolutional neural network and a filtered back-projection computer tomography algorithm to work with incomplete data.

2.5. Deep Learning-Based Refinement of Inverse Modeling Results

The problem of refining the inverse modeling result ${\mathbf{q}}^{(\infty )}$ can be approximated by the problem of inversion of $P\left(\mathbf{q}\right)$ on $Q^{(\ast )}$

$${\mathbf{q}}^{(•)}=\underset{\mathbf{q}\in Q^{(\ast )}\subset Q}{argmin}{∥P\left(\mathbf{q}\right)-{\mathbf{q}}^{(\infty )}∥}_Q.$$

(15)

To measure the quality of the refinement results, we use two metrics: the interpretation quality

$$\epsilon \left(\mathbf{q}\right)=\frac{{∥P\left(\mathbf{q}\right)-{\mathbf{q}}^{(\infty )}∥}_Q}{{∥{\mathbf{q}}^{(\infty )}∥}_Q},$$

(16)

and the complexity of the results $\#\left(\mathbf{q}\right)$. We measure the complexity on $Q^{(\ast )}$ as the number of point-wise sources (or positive elements in the discrete setting).

First, let us look at the action of the operator P on point-wise sources. In Figure 5, we present samples of pairs $\left\{P\left(\mathbf{q}\right),\mathbf{q}\right\}$, $\mathbf{q}\in Q^{(\ast )}$ for ${\Theta }_{Snapshot}^{\left(x\right)}\times {\Theta }_{Snapshot}^{\left(y\right)}=15\times 15$ and ${\Theta }_{Snapshot}^{\left(x\right)}\times {\Theta }_{Snapshot}^{\left(y\right)}=45\times 45$, correspondingly. We note that the approximate reconstruction result corresponding to a smaller number of projection functions ${\Theta }_{Snapshot}^{\left(x\right)}\times {\Theta }_{Snapshot}^{\left(y\right)}=15\times 15$ contains coarser artifacts compared to the ${\Theta }_{Snapshot}^{\left(x\right)}\times {\Theta }_{Snapshot}^{\left(y\right)}=45\times 45$ case. Analyzing Figure 5, we note that the problem resembles a deconvolution problem, i.e., the restoration of the original image after the distortion represented by a local convolution operation [42].

A need for deconvolution arises in various areas—for example, in the compensation of blur caused by the movement of a camera or object at shooting [43], or the correction of optical artifacts in microscopy [44] and astronomy [45]. The case in which the distorting function (convolution operator) is known in advance is “non-blind” deconvolution. In the absence of noise in a distorted image, non-blind deconvolution is achieved by inverting the convolution operator analytically (inverse filtering [42]). For noisy images, this approach leads to unsatisfactory results due to the appearance of significant artifacts caused by the added “deconvolution of noise” [42]. In such cases, the original image is restored using optimization algorithms such as Wiener filtering [46] or Tikhonov regularization [47] (in the frequency domain) or the Luce–Richardson method [48] (in the spatial domain).

Deconvolution for an unknown distorting function (“blind”) is performed by an iterative process, at each step of which the evaluation of the distorting function is refined, and, on its basis, the original image is evaluated using the methods described above [49]. For the evaluation of the distorting functions that one can use, such as machine learning methods, see [50]. In this context, if we consider the linear part of P solely that is the operator ${{\rm Y}}^{(\infty )}={\rm Y}[{\mathbf{q}}^{(\infty )},{\mathbf{q}}^{(\infty )}]$ (which makes sense as it acts between “increments”), our problem will be close to the problem of non-blind deconvolution in the absence of noise, but with the following reservations:

• the distortion function is spatially variant (i.e., not a local convolution);
• the distortion operator has a non-empty kernel (i.e., inverse filtering is impossible);
• the high-quality restoration of images of the special “several point-wise source” class is of critical importance.

We look for a solution ${\tilde{\mathbf{q}}}^{(•)}\in Q^{(\ast )}$ that provides “adequate” interpretation quality and complexity defined by constants ${\epsilon }^{(•)}$ and $N_{max}$:

$$\begin{array}{c}\epsilon \left({\tilde{\mathbf{q}}}^{(•)}\right)<{\epsilon }^{(•)},\end{array}$$

(17)

$$\begin{array}{c}\#\left({\tilde{\mathbf{q}}}^{(•)}\right)\le N_{max}.\end{array}$$

(18)

In [51], we used deep learning only to obtain a solution of (15). In the present work, we split the problem into two steps: localizing the sources and recovering the emission rates. The reason for this modification is that, provided with the source locations, the estimation of the emission rates is a more straightforward problem. Hence, in the modified algorithm, the first step is performed with the help of deep learning; the second involves solving a quadratic programming problem using the source locations obtained at the first step.

For the localization of sources, we use a neural network operator

$$T_{\theta }:Q\to Q^{(\ast )},$$

where $\theta$ are trainable parameters that have to be estimated with training samples ${\overline{Q}}^{(\ast )}\subset Q^{(\ast )}$:

$${\theta }^{(\times )}=\underset{\theta \in \Theta }{argmin}\sum _{\mathbf{q}\in {\overline{Q}}^{(\ast )}}{\rho }_{Q^{(\ast )}}(T_{\theta }\left(P\left(\mathbf{q}\right)\right),\mathbf{q}),$$

where ${\rho }_{Q^{(\ast )}}$ is the training loss function. Due to the complexity of the cost (error) function, we can obtain different optimization results by setting different (random) initial and optimization algorithm parameters in the training procedure.

At this step, we control the complexity of the resulting solution ${\mathbf{q}}^{(\times )}=T_{{\theta }^{(\times )}}\left({\mathbf{q}}^{(\infty )}\right)$. If $\#\left({\mathbf{q}}^{(\times )}\right)\le N_{max}$ is satisfied, ${\mathbf{q}}^{(\times )}$ is returned as the result of the step; otherwise, we repeat the learning procedure with different initial and optimization algorithm parameters. The localization step can be summarized as an operator $L:Q\to Q^{(\ast )}$ that maps ${\mathbf{q}}^{(\infty )}\in Q$ to ${\mathbf{q}}^{(\times )}\in Q^{(\ast )}$ such that $\#\left({\mathbf{q}}^{(\times )}\right)\le N_{max}$.

To recover the emission rates, we solve a quadratic programming problem and describe this step as an operator $G:Q^{(\ast )}\to Q^{(\ast )}$. Let an element $\mathbf{q}$ of $Q^{(\ast )}$ be

$$\mathbf{q}=\sum _{s=1}^S\sum _{l\in L_{src}}{q}_l^{\left(s\right)}{\mathbf{e}}_l\delta (x-x^{\left(s\right)})\delta (y-y^{\left(s\right)}).$$

Then, we consider $\mathbf{q}=\mathbf{q}\left(\overline{q}\right)$ as functions of rates $\overline{q}={\left\{q_l^{\left(s\right)}\right\}}_{s=\overline{1,S},l\in L_{src}}$ with fixed locations $(x^{\left(s\right)},y^{\left(s\right)})$ and solve the constrained quadratic optimization problem

$${\overline{q}}^{(+)}=\underset{\overline{q}\ge 0}{argmin}{∥P\left(\mathbf{q}\left(\overline{q}\right)\right)-{\mathbf{q}}^{(\infty )}∥}_Q^2.$$

(19)

In our case, $L_{src}=\left\{NO\right\}$ and the cost function in (19) is equivalent to a quadratic polynomial of ${\left\{q^{\left(s\right)}\right\}}_{s=\overline{1,S},}$

$$\begin{array}{c}{∥P\left(\mathbf{q}\left(\overline{q}\right)\right)-{\mathbf{q}}^{\left(\infty \right)}∥}^2=\sum _{m=1}^S\sum _{l=1}^S{q}^{\left(m\right)}{q}^{\left(l\right)}〈{{\rm Y}}^{\left(\infty \right)}{\delta }_l,{{\rm Y}}^{\left(\infty \right)}{\delta }_m〉\\ -2\sum _{l=1}^S{q}^{\left(l\right)}〈{{\rm Y}}^{\left(\infty \right)}{\mathbf{q}}^{\left(\infty \right)},{{\rm Y}}^{\left(\infty \right)}{\delta }_l〉+〈{{\rm Y}}^{\left(\infty \right)}{\mathbf{q}}^{\left(\infty \right)},{{\rm Y}}^{\left(\infty \right)}{\mathbf{q}}^{\left(\infty \right)}〉,\end{array}$$

where ${{\rm Y}}^{\left(\infty \right)}={\rm Y}[{\mathbf{q}}^{\left(\infty \right)},{\mathbf{q}}^{\left(\infty \right)}]$, ${\delta }_l=\delta (x-x^{\left(l\right)})\delta (y-y^{\left(l\right)})$. Finally,

$$G:\mathbf{q}\mapsto \mathbf{q}\left({\overline{q}}^{(+)}\right).$$

Considering these two steps sequentially, we obtain an approximation of ${\mathbf{q}}^{(•)}$:

$${\tilde{\mathbf{q}}}^{(•)}=G\left(L\left({\mathbf{q}}^{(\infty )}\right)\right).$$

Finally, we check the first requirement (17) on ${\tilde{\mathbf{q}}}^{(•)}$. If $\epsilon \left({\tilde{\mathbf{q}}}^{(•)}\right)<{\epsilon }^{(•)}$, then ${\tilde{\mathbf{q}}}^{(•)}$ is considered as an acceptable solution; otherwise, the procedure is repeated from the first step with different parameters in the learning procedure until both requirements (17) and (18) are satisfied. The algorithm is summarized in Figure 6.

2.6. Convolutional Neural Networks

To specify the operator $T_{\theta }$ at the localization step of the refinement algorithm, we need to construct a neural network that processes the image, transforming “spots” into “points” and filtering specific artifacts. Convolutional neural networks (CNN) have been reported to successfully solve various image reconstruction [3,52] and image processing tasks, including segmentation [53,54], deblurring [55,56,57], and denoising [58,59]. CNNs can be used in image [60,61] and multivariate time-series (1D image) classification tasks [62,63].

Keeping in mind these works, we tried several CNNs of increasing complexity: CNN3, CNN4, and CNN9, presented in Figure 7 and

Table 1. CNN3, CNN4, and CNN9 layers.

				CNN3	CNN4	CNN9
#	Layer Type	Filters	Kernel Size	Number of Trainable Parameters
1	Conv2D	64	$9\times 9$		5248	5248
2	Conv2D	32	$7\times 7$	1600	100,384	100,384
3	Conv2D	16	$5\times 5$	12,816	12,816	12,816
4	Conv2D	1	$3\times 3$	145	145	145
5	Conv2DTranspose	1	$3\times 3$			10
6	Conv2DTranspose	16	$5\times 5$			416
7	Conv2DTranspose	32	$7\times 7$			25,120
8	Conv2DTranspose	64	$9\times 9$			165,952
9	Conv2DTranspose	1	$3\times 3$			577
Total number of trainable parameters				14,561	118,593	310,668

. Layers with a larger filter size allow us to identify areas with blurring and to “shrink” the areas to standalone points with a smaller one. The largest network, CNN9, has an encoder–decoder architecture; CNN3 and CNN4 consist of layers of CNN9’s encoder.

To construct and train the CNN operator $T_{\theta }$, we used standard Tensorflow layers and training (optimization) algorithms. The first layer is an $N_x\times N_y$ input layer and the last one is an $N_x\times N_y$ output layer. The following parameters are the same for all the layers: strides = 1, activation = ’relu’, padding = ’same’; ’Adamax’ is the optimizer [64], ’mse’ is the loss and metric function. To reduce the learning rate, when the metric has stopped improving, ReduceLROnPlateau is used with the following parameters: factor = sqrt(0.1), cooldown = 0, patience = 5, min_lr = $0.5\times {10}^{-6}$.

We used a number of training epochs of 10, interpretation quality parameter ${\epsilon }^{(•)}=0.65$, complexity threshold $N_{max}=500$. To construct a training set ${\overline{Q}}^{(\ast )}$, we use all single point-wise sources localized in different points of the grid domain $\omega$:

$${\overline{Q}}^{(\ast )}=\left\{\sum _{l\in L_{src}}{q}_l{\mathbf{e}}_l\delta (x-x^{\left(s\right)})\delta (y-y^{\left(s\right)})|q_l>0,(x^{\left(s\right)},y^{\left(s\right)})\in \omega \right\}.$$

Some examples of ${\overline{Q}}^{(\ast )}$ elements are presented in Figure 5. We used training and validation samples of $0.8N_x{N}_y$ and $0.2N_x{N}_y$ elements, correspondingly, with a batch size of $0.8N_x{N}_y/8$.

3. Results

Figure 8 shows the relative error and the relative complexity $\#\left({\tilde{\mathbf{q}}}^{(•)}\right)/\#\left({\mathbf{q}}^{(\ast )}\right)$ of the solutions provided by the above-proposed algorithms. Relative complexity equal to 1 implies that the numbers of point-wise sources in “true” and refined solutions are equal. The error of the CNN-refined projection-based estimation allows us to evaluate the quality of the pure inversion of P on $P\left({\mathbf{q}}^{(\ast )}\right)$.

Figure 9 and Figure 10 show the source identification results obtained with the general algorithm ${\mathbf{q}}^{(\infty )}$ and the results of its refinement ${\tilde{\mathbf{q}}}^{(•)}$ with CNN3, CNN4, and CNN9 for different numbers of projection functions and source configurations.

Analyzing the results in Figure 8, we conclude that, from a quantitative point of view, the best results with respect to relative errors and complexity are obtained with CNN4. In all the cases, the refined solutions have smaller errors compared to the reconstruction results (Figure 8c). CNN3 produced excessively complex solutions (Figure 8b) and, in some cases (with “small” numbers of projection functions and larger artifacts), the error of the refined solution is larger than the error of the rough reconstruction. The performance of CNN9 can be assessed as intermediate between the two. From a qualitative point of view (Figure 9 and Figure 10), the results are similar up to some artifacts.

Since “true” solutions are not known in advance, let us compare the results produced by the above-considered ANN architectures with characteristics that do not take known “true” solutions. Figure 11 demonstrates the interpretation quality $\epsilon \left({\tilde{\mathbf{q}}}^{(•)}\right)$ (16) (the less, the better) and complexity $\#\left({\tilde{\mathbf{q}}}^{(•)}\right)$ (the less, the better). Analyzing Figure 11, we conclude that CNN3 almost always provides better solutions with respect to interpretation quality, but significantly concedes in providing simple (or compact) solutions. CNN4 and CNN9 provide similar solutions with respect to complexity, but CNN4 is almost always better than CNN9 in terms of interpretation quality. These results are in accordance with the results shown in Figure 8.

4. Discussion

Analyzing Figure 8, we conclude that the algorithm provided a significant refinement to the source reconstruction results. We can see that the relative reconstruction error decreases with an increasing projection function number, which reflects the increasing measurement data considered. On the contrary, the refinement results do not show similar monotonicity with respect to the input data and the basic reconstruction quality. This means that the performance of the refinement algorithm depends on the different characteristics of the input data. It may be connected to the quality of the reconstruction estimates, which shows the opposite dependence (Figure 3) on the number of projection functions. This unclear dependence can be considered as a limitation of the algorithm.

The hybrid algorithm performance depends on the properties of the sensitivity operators (e.g., the non-linearity of a sensitivity operator likely impacts the quality of $\Lambda$’s approximations). The sensitivity operators and corresponding right-hand sides of quasi-linear operator equations (6) are defined by the emission source identification problem statement (Section 2.1), which is composed of atmospheric process models, a measurement system, and an unknown function choice. The atmospheric model’s parameters depend on the climatic and geographical conditions of the region under consideration. Therefore, these conditions impact the sensitivity operators in various ways. In this paper, we present an illustration of the approach in realistic conditions for the Baikal region. A further investigation is needed to identify which conditions have significant effects on the sensitivity operator’s properties that are important for the hybrid algorithm’s performance and to estimate its generalizability and reliability for other regions (e.g., in a city air quality inverse modeling scenario, as in [35]).

The key point of the algorithm is the ability to construct a relevant and computationally cheap reconstruction result approximation $P\left(\mathbf{q}\right)$. In this paper, we provide an example of such an estimate and test its performance in the numerical experiments. Other estimates can also be considered. To reduce the risk of using inaccurate estimates, a strategy including both $\Lambda \left(\mathbf{q}\right)$ and its estimate $P\left(\mathbf{q}\right)$ can be developed.

The sensitivity operator that is evaluated in sensitivity operator-based algorithms provides a natural way to generate training samples of any size for various classes of solutions. In this work, we considered a complete set of single point-wise sources as a training set for the localization stage of the refinement algorithm. In future work, we plan to consider training sets containing multiple point-wise sources and piece-wise linear sources. Since the number of such sources is substantially higher than the number of single point-wise sources, which is $N_x\times N_y$, some reproducible procedure of constructing an affordable yet representative set of training samples should be developed, e.g., with some type of pseudo-random sequence. Another interesting question is whether it is possible to distinguish the main types of sources (point-wise, linear, and distributed), thus solving a source classification problem.

In contrast to many deep learning applications, where there is a given sample set and ANNs have to correctly process different inputs, in the present paper, we have a direct model that can potentially produce a sample set of any size and a trained ANN has to process only one input ${\mathbf{q}}^{(\infty )}$. This setting, in some sense, corresponds to the task of constructing a surrogate of a complicated model. In these terms, we construct a surrogate of the inverse $\Lambda$ on $Q^{(\ast )}$ using its approximation P, deep learning, and quadratic programming.

Using standard deep learning terminology, we can formally consider $\{{\mathbf{q}}^{(\infty )},{\mathbf{q}}^{(\ast )}\}$ as a testing set consisting of a single element. In real applications, ${\mathbf{q}}^{(\ast )}$ is unknown, but we can potentially use multi-point sources in the vicinity of ${\mathbf{q}}^{(\ast )}$ to construct a larger testing set. This may be the subject of future research. We can also consider the whole set of the above-presented 12 numerical experiments as a testing set for a hybrid algorithm.

The risk of overfitting in the proposed framework is reduced by the following hybrid algorithm’s features. In our setting, the training samples are significantly different from the target input $\{{\mathbf{q}}^{(\infty )},{\mathbf{q}}^{(\ast )}\}$: each training sample corresponds to a single point-wise source, while the target sample contains multiple point-wise sources. Furthermore, according to the algorithm (Figure 6), in the case of unsatisfactory results (by criteria (17) and (18)) for a target input, we can abandon the refinement result ${\tilde{\mathbf{q}}}^{(•)}$ and look for another. Thanks to the criteria, instead of an ANN, we can potentially use a random generator or any appropriate metaheuristic algorithm, but the ANN is supposed to produce more targeted results, thus decreasing the number of attempts. In our case, typical overfitting appears as a trained ANN producing a zero output for a target input ${\mathbf{q}}^{(\infty )}$, which can be identified with (17).

Another important question is the choice of the training loss metric. We used the standard mean squared error (mse) since it is fast enough and works with both Q (distributed) and $Q^{(\ast )}$ (point-wise) functions in discrete settings. A limitation of mse for this setting is that it does not measure the distance between point-wise sources. The distance metrics of Hausdorff, Chamfer, or Wassershtein appear more suitable, but additional work is needed to obtain their sufficiently fast implementation for the comparison of distributed and point-wise sources.

To update the emission rates at the second step of the refinement algorithm, we use the estimate $P\left(\mathbf{q}\right)$. To obtain a better estimate of the emission rates, it is possible to use (6) or consider an emission rate identification problem with fixed source locations. Additionally, in this paper, we consider an algorithm with only one iteration of the refinement procedure. Refinement can be done iteratively, i.e., the procedure can be repeated for the residual:

$${\mathbf{q}}^{(\infty ,k+1)}={\mathbf{q}}^{(\infty ,k)}-P\left(G\left(L\left({\mathbf{q}}^{(\infty ,k)}\right)\right)\right).$$

Interpretation quality $\epsilon \left(\mathbf{q}\right)$ (the less, the better) is quantified by (16), involving the operator P. A more rigorous evaluation of the interpretation quality can be carried out by using the operator $\Lambda$ instead of P in (16), which uses the solution of the inverse problem and therefore is more time-consuming (Figure 4). Evaluating the direct problem operator $\phi \left[{\tilde{\mathbf{q}}}^{(•)}\right]$ and comparing the corresponding values to measurement data I directly would be less time-consuming and more appropriate in the context of a source identification problem.

The upper boundary ${\epsilon }^{(•)}$ of admissible interpretation quality in (17) was experimentally chosen to be small enough to avoid overfitted ANN results (e.g., ${\mathbf{q}}^{(\times )}=0$) and large enough to reduce the number of iterations due to the violation of (17). The parameter $N_{max}<<N_x\times N_y$ of the second criterion (18) was chosen to avoid “smeared” solutions (e.g., if ${\mathbf{q}}^{(\times )}={\mathbf{q}}^{(\infty )}$, then $P\left({\mathbf{q}}^{(\infty )}\right)={\mathbf{q}}^{(\infty )}$, $\epsilon \left({\mathbf{q}}^{(\infty )}\right)=0<{\epsilon }^{(•)}$ and ${\mathbf{q}}^{(\infty )}$ fits the first criterion (17), but $\#\left({\mathbf{q}}^{(\infty )}\right)>N_{max}$). A more automated procedure for the selection of criteria parameters can be considered as a direction of further research.

In this paper, we evaluated the refinement of a solution obtained by the basic source identification algorithm. In order to evaluate the performance of the whole hybrid algorithm in the context of source identification, it should be compared to specialized point-wise source identification algorithms (e.g., [11,12,13,14,15,16]). This can be considered as a future research direction.

The question of choosing the correct ANN architecture is also an important one. We present a comparison of the results produced by the above-considered CNN architectures based on relative errors, interpretation quality, and solution complexity, but further investigation is also needed in this direction.

In this paper, we consider image-type measurements, when a spatial domain is equivalently visible by a measurement system. In the case of localized measurements, spatially different illumination becomes an important issue [31,32,33]. In this case, a source reconstruction algorithm tends to place maxima of the reconstructed sources in measurement points. To fix this distortion, the authors of [31,32,33] introduced a re-normalization concept to preserve the locations of the maxima. A refinement algorithm for localized measurements will be the subject of future research. Another important question is measurement noise. Machine learning denoising may be applied in this direction. In this case, it may be implemented as a pre-possessing algorithm in the space of the measurement results.

5. Conclusions

In this paper, we numerically tested a source identification algorithm that combines a general-type emission source identification stage and a post-processing refinement stage. Emission source identification by measurement data is carried out with a sensitivity operator-based algorithm. At the post-processing stage, the general-type emission source identified at the first stage is transformed into a source consisting of multiple point-wise sources. The second stage consists of two steps: point-wise source localization and emission rate estimation. The first step is carried out using deep learning with convolutional neural networks. Training samples are generated using the sensitivity operator obtained at the first step. The algorithm was tested in regional remote sensing emission source identification scenarios for the Lake Baikal region and was able to refine the emission source reconstruction results.

Hybrid approaches combining machine learning with traditional inverse problem solution methods emerge as a promising direction for the further development of inverse modeling algorithms. Moreover, aggregates used in traditional inverse problem solution algorithms can be successfully applied within machine learning frameworks to produce hybrid algorithms.