Averaging of Linear Quadratic Parabolic Optimal Control Problem

Abstract

This paper studies an averaged Linear Quadratic Regulator (LQR) problem for a parabolic partial differential equation (PDE), where the system dynamics are affected by uncertain parameters. Instead of assuming a deterministic operator, we model the uncertainty using a probability distribution over a set of possible system dynamics. This approach extends classical optimal control theory by incorporating an averaging framework to account for parameter uncertainty. We establish the existence and uniqueness of the optimal control solution and analyze its convergence as the probability distribution governing the system parameters changes. These results provide a rigorous foundation for solving optimal control problems in the presence of parameter uncertainty. Our findings lay the groundwork for further studies on optimal control in dynamic systems with uncertainty.

1. Introduction

Optimal control problems for partial differential equations (PDEs) are widely used in engineering, physics, and economics to model systems that evolve over time [1]. Among these, the Linear Quadratic Regulator (LQR) problem has been extensively studied due to its well-established theoretical properties and practical applicability. The LQR framework provides an optimal strategy for controlling systems governed by linear PDEs while minimizing a quadratic cost functional [1,2]. Recent advances also explore approximate bounded feedback synthesis in parabolic PDEs with nonlinear perturbations and semidefinite performance criteria, extending classical LQR concepts to weakly nonlinear systems [3].

In this paper, we consider an averaged LQR problem for a parabolic PDE, where the system dynamics contain uncertain parameters. Instead of assuming a single deterministic system, we model the uncertainty using a probability distribution over a set of possible dynamics. This approach is inspired by previous studies in optimal control and stochastic averaging methods [4]. Related work addresses optimal regulation under rapidly oscillating parameters using homogenized models and superposition-type cost structures [5].

Reinforcement learning (RL) has become a cornerstone of modern machine learning, operating alongside supervised and unsupervised learning to solve decision-making problems under uncertainty. In this paradigm, agents learn optimal policies by interacting with an environment, optimizing a long-term performance criterion [6]. The connection between RL and classical optimal control theory has long been recognized [7], and recent developments in RL are now significantly influencing the field of control theory [8].

A key distinction within RL lies between model-free and model-based methods. Model-free approaches directly approximate value functions or policies without constructing a model of the environment, while model-based methods aim to learn a model from data and use it for planning [6]. The latter often suffer from model bias, a challenge identified early on [9]. To overcome this, the PILCO algorithm was proposed [10,11], which models system dynamics probabilistically using Gaussian processes and performs policy improvement based on expected trajectories.

PILCO and its extensions [12,13,14] operate within a Bayesian model-based RL framework that integrates data-driven learning with optimal control. These methods have demonstrated remarkable data efficiency and robustness and inspired numerous theoretical developments [15,16,17]. The formalization of such Bayesian approaches using distributions over system dynamics is central to understanding and reducing model uncertainty [18,19].

In this context, averaged optimal control emerges as a powerful framework that interprets the expected behavior over a distribution of dynamics. This idea has strong theoretical roots in the Riemann–Stieltjes optimal control setting [20,21,22] and averaged controllability [23,24]. Additionally, the challenge of maintaining stability in distributed control systems under destabilizing factors has motivated algorithmic approaches to resilient network synthesis [25], further highlighting the importance of robust control strategies. Our work is motivated by these formulations and aims to explore how solutions to averaged optimal control problems relate to those of deterministic counterparts.

Additionally, reinforcement learning in continuous-time systems is gaining momentum in control engineering [26,27,28,29]. These studies provide a basis for the development of algorithms that can operate in real-world, high-frequency environments.

This paper contributes to this growing body of work by investigating the convergence of optimal policies derived from averaged linear quadratic regulator (LQR) problems to those of classical LQR problems as the distribution over dynamics concentrates. Our approach complements existing algorithms such as PILCO [11] by offering a theoretical justification for their observed empirical success [18].

The main contributions of this paper are as follows:

• We establish the existence and uniqueness of optimal solutions for the averaged control problem.
• We analyze the convergence of optimal controls as the probability measure representing system uncertainty becomes more concentrated.

The structure of this paper is as follows: First, we define the mathematical formulation of the problem. Then, we present theoretical preliminaries and key functional analysis tools. The main results, including proofs of existence, uniqueness, and convergence, follow. Finally, we summarize our findings and discuss potential future directions.

2. Setting of the Problem

For unknown functions $y=y(t,x)$, $u=u(t,x)\in L^2\left(Q_T\right)$, $t\in [0,T]$, $x\in \Omega ⋐{\mathbb{R}}^d$, and $Q_T=(0,T)\times \Omega$, we consider the linear quadratic optimal control problem:

$$\left\{\begin{array}{cc}{\displaystyle \frac{\partial y(t,x)}{\partial t}}-L_{A}y(t,x)\hfill & =u(t,x),\hfill \\ {y|}_{x\in \partial \Omega }\hfill & =0,\hfill \\ {y|}_{t=0}\hfill & =y_0\left(x\right),\hfill \end{array}\right.$$

(1)

where the cost function is given by

$$\begin{array}{c}\hfill J_{\pi }\left(u\right)={\mathbb{E}}_{\pi }\left[{\int }_0^T\left({\int }_{\Omega }{y}^2(t,x)dx+{\gamma }_1{\int }_{\Omega }{u}^2(t,x)dx\right)dt+{\gamma }_2{\int }_{\Omega }{y}^2(T,x)dx\right]=\\ \hfill ={\int }_{\mathcal{A}}\left[{\int }_0^T\left({\int }_{\Omega }{y}^2(t,x)dx+{\gamma }_1{\int }_{\Omega }{u}^2(t,x)dx\right)dt+{\gamma }_2{\int }_{\Omega }{y}^2(T,x)dx\right]d\pi \left(A\right)\to inf,\end{array}$$

(2)

where

$$L_A=\sum _{i,j=1}^d{a}_{ij}{\displaystyle \frac{{\partial }^2}{\partial x_i\partial x_j}},a_{ij}=a_{ji},$$

matrix $A=\left\{a_{ij}\right\}$, and operator $L_A$ satisfies the uniform ellipticity condition given by

$$\sum _{i,j=1}^d{a}_{ij}{\xi }_i{\xi }_j\ge {\gamma }_3\sum _{i=1}^d{\xi }_i^2,\forall \xi \in {\mathbb{R}}^d.$$

(3)

The uniform ellipticity condition ensures the well-posedness of the PDE and is a standard assumption in the study of elliptic and parabolic operators.

Here, ${\gamma }_1$ and ${\gamma }_2$ are positive numbers that represent the weight coefficients in the objective functional, ${\gamma }_3$ is a positive number that defines the uniform ellipticity of a differential operator, and $y_0\in L^2(\Omega )$.

$\mathcal{A}$ is a set of symmetric matrices satisfying condition (3) (with one and the same constant ${\gamma }_3$).

$\pi$ is a probability measure on $\mathcal{A}$.

In the following, we denote

$$\begin{array}{c}\hfill \parallel y\parallel ={\left({\int }_{\Omega }{y}^2\left(x\right)dx\right)}^{\frac{1}{2}},\\ \hfill (y,z)={\int }_{\Omega }y\left(x\right)z\left(x\right)dx,\\ \hfill {\parallel u\parallel }_{L^2\left(Q_T\right)}={\left({\int }_{Q_T}{u}^2(t,x)dtdx\right)}^{\frac{1}{2}}.\end{array}$$

For $A\in \mathcal{A}$, we denote by $\parallel A\parallel$ the standard Euclidean norm.

We will prove that for every $\pi$ problem (1), (2) has a unique solution $\{{\overline{y}}_{\pi },{\overline{u}}_{\pi }\}$ (here, ${\overline{y}}_{\pi }$ depends on A), and if

$${\pi }^n\to {\pi }^{\infty }\mathrm{weakly},$$

(4)

then

$$\begin{array}{c}\hfill \forall A\in \mathcal{A},{\overline{y}}_{\pi }^n\to {\overline{y}}_{\pi }^{\infty }\mathrm{in}C([0,T];L^2(\Omega )),\\ \hfill {\overline{u}}_{\pi }^n\to {\overline{u}}_{\pi }^{\infty }\mathrm{in}{L}^2(\Omega ).\end{array}$$

3. Preliminary Results

It is known that for every $A\in \mathcal{A}$ and $u\in L^2\left(Q_T\right)$, problem (1) has a unique solution in the weak sense ([30] [Theorem 3.1, p. 70]):

$$\exists !y\in W(0,T):=\{y\in L^2(0,T;H_0^1(\Omega ))|{\displaystyle \frac{\partial y}{\partial t}}\in L^2(0,T;H^{-1}(\Omega ))\}$$

such that ${y|}_{t=0}=y_0$ and $\forall v\in H_0^1(\Omega )$,

$${\displaystyle \frac{\partial }{\partial t}}(y\left(t\right),v)+(A\nabla y,\nabla v)=(u,v)\mathrm{a}.\mathrm{e}.\mathrm{on}(0,T).$$

Due to the embedding,

$$W(0,T)\subset C([0,T];L^2(\Omega ))$$

where equality ${y|}_{t=0}=y_0$ makes sense.

Moreover, for every weak solution, functions $t\mapsto \left(y\right(t),v)$ and $t\mapsto {\parallel y\left(t\right)\parallel }^2$ are absolutely continuous and

$${\displaystyle \frac{\partial }{\partial t}}(y\left(t\right),v)+(A\nabla y,\nabla v)=(u,v)\mathrm{a}.\mathrm{e}.\mathrm{on}(0,T).$$

$${\displaystyle \frac{1}{2}}{\displaystyle \frac{\partial }{\partial t}}{\parallel y\left(t\right)\parallel }^2+(A\nabla y,\nabla y)=(u,y)\mathrm{a}.\mathrm{e}.\mathrm{on}(0,T).$$

(5)

From (5), we derive the following estimates:

$$\forall t\in {[0,T]\parallel y\left(t\right)\parallel }^2\le ({\parallel y_0\parallel }^2+{\parallel u\parallel }_{L^2\left(Q_T\right)}^2)e^T,$$

(6)

$$2{\gamma }_3{\int }_0^T{\parallel \nabla y\parallel }^2\le {\parallel y_0\parallel }^2+{\parallel u\parallel }_{L^2\left(Q_T\right)}^2+({\parallel y_0\parallel }^2+{\parallel u\parallel }_{L^2\left(Q_T\right)}^2)e^{T}T.$$

(7)

If for a given $u\in L^2\left(Q_T\right)$, we denote by $y_1$ (cor. $y_2$) the solution of (1) with matrix $A_1$ (cor. $A_2$), then for $z=y_1-y_2$, we get

$${\displaystyle \frac{1}{2}}{\displaystyle \frac{\partial }{\partial t}}{\parallel z\left(t\right)\parallel }^2+((A_1-A_2)\nabla y_1,\nabla z)+(A_2\nabla z,\nabla z)=0.$$

Therefore,

$${\displaystyle \frac{1}{2}}{\displaystyle \frac{\partial }{\partial t}}{\parallel z\left(t\right)\parallel }^2\le \parallel A_1-A_2\parallel \parallel \nabla y_1\parallel \parallel \nabla z\parallel .$$

So, $\forall t\in [0,T]$ from (7), we deduce

$$\parallel y_1\left(t\right)-y_2{\left(t\right)\parallel }^2\le \parallel A_1-A_2\parallel {\displaystyle \frac{1}{{\gamma }_3}}({\parallel y_0\parallel }^2+{\parallel u\parallel }_{L^2\left(Q_T\right)}^2)(T{e}^T+1).$$

(8)

Finally, we note that weak convergence (4) can be described by the Wasserstein metric:

$$\rho ({\pi }_1,{\pi }_2)=\underset{\mu \in \Gamma ({\pi }_1,{\pi }_2)}{inf}{\int }_{\mathcal{A}\times \mathcal{A}}d(A_1,A_2)\partial \mu ,$$

where $(\mathcal{A},d)$ is a Polish metric space, and $\Gamma ({\pi }_1,{\pi }_2)$ is the collection of all probability measures on $\mathcal{A}\times \mathcal{A}$ with projections ${\pi }_1$ and ${\pi }_2$, respectively.

Then,

$${\pi }^n\stackrel{w}{\to }{\pi }^{\infty }\iff \rho ({\pi }^n,{\pi }^{\infty })\to 0\mathrm{as}n\to \infty .$$

4. Main Results

Theorem 1. 

For every measure π, the LQR optimal control problem (1), (2) has a unique solution $\{{\overline{y}}_{\pi },{\overline{u}}_{\pi }\}$ (here, ${\overline{y}}_{\pi }$ depends on A), and

$$\underset{t\in [0,T]}{max}\parallel {\overline{y}}_{\pi }\left(t\right)\parallel +{\parallel {\overline{u}}_{\pi }\parallel }_{L^2\left(Q_t\right)}\le C,$$

(9)

where constant $C>0$ does not depend on π and A.

Remark 1. 

${\overline{u}}_{\pi }$ depends on π but does not depend on A.

Proof of Theorem 1. 

Let $\left\{u_n\right\}$ be a minimizing sequence and $\left\{y_n\right\}$ be the corresponding solutions of (1). Then,

$$J_{\pi }(u_n)\le inf{J}_{\pi }\left(u\right)+{\displaystyle \frac{1}{n}}.$$

Taking control function $u\equiv 0$, for the corresponding solution y of (1), we have from (6),

$$J_{\pi }\left(0\right)={\int }_{\mathcal{A}}({\parallel y\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_2{\parallel y\left(T\right)\parallel }^2)d\pi \left(A\right)\le {\parallel y_0\parallel }^2{e}^T(T+{\gamma }_2).$$

So,

$$\begin{array}{c}\hfill {\gamma }_1{\parallel u_n\parallel }_{L^2\left(Q_T\right)}^2\le J_{\pi }\left(u_n\right)\le inf{J}_{\pi }\left(u\right)+{\displaystyle \frac{1}{n}}\le J_{\pi }\left(0\right)+{\displaystyle \frac{1}{n}}\\ \hfill \le J_{\pi }\left(0\right)+1\le {\parallel y_0\parallel }^2{e}^T(T+{\gamma }_2)+1.\end{array}$$

(10)

Equations (10) and (6) imply that

$${\parallel u_n\parallel }_{L^2\left(Q_T\right)}^2+\underset{t\in [0,T]}{max}{\parallel y_n\left(t\right)\parallel }^2\le C,$$

where

$$C={\displaystyle \frac{{\parallel y_0\parallel }^2{e}^T(T+{\gamma }_2)+1}{{\gamma }_1}}(1+e^T)+{\parallel y_0\parallel }^2{e}^T$$

does not depend on n, $\pi$, and A.

Moreover, from (7), we deduce that for every A, $\left\{y_n\right\}$ is bounded in $W(0,T)$. As $W(0,T)$ is compactly embedded in $L^2\left(Q_T\right)$, up to subsequence, for some $\{{\overline{y}}_A,\overline{u}\}$,

$$\begin{array}{c}\hfill u_n\to \overline{u}\mathrm{weakly}\mathrm{in}{L}^2\left(Q\right),\\ \hfill y_n\to {\overline{y}}_A\mathrm{weakly}\mathrm{in}W(0,T),\\ \hfill y_n\to {\overline{y}}_A\mathrm{in}{L}^2\left(Q_T\right)\mathrm{and}\mathrm{a}.\mathrm{e}.\mathrm{in}{Q}_T,\\ \hfill y_n\left(t\right)\to {\overline{y}}_A\left(t\right)\mathrm{weakly}\mathrm{in}{L}^2(\Omega )\forall t\in [0,T].\end{array}$$

(11)

Taking an arbitrary $\xi \in C_0^{\infty }(0,T)$ and passing the limit in the equality

$$-{\int }_0^T(y_n\left(t\right),v){\xi }_t+{\int }_0^T(A\nabla y_n,\nabla v)\xi ={\int }_0^T(u_n\left(t\right),v)\xi ,$$

we obtain that ${\overline{y}}_A$ is a solution of (1) with control $\overline{u}$ and matrix A. Due to the uniqueness of such a solution, the whole sequence $y_n$ tends to ${\overline{y}}_A$.

Moreover, due to (11), $\forall t\in [0,T]$, we get

$${\parallel \overline{u}\parallel }_{L^2\left(Q_T\right)}+\parallel \overline{y}{\left(t\right)\parallel }^2\le lim\{{\parallel {\overline{u}}_n\parallel }_{L^2\left(Q_T\right)}+\parallel {\overline{y}}_n\left(t\right){\parallel }^2\}\le C.$$

Also, using Fatou’s Lemma, we get

$$\begin{array}{c}\hfill lim{J}_{\pi }\left(u_n\right)=lim{\int }_{\mathcal{A}}({\parallel y_n\parallel }_{L^2\left(Q_T\right)}+{\gamma }_1{\parallel u_n\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_2\parallel y_n\left(T\right){\parallel }^2)d\pi \left(A\right)\\ \hfill \ge {\int }_{\mathcal{A}}(lim{\parallel y_n\parallel }_{L^2\left(Q_T\right)}+{\gamma }_{1}lim{\parallel u_n\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_{2}lim\parallel y_n\left(T\right){\parallel }^2)d\pi \left(A\right)\ge J_{\pi }\left(\overline{u}\right).\end{array}$$

Therefore, $\{\overline{y},\overline{u}\}\equiv \{{\overline{y}}_{\pi },{\overline{u}}_{\pi }\}$ is a solution of the LQR optimal control problem (1), (2), and (9) holds.

Because of the strict convexity of $u\mapsto J_{\pi }\left(u\right)$, we obtain uniqueness. Thus, the theorem is proved. □

Now, let us assume that

$${\pi }^n\stackrel{\mathrm{w}}{\to }{\pi }^{\infty }.$$

(12)

The convergence of optimal controls under the weak convergence of probability measures is analyzed using the Wasserstein metric, a powerful tool in probability and optimal transport theory [31].

Let $\{{\overline{y}}_{\pi }^n,{\overline{u}}_{\pi }^n\}$ and $\{{\overline{y}}_{\pi }^{\infty },{\overline{u}}_{\pi }^{\infty }\}$ be optimal solutions of (1), (2) for measures ${\pi }^n$ and ${\pi }^{\infty }$.

Theorem 2. 

Under assumption (12),

$$J_{{\pi }^n}\left({\overline{u}}_{{\pi }^n}\right)\to J_{{\pi }^{\infty }}\left({\overline{u}}_{{\pi }^{\infty }}\right),$$

(13)

$${\overline{u}}_{{\pi }^n}\to {\overline{u}}_{{\pi }^{\infty }}in{L}^2\left(Q_T\right),$$

(14)

$$\forall A\in \mathcal{A},{\overline{y}}_{{\pi }^n}\to {\overline{y}}_{{\pi }^{\infty }}inC([0,T];L^2(\Omega )).$$

(15)

Proof of Theorem 2. 

For every $u\in L^2\left(Q_T\right)$, due to (6) and (8),

$$\begin{array}{c}\hfill |{\int }_{\mathcal{A}}\left({\parallel y_{A_1}\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_1{\parallel u\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_2{\parallel y_{A_1}\left(T\right)\parallel }^2\right)d{\pi }^n\left(A_1\right)\\ \hfill -{\int }_{\mathcal{A}}\left({\parallel y_{A_2}\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_1{\parallel u\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_2{\parallel y_{A_2}\left(T\right)\parallel }^2\right)d{\pi }^{\infty }\left(A_2\right)|\\ \hfill =\left|{\int }_{\mathcal{A}\times \mathcal{A}}\left[({\parallel y_{A_1}\parallel }_{L^2\left(Q_T\right)}^2-{\parallel y_{A_2}\parallel }_{L^2\left(Q_T\right)}^2)+{\gamma }_2(\parallel y_{A_1}{\left(T\right)\parallel }^2-\parallel y_{A_2}\left(T\right){\parallel }^2)\right]d\mu \right|\\ \hfill \le C({\parallel u\parallel }_{L^2\left(Q_T\right)})·{\int }_{\mathcal{A}\times \mathcal{A}}{\parallel A_1-A_2\parallel }^{\frac{1}{2}}d\mu ,\end{array}$$

(16)

where $C({\parallel u\parallel }_{L^2\left(Q_T\right)})$ is bounded if ${\parallel u\parallel }_{L^2\left(Q_T\right)}$ is bounded, and $\mu$ is a probability measure on $\mathcal{A}\times \mathcal{A}$ with projections ${\pi }^N$ and ${\pi }^{\infty }$. If we take $d(A_1,A_2)={\parallel A_1-A_2\parallel }^2$, then space $(\mathcal{A},d)$ is a Polish metric space. Therefore, from (16), we get, for $u\in L^2\left(Q_T\right)$,

$$|J_{{\pi }^n}\left(u\right)-J_{{\pi }^{\infty }}\left(u\right)|\le C({\parallel u\parallel }_{L^2\left(Q_T\right)})\rho ({\pi }^N,{\pi }^{\infty }).$$

(17)

Let us denote

$$C=max\left\{C(\parallel {\overline{u}}_{{\pi }^n}{\parallel }_{L^2\left(Q_T\right)}),C(\parallel {\overline{u}}_{{\pi }^{\infty }}{\parallel }_{L^2\left(Q_T\right)})\right\}.$$

Due to (9), the number $C>0$ does not depend on n.

Then, the optimality of ${\overline{u}}_{{\pi }^n}$ and ${\overline{u}}_{{\pi }^{\infty }}$ implies

$$\begin{array}{c}\hfill inf{J}_{{\pi }^n}\left(u\right)-inf{J}_{{\pi }^{\infty }}\left(u\right)\le inf{J}_{{\pi }^n}\left(u\right)-J_{{\pi }^{\infty }}\left({\overline{u}}_{{\pi }^{\infty }}\right)\\ \hfill \le J_{{\pi }^n}\left({\overline{u}}_{{\pi }^{\infty }}\right)-J_{{\pi }^{\infty }}\left({\overline{u}}_{{\pi }^{\infty }}\right)\le C\rho ({\pi }^n,{\pi }^{\infty }).\end{array}$$

(18)

$$\begin{array}{c}\hfill inf{J}_{{\pi }^{\infty }}\left(u\right)-inf{J}_{{\pi }^n}\left(u\right)\le inf{J}_{{\pi }^{\infty }}\left(u\right)-J_{{\pi }^n}\left({\overline{u}}_{{\pi }^n}\right)\\ \hfill \le J_{{\pi }^{\infty }}\left({\overline{u}}_{{\pi }^n}\right)-J_{{\pi }^{\infty }}\left({\overline{u}}_{{\pi }^n}\right)\le C\rho ({\pi }^{\infty },{\pi }^n).\end{array}$$

(19)

Inequalities (18) and (19) guarantee (13).

In the following, we denote ${\overline{u}}^n:={\overline{u}}_{{\pi }^n}$, ${\overline{u}}^{\infty }:={\overline{u}}_{{\pi }^{\infty }}$.

Due to (7) and (9), we obtain that

$$\begin{array}{c}\hfill \left\{{\overline{u}}_n\right\}\mathrm{is}\mathrm{bounded}\mathrm{in}{L}^2\left(Q_T\right)\\ \hfill \forall A\left\{{\overline{y}}_n\right\}\mathrm{is}\mathrm{bounded}\mathrm{in}W(0,T)\end{array}$$

So, up to the subsequence, for some $\{\overline{u}$, $\overline{y}\}$, $\{{\overline{u}}^n,{\overline{y}}^n\}$ tends to $\{\overline{u},\overline{y}\}$ in the sense of (11).

Passing to the limit yields that $\overline{y}$ is a solution of (1) with control $\overline{u}$.

Due to (17),

$$J_{{\pi }^{\infty }}\left({\overline{u}}^n\right)\le J_{{\pi }^n}\left({\overline{u}}^n\right)+C\rho ({\pi }^n,{\pi }^{\infty }).$$

So, using (11), (13), and Fatou’s Lemma, we get

$$J_{{\pi }^{\infty }}\left(\overline{u}\right)\le lim{J}_{{\pi }^{\infty }}\left({\overline{u}}^n\right)\le lim(J_{{\pi }^n}\left({\overline{u}}^n\right)+C\rho ({\pi }^n,{\pi }^{\infty }))=J_{{\pi }^{\infty }}\left({\overline{u}}_{\infty }\right).$$

(20)

The uniqueness of the solution of (1), (2), and (20) implies that $\overline{u}={\overline{u}}_{\infty }$, and consequently, $\overline{y}={\overline{y}}_{\infty }$.

This inequality also implies that ${\overline{u}}^n\to \overline{u}$ in $L^2\left(Q_T\right)$.

Indeed, on the one hand,

$$\begin{array}{c}\hfill lim{J}_{{\pi }^{\infty }}\left({\overline{u}}^n\right)\le J_{{\pi }^{\infty }}\left({\overline{u}}^{\infty }\right)\\ \hfill ={\int }_{\mathcal{A}}\left(\parallel {\overline{y}}^{\infty }{\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_2{\parallel {\overline{y}}^{\infty }\left(T\right)\parallel }^2\right)d{\pi }^{\infty }\left(A\right)+{\gamma }_1{\parallel {\overline{u}}^{\infty }\parallel }_{L^2\left(Q_T\right)}^2.\end{array}$$

And on the other hand,

$$\begin{array}{c}\hfill lim{J}_{{\pi }^{\infty }}\left({\overline{u}}^n\right)\ge lim{\int }_{\mathcal{A}}\left(\parallel {\overline{y}}^n{\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_2{\parallel {\overline{y}}^n\left(T\right)\parallel }^2\right)d{\pi }^{\infty }\left(A\right)+{\gamma }_{1}lim{\parallel {\overline{u}}^n\parallel }_{L^2\left(Q_T\right)}^2\\ \hfill \ge {\int }_{\mathcal{A}}lim\left(\parallel {\overline{y}}^n{\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_2{\parallel {\overline{y}}^n\left(T\right)\parallel }^2\right)d{\pi }^{\infty }\left(A\right)+{\gamma }_{1}lim{\parallel {\overline{u}}^n\parallel }_{L^2\left(Q_T\right)}^2\\ \hfill ={\int }_{\mathcal{A}}\left(\parallel {\overline{y}}^{\infty }{\parallel }_{L^2\left(Q_T\right)}^2+{\gamma }_2{\parallel {\overline{y}}^{\infty }\left(T\right)\parallel }^2\right)d{\pi }^{\infty }\left(A\right)+{\gamma }_{1}lim{\parallel {\overline{u}}^n\parallel }_{L^2\left(Q_T\right)}^2.\end{array}.$$

So,

$$lim\parallel {\overline{u}}^n{\parallel }_{L^2\left(Q_T\right)}\le {\parallel {\overline{u}}^{\infty }\parallel }_{L^2\left(Q_T\right)}.$$

This inequality and weak convergence ${\overline{u}}^N$ to ${\overline{u}}^{\infty }$ in $L^2\left(Q_T\right)$ imply the strong convergence of (14).

Let us prove (15). We already know from (11) that

$$\forall A\in \mathcal{A},\parallel {\overline{y}}^N\left(t\right)\parallel \to \parallel {\overline{y}}^{\infty }\left(t\right)\parallel \mathrm{for}\mathrm{a}.\mathrm{e}.t\in (0,T),$$

and that functions $t\mapsto \parallel {\overline{y}}^n\left(t\right)\parallel$, $t\mapsto \parallel {\overline{y}}^{\infty }\left(t\right)\parallel$ are continuous.

From (1), we deduce

$${\displaystyle \frac{1}{2}}{\displaystyle \frac{\partial }{\partial t}}{\parallel {\overline{y}}^n\left(t\right)\parallel }^2\le 2({\overline{u}}^n\left(t\right),{\overline{y}}^n\left(t\right)).$$

Therefore, for all $t>s>0$,

$$\forall t\ge s\ge 0\parallel {\overline{y}}^n{\left(t\right)\parallel }^2\le {\parallel {\overline{y}}^n\left(s\right)\parallel }^2+2{\int }_s^t({\overline{u}}^n\left(\tau \right),{\overline{y}}^n\left(\tau \right))d\tau .$$

So, the function

$$t\mapsto J_n\left(t\right)={\parallel {\overline{y}}^n\left(t\right)\parallel }^2-2{\int }_0^t({\overline{u}}^n\left(\tau \right),{\overline{y}}^n\left(\tau \right))d\tau$$

is monotone, continuous, and

$$J_n\left(t\right)\to J_{\infty }\left(t\right)\mathrm{a}.\mathrm{e}.$$

Then, from Dini’s Theorem,

$$J_n\left(t\right)\to J_{\infty }\left(t\right)\mathrm{in}C\left([0,T]\right),$$

which implies (15). □

5. An Example

The aim of this section is to illustrate the obtained results using the scheme utilized in [4]. We assume that $\mathcal{A}$ is a finite set, i.e., $\mathcal{A}=\{A_1,..,A_M\}$ for some $M\in \mathbb{N}$. We consider a sequence of probability distributions ${\left\{{\pi }^n\right\}}_{n=1}^{\infty }$:

$${\pi }^n=\sum _{i=1}^M{\alpha }_i^n·{\delta }_{A_i},\mathrm{where}{\alpha }_i^n\ge 0,\sum _{i=1}^M{\alpha }_i^n=1,$$

where ${\delta }_{A_i}$ is a Dirac delta concentrated at $A_i$.

Assume that

$${\alpha }_1^n\to 1,{\alpha }_i^n\to 0,i=2\dots M\mathrm{as}n\to \infty .$$

Then, clearly,

$${\pi }^n\to {\delta }_{A_1}\mathrm{weakly}.$$

(21)

For the measure ${\pi }^{\infty }={\delta }_{A_1}$, problem (1), (2) is the following LQR optimal control problem:

$$\left\{\begin{array}{cc}{\displaystyle \frac{\partial y(t,x)}{\partial t}}-L_{A_1}y(t,x)\hfill & =u(t,x),\hfill \\ {y|}_{x\in \partial \Omega }\hfill & =0,\hfill \\ {y|}_{t=0}\hfill & =y_0\left(x\right),\hfill \end{array}\right.$$

(22)

$$J_{{\delta }_{A_1}}\left(u\right)={\int }_0^T\left({\int }_{\Omega }{y}^2(t,x)dx+{\gamma }_1{\int }_{\Omega }{u}^2(t,x)dx\right)dt+{\gamma }_2{\int }_{\Omega }{y}^2(T,x)dx\to inf,$$

(23)

According to Theorem 1, problems (22), (23) has a unique solution $\{{\overline{y}}_{{\pi }^{\infty }},{\overline{u}}_{{\pi }^{\infty }}\}$, which satisfies (9). For the measure ${\pi }^n={\sum }_{i=1}^M{\alpha }_i^n{\delta }_{A_i}$, we have the following problem of minimizing the functional

$$J_{{\pi }^n}\left(u\right)=\sum _{i=1}^M\left[{\alpha }_i^n·J_{{\delta }_{A_i}}\left(u\right)\right]\to inf$$

(24)

where

$$J_{{\delta }_{A_1}}\left(u\right)={\int }_0^T\left({\int }_{\Omega }{y}^2(t,x)dx+{\gamma }_1{\int }_{\Omega }{u}^2(t,x)dx\right)dt+{\gamma }_2{\int }_{\Omega }{y}^2(T,x)dx,$$

and y is a solution of (1) with $A=A_i$. With obvious changes, we can apply argument (10), (11) to problem (24) and obtain that such a problem has a unique solution ${\overline{u}}_{{\pi }^n}$, which together with y-components satisfies estimation (9) with constant C not depending on n. This means that

$$J_{{\delta }_{A_i}}\left({\overline{u}}_{{\pi }^n}\right)\le C^2(T+{\gamma }_1+{\gamma }_2),i=1\dots M,n\ge 1.$$

Therefore, using a well-known fact [3]—if in (1), $u_n\to u$ weakly in $L^2\left(Q_T\right)$, then $y_n\to y$ in $C([0,T];L^2(\Omega ))$—we can pass to the limit and obtain

$$J_{{\pi }^n}\left({\overline{u}}_{{\pi }^n}\right)={\alpha }_1^n·J_{{\delta }_{A_1}}\left({\overline{u}}_{{\pi }^n}\right)+\sum _{i=2}^M{\alpha }_i^n·J_{{\delta }_{A_i}}\left({\overline{u}}_{{\pi }^n}\right)\to J_{{\delta }_{A_1}}({\overline{u}}_{{\delta }_{A_1}}),n\to \infty .$$

(25)

We can give a numerical illustration in the simplest case: $d=1$, $\Omega =(0,\pi )$, ${\gamma }_1={\gamma }_2={\gamma }_3=1$, $M=2$, $A_1=1$, $A_2=2$, ${\alpha }_1^n=1-{\displaystyle \frac{1}{2^n}}$, ${\alpha }_2^n={\displaystyle \frac{1}{2^n}}$. Then, for $T=1$ and $y_0\left(x\right)=sinx$ for problem (22), (23), we have

$$\left\{\begin{array}{cc}{\displaystyle \frac{\partial y(t,x)}{\partial t}}-{\displaystyle \frac{{\partial }^{2}y(t,x)}{\partial x^2}}\hfill & =u(t,x),\hfill \\ {y|}_{x=0}\hfill & =0,\hfill \\ {y|}_{x=\pi }\hfill & =0,\hfill \\ {y|}_{t=0}\hfill & =sinx,\hfill \end{array}\right.$$

(26)

$$J_{{\delta }_{A_1}}\left(u\right)={\int }_0^1{\int }_0^{\pi }(y^2(t,x)+u^2(t,x))dxdt+y^2(1,x)\to inf,$$

(27)

Using Fourier analysis, we can reduce problem (26), (27) to the classical calculus of variations problem.

$$inf{J}_{{\delta }_{A_i}}\left(u\right)=\underset{\begin{array}{c}y\in C^1\left([0,1]\right)\\ y\left(0\right)=1\end{array}}{inf}\left[{\int }_0^1(y^2\left(t\right)+{(y^{\prime }+y)}^2)dt+y^2\left(1\right)\right].$$

(28)

For problem (24), we have

$$\begin{array}{c}\hfill inf{J}_{{\pi }_N}\left(u\right)=\underset{\begin{array}{c}{y}_1,y_2\in C^1\left([0,1]\right)\\ y_1\left(0\right)=y_2\left(0\right)=1\end{array}}{inf}[(1-{\displaystyle \frac{1}{2^n}})[{\int }_0^1(y_1^2\left(t\right)+{(y_1^{\prime }+y_1)}^2)dt+y_1^2\left(1\right)]\\ \hfill +{\displaystyle \frac{1}{2^n}}[{\int }_0^1(y_2^2\left(t\right)+{(y_2^{\prime }+y_2)}^2)dt+y_2^2\left(1\right)]]\end{array}$$

(29)

which, clearly, tends to the value of (28) as $n\to \infty$.

6. Conclusions

In this paper, we studied an averaged Linear Quadratic Regulator (LQR) problem for a parabolic partial differential equation (PDE), where the system dynamics are described by a probability distribution over possible operators. This formulation generalizes the classical LQR problem by incorporating uncertainty in the system parameters through an averaging approach.

The main contributions of this work are the following:

• We established the existence and uniqueness of the optimal control solution under appropriate assumptions.
• We proved the convergence of the optimal control as the probability distribution governing the system dynamics become more concentrated.

These results provide a rigorous theoretical foundation for analyzing control problems with uncertainty in system parameters.

In future research, we plan to generalize our results to multidimensional parabolic systems and evolution problems on infinite-time intervals. Moreover, it should be interesting to extend the results to hyperbolic partial differential equations (PDEs), which model wave-like and transport phenomena [32]. Exploring the control of such systems within a reinforcement learning framework may yield new methods for optimal control in dynamic environments [33], with potential applications in robotics, physics-based simulations, and real-time decision-making systems.

Beyond the realm of physical systems, intelligent control and learning frameworks are gaining traction in socio-technical domains. One notable example is the use of machine learning models to support automated recruitment and decision-making for hiring young professionals [34]. Additionally, the development of advanced database connectors underscores the relevance of scalable control and dataflow mechanisms in distributed computing systems [35].