# Singularly Perturbed Forward-Backward Stochastic Differential Equations: Application to the Optimal Control of Bilinear Systems

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

#### 1.1. Set-Up and Problem Statement

#### 1.2. Outline

## 2. Singularly Perturbed Bilinear Control Systems

**Remark**

**1.**

#### From Stochastic Control to Forward-Backward Stochastic Differential Equations

**Remark**

**2.**

## 3. Model Reduction

#### 3.1. Interpretation as an Optimal Control Problem

#### 3.2. Convergence of the Control Value

**Condition LQ**if the following holds:

- $(A,C)$ is controllable, and the range of $b\left(x\right)=Nx+B$ is a subspace of $\mathrm{range}\left(C\right)$.
- The matrix ${A}_{22}$ is Hurwitz (i.e., its spectrum lies entirely in the open left complex half-plane) and the matrix pair $({A}_{22},{C}_{2})$ is controllable.
- The driver of the FBSDE (15) is continuous and quadratically growing in Z.

**Theorem**

**1.**

#### 3.3. Formal Derivation of the Limiting FBSDE

## 4. Numerical Studies

#### 4.1. Numerical FBSDE Discretisation

#### 4.2. Least-Squares Solution of the Backward SDE

#### 4.3. Numerical Example

#### 4.4. Discussion

**Remark**

**3.**

## 5. Conclusions and Outlook

## Author Contributions

## Funding

## Conflicts of Interest

## Appendix A. Proofs and Technical Lemmas

#### Appendix A.1. Poisson Equation Lemma

**Lemma**

**A1.**

**Proof.**

#### Appendix A.2. Convergence of the Value Function

**Lemma**

**A2.**

**Proof.**

## References

- Fleming, W.H.; Mete Soner, H. Controlled Markov Processes and Viscosity Solutions, 2nd ed.; Springer: New York, NY, USA, 2006. [Google Scholar]
- Stengel, F.R. Optimal Control and Estimation; Dover Books on Advanced Mathematics; Dover Publications: New York, NY, USA, 1994. [Google Scholar]
- Dupuis, P.; Spiliopoulos, K.; Wang, H. Importance sampling for multiscale diffusions. Multiscale Model. Simul.
**2012**, 10, 1–27. [Google Scholar] [CrossRef] - Dupuis, P.; Wang, H. Importance sampling, large deviations, and differential games. Stoch. Rep.
**2004**, 76, 481–508. [Google Scholar] [CrossRef] - Davis, M.H.; Norman, A.R. Portfolio selection with transaction costs. Math. Oper. Res.
**1990**, 15, 676–713. [Google Scholar] [CrossRef] - Pham, H. Continuous-Time Stochastic Control and Optimization with Financial Applications; Springer: Berlin/Heidelberg, Germany, 2009. [Google Scholar]
- Hartmann, C.; Schütte, C. Efficient rare event simulation by optimal nonequilibrium forcing. J. Stat. Mech. Theor. Exp.
**2012**, 2012, 11004. [Google Scholar] [CrossRef] - Schütte, C.; Winkelmann, S.; Hartmann, C. Optimal control of molecular dynamics using markov state models. Math. Program. Ser. B
**2012**, 134, 259–282. [Google Scholar] [CrossRef] - Asplund, E.; Klüner, T. Optimal control of open quantum systems applied to the photochemistry of surfaces. Phys. Rev. Lett.
**2011**, 106, 140404. [Google Scholar] [CrossRef] [PubMed] - Steinbrecher, A. Optimal Control of Robot Guided Laser Material Treatment. In Progress in Industrial Mathematics at ECMI 2008; Fitt, A.D., Norbury, J., Ockendon, H., Wilson, E., Eds.; Springer: Berlin/Heidelberg, Germany, 2010; pp. 501–511. [Google Scholar]
- Zhang, W.; Wang, H.; Hartmann, C.; Weber, M.; Schütte, C. Applications of the cross-entropy method to importance sampling and optimal control of diffusions. SIAM J. Sci. Comput.
**2014**, 36, A2654–A2672. [Google Scholar] [CrossRef] - Antoulas, A.C. Approximation of Large-Scale Dynamical Systems; SIAM: Philadelphia, PA, USA, 2005. [Google Scholar]
- Baur, U.; Benner, P.; Feng, L. Model order reduction for linear and nonlinear systems: A system-theoretic perspective. Arch. Comput. Meth. Eng.
**2014**, 21, 331–358. [Google Scholar] [CrossRef] - Bensoussan, A.; Blankenship, G. Singular perturbations in stochastic control. In Singular Perturbations and Asymptotic Analysis in Control Systems; Lecture Notes in Control and Information Sciences; Kokotovic, P.V., Bensoussan, A., Blankenship, G.L., Eds.; Springer: Berlin/Heidelberg, Germany, 1987; Volume 90, pp. 171–260. [Google Scholar]
- Evans, L.C. The perturbed test function method for viscosity solutions of nonlinear PDE. Proc. R. Soc. Edinb. A
**1989**, 111, 359–375. [Google Scholar] [CrossRef] - Buckdahn, R.; Hu, Y. Probabilistic approach to homogenizations of systems of quasilinear parabolic PDEs with periodic structures. Nonlinear Anal.
**1998**, 32, 609–619. [Google Scholar] [CrossRef] - Ichihara, N. A stochastic representation for fully nonlinear PDEs and its application to homogenization. J. Math. Sci. Univ. Tokyo
**2005**, 12, 467–492. [Google Scholar] - Kushner, H.J. Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems; Birkhäuser: Boston, MA, USA, 1990. [Google Scholar]
- Kurtz, T.; Stockbridge, R.H. Stationary solutions and forward equations for controlled and singular martingale problems. Electron. J. Probab.
**2001**, 6, 5. [Google Scholar] [CrossRef] - Kabanov, Y.; Pergamenshchikov, S. Two-Scale Stochastic Systems: Asymptotic Analysis and Control; Springer: Berlin/Heidelberg, Germany; Paris, France, 2003. [Google Scholar]
- Kokotovic, P.V. Applications of singular perturbation techniques to control problems. SIAM Rev.
**1984**, 26, 501–550. [Google Scholar] [CrossRef] - Hartmann, C.; Schäfer-Bung, B.; Zueva, A. Balanced averaging of bilinear systems with applications to stochastic control. SIAM J. Control Optim.
**2013**, 51, 2356–2378. [Google Scholar] [CrossRef] - Pardalos, P.M.; Yatsenko, V.A. Optimization and Control of Bilinear Systems: Theory, Algorithms, and Applications; Springer: New York, NY, USA, 2010. [Google Scholar]
- Hartmann, C.; Latorre, J.; Pavliotis, G.A.; Zhang, W. Optimal control of multiscale systems using reduced-order models. J. Comput. Dyn.
**2014**, 1, 279–306. [Google Scholar] [CrossRef] - Peng, S. Backward Stochastic Differential Equations and Applications to Optimal Control. Appl. Math. Optim.
**1993**, 27, 125–144. [Google Scholar] [CrossRef] - Touzi, N. Optimal Stochastic Control, Stochastic Target Problem, and Backward Differential Equation; Springer: Berlin, Germany, 2013. [Google Scholar]
- Pardoux, E.; Peng, S. Adapted solution of a backward stochastic differential equation. Syst. Control Lett.
**1990**, 14, 55–61. [Google Scholar] [CrossRef] - Bahlali, K.; Kebiri, O.; Khelfallah, N.; Moussaoui, H. One dimensional BSDEs with logarithmic growth application to PDEs. Stochastics
**2017**, 89, 1061–1081. [Google Scholar] [CrossRef] - Duffie, D.; Epstein, L.G. Stochastic differential utility. Econometrica
**1992**, 60, 353–394. [Google Scholar] [CrossRef] - El Karoui, N.; Peng, S.; Quenez, M.C. Backward stochastic differential equations in finance. Math. Financ.
**1997**, 7, 1–71. [Google Scholar] [CrossRef] - Hu, Y.; Imkeller, P.; Müller, M. Utility maximization in incomplete markets. Ann. Appl. Probab.
**2005**, 15, 1691–1712. [Google Scholar] [CrossRef] [Green Version] - Hu, Y.; Peng, S. A stability theorem of backward stochastic differential equations and its application. Acad. Sci. Math.
**1997**, 324, 1059–1064. [Google Scholar] [CrossRef] - Kobylanski, M. Backward stochastic differential equations and partial differential equations with quadratic growth. Ann. Probab.
**2000**, 28, 558–602. [Google Scholar] [CrossRef] - Antonelli, F. Backward-forward stochastic differential equations. Ann. Appl. Probab.
**1993**, 3, 777–793. [Google Scholar] [CrossRef] - Bahlali, K.; Gherbal, B.; Mezerdi, B. Existence of optimal controls for systems driven by FBSDEs. Syst. Control Lett.
**1995**, 60, 344–349. [Google Scholar] [CrossRef] - Bahlali, K.; Kebiri, O.; Mtiraoui, A. Existence of an optimal Control for a system driven by a degenerate coupled Forward-Backward Stochastic Differential Equations. Comptes Rendus Math.
**2017**, 355, 84–89. [Google Scholar] [CrossRef] - Ma, J.; Protter, P.; Yong, J. Solving Forward-Backward Stochastic Differential Equations Explicitly—A Four Step Scheme. Probab. Theory Relat. Fields
**1994**, 98, 339–359. [Google Scholar] [CrossRef] - Zhen, W. Forward-backward stochastic differential equations, linear quadratic stochastic optimal control and nonzero sum differential games. J. Syst. Sci. Complex.
**2005**, 18, 179–192. [Google Scholar] - Hartmann, C.; Schütte, C.; Weber, M.; Zhang, W. Importance sampling in path space for diffusion processes with slow-fast variables. Probab. Theory Relat. Fields
**2017**, 170, 177–228. [Google Scholar] [CrossRef] [Green Version] - Bally, V. Approximation scheme for solutions of BSDE. In Backward Stochastic Differential Equations; El Karoui, N., Mazliak, L., Eds.; Addison Wesley Longman: Boston, MA, USA, 1997; pp. 177–191. [Google Scholar]
- Bender, C.; Steiner, J. Least-Squares Monte Carlo for BSDEs. In Numerical Methods in Finance; Springer: Berlin, Germany, 2012; pp. 257–289. [Google Scholar]
- Bouchard, B.; Elie, R.; Touzi, N. Discrete-time approximation of BSDEs and probabilistic schemes for fully nonlinear PDEs. Comput. Appl. Math.
**2009**, 8, 91–124. [Google Scholar] - Chevance, D. Numerical methods for backward stochastic differential equations. In Numerical Methods in Finance; Publications of the Newton Institute, Cambridge University Press: Cambridge, UK, 1997; pp. 232–244. [Google Scholar]
- Hyndman, C.B.; Ngou, P.O. A Convolution Method for Numerical Solution of Backward Stochastic Differential Equations. Methodol. Comput. Appl. Probab.
**2017**, 19, 1–29. [Google Scholar] [CrossRef] - Hartmann, C. Balanced model reduction of partially-observed Langevin equations: An averaging principle. Math. Comput. Model. Dyn. Syst.
**2011**, 17, 463–490. [Google Scholar] [CrossRef] - Fleming, W.H. Optimal investment models with minimum consumption criteria. Aust. Econ. Pap.
**2005**, 44, 307–321. [Google Scholar] [CrossRef] - Budhiraja, A.; Dupuis, P. A variational representation for positive functionals of infinite dimensional Brownian motion. Probab. Math. Stat.
**2000**, 20, 39–61. [Google Scholar] - Dai Pra, P.; Meneghini, L.; Runggaldier, J.W. Connections between stochastic control and dynamic games. Math. Control Signal Syst.
**1996**, 9, 303–326. [Google Scholar] [CrossRef] - Pavliotis, G.A.; Stuart, A.M. Multiscale Methods: Averaging and Homogenization; Springer: Berlin/Heidelberg, Germany, 2008. [Google Scholar]
- Anderson, B.D.O.; Liu, Y. Controller reduction: Concepts and approaches. IEEE Trans. Autom. Control
**1989**, 34, 802–812. [Google Scholar] [CrossRef] - Bensoussan, A.; Boccardo, L.; Murat, F. Homogenization of elliptic equations with principal part not in divergence form and hamiltonian with quadratic growth. Commun. Pure Appl. Math.
**1986**, 39, 769–805. [Google Scholar] [CrossRef] - Pardoux, E.; Peng, S. Backward stochastic differential equations and quasilinear parabolic partial differential equations. In Stochastic Partial Differential Equations and Their Applications; Lecture Notes in Control and Information Sciences 176; Rozovskii, B.L., Sowers, R.B., Eds.; Springer: Berlin, Germany, 1992. [Google Scholar]
- Freidlin, M.; Wentzell, A. Random Perturbations of Dynamical Systems; Springer: Berlin/Heidelberg, Germany, 2012. [Google Scholar]
- Khasminskii, R. Principle of averaging for parabolic and elliptic differential equations and for Markov processes with small diffusion. Theory Probab. Appl.
**1963**, 8, 1–21. [Google Scholar] [CrossRef] - Gobet, E.; Turkedjiev, P. Adaptive importance sampling in least-squares Monte Carlo algorithms for backward stochastic differential equations. Stoch. Proc. Appl.
**2005**, 127, 1171–1203. [Google Scholar] [CrossRef] - Turkedjiev, P. Numerical Methods for Backward Stochastic Differential Equations of Quadratic and Locally Lipschitz Type. Ph.D. Thesis, Humboldt-Universität zu Berlin, Berlin, Germany, 2013. [Google Scholar]
- Kebiri, O.; Neureither, L.; Hartmann, C. Adaptive importance sampling with forward-backward stochastic differential equations. arXiv, 2018; arXiv:1802.04981. [Google Scholar]
- Franzke, C.; Majda, A.J.; Vanden-Eijnden, E. Low-order stochastic mode reduction for a realistic barotropic model climate. J. Atmos. Sci.
**2005**, 62, 1722–1745. [Google Scholar] [CrossRef] - Lall, S.; Marsden, J.; Glavaški, S. A subspace approach to balanced truncation for model reduction of nonlinear control systems. Int. J. Robust. Nonlinear Control
**2002**, 12, 519–535. [Google Scholar] [CrossRef] [Green Version] - Briand, P.; Hu, Y. Probabilistic approach to singular perturbations of semilinear and quasilinear parabolic. Nonlinear Anal.
**1999**, 35, 815–831. [Google Scholar] [CrossRef] - Bensoussan, A.; Lions, J.L.; Papanicolaou, G. Asymptotic Analysis for Periodic Structures; American Mathematical Society: Washington, DC, USA, 1978; pp. 769–805. [Google Scholar]
- Pardoux, E.; Yu, A. Veretennikov: On the poisson equation and diffusion approximation 3. Ann. Probab.
**2005**, 33, 1111–1133. [Google Scholar] [CrossRef]

**Figure 1.**Plot of the mean of $E\left(\u03f5\right)\phantom{\rule{4pt}{0ex}}\pm $ its standard deviation $\sigma \left(E\right(\u03f5\left)\right)$ and for comparison we plot $\sqrt{\u03f5}$ against $\u03f5$ on a doubly logarithmic scale: we observe convergence of order $1/2$ as predicted by the theory.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Kebiri, O.; Neureither, L.; Hartmann, C.
Singularly Perturbed Forward-Backward Stochastic Differential Equations: Application to the Optimal Control of Bilinear Systems. *Computation* **2018**, *6*, 41.
https://doi.org/10.3390/computation6030041

**AMA Style**

Kebiri O, Neureither L, Hartmann C.
Singularly Perturbed Forward-Backward Stochastic Differential Equations: Application to the Optimal Control of Bilinear Systems. *Computation*. 2018; 6(3):41.
https://doi.org/10.3390/computation6030041

**Chicago/Turabian Style**

Kebiri, Omar, Lara Neureither, and Carsten Hartmann.
2018. "Singularly Perturbed Forward-Backward Stochastic Differential Equations: Application to the Optimal Control of Bilinear Systems" *Computation* 6, no. 3: 41.
https://doi.org/10.3390/computation6030041