Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation
Abstract
:1. Introduction
2. The Model
- (i)
- with (), for all , and for all . Additionally, is bounded continuous function for all
- (ii)
- The drift term and the diffusion term are continuous functions. Moreover, both of them are Lipschitz continuous in their first component, uniformly for all and , with Lipschitz constant .
- (iii)
- The system is non-degenerate, i.e., for suitable constant , where represents the identity matrix.
The Risk-Sensitive Criterion
- (i)
- The running cost function is continuous in , and , for suitable , where is a given positive function and twice continuously differentiable in for each , and , for all .
- (ii)
- There have two constants such that and
- (iii)
- And assume that
3. The Main Results
3.1. The Optimality Equation and Viscosity Property
- (i)
- , for all ;
- (ii)
- , if , with ;
- (iii)
- and for each , we have
3.2. Comparison Result
4. The Approximation Scheme
4.1. Existence of -Optimal Controls of Finite-Difference-Type
4.2. Numerical Simulation
5. Discussion
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
- (i)
- The set of second-order superdifferentials of w at for each α is
- (ii)
- The set of second-order subdifferentials of w at for each α is
References
- Howard, R.A.; Matheson, J.E. Risk sensitive markov decision processes. Manag. Sci. 1972, 18, 356–369. [Google Scholar] [CrossRef]
- Jacobson, D.H. Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games. IEEE Trans. Automat. Control 1973, 18, 124–131. [Google Scholar] [CrossRef]
- Di Masi, G.B.; Stettner, Ł. Infinite horizon risk sensitive control of discrete time markov processes with small risk. Syst. Control Lett. 2000, 40, 15–20. [Google Scholar] [CrossRef]
- Di Masi, G.B.; Stettner, Ł. Infinite horizon risk sensitive control of discrete time markov processes under minorization property. SIAM J. Control Optim. 2007, 46, 231–252. [Google Scholar] [CrossRef]
- Ghosh, M.K.; Saha, S. Risk-sensitive control of continuous time markov chains. Stochastics 2014, 86, 655–675. [Google Scholar] [CrossRef]
- Guo, X.; Liao, Z. Risk-sensitive discounted continuous-time markov decision processes with unbounded rates. SIAM J. Control Optim. 2019, 57, 3857–3883. [Google Scholar] [CrossRef]
- Guo, X.; Zhang, J. Risk-sensitive continuous-time markov decision processes with unbounded rates and borel spaces. Discrete Event Dyn. Syst. 2019, 29, 445–471. [Google Scholar] [CrossRef]
- Kumar, K.S.; Pal, C. Risk-sensitive control of pure jump process on countable space with near monotone cost. Appl. Math. Optim. 2013, 68, 311–331. [Google Scholar]
- Huang, Y.; Lian, Z.; Guo, X. Risk-sensitive finite-horizon piecewise deterministic markov decision processes. Oper. Res. Lett. 2020, 48, 96–103. [Google Scholar] [CrossRef]
- Arapostathis, A.; Biswas, A. Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions. Stoch. Process. Appl. 2018, 128, 1485–1524. [Google Scholar] [CrossRef]
- Arapostathis, A.; Biswas, A. A variational formula for risk-sensitive control of diffusions in Rd. SIAM J. Control Optim. 2020, 58, 85–103. [Google Scholar] [CrossRef]
- Gauttam, S.K.; Kumar, K.S.; Pal, C. Risk-sensitive control of reflected diffusion processes on orthrant. Pure Appl. Funct. Anal. 2017, 2, 477–510. [Google Scholar]
- Menaldi, J.; Robin, M. Remarks on risk-sensitive control problems. Appl. Math. Optim. 2005, 52, 297–310. [Google Scholar] [CrossRef]
- Borkar, V.S. Q-learning for risk-sensitive control. Math. Oper. Res. 2002, 27, 294–311. [Google Scholar] [CrossRef]
- Fleming, W.H.; Sheu, S.J. Risk-sensitive control and an optimal investment model II. Ann. Appl. Probab. 2002, 12, 730–767. [Google Scholar] [CrossRef]
- Grandits, P.; Hubalek, F.; Schachermayer, W.; Žigo, M. Optimal expected exponential utility of dividend payments in a brownian risk model. Scand. Actuatial J. 2007, 2, 73–107. [Google Scholar] [CrossRef]
- Speyer, J. An adaptive terminal guidance scheme based on an exponential cost criterion with application to homing missile guidance. IEEE Trans. Autom. Control 1976, 21, 371–375. [Google Scholar] [CrossRef]
- Nguyen, D.H.; Yin, G.; Zhu, C. Certain properties related to well posedness of switching diffusions. Stochastic Process. Appl. 2017, 127, 3135–3158. [Google Scholar] [CrossRef]
- Yin, G.; Zhu, C. Properties of solutions of stochastic differential equations with continuous-state-dependent switching. J. Differ. Eqs. 2010, 249, 2409–2439. [Google Scholar] [CrossRef]
- Ghosh, M.K.; Arapostathis, A.; Marcus, S.I. Optimal control of switching diffusions with application to flexible manufacturing systems. SIAM J. Control Optim. 1993, 31, 1183–1204. [Google Scholar] [CrossRef]
- Sethi, S.P.; Zhang, Q. Hierarchical Decision Making in Stochastic Manufacturing Systems; Birkhäuser Boston, Inc.: Boston, MA, USA, 1994. [Google Scholar]
- Yin, G.; Dey, S. Weak convergence of hybrid filtering problems involving nearly completely decomposable hidden markov chains. SIAM J. Control Optim. 2003, 41, 1820–1842. [Google Scholar] [CrossRef]
- Yin, G.; Liu, R.H.; Zhang, Q. Recursive algorithms for stock liquidation: A stochastic optimization approach. SIAM J. Optim. 2002, 13, 240–263. [Google Scholar] [CrossRef]
- Mao, X.; Yuan, C. Stochastic Differential Equations with Markovian Switching; Imperial Colledge Press: London, UK, 2006. [Google Scholar]
- Yin, G.G.; Zhu, C. Hybrid Switching Diffusions: Properties and Applications; Springer: New York, NY, USA, 2010; Volume 63. [Google Scholar]
- Zhu, C.; Yin, G. On competitive lotka-volterra model in random environments. J. Math. Anal. Appl. 2009, 357, 154–170. [Google Scholar] [CrossRef]
- Lu, X. Constrained optimality for controlled switching diffusions with an application to stock purchasing. Quant. Financ. 2019, 19, 2069–2085. [Google Scholar] [CrossRef]
- Fleming, W.H.; Soner, H.M. Controlled Markov Processes and Viscosity Solutions, 2nd ed.; Springer: Berlin/Heidelberg, Germany, 2006. [Google Scholar]
- Yong, J.; Zhou, X. Stochastic Controls: Hamiltonian Systems and HJB Equations; Springer: Berlin/Heidelberg, Germany, 1999. [Google Scholar]
- Ksendal, B. Stochastic Differential Equations; Springer: New York, NY, USA, 1998. [Google Scholar]
- Deng, Y.; Liang, Z. Random Point Processes and Applications; Science Press: Beijing, China, 1992. (In Chinese) [Google Scholar]
- Ghosh, M.K.; Arapostathis, A.; Marcus, S.I. Ergodic control of switching diffusions. SIAM J. Control Optim. 1997, 35, 1952–1988. [Google Scholar] [CrossRef]
- Crandall, M.G.; Ishii, H.; Lions, P.L. User’s guide to viscosity solutions of second order partial differential equations. Bull. New Ser. Am. Math. Soc. 1992, 27, 1–67. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lu, X.; Sun, L. Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation. Mathematics 2024, 12, 38. https://doi.org/10.3390/math12010038
Lu X, Sun L. Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation. Mathematics. 2024; 12(1):38. https://doi.org/10.3390/math12010038
Chicago/Turabian StyleLu, Xianggang, and Lin Sun. 2024. "Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation" Mathematics 12, no. 1: 38. https://doi.org/10.3390/math12010038
APA StyleLu, X., & Sun, L. (2024). Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation. Mathematics, 12(1), 38. https://doi.org/10.3390/math12010038