# A Discrete-Time Homing Problem with Two Optimizers

## Abstract

**:**

## 1. Introduction

**Remark**

**1.**

## 2. Dynamic Programming

**Proposition**

**1.**

**Remark**

**2.**

**Remark**

**3.**

**Proposition**

**2.**

**Remark**

**4.**

**Proposition**

**3.**

**Proof.**

## 3. Optimal Choice for ${\mathit{u}}_{\mathit{n}}$

**Remark**

**5.**

**Proposition**

**4.**

**Remark**

**6.**

## 4. Numerical Examples

#### 4.1. Critical Value of $\mathit{\lambda}$

#### 4.2. Solution of the Non-Linear Difference Equation

## 5. Conclusions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Hämäläinen, R.P. Nash and Stackelberg solutions to general linear-quadratic two player difference games. I: Open-loop and feedback strategies. Kybernetika
**1978**, 14, 38–56. [Google Scholar] - Reddy, P.V.; Zaccour, G. Feedback Nash equilibria in linear-quadratic difference games with constraints. IEEE Trans. Automat. Control
**2017**, 62, 590–604. [Google Scholar] [CrossRef] - Lin, Y. Feedback Stackelberg strategies for the discrete-time mean-field stochastic systems in infinite horizon. J. Frank. Inst.
**2019**, 356, 5222–5239. [Google Scholar] [CrossRef] - Liu, X.; Liu, R.; Li, Y. Infinite time linear quadratic Stackelberg game problem for unknown stochastic discrete-time systems via adaptive dynamic programming approach. Asian J. Control
**2020**, 2, 937–948. [Google Scholar] [CrossRef] - Ju, P.; Li, X.; Lei, J.; Li, T. Optimal linear strategy for stochastic games with one-side control-sharing information. Int. J. Control
**2022**, 1–7. [Google Scholar] [CrossRef] - Whittle, P. Optimization over Time; Wiley: Chichester, UK, 1982; Volume 1. [Google Scholar]
- Whittle, P. Risk-Sensitive Optimal Control; Wiley: Chichester, UK, 1990. [Google Scholar]
- Kuhn, J. The risk-sensitive homing problem. J. Appl. Probab.
**1985**, 22, 796–803. [Google Scholar] [CrossRef] - Makasu, C. Risk-sensitive control for a class of homing problems. Automatica
**2009**, 45, 2454–2455. [Google Scholar] [CrossRef] - Lefebvre, M. An explicit solution to a discrete-time stochastic optimal control problem. WSEAS Trans. Syst.
**2023**, 22, Art. #40. 368–371. [Google Scholar] [CrossRef] - Lefebvre, M.; Kounta, M. Discrete homing problems. Arch. Control Sci.
**2013**, 23. [Google Scholar] [CrossRef] - Lefebvre, M. The homing problem for autoregressive processes. IMA J. Math. Control Inform.
**2022**, 39, 322–344. [Google Scholar] [CrossRef] - Kounta, M.; Dawson, N.J. Linear quadratic Gaussian homing for Markov processes with regime switching and applications to controlled population growth/decay. Methodol. Comput. Appl. Probab.
**2021**, 23, 1155–1172. [Google Scholar] [CrossRef] - Feller, W. An Introduction to Probability Theory and its Applications; Wiley: New York, NY, USA, 1968; Volume I. [Google Scholar]

**Figure 1.**Functions $G\left(1\right)$ (solid line) and $H\left(1\right)$ when $\lambda \in [1.95,2.05]$.

**Figure 2.**Functions $G\left(10\right)$ (solid line) and $H\left(10\right)$ when $\lambda \in [2.157,2.158]$.

**Table 1.**Functions $F\left(x\right)$, $\mathsf{\Phi}\left(x\right)$, $\mathsf{\Psi}\left(x\right)$, $G\left(x\right)$ and $H\left(x\right)$, and optimal control ${u}_{0}^{*}\left(x\right)$ for $x=1,2,\dots ,10$ when $\lambda =1/2$.

x | $\mathit{F}\left(\mathit{x}\right)$ | $\mathsf{\Phi}\left(\mathit{x}\right)$ | $\mathsf{\Psi}\left(\mathit{x}\right)$ | $\mathit{G}\left(\mathit{x}\right)$ | $\mathit{H}\left(\mathit{x}\right)$ | ${\mathit{u}}_{0}^{*}\left(\mathit{x}\right)$ |
---|---|---|---|---|---|---|

1 | $-1$ | $0.5$ | $-1$ | $0.5$ | $-1$ | 0 |

2 | $-1$ | $0.75$ | $-1$ | 0 | $-1$ | 0 |

3 | $-2$ | $0.875$ | $-2$ | 0 | $-2$ | 0 |

4 | $-2$ | $1.1875$ | $-2$ | $-1$ | $-2$ | 0 |

5 | $-3$ | $1.4687$ | $-3$ | $-1$ | $-3$ | 0 |

6 | $-3$ | $1.6719$ | $-3$ | $-2$ | $-3$ | 0 |

7 | $-4$ | $1.9297$ | $-4$ | $-2$ | $-4$ | 0 |

8 | $-4$ | $2.1992$ | $-4$ | $-3$ | $-4$ | 0 |

9 | $-5$ | $2.4355$ | $-5$ | $-3$ | $-5$ | 0 |

10 | $-5$ | $2.6826$ | $-5$ | $-4$ | $-5$ | 0 |

**Table 2.**Functions $F\left(x\right)$, $\mathsf{\Phi}\left(x\right)$, $\mathsf{\Psi}\left(x\right)$, $G\left(x\right)$ and $H\left(x\right)$, and optimal control ${u}_{0}^{*}\left(x\right)$ for $x=1,2,\dots ,10$ when $\lambda =1$.

x | $\mathit{F}\left(\mathit{x}\right)$ | $\mathsf{\Phi}\left(\mathit{x}\right)$ | $\mathsf{\Psi}\left(\mathit{x}\right)$ | $\mathit{G}\left(\mathit{x}\right)$ | $\mathit{H}\left(\mathit{x}\right)$ | ${\mathit{u}}_{0}^{*}\left(\mathit{x}\right)$ |
---|---|---|---|---|---|---|

1 | 0 | 1 | 0 | 1 | 0 | 0 |

2 | 0 | $1.5$ | 0 | 1 | 0 | 0 |

3 | 0 | $1.75$ | 0 | 1 | 0 | 0 |

4 | 0 | $2.375$ | 0 | 1 | 0 | 0 |

5 | 0 | $2.937$ | 0 | 1 | 0 | 0 |

6 | 0 | $3.344$ | 0 | 1 | 0 | 0 |

7 | 0 | $3.859$ | 0 | 1 | 0 | 0 |

8 | 0 | $4.398$ | 0 | 1 | 0 | 0 |

9 | 0 | $4.871$ | 0 | 1 | 0 | 0 |

10 | 0 | $5.365$ | 0 | 1 | 0 | 0 |

**Table 3.**Functions $F\left(x\right)$, $\mathsf{\Phi}\left(x\right)$, $\mathsf{\Psi}\left(x\right)$, $G\left(x\right)$ and $H\left(x\right)$, and optimal control ${u}_{0}^{*}\left(x\right)$ for $x=1,2,\dots ,10$ when $\lambda =2$.

x | $\mathit{F}\left(\mathit{x}\right)$ | $\mathsf{\Phi}\left(\mathit{x}\right)$ | $\mathsf{\Psi}\left(\mathit{x}\right)$ | $\mathit{G}\left(\mathit{x}\right)$ | $\mathit{H}\left(\mathit{x}\right)$ | ${\mathit{u}}_{0}^{*}\left(\mathit{x}\right)$ |
---|---|---|---|---|---|---|

1 | 2 | 2 | 2 | 2 | 2 | $-1$ or 0 |

2 | 2 | 3 | 2 | 3 | 2 | 0 |

3 | 3 | $3.5$ | 4 | 3 | $3.75$ | $-1$ |

4 | 4 | $4.75$ | 4 | $4.75$ | 4 | 0 |

5 | 5 | $5.875$ | 6 | 5 | $5.687$ | $-1$ |

6 | 6 | $6.687$ | 6 | $6.687$ | 6 | 0 |

7 | 7 | $7.719$ | 8 | 7 | $7.797$ | $-1$ |

8 | 8 | $8.797$ | 8 | $8.797$ | 8 | 0 |

9 | 9 | $9.742$ | 10 | 9 | $9.730$ | $-1$ |

10 | 10 | $10.730$ | 10 | $10.730$ | 10 | 0 |

**Table 4.**Functions $F\left(x\right)$, $\mathsf{\Phi}\left(x\right)$, $\mathsf{\Psi}\left(x\right)$, $G\left(x\right)$ and $H\left(x\right)$, and optimal control ${u}_{0}^{*}\left(x\right)$ for $x=1,2,\dots ,10$ when $\lambda =10$.

x | $\mathit{F}\left(\mathit{x}\right)$ | $\mathsf{\Phi}\left(\mathit{x}\right)$ | $\mathsf{\Psi}\left(\mathit{x}\right)$ | $\mathit{G}\left(\mathit{x}\right)$ | $\mathit{H}\left(\mathit{x}\right)$ | ${\mathit{u}}_{0}^{*}\left(\mathit{x}\right)$ |
---|---|---|---|---|---|---|

1 | 10 | 10 | 18 | 10 | 14 | $-1$ |

2 | 15 | 15 | 18 | 15 | $16.5$ | $-1$ |

3 | $17.5$ | $17.5$ | 36 | $17.5$ | $22.75$ | $-1$ |

4 | $23.75$ | $23.75$ | 36 | $23.75$ | $28.375$ | $-1$ |

5 | $29.375$ | $29.375$ | 54 | $29.375$ | $32.437$ | $-1$ |

6 | $33.437$ | $33.437$ | 54 | $33.437$ | $37.594$ | $-1$ |

7 | $38.594$ | $38.594$ | 72 | $38.594$ | $42.984$ | $-1$ |

8 | $43.984$ | $43.984$ | 72 | $43.984$ | $47.711$ | $-1$ |

9 | $48.711$ | $48.711$ | 90 | $48.711$ | $52.652$ | $-1$ |

10 | $53.652$ | $53.652$ | 90 | $53.652$ | $57.818$ | $-1$ |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Lefebvre, M.
A Discrete-Time Homing Problem with Two Optimizers. *Games* **2023**, *14*, 68.
https://doi.org/10.3390/g14060068

**AMA Style**

Lefebvre M.
A Discrete-Time Homing Problem with Two Optimizers. *Games*. 2023; 14(6):68.
https://doi.org/10.3390/g14060068

**Chicago/Turabian Style**

Lefebvre, Mario.
2023. "A Discrete-Time Homing Problem with Two Optimizers" *Games* 14, no. 6: 68.
https://doi.org/10.3390/g14060068