# Dynamic Multiagent Incentive Contracts: Existence, Uniqueness, and Implementation

^{1}

^{2}

^{*}

^{†}

^{‡}

## Abstract

**:**

## 1. Introduction

- The principal desires to stimulate Agent 1 to exert output A and Agent 2 to exert output B. The outcomes of signing contracts are represented by the matrices in Table 1, where each entry is the principal’s and agent’s utility received from the contract. If these two contracts are signed separately, the unique equilibria are $\{L,A\}$ with Agent 1 and $\{L,B\}$ with Agent 2.
- We now assume that two agents’ outputs are aggregated in a linearly additive way. In this case, the principal’s dominant policy is $[{c}_{1},{c}_{2}]=[L,L]$. Notice that the existence and the number of equilibria may vary with the agents’ utility functions ${u}_{i}({c}_{i},{X}_{i},{X}_{-i})$. Three possible outcomes for the contracts are below:
- Unique Nash equilibrium: Assume that the utility of each agent is only dependent on its payoff, i.e., ${u}_{i}({c}_{i},{X}_{i},{X}_{-i})={c}_{i}$. The agents’ best responses are $[{X}_{1},{X}_{2}]=[A,B]$. With a fixed $[{c}_{1},{c}_{2}]=[L,L]$, their utility follows Table 2.
- Multiple Nash equilibria: Assuming that the principal rewards whoever delivers B an additional unit of compensation, there exists two Nash equilibria, $[{X}_{1},{X}_{2}]=[A,B]$ and $[{X}_{1},{X}_{2}]=[B,B]$, for which their utility follows Table 3.
- No Nash equilibrium: Assuming that the utility of each agent is affected by the other’s action such that the principal would reward the agents when their outputs match, i.e., ${u}_{i}({c}_{i},{X}_{i},{X}_{-i})={c}_{i}+2$ if ${X}_{i}={X}_{-i}$, then there is no Nash equilibrium, as seen in Table 4.

## 2. Setting

#### 2.1. Output Processes and Terminal Conditions

- The drift term ${f}_{i}:\mathcal{A}\to {\mathbb{R}}_{+}$ in (1) is in an ${L}^{2}$ space such that ${\int}_{0}^{T}{f}_{i}^{2}ds<\infty $ for all $i\in \left[n\right]$.
- ${f}_{i}$ is partially differentiable almost everywhere with respect to ${a}_{i}\left(t\right)$ for all $i\in \left[n\right]$.
- The diffusion term ${\sigma}_{i}$ is a known constant for all $i\in \left[n\right]$.
- The Brownian motions $\mathit{B}\left(t\right)={[{B}_{1}\left(t\right),\cdots ,{B}_{n}\left(t\right)]}^{T}$ are correlated with the correlation matrix $E\left(\mathit{B}\left(t\right)\mathit{B}{\left(t\right)}^{T}\right)=\mathsf{\Sigma}$, strongly positive definite, i.e., ${\mathit{x}}^{T}\mathsf{\Sigma}\mathit{x}\ge \mathit{\alpha}{\parallel \mathit{x}\parallel}^{2}$ for all $\mathit{x}\in {R}^{n}$ and some constant $\mathit{\alpha}>0$.

#### 2.2. Solving Optimal Contracts

## 3. Incentive-Compatible Constraints

#### 3.1. Parametrization of the Individual Agent’s Problem

**Proposition**

**1.**

**Proof.**

**Proposition**

**2.**

**Proof.**

#### 3.2. Multiagent Nash Equilibrium

- ${u}_{i}:\mathcal{A}\times {\mathcal{C}}_{i}\to \mathbb{R}$ is twice continuously differentiable, decreasing in ${c}_{i}$, and concave in ${a}_{i}$.
- ${f}_{i}:\mathcal{A}\to {\mathbb{R}}_{+}$ is twice continuously differentiable, increasing and concave in ${a}_{i}$.
- For each i and $\mathit{a}$, $\frac{\partial {f}_{i}({\mathit{a}}_{i},0)}{\partial {a}_{i}}\ne 0$ and ${f}_{i}\left(\mathit{a}\right)\to \infty $ while $\frac{\partial f\left(\mathit{a}\right)}{\partial {a}_{i}}\to 0$ as ${a}_{i}\to \infty $.
- The set ${\cap}_{i}\{(\mathit{a},\mathit{c}):{u}_{i}(\mathit{a},{c}_{i})\ge 0\mathrm{for}\mathrm{all}i\}$ is nonempty and compact.
- There exists an $m>0$ such that $m<{sup}_{x}{u}_{i}({\mathit{a}}_{-i},x,{c}_{i})$, and ${u}_{i}\to -\infty $ as $x\to \infty $, for all i and ${\mathit{a}}_{-i},{c}_{i}$.
- ${u}_{i}({\mathit{a}}_{-i},0,{c}_{i})\ge 0$ for each ${\mathit{a}}_{-i},{c}_{i}$.

**Lemma**

**1.**

**Proof.**

**Definition**

**1.**

**Lemma**

**2.**

**Proof.**

**Corollary**

**1.**

**Proof.**

**Theorem**

**1.**

**Proof.**

#### 3.3. On the Uniqueness of the Nash Equilibrium in Multiagent Contracts

- ${u}_{i}$ is strictly concave in ${a}_{i}$ and ${u}_{i}^{\prime}({\mathit{a}}_{-i},{a}_{i},{c}_{i}):=\frac{\partial {u}_{i}({\mathit{a}}_{-i},{a}_{i},{c}_{i})}{\partial {a}_{i}}<0$ for each $i\in \left[n\right]$ and each ${\mathit{a}}_{-i}$.
- Let for each $i\in \left[n\right]$, ${u}_{ij}^{{}^{\u2033}}:=\frac{{\partial}^{2}{u}_{i}}{\partial {a}_{i}\partial {a}_{j}}$ for each $i,j$ and, similarly, ${f}_{ij}^{{}^{\u2033}}$. The matrix ${D}^{2}{u}_{i}$ (${D}^{2}{f}_{i}$) is such that its ith row is strictly diagonally dominant (diagonally dominant) in variables $\mathit{a}$, i.e.,$$-{u}_{ii}^{{}^{\u2033}}>\sum _{i\ne j}|{u}_{ij}^{{}^{\u2033}}\left|\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\right(-{f}_{ii}^{{}^{\u2033}}\ge \sum _{i\ne j}|{f}_{ij}^{{}^{\u2033}}\left|\right).$$

**Remark**

**1.**

- 1.
- Condition 1 stipulates that the optimal effort the agents exert is unique and also has a negative effect on their instantaneous utility, i.e., the marginal utility as a function of the agent i’s effort ${a}_{i}$ is negative.
- 2.
- Condition 2 states that agent i’s particular decision mostly affects the decrease in his or her marginal utility. In contrast, the other agents’ efforts have a minor effect (note the strict concavity implies that ${u}_{ii}^{{}^{\u2033}}$ is negative).
- 3.
- The signs of ${u}_{ij}^{\u2033}$ are related to whether ${a}_{i}$ is a strategic complement or a strategic substitute [36]. Diagonal dominance thus assumes that the magnitude of the effect of any agent’s actions exceeds the magnitude of the combined strategic effects of all the other agents’ actions.

**Lemma**

**3.**

**Proof.**

**Theorem**

**2.**

**Proof.**

## 4. The Optimal Multiagent Contracts

**Assumption**

**1.**

**Remark**

**2.**

**Theorem**

**3.**

**Proof.**

#### Iterative Algorithm for Solving Multiagent Contracts

- Initialize the terminal condition $F(T,\mathit{s})=-{\mathbf{1}}^{\u22ba}\mathsf{\Phi}\left(\mathit{z}\right)$.
- While $t=T-\tau \Delta t\ge 0$, with a fixed $\u03f5>0$,
- (a)
- For each state $\mathit{w},\mathit{x},\mathit{z}$, start with an arbitrary contract ${\mathit{v}}_{0}=\{{\mathit{c}}_{0},{\mathit{y}}_{0}\}$.
- (b)
- Solve a fixed point problem such that ${\mathit{a}}^{\ast}\left(t\right)\in \mathsf{\Theta}\left({\mathit{v}}_{0}\right)$. If the conditions in Section 3.3 are satisfied, the equilibrium is unique.
- (c)
- (d)
- Optimize the objective value $\tilde{F}(t,\mathit{s})$ for each state $\mathit{s}=(\mathit{w},\mathit{x},\mathit{z})$ by the gradient ascent method. The gradient is ${\nabla}_{\mathit{v}}\tilde{F}\in {\mathbb{R}}^{2n}$, and the step size $\gamma $ can be determined by a line-search method. If $\parallel {\nabla}_{\mathit{v}}\tilde{F}\parallel \ge \u03f5$, go back to (b) with the new contracts ${\mathit{v}}_{0}\leftarrow (\mathit{c},\mathit{y})$.
- (e)
- Go to Step 3 if $\parallel {\nabla}_{\mathit{v}}\tilde{F}\parallel <\u03f5$.

- Update the contracts $\left\{\mathit{c}\right(t),\mathit{y}(t\left)\right\}$ and continuation value $F(t,\mathit{s})$. Go to Step 2 with $\tau \leftarrow \tau +1$.

**Lemma**

**4.**

**Proof.**

## 5. Conclusions

- The Martingale approach is restricted to the SDE output process, where the each agent’s decision only affects the drift term. An extension to controlling the diffusion of output process may cause significant technical difficulties even in the single-agent case.
- The coupled gradient-based and fixed-point optimization restricts the computational efficiency of solving the contracts. In the absence of a unique multiagent Nash equilibrium, the proposed algorithm can only compute local optimum contracts, and thus, the verification theorem in Theorem 3 fails. Developing more efficient algorithms for multiagent contracts and with multiple Nash equilibria is a meaningful future direction.

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

IC | incentive-compatible |

IR | individual-rational |

PDE | partial differential equation |

SPNE | subgame perfect Nash equilibrium |

HJB | Hamilton–Jacobi–Bellman equation |

## References

- Aïd, R.; Possamaï, D.; Touzi, N. Optimal electricity demand response contracting with responsiveness incentives. arXiv
**2018**, arXiv:1810.09063. [Google Scholar] - Brunnermeier, M.K.; Sannikov, Y. A macroeconomic model with a financial sector. Am. Econ. Rev.
**2014**, 104, 379–421. [Google Scholar] [CrossRef][Green Version] - Cvitanic, J.; Zhang, J. Contract Theory in Continuous-Time Models; Springer Science & Business Media: Berlin, Germany, 2012. [Google Scholar]
- DeMarzo, P.M.; Sannikov, Y. Optimal security design and dynamic capital structure in a continuous-time agency model. J. Financ.
**2006**, 61, 2681–2724. [Google Scholar] [CrossRef] - Faingold, E.; Sannikov, Y. Reputation in continuous-time games. Econometrica
**2011**, 79, 773–876. [Google Scholar] [CrossRef] - Fuchs, W. Contracting with repeated moral hazard and private evaluations. Am. Econ. Rev.
**2007**, 97, 1432–1448. [Google Scholar] [CrossRef][Green Version] - Guo, L.; Ye, J.J. Necessary optimality conditions for optimal control problems with equilibrium constraints. SIAM J. Control Optim.
**2016**, 54, 2710–2733. [Google Scholar] [CrossRef][Green Version] - Luo, Q.; Saigal, R.; Chen, Z.; Yin, Y. Accelerating the adoption of automated vehicles by subsidies: A dynamic games approach. Transp. Res. Part B Methodol.
**2019**, 129, 226–243. [Google Scholar] [CrossRef] - Mastrolia, T.; Ren, Z. Principal-Agent problem with common agency without communication. SIAM J. Financ. Math.
**2018**, 9, 775–799. [Google Scholar] [CrossRef][Green Version] - Nadtochiy, S.; Zariphopoulou, T. Optimal Contract for a Fund Manager with Capital Injections and Endogenous Trading Constraints. SIAM J. Financ. Math.
**2019**, 10, 698–722. [Google Scholar] [CrossRef][Green Version] - Laffont, J.J.; Martimort, D. The Theory of Incentives: The Principal-Agent Model; Princeton University Press: Princeton, NJ, USA, 2009. [Google Scholar]
- Piskorski, T.; Tchistyi, A. Optimal mortgage design. Rev. Financ. Stud.
**2010**, 23, 3098–3140. [Google Scholar] [CrossRef] - Williams, N. A solvable continuous time dynamic principal–agent model. J. Econom. Theory
**2015**, 159, 989–1015. [Google Scholar] [CrossRef][Green Version] - Demarzo, P.M.; Sannikov, Y. Learning, termination, and payout policy in dynamic incentive contracts. Rev. Econ. Stud.
**2016**, 84, 182–236. [Google Scholar] [CrossRef][Green Version] - Cvitanić, J.; Possamaï, D.; Touzi, N. Dynamic programming approach to principal–agent problems. Financ. Stoch.
**2018**, 22, 1–37. [Google Scholar] [CrossRef][Green Version] - Spear, S.E.; Srivastava, S. On repeated moral hazard with discounting. Rev. Econ. Stud.
**1987**, 54, 599–617. [Google Scholar] [CrossRef] - Holmstrom, B.; Milgrom, P. Aggregation and linearity in the provision of intertemporal incentives. Econometrica
**1987**, 55, 303–328. [Google Scholar] [CrossRef] - Schättler, H.; Sung, J. The first-order approach to the continuous-time principal–agent problem with exponential utility. J. Econom. Theory
**1993**, 61, 331–371. [Google Scholar] [CrossRef] - Sung, J. Linearity with project selection and controllable diffusion rate in continuous-time principal-agent problems. RAND J. Econ.
**1995**, 26, 720–743. [Google Scholar] [CrossRef] - Müller, H.M. Asymptotic efficiency in dynamic principal-agent problems. J. Econom. Theory
**2000**, 91, 292–301. [Google Scholar] [CrossRef][Green Version] - Sannikov, Y. A continuous-time version of the principal-agent problem. Rev. Econ. Stud.
**2008**, 75, 957–984. [Google Scholar] [CrossRef] - Cvitanić, J.; Possamaï, D.; Touzi, N. Moral hazard in dynamic risk management. Manag. Sci.
**2016**, 63, 3328–3346. [Google Scholar] [CrossRef][Green Version] - Keun Koo, H.; Shim, G.; Sung, J. Optimal Multi-Agent Performance Measures for Team Contracts. Math. Financ.
**2008**, 18, 649–667. [Google Scholar] [CrossRef] - Thakur, A. Continuous-Time Principal Multi-Agent Problem: Moral Hazard in Teams & Fiscal Federalism; Working Paper; Princeton University: Princeton, NJ, USA, 2015. [Google Scholar]
- Georgiadis, G. Projects and team dynamics. Rev. Econ. Stud.
**2014**, 82, 187–218. [Google Scholar] [CrossRef] - Elie, R.; Mastrolia, T.; Possamaï, D. A Tale of a Principal and Many, Many Agents. Math. Oper. Res.
**2018**, 377–766. [Google Scholar] [CrossRef] - Elie, R.; Possamaï, D. Contracting theory with competitive interacting agents. SIAM J. Control Optim.
**2019**, 57, 1157–1188. [Google Scholar] [CrossRef][Green Version] - Englmaier, F.; Wambach, A. Optimal incentive contracts under inequity aversion. Games Econom. Behav.
**2010**, 69, 312–328. [Google Scholar] [CrossRef][Green Version] - Goukasian, L.; Wan, X. Optimal incentive contracts under relative income concerns. Math. Financ. Econ.
**2010**, 4, 57–86. [Google Scholar] [CrossRef][Green Version] - Ma, C.T. Unique implementation of incentive contracts with many agents. Rev. Econ. Stud.
**1988**, 55, 555–572. [Google Scholar] [CrossRef] - Demski, J.S.; Sappington, D. Optimal incentive contracts with multiple agents. J. Econom. Theory
**1984**, 33, 152–171. [Google Scholar] [CrossRef] - Holmstrom, B.; Milgrom, P. Multitask principal-agent analyses: Incentive contracts, asset ownership, and job design. J. Econ. Org.
**1991**, 7, 24. [Google Scholar] [CrossRef][Green Version] - Gale, D.; Nikaido, H. The Jacobian matrix and global univalence of mappings. Math. Ann.
**1965**, 159, 81–93. [Google Scholar] [CrossRef] - Kojima, M.; Saigal, R. A study of PC
^{1}homeomorphisms on subdivided polyhedrons. SIAM J. Math. Anal.**1979**, 10, 1299–1312. [Google Scholar] [CrossRef] - Shan, Y. Optimal contracts for research agents. RAND J. Econ.
**2017**, 48, 94–124. [Google Scholar] [CrossRef] - Osborne, M.J.; Rubinstein, A. A Course in Game Theory; MIT Press: Cambridge, MA, USA, 1994. [Google Scholar]
- Householder, A.S. The Theory of Matrices in Numerical Analysis; Courier Corporation: North Chelmsford, MA, USA, 2013. [Google Scholar]
- Jensen, M.; Smears, I. On the convergence of finite element methods for Hamilton–Jacobi–Bellman equations. SIAM J. Numer. Anal.
**2013**, 51, 137–162. [Google Scholar] [CrossRef][Green Version] - Eaves, B.C.; Saigal, R. Homotopies for computing fixed points in unbounded regions. Math. Program.
**1972**, 3, 225–237. [Google Scholar] [CrossRef] - Evans, L.C. Partial Differential Equations; American Mathematical Society: Providence, RI, USA, 2010; Volume 19. [Google Scholar]
- Chernov, A. On Some Approaches to Find Nash Equilibrium in Concave Games. Autom. Remote Control
**2019**, 80, 964–988. [Google Scholar] [CrossRef] - Yin, Y.; Luo, Q.; Zhan, J.; Saigal, R. Dynamic Subsidies for a Synergy between Charging Infrastructure Development and Electric Vehicle Adoption; Working Paper; University of Michigan: Ann Arbor, MI, USA, 2020. [Google Scholar]

Agent 1 | Agent 2 | |||
---|---|---|---|---|

$\mathit{A}$ | $\mathit{B}$ | $\mathit{A}$ | $\mathit{B}$ | |

L | 4,2 | 2,1 | 2,1 | 4,2 |

H | 3,3 | 1,2 | 1,2 | 3,3 |

A | B | |
---|---|---|

A | 2,1 | 2,2 |

B | 1,1 | 1,2 |

A | B | |
---|---|---|

A | 2,1 | 2,3 |

B | 2,1 | 2,3 |

A | B | |
---|---|---|

A | 4,3 | 2,2 |

B | 1,1 | 3,4 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Luo, Q.; Saigal, R. Dynamic Multiagent Incentive Contracts: Existence, Uniqueness, and Implementation. *Mathematics* **2021**, *9*, 19.
https://doi.org/10.3390/math9010019

**AMA Style**

Luo Q, Saigal R. Dynamic Multiagent Incentive Contracts: Existence, Uniqueness, and Implementation. *Mathematics*. 2021; 9(1):19.
https://doi.org/10.3390/math9010019

**Chicago/Turabian Style**

Luo, Qi, and Romesh Saigal. 2021. "Dynamic Multiagent Incentive Contracts: Existence, Uniqueness, and Implementation" *Mathematics* 9, no. 1: 19.
https://doi.org/10.3390/math9010019