A Generalized Finite Difference Method for Solving Hamilton–Jacobi–Bellman Equations in Optimal Investment

Abstract

This paper studies the numerical algorithm of stochastic control problems in investment optimization. Investors choose the optimal investment to maximize the expected return under uncertainty. The optimality condition, the Hamilton–Jacobi–Bellman (HJB) equation, satisfied by the value function and obtained by the dynamic programming method, is a partial differential equation coupled with optimization. One of the major computational difficulties is the irregular boundary conditions presented in the HJB equation. In this paper, two mesh-free algorithms are proposed to solve two different cases of HJB equations with regular and irregular boundary conditions. The model of optimal investment under uncertainty developed by Abel is used to study the efficacy of the proposed algorithms. Extensive numerical studies are conducted to test the impact of the key parameters on the numerical efficacy. By comparing the numerical solution with the exact solution, the proposed numerical algorithms are validated.

1. Introduction

The stochastic optimal control problem of investment optimization, which involves making investment strategies under uncertainty, has always been an important issue in finance and economics. In 1970, Merton [1] conducted a pioneering study on stochastic optimal control in finance, which garnered the application of optimal control in finance widespread attention. Similar theoretical and applied analysis in capital investment includes Abel [2], Karatzas [3], Dixit and Pindyck [4], and Zhou and Li [5].

Stochastic optimal control problems are usually difficult to solve directly in terms of control and state variables. Therefore, most existing studies use indirect methods which are based on the optimality conditions derived from the problem. The indirect methods include the dynamic programming [6] and the maximum principle [7,8,9]. The dynamic programming method derives the Hamilton-Jacobi–Bellman (HJB) equation satisfied by the value function that maximizes the objective function. The optimal control problem is then converted into a problem solving the partial differential equations (PDEs) coupled with optimization. The resulting PDEs are often nonlinear, and thus remain challenging to solve. By solving the corresponding HJB equations, the optimal control and value function of the control problems are derived. This approach is the most commonly used solution technique in finance [10,11,12].

The classical numerical algorithms for HJB equations are grid methods (mesh methods) such as the finite difference method (FDM). Wang et al. [13] approximated the viscosity solution of the HJB equation using the upwind finite difference. Peyrl et al. [14] proposed a successive approximation algorithm, where the partial differential equations are solved with the upwind difference scheme. Ma and Ma [15] developed the iterative finite difference methods with policy iterations to solved the sequence of decoupled HJB equations. Inoue et al. [16] investigated the convergence properties of the upwind difference scheme for HJB equations. Forsyth et al. [17] studied the optimal control problem in nonlinear option pricing, using Newton’s iteration method to solve the implicit discretized control equations. Wang et al. [12] used a fully implicit time-stepping finite difference method to solve a nonlinear HJB equation derived from an asset allocation problem based on the mean-variance approach. In addition, there are the finite element method [18,19,20,21,22,23] and the finite volume method [24,25,26].

Mesh methods have been widely studied and applied in partial differential equations. However, mesh methods require a regular mesh of grids and are only applicable to regular boundaries, whereas, in dynamic stochastic economic models, the natural modeling domain may have an irregular shape [27,28]. For example, in the option pricing problem, the pricing model for American backdated options can be described as a two-dimensional free boundary problem [29,30,31]. De Angelis et al. [32] considered the problem of optimal entry into an irreversible investment scheme with a cost function that is non-convex for the control variables, where it is necessary to study optimal entry policies with an irregular boundary. If the state space is irregular, the special handling of region interiors and boundaries is required, which may increase some computational complexity [33].

As a consequence, mesh-free numerical methods have been proposed. Relevant research on meshless or semi-meshless methods for solving partial differential equations includes the global radial basis function method [34], the least squares configuration radial basis function method [35], the Haar wavelet collocation method [36], the Chebyshev method [37], the generalized finite difference method (hereafter GFDM) (see Benito et al. [38,39]), etc. Among them, the GFDM is one of the most promising methods. Based on Taylor series expansion and moving least squares approximation, the GFDM approximates the spatial derivatives at each discrete node as the linear combination of node values in its neighborhood. It overcomes the dependence of the traditional difference method on a regular grid and allows more flexibility in obtaining approximations for functions. Existing research work, such as nonlinear parabolic and hyperbolic partial differential equations studied by Benito et al. [40] and Ureña et al. [41], and linear elliptic partial differential equations studied by Gavete et al. [39], showed that the GFDM is very effective in solving second-order partial differential equations.

Our major contribution is to solve the HJB equation using a numerical scheme based on a meshless method, the GFDM, which is suitable for handling both regular and irregular boundaries. We propose two numerical algorithms, one for the general cases and the other for the case where the optimal control in the HJB equation can be explicitly obtained. For general cases, we propose a successive approximation algorithm which combines the GFDM discretization scheme with the optimization algorithm. For special cases with the explicit expression of optimal control, we first obtain the discretization scheme using the GFDM and then solve the discretized nonlinear equations using Newton’s iterative method. We carry out experiments under regular and irregular boundaries, and verify the validity of the proposed numerical algorithms. We also test the effect of key parameters on the accuracy of the proposed algorithms including total points, weighting functions, and irregular index.

The remainder of this paper is organized as follows. Section 2 presents a review of the HJB equation and the GFDM. Section 3 develops the numerical methods to solve the HJB equation. In Section 4, an example of the optimal investment problem is applied to demonstrate the efficiency and accuracy of the numerical methods proposed. Moreover, the key parameters of the algorithms are also tested to verify the validity of the algorithms. Section 5 provides concluding remarks.

2. Review of HJB Equation and GFDM

2.1. Review of Stochastic Optimal Control Problem and HJB Equation

In this section, we will give a review of stochastic optimal control problems over an infinite time horizon. Let us consider the state variable $x\left(s\right)$ which satisfies the following stochastic differential equation

$$\left\{\begin{array}{c}dx\left(s\right)=g(x,u)ds+\sigma (x,u)dw,\hfill \\ x\left(t\right)=x,\hfill \end{array}\right.$$

(1)

together with the infinite horizon objective functional

$$J(t,x,u(.))=\mathbb{E}{\int }_t^{+\infty }{e}^{-\alpha (s-t)}f(x,u(.))ds.$$

(2)

The goal of the optimal control problem is to maximize $J(t,x,u(.\left)\right)$, that is

$$\underset{u(.)}{sup}J(t,x,u(.)),$$

(3)

where we assume that u is the control and the set of admissible controls has the form

$${\mathbb{U}}_{ad}=\{u\in \mathbb{U}|u\left(t\right)\in U_{ad},t\ge 0\},$$

with $\mathbb{U}=L^2(0,\infty ;{\mathbb{R}}^m)$, and ${\mathbb{U}}_{ad}\subset {\mathbb{R}}^m$ denotes a compact convex subset. The known constant $\alpha$ is a discount rate. $w\left(t\right)$ is a k-dimensional standard Brownian motion on a probability space $(\Omega ,\mathcal{F},P)$, $g(x,u):G\times \mathbb{U}\to {\mathbb{R}}^n$ and $\sigma (x,u):G\times \mathbb{U}\to {\mathbb{R}}^{n\times k}$ denote the drift and diffusion terms, respectively, which satisfy the linear growth condition and Lipschitz condition such that the stochastic differential equation for the state variable $x\left(t\right)\in G$ has a unique strong solution [42], where G is some open and bounded set on ${\mathbb{R}}^n$.

We define the value function of the problem as follows:

$$V(t,x)=\underset{u(.)}{sup}\mathbb{E}{\int }_t^{+\infty }{e}^{-\alpha (s-t)}f(x,u(.))ds,$$

(4)

and set $V\left(x\right):=V(0,x)$. This value function gives the best value for every initial condition, given the set of admissible control. Define the Hamiltonian as

$$H(x,DV,D^{2}V)=\underset{u(.)}{sup}[f(x,u)+\mathcal{A}(x,u)V\left(x\right)].$$

(5)

It is characterized as the solution of the Hamilton–Jacobi–Bellman (HJB) equation

$$\alpha V\left(x\right)=\underset{u(.)}{sup}[f(x,u)+\mathcal{A}(x,u)V\left(x\right)],x\in G,$$

(6)

where the operator $\mathcal{A}$ is defined as

$$\mathcal{A}(x,u)=\frac{1}{2}\sum _{i,j}{a}_{ij}\frac{{\partial }^2}{\partial x_i\partial x_j}+\sum _i{g}_i\frac{\partial }{\partial x_i},$$

(7)

and $a=\sigma (x,u)\sigma {(x,u)}^T$ is a symmetric matrix.

The HJB Equation (6) includes two subproblems to be solved: the Hamiltonian operator to be optimized and the partial differential equation.

2.2. Review of Generalized Finite Difference Method

In this paper, the GFDM, mainly based on the Taylor series expansion and the weighted least squares method, is used to approximate the spatial derivatives. In order to derive the GFD scheme for the HJB Equation (6), we first review the basic principles of GFDM in Benito et al. [38,39].

Assuming that the solution domain for an unknown function V is $\mathit{Z}\subset {\mathbb{R}}^2$, we need to discretize the solution domain Z into N points, namely $\mathit{Z}=\{{\mathit{x}}_{\mathbf{1}},{\mathit{x}}_{\mathbf{2}},\dots ,{\mathit{x}}_{\mathit{N}}\}$. For each ${\mathit{x}}_{\mathit{c}}=({\mathit{y}}_{\mathit{c}},{\mathit{K}}_{\mathit{c}})\in Z$, we define the subdomain $\mathit{M}=\{{\mathit{x}}_{\mathit{c}};{\mathit{x}}_{\mathbf{1}},{\mathit{x}}_{\mathbf{2}},\dots ,{\mathit{x}}_{\mathit{p}}\}\subset \mathit{Z}$ where the center point ${\mathit{x}}_{\mathit{c}}\in Z$, and ${\mathit{x}}_{\mathit{i}}=({\mathit{y}}_{\mathit{i}},{\mathit{K}}_{\mathit{i}})\in \mathit{Z}$ where $\mathit{i}=1,\dots ,\mathit{p}$ is a set of points located in the neighborhood of ${\mathit{x}}_{\mathit{c}}$ selected based on certain criteria such as a four-quadrant method or distance criterion. Define ${\mathit{V}}_{\mathit{c}}=\mathit{V}\left({\mathit{x}}_{\mathit{c}}\right)$ and ${\mathit{V}}_{\mathit{i}}=\mathit{V}\left({\mathit{x}}_{\mathit{i}}\right)$. By Taylor series expansion, $V_i$ expanded at the center point ${\mathit{x}}_{\mathit{c}}$ is

$$\begin{array}{c}\hfill V_i=V_c+h_i{\displaystyle \frac{\partial V_c}{\partial y}}+l_i{\displaystyle \frac{\partial V_c}{\partial K}}+\frac{1}{2}(h_i^2\frac{{\partial }^2{V}_c}{\partial y^2}+l_i^2\frac{{\partial }^2{V}_c}{\partial K^2}+2h_i{l}_i\frac{{\partial }^2{V}_c}{\partial y\partial K})+\cdots ,\end{array}$$

(8)

where $h_i=y_i-y_c,l_i=K_i-K_c$ with $i=1,2,\dots ,p$.

We only retain the Taylor expansion formula to the second-order term, so that the approximate value with second-order accuracy of $V_i$ can be obtained, and the approximate value of $V_i$ truncated at the second order is set to $v_i$. Based on the formula (8), the weighted residual function can be defined as

$$\begin{array}{c}\hfill F\left(v\right)=\sum _{i=1}^p\left\{w_i^2{[-v_i+v_c+h_i\frac{\partial v_c}{\partial y}+l_i\frac{\partial v_c}{\partial K}+\frac{1}{2}(h_i^2\frac{{\partial }^2{v}_c}{\partial y^2}+l_i^2\frac{{\partial }^2{v}_c}{\partial K^2}+2h_i{l}_i\frac{{\partial }^2{v}_c}{\partial y\partial K})]}^2\right\},\end{array}$$

(9)

where $w_i=w\left({\mathit{x}}_{\mathit{i}}\right)$ denotes the weighting function at point ${\mathit{x}}_{\mathit{i}}$. The weighting functions are usually in the form: $e^{-n{\left(dist\right)}^2}$ or $\frac{1}{dis{t}^n}$, where $dist$ is the distance between ${\mathit{x}}_{\mathit{i}}$ and ${\mathit{x}}_{\mathit{c}}$, $n\in \mathbb{N}$.

We define

$$\begin{array}{c}\hfill \mathit{H}={\left[\frac{\partial v_c}{\partial y},\frac{\partial v_c}{\partial K},\frac{{\partial }^2{v}_c}{\partial y^2},\frac{{\partial }^2{v}_c}{\partial K^2},\frac{{\partial }^2{v}_c}{\partial y\partial K}\right]}^T.\end{array}$$

(10)

The goal is to minimize the weighted residual function (9) with respect to the partial derivatives. Taking the partial derivatives of function (9) with respect to (10) and setting them equal to 0, i.e., $\frac{\partial F}{\mathbf{\partial }\mathit{H}}=\mathbf{0}$, we have

$$\begin{array}{cc}\hfill \frac{\partial F}{\partial \left(\frac{\partial v_c}{\partial y}\right)}& =2\sum _{i=1}^p{w}_i^2{h}_i\Phi =0,\hfill \\ \hfill \frac{\partial F}{\partial \left(\frac{\partial v_c}{\partial K}\right)}& =2\sum _{i=1}^p{w}_i^2{l}_i\Phi =0,\hfill \\ \hfill \frac{\partial F}{\partial \left(\frac{{\partial }^2{v}_c}{\partial y^2}\right)}& =2\sum _{i=1}^p{w}_i^2\frac{h_i^2}{2}\Phi =0,\hfill \\ \hfill \frac{\partial F}{\partial \left(\frac{{\partial }^2{v}_c}{\partial K^2}\right)}& =2\sum _{i=1}^p{w}_i^2\frac{l_i^2}{2}\Phi =0,\hfill \\ \hfill \frac{\partial F}{\partial \left(\frac{{\partial }^2{v}_c}{\partial y\partial K}\right)}& =2\sum _{i=1}^p{w}_i^2{h}_i{l}_i\Phi =0,\hfill \end{array}$$

(11)

where

$$\Phi =-v_i+v_c+h_i\frac{\partial v_c}{\partial y}+l_i\frac{\partial v_c}{\partial K}+\frac{1}{2}(h_i^2\frac{{\partial }^2{v}_c}{\partial y^2}+l_i^2\frac{{\partial }^2{v}_c}{\partial K^2}+2h_i{l}_i\frac{{\partial }^2{v}_c}{\partial y\partial K}).$$

From (11), each equation is linear with respect to the five unknown partial derivatives. In order to facilitate the calculation, the above formulae can be rewritten as a system of matrix equations $\mathit{AH}=\mathit{b}$, where

$$\mathit{A}=\left(\begin{array}{cccc}{h}_1& h_2& \cdots & h_p\\ l_1& l_2& \cdots & l_p\\ \frac{h_1^2}{2}& \frac{h_2^2}{2}& \cdots & \frac{h_p^2}{2}\\ \frac{l_1^2}{2}& \frac{l_2^2}{2}& \cdots & \frac{l_p^2}{2}\\ h_1{l}_1& h_2{l}_2& \cdots & h_p{l}_p\end{array}\right)\left(\begin{array}{cccc}{w}_1^2& & & \\ & w_2^2& & \\ & & \ddots & \\ & & & w_p^2\end{array}\right)\left(\begin{array}{ccccc}{h}_1& l_1& \frac{h_1^2}{2}& \frac{l_1^2}{2}& h_1{l}_1\\ h_2& l_2& \frac{h_2^2}{2}& \frac{l_2^2}{2}& h_2{l}_2\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ h_p& l_p& \frac{h_p^2}{2}& \frac{l_p^2}{2}& h_p{l}_p\end{array}\right),$$

(12)

$$\mathit{b}=\left(\begin{array}{cccc}{h}_1& h_2& \cdots & h_p\\ l_1& l_2& \cdots & l_p\\ \frac{h_1^2}{2}& \frac{h_2^2}{2}& \cdots & \frac{h_p^2}{2}\\ \frac{l_1^2}{2}& \frac{l_2^2}{2}& \cdots & \frac{l_p^2}{2}\\ h_1{l}_1& h_2{l}_2& \cdots & h_p{l}_p\end{array}\right)\left(\begin{array}{cccc}{w}_1^2& & & \\ & w_2^2& & \\ & & \ddots & \\ & & & w_p^2\end{array}\right)\left(\begin{array}{c}{v}_1-v_c\\ v_2-v_c\\ ⋮\\ v_p-v_c\end{array}\right).$$

(13)

Let

$$\mathit{Q}=\left(\begin{array}{cccc}{h}_1& h_2& \cdots & h_p\\ l_1& l_2& \cdots & l_p\\ \frac{h_1^2}{2}& \frac{h_2^2}{2}& \cdots & \frac{h_p^2}{2}\\ \frac{l_1^2}{2}& \frac{l_2^2}{2}& \cdots & \frac{l_p^2}{2}\\ h_1{l}_1& h_2{l}_2& \cdots & h_p{l}_p\end{array}\right),\mathit{W}=\left(\begin{array}{cccc}{w}_1^2& & & \\ & w_2^2& & \\ & & \ddots & \\ & & & w_p^2\end{array}\right),$$

and

$$\mathit{v}={(v_1,v_2,\cdots ,v_p)}^T,\mathbf{1}=\underset{p}{\underbrace{{(1,1,\cdots ,1)}^T}},$$

then we have $\mathit{A}={\mathit{QWQ}}^{\mathit{T}}$ and $\mathit{b}=\mathit{QW}(\mathit{v}-v_c\mathbf{1})$. Assuming that all the points in each support domain are different, then it is easy to prove that the solution of linear equations $\mathit{AH}=\mathit{b}$ is unique and $\mathit{H}$ can be explicitly solved as

$$\mathit{H}={\mathit{A}}^{-1}\mathit{b}.$$

(14)

By introducing canonical base vectors ${\mathit{e}}_i(i=1,2,\dots ,p)$, vector $\mathit{H}$ can be expressed as

$$\mathit{H}={\mathit{A}}^{-\mathbf{1}}{\mathit{QWe}}_{\mathbf{1}}{v}_1+\dots +{\mathit{A}}^{-\mathbf{1}}{\mathit{QWe}}_{\mathit{p}}{v}_p-{\mathit{A}}^{-\mathbf{1}}\mathit{QW}({\mathit{e}}_{\mathbf{1}}+\dots +{\mathit{e}}_{\mathit{p}})v_c=-{\mathit{m}}_{\mathit{c}}{v}_c+\sum _{i=1}^p{\mathit{m}}_{\mathit{i}}{v}_i,$$

(15)

where ${\mathit{m}}_{\mathit{i}}={\mathit{A}}^{-\mathbf{1}}{\mathit{QWe}}_{\mathit{i}}={\{m_{i1},m_{i2},m_{i3},m_{i4},m_{i5}\}}^T$, ${\mathit{m}}_{\mathit{c}}={\{m_{c1},m_{c2},m_{c3},m_{c4},m_{c5}\}}^T$, and ${\mathit{m}}_{\mathit{c}}={\sum }_{i=1}^p{\mathit{m}}_{\mathit{i}}$. We can write the explicit expressions for the value of the five unknown partial derivatives as    

$$\begin{array}{c}\hfill \frac{\partial v\left({\mathit{x}}_{\mathit{c}}\right)}{\partial y}=\sum _{i=1}^p{m}_{i1}{v}_i-m_{c1}{v}_c+o(h_i^2,l_i^2),\mathrm{with}{m}_{c1}=\sum _{i=1}^p{m}_{i1},\\ \hfill \frac{\partial v\left({\mathit{x}}_{\mathit{c}}\right)}{\partial K}=\sum _{i=1}^p{m}_{i2}{v}_i-m_{c2}{v}_c+o(h_i^2,l_i^2),\mathrm{with}{m}_{c2}=\sum _{i=1}^p{m}_{i2},\\ \hfill \frac{{\partial }^{2}v\left({\mathit{x}}_{\mathit{c}}\right)}{\partial y^2}=\sum _{i=1}^p{m}_{i3}{v}_i-m_{c3}{v}_c+o(h_i^2,l_i^2),\mathrm{with}{m}_{c3}=\sum _{i=1}^p{m}_{i3},\\ \hfill \frac{{\partial }^{2}v\left({\mathit{x}}_{\mathit{c}}\right)}{\partial K^2}=\sum _{i=1}^p{m}_{i4}{v}_i-m_{c4}{v}_c+o(h_i^2,l_i^2),\mathrm{with}{m}_{c4}=\sum _{i=1}^p{m}_{i4},\\ \hfill \frac{{\partial }^{2}v\left({\mathit{x}}_{\mathit{c}}\right)}{\partial y\partial K}=\sum _{i=1}^p{m}_{i5}{v}_i-m_{c5}{v}_c+o(h_i^2,l_i^2),\mathrm{with}{m}_{c5}=\sum _{i=1}^p{m}_{i5}.\end{array}$$

(16)

From (16), it can be seen that the spatial partial derivatives of each point can be approximated as a linear combination of the function values of each node in its support domain, which is similar to the traditional finite difference method.

3. Numerical Scheme of HJB Equations

In this section, we present the numerical methods of the HJB equation with the Dirichlet boundary condition arising from the corresponding infinite horizon optimal control problem

$$\left\{\begin{array}{c}\alpha V\left(x\right)=\underset{u}{sup}[f(x,u)+\mathcal{A}(x,u)V\left(x\right)],x\in Z\subset {\mathbb{R}}^n,\hfill \\ V\left(x\right)=B\left(x\right),x\in \partial Z.\hfill \end{array}\right.$$

(17)

We propose two numerical algorithms for different cases of HJB equations:

• For the general case of the HJB equation coupled with optimization, we propose a successive approximation algorithm which combines the GFDM discretization scheme with the optimization algorithm.
• For the special case where one can explicitly express the optimal control u by maximizing the Hamiltonian, we propose an algorithm combining the GFDM and Newton’s iterative method.

3.1. The General Case

Assuming that the state variable is $x=(y,K)$, we consider the HJB Equation (17) in two-dimensional space, which can be written in the general form

$$\alpha V\left(x\right)-\underset{u\in U}{sup}[f(x,u)+\frac{1}{2}[a_{11}\frac{{\partial }^{2}V\left(x\right)}{\partial y^2}+2a_{12}\frac{{\partial }^{2}V\left(x\right)}{\partial y\partial K}+a_{22}\frac{{\partial }^{2}V\left(x\right)}{\partial K^2}]+g_1\frac{\partial V\left(x\right)}{\partial y}+g_2\frac{\partial V\left(x\right)}{\partial K}]=0,$$

(18)

for $x\in Z$, where $a_{ij}(x,u)={\left(\sigma (x,u)\sigma {(x,u)}^T\right)}_{ij},i=1,2.$

We will give a discretization scheme which is based on the GFDM for the spatial variables in the HJB Equation (18). Assuming that the number of interior points in the solution domain is M, then for $c=1,2,\dots ,M$, we can obtain the discrete equation at each point $x_c$ as

$$\begin{array}{cc}\hfill & \alpha V_c-\underset{u_c\in U}{sup}[f(x_c,u_c)+\frac{1}{2}[a_{11}(x_c,u_c)\frac{{\partial }^2{V}_c}{\partial y^2}+2a_{12}(x_c,u_c)\frac{{\partial }^2{V}_c}{\partial y\partial K}+a_{22}(x_c,u_c)\frac{{\partial }^2{V}_c}{\partial K^2}]\hfill \\ \hfill & +g_1(x_c,u_c)\frac{\partial V_c}{\partial y}+g_2(x_c,u_c)\frac{\partial V_c}{\partial K}]=0,x_c\in Z,\hfill \end{array}$$

(19)

with the boundary value $V_j$ at the boundary points $x_j$ for $j=M+1,M+2,\dots ,N,$

$$V_j=B\left(x_j\right),x_j\in \partial Z.$$

(20)

Using the formulae (16), and denoting the approximate value of $V_c$ by $v_c$ ($c=1,2,\dots ,M$), we have

$$\begin{array}{cc}\hfill & \alpha v_c-\underset{u_c\in U}{sup}[f(x_c,u_c)+\frac{1}{2}[a_{11}(x_c,u_c)(\sum _{i=1}^p{m}_{i3}{v}_i-m_{c3}{v}_c)\hfill \\ \hfill & +2a_{12}(x_c,u_c)(\sum _{i=1}^p{m}_{i5}{v}_i-m_{c5}{v}_c)+a_{22}(x_c,u_c)(\sum _{i=1}^p{m}_{i4}{v}_i-m_{c4}{v}_c)]\hfill \\ \hfill & +g_1(x_c,u_c)(\sum _{i=1}^p{m}_{i1}{v}_i-m_{c1}{v}_c)+g_2(x_c,u_c)(\sum _{i=1}^p{m}_{i2}{v}_i-m_{c2}{v}_c)]=0,x_c\in Z.\hfill \end{array}$$

(21)

Denote the left-hand side of Equation (21) as $F_c\left(\mathit{v}\right)$, where $\mathit{v}={(v_1,v_2,\dots ,v_M)}^T.$ Then, each discrete point ${\mathit{x}}_{\mathit{c}}$ corresponds to a discrete equation

$$F_c\left(\mathit{v}\right)=0.$$

(22)

The Hamiltonian in Equation (21) can be solved using the standard optimization tools such as “fminbnd” in Matlab software which is based on a golden section search and parabolic interpolation method. After solving the optimization for every discrete point, it turns out that the equations are only related to unknown discrete value functions. Therefore, we need to solve a system of equations consisting of M discrete linear algebra equations with M unknown variables $v_c$, that is

$$\mathit{F}\left(\mathit{v}\right)={[F_1\left(\mathit{v}\right),F_2\left(\mathit{v}\right),\dots ,F_M\left(\mathit{v}\right)]}^T=\mathbf{0}.$$

(23)

Using the GFDM discretization scheme and the optimization method given above, we propose an algorithm alternately updating the control $\mathit{u}$ and the value function $\mathit{v}$. Algorithm 1 summarizes the general successive approximation algorithm which is an iterative method.

Algorithm 1 Successive Approximation Algorithm

Input: Initial control law ${\tilde{\mathit{u}}}^0$;   given tolerance $\epsilon$ Output: Approximation of control law $\mathit{u}$,   value function $\mathit{v}$  1: Initialize $k=0$;   ${\mathit{u}}^0={\tilde{\mathit{u}}}^0$  2: repeat  3:    Solve the boundary value problem using GFDM scheme in Section 3.1 for the given control $u_c^k$,   i.e.,    solve equations $\mathit{F}\left({\mathit{v}}^k\right)=\mathbf{0}$, where $F_c\left({\mathit{v}}^k\right)=\alpha v_c^k-[f(x_c,u_c^k)+\mathcal{A}(x_c,u_c^k)v_c^k]$ for $c=1,2,\dots ,M$, $x_c\in Z$;   with the boundary condition $v_j=B\left(x_j\right),x_j\in \partial Z$.  4:    Compute ${\mathit{u}}^{k+1}$ by solving the optimization for every discrete point $x_c$: $u_c^{k+1}=arg{\displaystyle \underset{u_c\in U}{sup}}[f(x_c,u_c)+\mathcal{A}(x_c,u_c)v_c^k]$,$c=1,2,\dots ,M$  5:    $k=k+1$  6: until $\left|\right|{\mathit{u}}^{k+1}-{\mathit{u}}^k\left|\right|<\epsilon$  7: $\mathit{u}={\mathit{u}}^{k+1}$, $\mathit{v}={\mathit{v}}^{k+1}$  8: return $\mathit{u},\mathit{v}$

3.2. The Special Case

For some special cases, one can explicitly express the optimal control u by maximizing the Hamiltonian in the form

$$\widehat{u}=\widehat{u}(x,DV,D^{2}V).$$

(24)

Substituting (24) into (18), we have

$$\begin{array}{cc}\hfill & \alpha V\left(x\right)-[f(x,\widehat{u})+\frac{1}{2}[a_{11}(x,\widehat{u})\frac{{\partial }^{2}V\left(x\right)}{\partial y^2}+2a_{12}(x,\widehat{u})\frac{{\partial }^{2}V\left(x\right)}{\partial y\partial K}+a_{22}(x,\widehat{u})\frac{{\partial }^{2}V\left(x\right)}{\partial K^2}]\hfill \\ \hfill & +g_1(x,\widehat{u})\frac{\partial V\left(x\right)}{\partial y}+g_2(x,\widehat{u})\frac{\partial V\left(x\right)}{\partial K}]=0,\hfill \end{array}$$

(25)

for $x\in Z$. That is, the HJB equation is transformed into a partial differential equation without the optimization. Consequently, we can obtain the optimal control and optimal value by directly solving the partial differential equation. However, quite often, the obtained equations are nonlinear partial differential equations, presenting computational challenges. We will use Newton’s iteration method to solve the system of nonlinear generalized finite difference equations.

Denote the left-hand side of Equation (25) by ${\mathcal{F}}_c\left(\mathit{v}\right)$. Similarly to the discretization scheme in the general case, we can obtain the discrete equations for each discrete point

$$\mathcal{F}\left(\mathit{v}\right)={[{\mathcal{F}}_1\left(\mathit{v}\right),{\mathcal{F}}_2\left(\mathit{v}\right),\dots ,{\mathcal{F}}_M\left(\mathit{v}\right)]}^T=\mathbf{0}.$$

(26)

The idea of the iterative method is to treat the system as the zero of the operator $\mathcal{F}$: ${\mathbb{R}}^M\to {\mathbb{R}}^M$,

$$\mathit{v}solves\left(26\right)\iff \mathcal{F}\left(\mathit{v}\right)=\mathbf{0}.$$

(27)

Define D as the differential operator. The differentiation of $\mathcal{F}$ is then denoted by $D\mathcal{F}$. Algorithm 2 summarizes how to find the zero of $\mathcal{F}$ using Newton’s iterative method.

Algorithm 2 Newton Iterations

Input: Initial guess ${\tilde{\mathit{v}}}^0$;   max number of iterations $maxIter$;   given tolerance $\epsilon$ Output: Approximation of $\mathit{v}$  Initialize $k=0$;   ${\mathit{v}}^0={\tilde{\mathit{v}}}^0$  while $k<maxIter$ do    Obtain ${\mathcal{F}}_c\left(\mathit{v}\right)$ for every discrete point using Formula (25) and (16) arising from GFDM    Compute $D\mathcal{F}\left({\mathit{v}}^k\right)$ and $\mathcal{F}\left({\mathit{v}}^k\right)$ based on each discrete equation ${\mathcal{F}}_c\left(\mathit{v}\right)$    Let ${\mathit{y}}^k$ solve: $D\mathcal{F}\left({\mathit{v}}^k\right){\mathit{y}}^k=-\mathcal{F}\left({\mathit{v}}^k\right)$    ${\mathit{v}}^{k+1}={\mathit{v}}^k+{\mathit{y}}^k$    if $\left|\right|{\mathit{v}}^{k+1}-{\mathit{v}}^k\left|\right|<\epsilon$ then      $\mathit{v}={\mathit{v}}^{k+1}$      break    end if    $k=k+1$   end while   return $\mathit{v}$

4. Case Study

4.1. Optimal Investment Problem

The case study presents an optimal investment model under uncertainty in Abel [2]. Consider the problem of a firm making optimal investment decisions under uncertainty. The evolution of the firm’s capital stock $K\left(t\right)$ over time satisfies the following differential equation

$$\begin{array}{c}\hfill dK\left(t\right)=(u\left(t\right)-\delta K\left(t\right))dt;K\left(0\right)=K_0,\end{array}$$

(28)

where $u\left(t\right)$ is the control variable denoting the amount of capital investment, and $\delta \ge 0$ denotes the depreciation rate of the capital stock. The output price $y\left(t\right)$ satisfies the following stochastic process

$$dy\left(t\right)=\mu y\left(t\right)dt+\sigma y\left(t\right)dw\left(t\right);y\left(0\right)=y_0,$$

(29)

where $w\left(t\right)$ is a standard Brownian motion on a probability space $(\Omega ,\mathcal{F},P)$, and the coefficients $\mu$ and $\sigma$ positively represent the growth rate and volatility, respectively.

The output technology is assumed to be of the Cobb–Douglas form. The elements of production considered are the capital stock $K\left(t\right)$ and the labor $L\left(t\right)$. Assuming returns to scale are constant, then the form of the Cobb–Douglas production function is $F(K\left(t\right),L\left(t\right))=L{\left(t\right)}^{\xi }K{\left(t\right)}^{1-\xi }$, where $0<\xi <1$. The profit at time t is

$$\pi \left(t\right)=y\left(t\right)L{\left(t\right)}^{\xi }K{\left(t\right)}^{1-\xi }-\mathrm{w}L\left(t\right).$$

(30)

The firm pays a fixed wage w, and the labor can be instantaneously at no cost, so the firm chooses $L\left(t\right)$ to maximize the instantaneous operating income at time t. Take the partial derivative of formula (30) with respect to $L\left(t\right)$, and set the partial derivative to be 0; it then yields

$$L\left(t\right)={\mathrm{w}}^{\frac{1}{\xi -1}}y{\left(t\right)}^{\frac{1}{1-\xi }}K\left(t\right){\xi }^{\frac{1}{1-\xi }}.$$

(31)

Substituting formula (31) into (30), the maximized total profit at time t becomes

$$\pi \left(t\right)=cy{\left(t\right)}^{\gamma }K\left(t\right),$$

(32)

where c and $\gamma$ are constants, and $\gamma =\frac{1}{1-\xi }$, $c={\gamma }^{-\gamma }{(\gamma -1)}^{\gamma -1}{\mathrm{w}}^{1-\gamma }$.

The firm will incur certain adjustment costs in its operation. Assuming that the convex adjustment cost generated by capital is $h\left(u\right(t\left)\right)$, the actual profit at time t is

$$p\left(t\right)=cy{\left(t\right)}^{\gamma }K\left(t\right)-h\left(u\left(t\right)\right).$$

(33)

The firm’s goal is to choose the capital investment $u\left(t\right)$ that maximizes the expected discount cash flow at a given discount rate $\alpha$ over an infinite time horizon. This problem is a standard stochastic optimal control problem, and the value function is

$$\begin{array}{c}\hfill V(y_0,K_0)=\underset{u\left(t\right)\in \mathcal{U}}{sup}\left\{\mathbb{E}{\int }_0^{+\infty }{e}^{-\alpha t}\left[cy{\left(t\right)}^{\gamma }K\left(t\right)-h\left(u\left(t\right)\right)\right]dt\right\},\end{array}$$

(34)

where $\mathcal{U}$ represents the set of all admissible controls.

Assume that the convex adjustment cost is $h\left(u\left(t\right)\right)=\frac{u{\left(t\right)}^2}{2}$. Using the dynamic programming principle and Ito’s formula, we can obtain that the value function V satisfies the following HJB equation

$$\begin{array}{cc}\hfill -{\displaystyle \frac{1}{2}}{\sigma }^2{y}^2{\displaystyle \frac{{\partial }^{2}V}{\partial y^2}}+\alpha V& =\underset{u(.)}{sup}[\frac{\partial V}{\partial x}·g(x,u)+f(x,u)]\hfill \\ \hfill & =\underset{u(.)}{sup}[\mu y{\displaystyle \frac{\partial V}{\partial y}}+(-\delta K+u){\displaystyle \frac{\partial V}{\partial K}}+c{y}^{\gamma }K-\frac{1}{2}{u}^2].\hfill \end{array}$$

(35)

In order to solve the Equation (35) using numerical methods, a Dirichlet condition is imposed at the boundary $\partial Z$: $V(y,K)=B(y,K)$. Then, the problem to be solved is

$$\left\{\begin{array}{cc}\hfill -{\displaystyle \frac{1}{2}}{\sigma }^2{y}^2{\displaystyle \frac{{\partial }^{2}V}{\partial y^2}}+\alpha V& =\underset{u(.)}{sup}[\mu y{\displaystyle \frac{\partial V}{\partial y}}+(-\delta K+u){\displaystyle \frac{\partial V}{\partial K}}+c{y}^{\gamma }K-\frac{1}{2}{u}^2],(y,K)\in Z,\hfill \\ \hfill V(y,K)& =B(y,K),(y,K)\in \partial Z,\hfill \end{array}\right.$$

(36)

where

$$B(y,K)=AK{y}^{\gamma }+\frac{A^2}{2[\alpha -2\gamma \mu -\gamma (2\gamma -1){\sigma }^2]}{y}^{2\gamma },$$

(37)

with

$$A={\displaystyle \frac{c}{\alpha +\delta -\gamma \mu -{\displaystyle \frac{1}{2}}\gamma (\gamma -1){\sigma }^2}}.$$

4.2. Numerical Results

From (35), we are able to obtain the explicit expression of the optimal control given by $\widehat{u}=\frac{\partial V}{\partial K}$. Therefore, we are able to use both algorithms proposed in Section 3 to obtain the numerical approximation of $\mathit{V}$ and $\mathit{u}$ in problem (36). The effectiveness of the proposed schemes is verified in the cases of regular and node distribution, and the influence of related parameters on numerical accuracy is explored.

We first set the basic parameters involved in the problem (34) as follows:

$$\mu =0.01,\sigma =0.2,\delta =0.1,\xi =0.2,\mathrm{w}=5,\alpha =0.2,y_0=0.5,K_0=1.5.$$

The errors are evaluated using the following formulae:

$$\begin{array}{cc}\hfill globalerror& =\frac{\sqrt{\frac{1}{N}{\sum }_{i=1}^N{[approx\left(i\right)-exact\left(i\right)]}^2}}{|exac{t}_{max}|},\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill meanrelativeerror\left(mre\right)& =\frac{1}{N}\sum _{i=1}^N\left|\frac{approx\left(i\right)-exact\left(i\right)}{exact\left(i\right)}\right|,\hfill \end{array}$$

(39)

where N represents the total number of points considered in the solution domain, $approx\left(i\right)$ and $exact\left(i\right)$ represent the approximate solution and exact solution at point ${\mathit{x}}_{\mathit{i}}$, respectively, and $exac{t}_{max}$ represents the maximum value of the exact values of the points of all support domains. The analytical solution of Equation (35) is

$$V(y,K)={\displaystyle \frac{c}{\alpha +\delta -\gamma \mu -{\displaystyle \frac{1}{2}}\gamma (\gamma -1){\sigma }^2}}K{y}^{\gamma }+\frac{{\left[{\displaystyle \frac{c}{\alpha +\delta -\gamma \mu -{\displaystyle \frac{1}{2}}\gamma (\gamma -1){\sigma }^2}}\right]}^2}{2[\alpha -2\gamma \mu -\gamma (2\gamma -1){\sigma }^2]}{y}^{2\gamma },$$

(40)

and

$$\widehat{u}={\displaystyle \frac{c}{\alpha +\delta -\gamma \mu -{\displaystyle \frac{1}{2}}\gamma (\gamma -1){\sigma }^2}}{y}^{\gamma }.$$

(41)

The threshold $\epsilon$ used to terminate the iteration is set as ${10}^{-4}$, and the maximum number of iterations $maxIter$ is set as 20. All examples in this paper are implemented using Matlab R2017b on a 2.10 GHz desktop computer with 12 GB of memory.

The default weighting function is chosen as $e^{-dis{t}^2}$, and the points that constitute the support domain are selected based on the distance criterion (see Figure 1).

According to [43], the degree of the irregularity of node clouds will also have an impact on the accuracy of the results. Therefore, under the irregular node distribution, the influence of the irregular index of the cloud of nodes on the numerical solution is mainly considered. Following [43], we define the irregularity index of the clouds of nodes (IIC) as follows. The distance from the node ${\mathit{x}}_{\mathit{i}}(i=1,2,\dots ,p)$ to the center node ${\mathit{x}}_{\mathit{k}}$ in each support domain is

$$d_i=\sqrt{h_i^2+l_i^2}.$$

(42)

Taking each point ${\mathit{x}}_{\mathit{k}}$ in the solution domain as the central node, the average distance of the support domain where it is located is

$$r{m}_k=\frac{{\sum }_{i=1}^p{d}_i}{p},$$

(43)

where p represents the number of nodes in each support domain except the central node.

Then, the average distance from all support domains in the entire solution domain is

$$r{m}_c=\frac{{\sum }_{k=1}^{M}r{m}_k}{M},$$

(44)

where M represents the number of interior points in the solution domain. Thus, the irregularity index of clouds of nodes can be defined as

$$IIC=\sqrt{\frac{{\sum }_{k=1}^M{(r{m}_k-r{m}_c)}^2}{M}}.$$

(45)

To discuss the convergence of the proposed algorithms, we evaluate the order of convergence using the following formula:

$$order\left(V_k\right)=\frac{log\frac{\left|\right|V_{k-1}-V^{*}{\left|\right|}_2}{\left|\right|V_k-V^{*}{\left|\right|}_2}}{log\frac{h_{k-1}}{h_k}},$$

(46)

where $V^{*}$, $V_k$, $\left|\right|V_k-V^{*}{\left|\right|}_2$, and $h_k$ denote the exact value of the function V, approximate solution of the function V, the $L^2$ norm of the error, and the step size of the k-th set of points, respectively.

4.2.1. The Regular Node Distributions

Consider the regular node distribution given in Figure 2. We set the solution domain as a half-unit side square area $[0.5,1]\times [1.5,2]$, and the total number of nodes N are set as $64,225,361$ and 529, respectively. The errors of the value function $\mathit{V}$ and the optimal control $\mathit{u}$ are listed in

Table 1. Numerical results of both algorithms under regular node distributions.

N	Algorithm	Global Error (V)	mre (V)	Global Error (u)	mre (u)
64	1	2.0781 × 10${}^{-5}$	2.9544 × 10${}^{-5}$	5.6422 × 10${}^{-3}$	4.0917 × 10${}^{-3}$
	2	1.8414 × 10${}^{-5}$	4.0311 × 10${}^{-5}$	4.7054 × 10${}^{-3}$	3.4200 × 10${}^{-3}$
225	1	1.9742 × 10${}^{-5}$	2.5480 × 10${}^{-5}$	4.9443 × 10${}^{-3}$	2.5629 × 10${}^{-3}$
	2	1.0119 × 10${}^{-5}$	1.4615 × 10${}^{-5}$	1.1014 × 10${}^{-3}$	7.8821 × 10${}^{-4}$
361	1	1.2136 × 10${}^{-5}$	1.5951 × 10${}^{-5}$	3.4399 × 10${}^{-3}$	1.5857 × 10${}^{-3}$
	2	7.1307 × 10${}^{-6}$	1.2235 × 10${}^{-5}$	6.0667 × 10${}^{-4}$	7.0508 × 10${}^{-4}$
529	1	8.2071 × 10${}^{-6}$	1.0909 × 10${}^{-5}$	2.5735 × 10${}^{-3}$	1.0818 × 10${}^{-3}$
	2	2.6321 × 10${}^{-6}$	4.7869 × 10${}^{-6}$	4.0350 × 10${}^{-4}$	2.6199 × 10${}^{-4}$

. It can be seen that our proposed algorithms perform well over the regular domain, and the greater the number of nodes, the smaller the errors. Since we use the exact form of optimal control in conducting the Algorithm 2, it is not surprising that the errors of $\mathit{u}$ obtained by Algorithm 2 are smaller than those obtained by Algorithm 1. The CPU time in seconds and the order of convergence of $\mathit{v}$ are listed in

Table 2. Comparison of both algorithms on the order of convergence and computation time (s).

N	${\mathit{h}}_{\mathit{k}}$		Algorithm 1			Algorithm 2
		$\left|\right|{\mathit{V}}_{\mathit{k}}-{\mathit{V}}^{*}{\left|\right|}_2$	$\mathit{Order}\left({\mathit{V}}_{\mathit{k}}\right)$	Time (s)	$\left|\right|{\mathit{V}}_{\mathit{k}}-{\mathit{V}}^{*}{\left|\right|}_2$	$\mathit{Order}\left({\mathit{V}}_{\mathit{k}}\right)$	Time (s)
64	1/14	1.1794 × 10${}^{-2}$	1.906	0.46	5.4618 × 10${}^{-3}$	1.703	0.32
225	1/28	3.1493 × 10${}^{-3}$	1.905	1.86	1.7476 × 10${}^{-3}$	1.644	1.10
361	1/36	1.9458 × 10${}^{-3}$	1.916	3.09	1.0931 × 10${}^{-3}$	1.867	1.57
529	1/44	1.3199 × 10${}^{-3}$	1.934	4.36	7.4195 × 10${}^{-4}$	1.931	2.73

. The results show that our algorithms are stable and are close to the second-order convergence.

In addition, we implement the standard FDM [44] to solve our problem, where the central-difference approximation for the first- and the second-order derivatives are adopted, and then, the results obtained by the GFDM and by the FDM are compared in

Table 3. Comparison of the numerical errors with the FDM and the GFDM using Algorithm 1.

N	Algorithm	Global Error (V)	mre (V)	Global Error (u)	mre (u)
64	GFDM	2.0781 × 10${}^{-5}$	2.9544 × 10${}^{-5}$	5.6422 × 10${}^{-3}$	4.0917 × 10${}^{-3}$
	FDM	2.1321 × 10${}^{-5}$	4.4319 × 10${}^{-5}$	7.1449 × 10${}^{-3}$	9.1369 × 10${}^{-3}$
225	GFDM	1.9742 × 10${}^{-5}$	2.5480 × 10${}^{-5}$	4.9443 × 10${}^{-3}$	2.5629 × 10${}^{-3}$
	FDM	2.0754 × 10${}^{-5}$	3.0998 × 10${}^{-5}$	5.1921 × 10${}^{-3}$	6.3504 × 10${}^{-3}$
361	GFDM	1.2136 × 10${}^{-5}$	1.5951 × 10${}^{-5}$	3.4399 × 10${}^{-3}$	1.5857 × 10${}^{-3}$
	FDM	1.8986 × 10${}^{-5}$	2.4111 × 10${}^{-5}$	4.1956 × 10${}^{-3}$	5.2431 × 10${}^{-3}$
529	GFDM	8.2071 × 10${}^{-6}$	1.0909 × 10${}^{-5}$	2.5735 × 10${}^{-3}$	1.0818 × 10${}^{-3}$
	FDM	9.5459 × 10${}^{-6}$	1.2736 × 10${}^{-5}$	3.7906 × 10${}^{-3}$	4.5419 × 10${}^{-3}$

. It is observed that the results from the GFDM has a greater level of accuracy than the results obtained by the FDM.

We then investigate the stability property of proposed algorithms.

Table 4. Robustness analysis of Algorithm 1 for the number of nodes and weighting functions.

N	Weighting Functions	Global Error (V)	mre (V)	Global Error (u)	mre (u)
	$e^{-0.5{\left(dist\right)}^2}$	2.0816 × 10${}^{-5}$	2.9594 × 10${}^{-5}$	5.6588 × 10${}^{-3}$	4.1064 × 10${}^{-3}$
	$e^{-1.5{\left(dist\right)}^2}$	2.0746 × 10${}^{-5}$	2.9494 × 10${}^{-5}$	5.6256 × 10${}^{-3}$	4.0771 × 10${}^{-3}$
64	$\frac{1}{dis{t}^{0.5}}$	1.8371 × 10${}^{-5}$	2.6113 × 10${}^{-5}$	4.7302 × 10${}^{-3}$	3.3365 × 10${}^{-3}$
	$\frac{1}{dis{t}^1}$	1.5793 × 10${}^{-5}$	2.2421 × 10${}^{-5}$	3.8283 × 10${}^{-3}$	2.6050 × 10${}^{-3}$
	$e^{-0.5{\left(dist\right)}^2}$	1.9751 × 10${}^{-5}$	2.5492 × 10${}^{-5}$	4.9469 × 10${}^{-3}$	2.5643 × 10${}^{-3}$
	$e^{-1.5{\left(dist\right)}^2}$	1.9732 × 10${}^{-5}$	2.5468 × 10${}^{-5}$	4.9419 × 10${}^{-3}$	2.5614 × 10${}^{-3}$
225	$\frac{1}{dis{t}^{0.5}}$	9.9952 × 10${}^{-6}$	1.1013 × 10${}^{-5}$	2.4839 × 10${}^{-3}$	1.1695 × 10${}^{-3}$
	$\frac{1}{dis{t}^1}$	1.4409 × 10${}^{-5}$	1.8664 × 10${}^{-5}$	3.6017 × 10${}^{-3}$	1.8337 × 10${}^{-3}$
	$e^{-0.5{\left(dist\right)}^2}$	1.2139 × 10${}^{-5}$	1.5955 × 10${}^{-5}$	3.4409 × 10${}^{-3}$	1.5862 × 10${}^{-3}$
	$e^{-1.5{\left(dist\right)}^2}$	1.2132 × 10${}^{-5}$	1.5946 × 10${}^{-5}$	3.4389 × 10${}^{-3}$	1.5851 × 10${}^{-3}$
361	$\frac{1}{dis{t}^{0.5}}$	1.0514 × 10${}^{-5}$	1.3844 × 10${}^{-5}$	2.9795 × 10${}^{-3}$	1.3648 × 10${}^{-3}$
	$\frac{1}{dis{t}^1}$	8.8539 × 10${}^{-6}$	1.1680 × 10${}^{-5}$	2.5104 × 10${}^{-3}$	1.1440 × 10${}^{-3}$
	$e^{-0.5{\left(dist\right)}^2}$	8.2087 × 10${}^{-6}$	1.0911 × 10${}^{-5}$	2.5740 × 10${}^{-3}$	1.0820 × 10${}^{-3}$
	$e^{-1.5{\left(dist\right)}^2}$	8.2055 × 10${}^{-6}$	1.0907 × 10${}^{-5}$	2.5730 × 10${}^{-3}$	1.0815 × 10${}^{-3}$
529	$\frac{1}{dis{t}^{0.5}}$	7.1091 × 10${}^{-6}$	9.4666 × 10${}^{-6}$	2.2295 × 10${}^{-3}$	9.3215 × 10${}^{-4}$
	$\frac{1}{dis{t}^1}$	5.9865 × 10${}^{-6}$	7.9877 × 10${}^{-6}$	1.8787 × 10${}^{-3}$	7.8164 × 10${}^{-4}$

gives the global error and the mean relative error under different numbers of nodes and weighting functions with the implementation of Algorithm 1. It shows the consistent accuracy across various N and weighting functions. The standard deviation of the global errors of the value function $\mathit{V}$ and that of the optimal control $\mathit{u}$ are 5.2468 × 10${}^{-6}$ and 1.2410 × 10${}^{-3}$, respectively. Thus, Algorithm 1 is stable in the numerical approximation. Since Algorithm 2 differs from Algorithm 1 in replacing the optimization search with the Newton iteration due to the known expression in the optimal control, the stability of Algorithm 2 is inferred. Therefore, both algorithms are efficient and robust.

The convergence histories using Algorithm 1 are shown in Figure 3. Figure 4 and Figure 5 show the comparison of the exact solution and numerical solutions of $\mathit{V}$ and $\mathit{u}$, respectively, with $N=529$. In Figure 3, we see that Algorithm 1 converges to the solution of the problem after a few iterations, which demonstrates the fast convergence of the algorithm.

From Table 1 as well as Figure 4 and Figure 5, we observe that Algorithms 1 and 2 perform equally well, so we can conclude that both algorithms can be used. However, we do find that Algorithm 1 takes a longer computational time than Algorithm 2, because Algorithm 1 requires solving the optimization in each iteration as shown in Table 2.

4.2.2. The Irregular Node Distributions

We conduct experiments under two irregular node distributions with the total points $N=361$ (we view the rectangular boundaries used by traditional mesh algorithms as regular boundaries, and those with non-rectangular boundaries as irregular ones). The shapes of the nodes are shown in Figure 6. The solution domains of both clouds of nodes are given as

• Shape a: ${\Omega }_a=\{(y,K)\in {\mathbb{R}}^2|{(y-0.75)}^2+{(K-1.75)}^2\le 0.125\}.$
• Shape b: ${\Omega }_b=\{(y,K)\in {\mathbb{R}}^2|1.5\le K\le -8{(y-0.75)}^2+2,0.5\le y\le 1\}$.

The accuracy of the numerical solutions under different shapes is shown in

Table 5. Numerical errors of both algorithms under irregular node distributions.

Shape	Algorithm	Global Error (V)	mre (V)	Global Error (u)	mre (u)
a	1	1.4845 × 10${}^{-5}$	1.6205 × 10${}^{-5}$	2.3411 × 10${}^{-3}$	1.7330 × 10${}^{-3}$
	2	1.7672 × 10${}^{-5}$	2.0889 × 10${}^{-5}$	7.7989 × 10${}^{-4}$	7.8551 × 10${}^{-4}$
b	1	4.6104 × 10${}^{-5}$	4.6373 × 10${}^{-5}$	4.5303 × 10${}^{-3}$	4.0952 × 10${}^{-3}$
	2	8.0699 × 10${}^{-6}$	1.0453 × 10${}^{-5}$	1.5865 × 10${}^{-3}$	1.0809 × 10${}^{-3}$

. The $L^2$ difference between the two successive iterations obtained by Algorithm 1 is shown in Figure 7. Similarly to the case of regular nodes, Algorithm 1 converges to the solution of the problem after several iterations.

Figure 8, Figure 9, Figure 10 and Figure 11 show the exact solutions and numerical solutions of $\mathit{V}$ and $\mathit{u}$ under different shapes, respectively. It can be seen that the results are also accurate under irregular points as in regular ones.

We will then test the influence of the irregular index of the clouds of points (IIC) and a key parameter, the weighting functions, on the accuracy of the numerical solution using Algorithm 2.

For each support domain, $p=15$ nodes were selected.

Table 6. Numerical errors of Algorithm 2 under irregular nodes versus IIC.

Shape	IIC	Global Error (V)	mre (V)	Global Error (u)	mre (u)
a	2.4362 × 10${}^{-2}$	1.8241 × 10${}^{-5}$	3.1043 × 10${}^{-5}$	1.6932 × 10${}^{-3}$	1.8460 × 10${}^{-3}$
b	2.1311 × 10${}^{-2}$	8.0699 × 10${}^{-6}$	1.0453 × 10${}^{-5}$	1.5865 × 10${}^{-3}$	1.0809 × 10${}^{-3}$

shows the numerical results for different values of IIC. It can be seen that the IIC obtained for the circular region is greater, and the error of the results is larger.

From the basic principle of GFDM, it is known that the approximated value of the central node of each support domain is the weighted sum of the function values of the rest of the nodes, and the nodes closer to the central node have a greater impact on the function approximation, so the nodes that are closer to the central node should be given higher weights. Based on this feature, this paper uses two different weighting functions: the exponential function $e^{-n{\left(dist\right)}^2}$ and the potential function $\frac{1}{dis{t}^n}$ for comparison.

Table 7. Numerical errors of Algorithm 2 under irregular nodes versus weighting functions.

Shape	Weighting Functions	n	Global Error (V)	mre (V)	Global Error (u)	mre (u)
		0.5	1.7631 × 10${}^{-5}$	2.0844 × 10${}^{-5}$	7.7985 × 10${}^{-4}$	7.8864 × 10${}^{-4}$
	$e^{-n{\left(dist\right)}^2}$	1	1.7672 × 10${}^{-5}$	2.0889 × 10${}^{-5}$	7.7989 × 10${}^{-4}$	7.8551 × 10${}^{-4}$
a		1.5	1.7718 × 10${}^{-5}$	2.0936 × 10${}^{-5}$	7.7993 × 10${}^{-4}$	7.8838 × 10${}^{-4}$
		0.5	2.5221 × 10${}^{-5}$	2.3074 × 10${}^{-5}$	7.4626 × 10${}^{-4}$	7.2931 × 10${}^{-4}$
	$\frac{1}{dis{t}^n}$	1	1.8287 × 10${}^{-5}$	1.6347 × 10${}^{-5}$	7.3280 × 10${}^{-4}$	7.1948 × 10${}^{-4}$
		1.5	1.3395 × 10${}^{-5}$	1.4910 × 10${}^{-5}$	7.3636 × 10${}^{-4}$	7.5369 × 10${}^{-4}$
		0.5	8.0773 × 10${}^{-6}$	1.0463 × 10${}^{-5}$	1.5875 × 10${}^{-3}$	1.0816 × 10${}^{-3}$
	$e^{-n{\left(dist\right)}^2}$	1	8.0699 × 10${}^{-6}$	1.0453 × 10${}^{-5}$	1.5865 × 10${}^{-3}$	1.0809 × 10${}^{-3}$
b		1.5	8.0625 × 10${}^{-6}$	1.0443 × 10${}^{-5}$	1.5855 × 10${}^{-3}$	1.0801 × 10${}^{-3}$
		0.5	5.7602 × 10${}^{-6}$	7.7763 × 10${}^{-6}$	1.4208 × 10${}^{-3}$	8.9286 × 10${}^{-4}$
	$\frac{1}{dis{t}^n}$	1	1.3814 × 10${}^{-5}$	9.3381 × 10${}^{-6}$	3.4880 × 10${}^{-3}$	1.5311 × 10${}^{-3}$
		1.5	4.9254 × 10${}^{-6}$	5.8692 × 10${}^{-6}$	1.7206 × 10${}^{-3}$	1.0926 × 10${}^{-3}$

gives the numerical results under different weighting functions. Both forms of weighting functions, the exponential function and the potential function, yield small errors. Moreover, the exponential weighting function produces more stable results with respect to the changes in n than the potential function.

In sum, our two mesh-free numerical methods are robust in solving HJB equations (both general and some special form) with regular and irregular boundary conditions. The effectiveness and the accuracy of the proposed methods are evidenced by comprehensive numerical analyses.

5. Conclusions

In this paper, we present two algorithms for computing different cases of HJB equations with boundary conditions. The first algorithm, combining the mesh-free GFD method and successive approximation method, is proposed to solve the general case of HJB equations. The second algorithm, combining the mesh-free GFD method and Newton’s iterative method, is proposed to solve some special case of HJB equations. Numerical examples show that both algorithms are efficient and have high accuracy over regular and irregular domains. We also tested the effect of key parameters on the accuracy of the proposed algorithm under regular node distributions and irregular node distributions, respectively, including the total points, weighting functions and irregular index.

Due to the meshless nature of the generalized finite difference method, the proposed algorithms can be easily applied for solving nonlinear PDEs over complicated and realistic boundaries. Additionally, it can also be extended to solve HJB-FP systems in the mean field type control problem, which will be investigated in our future study.