Methods of Moment and Maximum Entropy for Solving Nonlinear Expectation

Abstract

In this paper, we consider a special nonlinear expectation problem on the special parameter space and give a necessary and sufficient condition for the existence of the solution. Meanwhile, we generalize the necessary and sufficient condition to the two-dimensional moment problem. Moreover, we use the maximum entropy method to carry out a kind of concrete solution and analyze the convergence for the maximum entropy solution. Numerical experiments are presented to compute the maximum entropy density functions.

1. Introduction

The sublinear expectation $\widehat{\mathbb{E}}$ introduced by Peng [1,2] can be regarded as the supremum of a family of linear expectations $\{E_{\mathit{\theta }}:\mathit{\theta }\in \Theta \}$, that is,

$${\displaystyle \widehat{\mathbb{E}}\left[\phi \left(X\right)\right]=\underset{\mathbf{\theta }\in \Theta }{\sup }{E}_{\mathbf{\theta }}\left[\phi \left(X\right)\right],}$$

(1)

where $\phi \left(x\right)$ is a local Lipschitz continuous function and $\Theta$ is the parameter space.

It is evident that the sublinear expectation defined by (1) depends on the choice of parameter space $\Theta$. Different spaces will result in different nonlinear expectations. In particular, let $\phi \left(x\right)=x^n$ and:

$${\overline{\mu }}_n=\widehat{\mathbb{E}}\left[X^n\right]\ge {\underline{\mu }}_n=-\widehat{\mathbb{E}}[-X^n],n=1,2,\cdots ,N,$$

(2)

then the parameter space $\Theta$ can be chosen as the following form:

$$\Theta =[{\underline{\mu }}_1,{\overline{\mu }}_1]\times [{\underline{\mu }}_2,{\overline{\mu }}_2]\times \cdots \times [{\underline{\mu }}_N,{\overline{\mu }}_N].$$

(3)

When $N=1$, Peng [3] gave the definition of the independent and identically distributed random variable and proved the weak law of large numbers (LLN) under the sublinear expectation and Condition (2). Furthermore, if $\Theta =\left\{0\right\}\times [{\underline{\mu }}_2,{\overline{\mu }}_2]$, then Peng [4,5,6,7,8] defined the G-normal distribution and presented a new central limit theorem (CLT) under $\widehat{\mathbb{E}}$. The new LLN and CLT are the theoretical foundations in the framework of sublinear expectation.

The calculation of $\widehat{\mathbb{E}}\left(\phi \left(X\right)\right)$ can be performed by solving the following nonlinear partial differential equation:

$$\left\{\begin{array}{cc}{\displaystyle \frac{\partial u}{\partial t}-\frac{1}{2}({\left(\frac{{\partial }^{2}u}{\partial x^2}\right)}^{+}-{\sigma }^2{\left(\frac{{\partial }^{2}u}{\partial x^2}\right)}^{-})=0,}\hfill & (t,x)\in [0,+\infty )\times \mathbb{R},\hfill \\ u(0,x)=\phi \left(x\right),\hfill & x\in \mathbb{R},\hfill \end{array}\right.$$

(4)

whose solution is $u(t,x)=\widehat{\mathbb{E}}\left[\phi (x+\sqrt{t}X)\right]$. When the initial value $\phi \left(x\right)$ is a convex function, Peng [8] gave the expression of $\widehat{\mathbb{E}}\left[\phi \left(X\right)\right]$ as follows:

$${\displaystyle \widehat{\mathbb{E}}\left[\phi \left(X\right)\right]=\frac{1}{\sqrt{2\pi {\overline{\mu }}_2^2}}{\int }_{-\infty }^{+\infty }\phi \left(x\right)exp\{-\frac{x^2}{2{\overline{\mu }}_2^2}\}dx.}$$

(5)

If $\phi \left(x\right)$ is a concave function, the variance ${\overline{\mu }}_2^2$ in (5) will be replaced by ${\underline{\mu }}_2^2$. For neither the concave nor the convex case, Hu [9] derived the explicit solutions of Problem (4) with the initial condition $\phi \left(x\right)=x^n$, $n\ge 1$. Gong [10] used the fully-implicit numerical scheme to compute the nonlinear probability under the G-expectation determined by the G-heat Equation (4) with the initial condition $\phi \left(x\right)={\mathbf{1}}_{\{x<0\}},x\in \mathbb{R}$. Here, ${\mathbf{1}}_{\{x<0\}}$ denotes the indicator function of the set $\left\{x\right|x<0\}$. What all these methods mentioned above have in common is that G-expectation is calculated via solving the nonlinear partial differential Equation (4) with the particular initial condition $\phi \left(x\right)$. However, because of the nonlinear term, it is not easy to find a solution of (4) with general continuous initial function $\phi \left(x\right)$.

Based on the above reasons, in this paper, we consider a kind of special parameter space $\Theta$ defined in (3) and convert the sublinear expectation in (1) into the following two series of moment problems: find the probability density functions $\overline{p}\left(x\right)$ and $\underline{p}\left(x\right)$ such that:

$${\displaystyle {\overline{\mu }}_n={\int }_{-\infty }^{+\infty }{x}^n\overline{p}\left(x\right)dx}$$

(6)

and:

$${\displaystyle {\underline{\mu }}_n={\int }_{-\infty }^{+\infty }{x}^n\underline{p}\left(x\right)dx,}$$

(7)

respectively, where $n=0,1,\cdots ,N$ and $N=2M$. That is, we will approximately find a special class of nonlinear expectations $\widehat{\mathbb{E}}$, which satisfies:

$${\displaystyle \widehat{\mathbb{E}}\left[X^n\right]=\underset{\mathbf{\theta }\in \Theta }{\sup }{\int }_{\mathbb{R}}{x}^n{p}_{\mathbf{\theta }}\left(x\right)dx=\underset{{\underline{\mu }}_n\le {\theta }_n\le {\overline{\mu }}_n}{\sup }{\int }_{\mathbb{R}}{x}^n{p}_{\mathbf{\theta }}\left(x\right)dx={\overline{\mu }}_n}$$

and:

$${\displaystyle -\widehat{\mathbb{E}}[-X^n]=-\underset{\mathbf{\theta }\in \Theta }{\sup }{\int }_{\mathbb{R}}(-x^n)p_{\mathbf{\theta }}\left(x\right)dx=-\underset{{\underline{\mu }}_n\le {\theta }_n\le {\overline{\mu }}_n}{inf}{\int }_{\mathbb{R}}{x}^n{p}_{\mathbf{\theta }}\left(x\right)dx={\underline{\mu }}_n,}$$

where $\theta =({\theta }_0,{\theta }_1,\cdots ,{\theta }_N)$.

The rest of this article is organized as follows. In Section 2, we present an alternative sufficient and necessary condition of the existence of solutions $\overline{p}\left(x\right)$ and $\underline{p}\left(x\right)$ that satisfy (6) and (7), respectively. In Section 3, we use the maximum entropy method to find the concrete solutions and analyze the convergence of maximum entropy solutions. In Section 4, we conduct the numerical simulations to calculate the maximum entropy density functions.

2. Existence of Solutions for Moment Problems

According to Theorems 1.35 and 1.36 in Akihito [11], the sequences ${\left\{{\overline{\mu }}_j\right\}}_{j=0}^N$ and ${\left\{{\underline{\mu }}_j\right\}}_{j=0}^N$ should satisfy some conditions if we use them to determine the probability density functions. Therefore, in this section, we consider the sufficient and necessary conditions to the existence of solutions for the moment Problems (6) and (7), respectively.

Let:

$${\overline{\Delta }}_{2M}=\left(\begin{array}{cccc}{\overline{\mu }}_0& {\overline{\mu }}_1& \cdots & {\overline{\mu }}_M\\ {\overline{\mu }}_1& {\overline{\mu }}_2& \cdots & {\overline{\mu }}_{M+1}\\ ⋮& ⋮& ⋮\\ {\overline{\mu }}_M& {\overline{\mu }}_{M+1}& \cdots & {\overline{\mu }}_{2M}\end{array}\right)\mathrm{and}{\underline{\Delta }}_{2M}=\left(\begin{array}{cccc}{\underline{\mu }}_0& {\underline{\mu }}_1& \cdots & {\underline{\mu }}_M\\ {\underline{\mu }}_1& {\underline{\mu }}_2& \cdots & {\underline{\mu }}_{M+1}\\ ⋮& ⋮& ⋮\\ {\underline{\mu }}_M& {\underline{\mu }}_{M+1}& \cdots & {\underline{\mu }}_{2M}\end{array}\right)$$

(8)

be the Hankel matrices of the given sequences ${\left\{{\overline{\mu }}_j\right\}}_{j=0}^{2M}$ and ${\left\{{\underline{\mu }}_j\right\}}_{j=0}^{2M}$, respectively.

The following theorem gives an alternative sufficient and necessary condition for the existence of $\overline{p}\left(x\right)$, which satisfies (6).

Theorem 1.

A sufficient and necessary condition that the sequence ${\left\{{\overline{\mu }}_j\right\}}_{j=0}^N$ determines the probability density function $\overline{p}\left(x\right)$ is that ${\left\{{\overline{\mu }}_j\right\}}_{j=0}^N$ satisfies one of the following two conditions:

(a) For any nonvanishing vector ${\mathit{X}}_m^T=(x_0,x_1,\cdots ,x_m)(1\le m\le M)$,

$${\overline{\mu }}_{2m}{\mathbf{X}}_{m-1}^T{\overline{\Delta }}_{2(m-1)}{\mathbf{X}}_{m-1}>{\left({\overline{U}}_m^T{\mathbf{X}}_{m-1}\right)}^2;$$

(b) If there exist $m_0\ge 1$ and nonvanishing vector ${\mathit{X}}_{m_0}^T=(x_0,x_1,\cdots ,x_{m_0})$ such that ${\mathit{X}}_{m_0}^T{\overline{\Delta }}_{2m_0}{\mathit{X}}_{m_0}=0$, then:

$$|{\overline{\Delta }}_{2m}|=0$$

for $m_0\le m\le M$, where ${\overline{U}}_m^T=({\overline{\mu }}_m,\cdots ,{\overline{\mu }}_{2m-1})$.

Proof. 

We prove first the necessity. Let X be a continuous random variable with density function $\overline{p}\left(x\right)$, whose original moments are:

$${\overline{\mu }}_m=E\left[X^m\right]={\int }_R{x}^m\overline{p}\left(x\right)dx,m=0,1,\cdots ,2M,$$

(9)

where ${\overline{\mu }}_0=1$.

For any $m+1$-dimensional nonvanishing vector ${\mathit{X}}_m={(x_0,x_1,\cdots ,x_m)}^T$, we can check that:

$${\displaystyle {\mathbf{X}}_m^T{\overline{\Delta }}_{2m}{\mathbf{X}}_m=E\left[{\left(\sum _{j=0}^m{x}_j{X}^j\right)}^2\right],m=0,1,\cdots ,M.}$$

(10)

In fact, taking $m=0$, we have:

$${\mathbf{X}}_0^T{\overline{\Delta }}_0{\mathbf{X}}_0=x_0^2=E\left[x_0^2\right],$$

(11)

which means that (10) holds for $m=0$.

Assume that the Equation (10) holds for $0\le m\le M-1$, that is,

$${\displaystyle {\mathbf{X}}_{M-1}^T{\overline{\Delta }}_{2(M-1)}{\mathbf{X}}_{M-1}=E\left[{\left(\sum _{j=0}^{M-1}{x}_j{X}^j\right)}^2\right].}$$

(12)

Then, we consider the case of $m=M$ and note that the matrix ${\overline{\Delta }}_{2M}$ can be rewritten as:

$${\overline{\Delta }}_{2M}=\left(\begin{array}{cc}{\overline{\Delta }}_{2(M-1)}& {\overline{U}}_M\\ {\overline{U}}_M^T& {\overline{\mu }}_{2M}\end{array}\right).$$

(13)

Multiply (13) by ${\mathit{X}}_M^T$ and ${\mathit{X}}_M$, respectively; it follows from (9) and (12) that:

$$\begin{array}{c}{\mathbf{X}}_M^T{\overline{\Delta }}_{2M}{\mathbf{X}}_M=({\mathbf{X}}_{M-1}^T,x_M)\left(\begin{array}{cc}{\overline{\Delta }}_{2(M-1)}& {\overline{U}}_M\\ {\overline{U}}_M^T& {\overline{\mu }}_{2M}\end{array}\right)\left(\begin{array}{c}{\mathbf{X}}_{M-1}\\ x_M\end{array}\right)\hfill \\ ={\mathbf{X}}_{M-1}^T{\overline{\Delta }}_{2(M-1)}{\mathbf{X}}_{M-1}+2x_M{\overline{U}}_M^T{X}_{M-1}+{\overline{\mu }}_{2M}{x}_M^2\hfill \\ {\displaystyle =E\left[{\left(\sum _{j=0}^{M-1}{x}_j{X}^j\right)}^2\right]+2x_M\sum _{j=0}^{M-1}{\overline{\mu }}_{M+j}{x}_j+{\overline{\mu }}_{2M}{x}_M^2}\hfill \\ {\displaystyle =E\left[{\left(\sum _{j=0}^{M-1}{x}_j{X}^j\right)}^2\right]+2x_M\sum _{j=0}^{M-1}{x}_{j}E\left[X^{M+j}\right]+x_M^{2}E\left[X^{2M}\right]}\hfill \\ {\displaystyle =E\left[{\left(\sum _{j=0}^M{x}_j{X}^j\right)}^2\right]\ge 0.}\hfill \end{array}$$

(14)

By using mathematical induction, we know that (10) holds for $m=0,1,\cdots ,M$. Moreover, Equation (10) means that the Hankel matrices are nonnegative definite.

Now, we prove the necessary condition. Because of the nonnegative definiteness of ${\overline{\Delta }}_{2m}(0\le m\le M)$, there are two possible cases for ${\mathit{X}}_m^T{\overline{\Delta }}_{2m}{\mathit{X}}_m$, that is ${\mathit{X}}_m^T{\overline{\Delta }}_{2m}{\mathit{X}}_m>0$ for any m-dimensional nonzero vector ${\mathit{X}}_m$ or ${\mathit{X}}_{m_0}^T{\overline{\Delta }}_{2m_0}{\mathit{X}}_{m_0}=0$ for some $m_0$-dimensional nonzero vector ${\mathit{X}}_{m_0}$. Thus, we divide the proof into two cases.

Case 1: Let ${\mathit{X}}_m^T{\overline{\Delta }}_{2m}{\mathit{X}}_m{\displaystyle =E\left[{\left(\sum _{j=0}^m{x}_j{X}^j\right)}^2\right]>0}$ for any nonzero vector ${\mathit{X}}_m(1\le m\le M)$; by (14), we have:

$${\displaystyle E\left[{\left(\sum _{j=0}^m{x}_j{X}^j\right)}^2\right]={\mathbf{X}}_{m-1}^T{\overline{\Delta }}_{2(m-1)}{\mathbf{X}}_{m-1}+2x_m{\overline{U}}_m^T{X}_{m-1}+{\overline{\mu }}_{2m}{x}_m^2.}$$

(15)

We regard (15) as a quadratic equation in one variable with respect to $x_m$. Since the value of (15) is greater than zero, so its discriminant is less than zero, i.e.,

$${\left({\overline{U}}_m^T{\mathbf{X}}_{m-1}\right)}^2-{\overline{\mu }}_{2m}{\mathbf{X}}_{m-1}^T{\overline{\Delta }}_{2(m-1)}{\mathbf{X}}_{m-1}<0.$$

(16)

This implies that (a) in Theorem 1 holds for $m=1,2,\cdots ,M$.

Case 2: Let ${\mathit{X}}_{m_0-1}^T{\overline{\Delta }}_{2(m_0-1)}{\mathit{X}}_{m_0-1}=0$ for some nonvanishing vector ${\mathit{X}}_{m_0-1}(2\le m_0\le M+1)$. For $m>m_0-1$, let $m=m_0-1+k_0$, $k_0=1,2,\cdots ,M-m_0+1$. We choose an $m\times 1$ vector ${\mathit{X}}_m^T=({\mathit{X}}_{m_0-1}^T,\underset{k_0}{\underbrace{0,\cdots ,0}})$ and multiply ${\overline{\Delta }}_{2m}$ by $X_m^T$ and $X_m$. Here, we consider only the case of $k_0=1$ as an example, and the same method can be used to analyze for $2\le k_0\le M-m_0+1$. Then,

$$\begin{array}{cc}\hfill {\mathbf{X}}_{m_0}^T{\overline{\Delta }}_{2m_0}{\mathbf{X}}_{m_0}& =({\mathbf{X}}_{m_0-1}^T,0)\left(\begin{array}{cc}{\overline{\Delta }}_{2(m_0-1)}& {\overline{U}}_{m_0}\\ {\overline{U}}_{m_0}^T& {\overline{\mu }}_{2m_0}\end{array}\right)\left(\begin{array}{c}{\mathbf{X}}_{m_0-1}\\ 0\end{array}\right)\hfill \\ & ={\mathbf{X}}_{m_0-1}^T{\overline{\Delta }}_{2(m_0-1)}{\mathbf{X}}_{m_0-1}.\hfill \end{array}$$

(17)

Since ${\overline{\Delta }}_{2m_0}$ is nonnegative definite and ${\mathit{X}}_{m_0-1}^T{\overline{\Delta }}_{2(m_0-1)}{\mathit{X}}_{m_0-1}=0$, by (17), it follows that $|{\overline{\Delta }}_{2m_0}|=0$, that is (b) holds.

Next, we prove the sufficiency via its converse-negative proposition. In other words, we need to show that there is no probability density function $p\left(x\right)$ such that the corresponding moments ${\left\{{\overline{\mu }}_j\right\}}_{j=0}^N$ satisfy: there exist a positive integer $m_0(2\le m_0\le M+1)$ and an $m_0$-dimensional nonzero vector ${\mathit{X}}_{m_0-1}$ such that:

(i) ${\overline{\mu }}_{2m_0}{\mathit{X}}_{m_0-1}^T{\overline{\Delta }}_{2(m_0-1)}{\mathit{X}}_{m_0-1}\le {\left({\overline{U}}_{m_0}^T{\mathit{X}}_{m_0-1}\right)}^2$;

(ii) if ${\mathit{X}}_{m_0-1}^T{\overline{\Delta }}_{2(m_0-1)}{\mathit{X}}_{m_0-1}=0$, then there exists a positive integer $m_1>m_0-1$ such that $|{\overline{\Delta }}_{2m_1}|>0$.

Now, we carry out the proof by contradiction. Suppose such a density function $\overline{p}\left(x\right)$ exists. If ${\mathit{X}}_{m_0}^T{\overline{\Delta }}_{2m_0}{\mathit{X}}_{m_0}>0$ holds for any $m_0+1$-dimensional nonvanishing vector, then it follows from (15) that:

$${\left({\overline{U}}_{m_0}^T{\mathbf{X}}_{m_0-1}\right)}^2-{\overline{\mu }}_{2m_0}{\mathbf{X}}_{m_0-1}^T{\overline{\Delta }}_{2(m_0-1)}{\mathbf{X}}_{m_0-1}<0.$$

(18)

The above inequality contradicts Condition (i).

If the moments ${\overline{\mu }}_j$ satisfy Condition (ii), then there exists an $m_0$-dimensional nonzero vector such that ${\mathit{X}}_{m_0-1}^T{\overline{\Delta }}_{2(m_0-1)}{\mathit{X}}_{m_0-1}=0$. By Condition (ii), without loss of generality, we let $m_1=m_0$ and repeat the same procedure for the derivation of (17) to get that:

$$0<{\mathbf{X}}_{m_0}^T{\overline{\Delta }}_{2m_0}{\mathbf{X}}_{m_0}={\mathbf{X}}_{m_0-1}^T{\overline{\Delta }}_{2(m_0-1)}{\mathbf{X}}_{m_0-1}=0,$$

(19)

where ${\mathit{X}}_{m_0}^T=({\mathit{X}}_{m_0-1}^T,0)$. Obviously, the relationship in (19) is contradictory.

Then, we complete the proof of necessity.  □

Remark 1.

We use ${\underline{\mu }}_m$, ${\underline{\Delta }}_{2m}$, and ${\underline{U}}_m$ to replace ${\overline{\mu }}_{2m}$, ${\overline{\Delta }}_{2m}$, and ${\overline{U}}_m$, respectively, in Theorem 1, then we get the sufficient and necessary conditions to the existence of the probability density function $\underline{p}\left(x\right)$.

Note that Theorem 1 is presented in one-dimensional random variable space. Now, we extend this theorem to two-dimensional case. Let X and Y be any continuous random variables with the joint probability density function $p_{\mathit{\theta }}(x,y)$. The two-dimensional moment problems are defined by: find the joint probability density function $\overline{p}(x,y)$ and $\underline{p}(x,y)$ such that:

$${\overline{\mu }}_{ij}={\int }_{\mathbb{R}}{\int }_{\mathbb{R}}{x}^i{y}^j\overline{p}(x,y)dxdy,i,j\in \mathcal{M}$$

(20)

and:

$${\displaystyle {\underline{\mu }}_{ij}={\int }_{\mathbb{R}}{\int }_{\mathbb{R}}{x}^i{y}^j\underline{p}(x,y)dxdy,i,j\in \mathcal{M},}$$

(21)

where the set:

$$\mathcal{M}=\left\{\right(i,j\left)\right|i,j=0,1,\cdots ,2M\mathrm{and}0\le i+j\le 2M\}.$$

Here, we still take the sequence ${\left\{{\overline{\mu }}_{ij}\right\}}_{i,j\in \mathcal{M}}$ for example and present the main results in the following theorem without proof. Let:

$${\overline{\Gamma }}_{2m+1}:=\left(\begin{array}{cccccccc}{\overline{\mu }}_{00}& {\overline{\mu }}_{10}& \cdots & {\overline{\mu }}_{m0}& {\overline{\mu }}_{01}& {\overline{\mu }}_{02}& \cdots & {\overline{\mu }}_{0m}\\ {\overline{\mu }}_{10}& {\overline{\mu }}_{20}& \cdots & {\overline{\mu }}_{m+1,0}& {\overline{\mu }}_{11}& {\overline{\mu }}_{12}& \cdots & {\overline{\mu }}_{1m}\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ {\overline{\mu }}_{m0}& {\overline{\mu }}_{m+1,0}& \cdots & {\overline{\mu }}_{2m,0}& {\overline{\mu }}_{m1}& {\overline{\mu }}_{m2}& \cdots & {\overline{\mu }}_{mm}\\ {\overline{\mu }}_{01}& {\overline{\mu }}_{11}& \cdots & {\overline{\mu }}_{m1}& {\overline{\mu }}_{02}& {\overline{\mu }}_{02}& \cdots & {\overline{\mu }}_{0,m+1}\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ {\overline{\mu }}_{0m}& {\overline{\mu }}_{1m}& \cdots & {\overline{\mu }}_{mm}& {\overline{\mu }}_{0,m+1}& {\overline{\mu }}_{0,m+2}& \cdots & {\overline{\mu }}_{0,2m}\end{array}\right)$$

(22)

be the Hankel matrix generated by the given sequence ${\left\{{\overline{\mu }}_{ij}\right\}}_{i,j\in \mathcal{M}}$.

Theorem 2.

The sequence ${\left\{{\overline{\mu }}_{ij}\right\}}_{i,j\in \mathcal{M}}$ can determine the density functions ${\overline{p}}_N(x,y)$ if and only if it satisfies one of the following conditions:

(a) For any $m-1$-dimensional vectors ${\overline{\mathit{X}}}_{m-1}^T=(x_1,\cdots ,x_{m-1})$, ${\mathit{Y}}_{m-1}^T=(y_1,\cdots ,y_{m-1})$ and m-dimensional vectors ${\mathit{X}}_m^T=(x_0,\cdots ,x_{m-1})$, ${\overline{\mathit{Y}}}_m^T=(y_1,y_2\cdots ,y_m)(1\le m\le M)$,

$${\overline{\mu }}_{2m,0}{\mathbf{Z}}_{2m+1}^T{\overline{\Gamma }}_{2m+1}{\mathbf{Z}}_{2m+1}>{({\overline{U}}_{0·}^T{\mathbf{X}}_m+{\overline{U}}_{m·}^T{\mathbf{Y}}_{m-1})}^2$$

and:

$${\overline{\mu }}_{0,2m}{\mathbf{Z}}_{2m+1}^T{\overline{\Gamma }}_{2m+1}{\mathbf{Z}}_{2m+1}>{({\overline{U}}_{·m}^T{\overline{\mathbf{X}}}_{m-1}+{\overline{U}}_{0·}^T{\overline{\mathbf{Y}}}_m)}^2;$$

(b) If there exists $m_0\ge 1$ such that ${\mathit{Z}}_{2m_0+1}^T{\overline{\Gamma }}_{2m_0+1}{\mathit{Z}}_{2m_0+1}=0$, then:

$$|{\overline{\Gamma }}_{2l+1}|=0$$

for $m_0\le l\le M$, where the vectors ${\mathit{Z}}_{2m-1}^T=(X_m^T,x_m,Y_{m-1}^T,y_m)$ and ${\overline{U}}_{·0}^T=({\overline{\mu }}_{m0},\cdots ,{\overline{\mu }}_{2m-1,0})$, ${\overline{U}}_{0·}^T=({\overline{\mu }}_{0m},\cdots ,{\overline{\mu }}_{0,2m-1})$ and ${\overline{U}}_{·m}^T=({\overline{\mu }}_{1m},\cdots ,{\overline{\mu }}_{m-1,m})$, ${\overline{U}}_{m·}^T=({\overline{\mu }}_{m1},\cdots ,{\overline{\mu }}_{m,m-1})$.

3. Maximum Entropy for Moment Problems

In Section 2, we discussed the existence of solutions of the moment problems. Now, we will use the maximum entropy method to get the solutions for Problems (6) and (7). Moreover, we will consider the convergence of maximum entropy density functions.

Given the first $N+1$ moments ${\overline{\mu }}_0,{\overline{\mu }}_1,\cdots ,{\overline{\mu }}_N$, the core idea of the maximum entropy method is to find the probability density function ${\overline{p}}_N\left(x\right)$, such that:

$${\displaystyle {\overline{\mu }}_j={\int }_{\mathbb{R}}{x}^j{\overline{p}}_N\left(x\right)dx,j=0,1,\cdots ,N,}$$

(23)

where ${\overline{\mu }}_0=1$.

The Lagrange operator can be defined as:

$${\displaystyle L(\overline{p},{\overline{\lambda }}_0,\cdots ,{\overline{\lambda }}_N)=-{\int }_{\mathbb{R}}\overline{p}\left(x\right)\ln \overline{p}\left(x\right)dx+\sum _{j=0}^N{\overline{\lambda }}_j({\int }_{\mathbb{R}}{x}^j\overline{p}\left(x\right)dx-{\overline{\mu }}_j),}$$

(24)

where ${\overline{\lambda }}_j$, $j=0,1,\cdots ,N$ are the Lagrange multipliers.

By the functional variation with respect to $\overline{p}\left(x\right)$, we have:

$${\displaystyle {\overline{p}}_N\left(x\right)=exp\{-\sum _{j=0}^N{\overline{\lambda }}_j{x}^j\}=\underset{\overline{p}}{\max }\{-{\int }_{\mathbb{R}}\overline{p}\left(x\right)\ln \overline{p}\left(x\right)dx\}}$$

(25)

and (23) holds. The values of ${\overline{\lambda }}_j$ can be calculated by solving a system of $N+1$ equations resulting from the moments conditions (23). Here, we take $N=2$ as an example and work out the values of ${\overline{\lambda }}_j(j=0,1,2)$ as:

$${\overline{\lambda }}_0={\overline{\mu }}_1^2+\ln \sqrt{2\pi ({\overline{\mu }}_2-{\overline{\mu }}_1^2)},{\overline{\lambda }}_1=-\frac{{\overline{\mu }}_1}{{\overline{\mu }}_2-{\overline{\mu }}_1^2},{\overline{\lambda }}_2=\frac{1}{2({\overline{\mu }}_2-{\overline{\mu }}_1^2)}.$$

So far, we have considered the existence of the solutions of the moment Problems (6) and (7) for all $N\ge 1$. According to Theorem 3.3.11 in Durrett [12], we know that the probability density function $\overline{p}\left(x\right)$ is also unique in the weak convergence (see (27)) as long as its moments ${\left\{{\overline{\mu }}_j\right\}}_{j=0}^{+\infty }$ satisfy the conditions (a) and (b) in Theorem 1 and:

$${\displaystyle \underset{m\to +\infty }{\lim \sup }\frac{{\overline{\mu }}_{2m}^{1/2m}}{2m}<\infty .}$$

(26)

By the analysis in Frontini [13], we have the weak convergence for the maximum entropy solution ${\overline{p}}_N\left(x\right)$ as follows:

$${\displaystyle \underset{N\to +\infty }{\lim }{\int }_{\mathbb{R}}{\overline{p}}_N\left(x\right)\ln {\overline{p}}_N\left(x\right)dx={\int }_{\mathbb{R}}\overline{p}\left(x\right)\ln \overline{p}\left(x\right)dx.}$$

(27)

Remark 2.

By an argument analogous to the one-dimensional case, we can use the maximum entropy method to solve a concrete joint probability density function ${\overline{p}}_N(x,y)$ for the two-dimensional moment problem (20). The detailed process is shown in the next section and need not be repeated here.

By Theorem 2 and Theorem 14.20 in Schmüdgen [14], if the marginal moments ${\left\{{\overline{\mu }}_{m·}\right\}}_{2m=0}^{+\infty }$ and ${\left\{{\overline{\mu }}_{·m}\right\}}_{2m=0}^{+\infty }$ satisfy Conditions (a) and (b) in Theorem 2 and the following multivariate Carleman condition [14]:

$${\displaystyle \sum _{m=0}^{+\infty }{\overline{\mu }}_{2m,i}^{-\frac{1}{2m}}=\sum _{m=0}^{+\infty }{\overline{\mu }}_{i,2m}^{-\frac{1}{2m}}=+\infty }$$

for $0\le i\le N$, then there exists a unique joint probability density function $\overline{p}(x,y)$ satisfying (20).

The convergence rate for the maximum entropy density ${\overline{p}}_N(x,y)$ has been analyzed in Frontini [13], and the results are analogous to (27).

4. Numerical Experiments

In this section, we conduct numerical experiments to calculate the two-dimensional maximum entropy density functions. In the numerical experiments, we use the maximum entropy method proposed in Section 3 to calculate the joint probability density functions for the weekly closing price of stock X and the weekly rate of return Y of Shanghai A shares.

By random sampling, we collect the data about the maximum (Table 1) and minimum (Table 2) values of the mixed sample moments of orders one and two for X and Y, respectively. That is,

Table 1. The values for the maximum value of the moments.

Moments	${\overline{\mathit{\mu }}}_{00}$	${\overline{\mathit{\mu }}}_{10}$	${\overline{\mathit{\mu }}}_{20}$	${\overline{\mathit{\mu }}}_{11}$	${\overline{\mathit{\mu }}}_{01}$	${\overline{\mathit{\mu }}}_{02}$
Values	1	30.781	1.5095 × 103	1.0158	0.033	0.0086

Table 2. The values for the minimum value of the moments.

Moments	${\underline{\mathit{\mu }}}_{00}$	${\underline{\mathit{\mu }}}_{10}$	${\underline{\mathit{\mu }}}_{20}$	${\underline{\mathit{\mu }}}_{11}$	${\underline{\mathit{\mu }}}_{01}$	${\underline{\mathit{\mu }}}_{02}$
Values	1	14.311	343.1627	−1.2451	−0.087	0.0031

The two-dimensional maximum entropy problem is defined as: find a probability density function ${\overline{p}}_2(x,y)$ such that:

$${\displaystyle {\overline{p}}_2(x,y)=\underset{\overline{p}}{\max }\{-{\int }_{\mathbb{R}}{\int }_{\mathbb{R}}\overline{p}(x,y)\ln \overline{p}(x,y)dxdy\}}$$

(28)

and:

$${\overline{\mu }}_{kj}={\int }_{\mathbb{R}}{\int }_{\mathbb{R}}{x}^k{y}^j{\overline{p}}_2(x,y)dxdy$$

(29)

for $k,j=0,1,2$ and $0\le k+j\le 2$.

Take notice of the formula as:

$${\overline{p}}_2(x,y)={\overline{p}}_2\left(y\right|x){\overline{p}}_2\left(x\right),$$

(30)

where ${\overline{p}}_2\left(y\right|x)$ denotes the maximum entropy conditional density function. From (30), we can obtain the joint density function ${\overline{p}}_2\left(y\right|x)$ as long as we deduce ${\overline{p}}_2\left(y\right|x)$ and ${\overline{p}}_2\left(x\right)$. Let:

$${\displaystyle {\overline{\mu }}_{0j}\left(x\right)={\int }_R{y}^j{\overline{p}}_2\left(y\right|x)dy,j=0,1,2,}$$

(31)

be the conditional moments of the random variable Y under X, which satisfy:

$${\displaystyle {\int }_{\mathbb{R}}{x}^k{\overline{\mu }}_{0j}{\overline{p}}_2\left(x\right)dx={\overline{\mu }}_{kj}}$$

(32)

for $k,j=0,1,2$ and $0\le k+j\le 2$.

According to Table 1 and the definition (22), we have:

$${\overline{\Gamma }}_3=\left(\begin{array}{ccc}1& 30.781& 0.033\\ 30.781& 1509.5& 1.0158\\ 0.033& 1.0158& 0.0086\end{array}\right)\mathrm{and}{\overline{\Delta }}_{2·}=\left(\begin{array}{cc}1& 30.781\\ 30.781& 1509.5\end{array}\right).$$

(33)

It is easy to verify that ${\overline{\Gamma }}_3$ and ${\overline{\Delta }}_{2·}$ are positive definite. Hence, by Theorem 1, there exists a density function ${\overline{p}}_2\left(x\right)$ determined by $\{{\overline{\mu }}_{i0},i=0,1\}$.

To derive the explicit expressions of ${\overline{p}}_2(x,y)$, we need to construct the conditional moments ${\left\{{\overline{\mu }}_{0j}\left(x\right)\right\}}_{j=0}^2$ on the base of (32). Without loss of generality, we suppose ${\overline{\mu }}_{01}\left(x\right)$ is a constant. Combining (32) yields that:

$$\left\{\begin{array}{cc}{\overline{\mu }}_{00}\left(x\right)=1,\hfill & \\ {\overline{\mu }}_{01}\left(x\right)={\overline{\mu }}_{01},\hfill & \\ {\overline{\mu }}_{02}\left(x\right)=e^{-\overline{\delta }{x}^2}+{\overline{\mu }}_{01}^2,\hfill & \overline{\delta }=40.7624.\hfill \end{array}\right.$$

(34)

By calculation, we derive the expressions of ${\overline{p}}_2(x,y)$ as follows:

$$\begin{array}{c}{\overline{p}}_2(x,y)=\frac{1}{\sqrt{2\pi }}exp\{-1-{\overline{\lambda }}_0-{\overline{\lambda }}_{1}x-{\overline{\lambda }}_2{x}^2\hfill \\ -\frac{1}{2}{\overline{\mu }}_{01}^2{e}^{-\overline{\delta }{x}^2}+{\overline{\mu }}_{01}y{e}^{-\overline{\delta }{x}^2}-\frac{1}{2}{y}^2{e}^{-\overline{\delta }{x}^2}\},\hfill \end{array}$$

(35)

where ${\overline{\lambda }}_0=3.9276$, ${\overline{\lambda }}_1=-0.0548$, and ${\overline{\lambda }}_2=0.0009$.

In the same way, with the data in Table 2, we can obtain the following probability density function ${\underline{p}}_2(x,y)$ determined by $\{{\underline{\mu }}_{k,j},k,j=0,1,2\mathrm{and}0\le k+j\le 2\}$.

$$\begin{array}{c}{\displaystyle {\underline{p}}_2(x,y)=\frac{1}{\sqrt{2\pi }}exp\{-1-{\underline{\lambda }}_0-{\underline{\lambda }}_{1}x-{\underline{\lambda }}_2{x}^2}\hfill \\ -\frac{1}{2}{\underline{\mu }}_{01}^2{e}^{-\underline{\delta }{x}^2}+{\underline{\mu }}_{01}y{e}^{-\underline{\delta }{x}^2}-\frac{1}{2}{y}^2{e}^{-\underline{\delta }{x}^2}\},\hfill \end{array}$$

(36)

where ${\underline{\lambda }}_0=3.1240$, ${\underline{\lambda }}_1=-0.1034$, ${\underline{\lambda }}_2=0.0036$, and $\underline{\delta }=773.7203$.

Figure 1 and Figure 2 show visually the sketches of the marginal entropy density functions ${\overline{p}}_2\left(x\right)$, ${\underline{p}}_2\left(x\right)$ (derived in (25) with $N=2$) and the joint entropy density functions ${\overline{p}}_2(x,y)$, ${\underline{p}}_2(x,y)$ (derived in (35) and (36)), respectively.

Figure 1. The figures above show the figures of the maximum entropy marginal density ${\overline{p}}_2\left(x\right)$ (left) and joint density ${\overline{p}}_2(x,y)$ (right) determined by ${\left\{{\overline{\mu }}_{j0}\right\}}_{j=0}^2$ and ${\left\{{\overline{\mu }}_{ij}\right\}}_{i,j\in \mathcal{M}}^2$, respectively.

Figure 2. The figures above show the figures of the maximum entropy marginal density ${\underline{p}}_2\left(x\right)$ (left) and joint density ${\underline{p}}_2(x,y)$ (right) determined by ${\left\{{\underline{\mu }}_{j0}\right\}}_{j=0}^2$ and ${\left\{{\underline{\mu }}_{ij}\right\}}_{i,j\in \mathcal{M}}^2$, respectively.