First-Passage Times and Optimal Control of Integrated Jump-Diffusion Processes

: Let Y ( t ) be a one-dimensional jump-diffusion process and X ( t ) be deﬁned by d X ( t ) = ρ [ X ( t ) , Y ( t )] d t , where ρ ( · , · ) is either a strictly positive or negative function. First-passage-time problems for the degenerate two-dimensional process ( X ( t ) , Y ( t )) are considered in the case when the process leaves the continuation region at the latest at the moment of the ﬁrst jump, and the problem of optimally controlling the process is treated as well. A particular problem, in which ρ [ X ( t ) , Y ( t )] = Y ( t ) − X ( t ) and Y ( t ) is a standard Brownian motion with jumps, is solved explicitly.


Introduction
Diffusion processes are used as models in various applications, in particular in neuroscience to emulate the dynamics of the membrane potential of a neuron [1].Moreover, to take into account the spikes of the neuron, jump-diffusion processes have been proposed by Jahn et al. [2] and Melanson and Longtin [3], among others.
Presently, diffusion and jump-diffusion processes both increase and decrease in any interval, however small.However, in some applications, it is not realistic to assume that the process can decrease or increase.For example, if X(t) represents the wear of a machine at time t, the stochastic process {X(t), t ≥ 0} should increase with time.
One way to obtain a strictly increasing or decreasing process is to consider degenerate two-dimensional diffusion processes (X(t), Y(t)) defined by where ρ(•, •) is either a strictly positive or negative function and {B(t), t ≥ 0} is a standard Brownian motion.The functions f and v are such that {Y(t), t ≥ 0} is a diffusion process.
When ρ[X(t), Y(t)] = Y(t), the process {X(t), t ≥ 0} is called an integrated diffusion process.We can of course generalize the definition to the case when {Y(t), t ≥ 0} is a jump-diffusion process.
The author has published a number of papers on integrated diffusion processes; see, for instance, Lefebvre [4] for a recent one.Other papers on this topic include those by Lachal [5], Makasu [6], Metzler [7], Caravelli et al. [8] and Levy [9].
Let φ(ξ, η; x, y) be the joint probability density function (pdf) of the random vector (X(t), Y(t)), with (X(0), Y(0)) = (x, y).As is well known (see, for example, Cox and Miller [10], p. 247), the function φ satisfies the Kolmogorov backward equation Moreover, the pdf g(t; x, y) of the random variable T(x, y) satisfies the same partial differential equation (PDE): (subject to different boundary conditions).It follows that the moment-generating function of the random variable T(x, y), namely where α > 0, is a solution of the following PDE: where M yy := ∂ 2 M(x, y; α)/∂y 2 , etc. Furthermore, this equation is subject to the boundary condition We now replace the diffusion process {Y(t), t ≥ 0} by the jump-diffusion process defined by where {N(t), t ≥ 0} is a Poisson process with rate λ.The random variables Z 1 , Z 2 , . . .are assumed to be independent and identically distributed (i.i.d.), and also independent of the Poisson process.We can state the following proposition.
Proposition 1.The function M(x, y; α) satisfies the integro-differential equation (dropping the dependence on α from the notation) for (x, y) ∈ C, where f Z (z) is the common density function of the Z n s.As above, the equation is subject to the boundary condition (8).
Proof.This result is obtained by generalizing the infinitesimal generator of the jump-diffusion process {Y(t), t ≥ 0} in Kou and Wang [11] to the case when f (y) and v(y) are not necessarily constant functions.
Remark 1. See also the remark after the proof of Proposition 2.
There are still few explicit solutions to first-passage problems for jump-diffusion processes.Kou and Wang [11] obtained explicit formulae for the Laplace transform of the pdf of the first-passage time τ to a constant boundary for a Wiener process with jumps having a double exponential distribution.This model was generalized or modified in various papers.In Chen et al. [12], τ was the first-exit time from a finite interval, while in Yin et al. [13] the jumps were mixed-exponential random variables.Karnaukh [14] considered the case when the parameters of the Wiener process depend on a finite Markov chain.In Lefebvre [15], the jump sizes were assumed to be uniformly distributed, while in Abundo [16] the jumps (positive and/or negative) were of a constant size and the boundaries were time-dependent.Because obtaining exact analytical solutions to such problems is difficult, some authors presented numerical techniques to obtain the quantities of interest; see, for example, Belkaid and Utzet [17].
Di Crescenzo et al. [18] computed bounds for first-crossing-time probabilities of a Wiener process with random jumps driven by a counting process.Fernández et al. [19] proposed algorithms to compute double-barrier first-passage-time probabilities of a jumpdiffusion process with an arbitrary jump size distribution.
D'Onofrio and Lanteri [20] obtained numerical approximations for the density functions of first-passage times in the case of diffusion processes with state-dependent jumps.Finally, in Lefebvre [21], the author was able to obtain exact solutions to optimal control problems for Wiener processes with exponential jumps.
In the current paper, explicit results will be presented for the first-passage time of a two-dimensional jump-diffusion process.
In the next section, the special case when the two-dimensional process (X(t), Y(t)) is killed at the latest at the moment of the first jump, will be considered.We are also interested in the mean value of T(x, y), as well as in the probability that (X(t), Y(t)) will leave the continuation region through a given part of its boundary ∂C.In Section 3, the problem of maximizing or minimizing the time the controlled version of the process (X(t), Y(t)) spends in the continuation region C will be treated.A particular problem will be solved explicitly in Section 4. Finally, we will conclude with a few remarks in Section 5.

Killed Processes
Assume that the random variables Z 1 , Z 2 , . . .are such that no overshoot is possible.That is, the degenerate two-dimensional jump-diffusion process (X(t), Y(t)) cannot jump over the boundary of the continuation region C. Let m(x, y) := E[T(x, y)] and p(x, y) : where ∂C 0 ⊂ ∂C.We have the following corollaries.
Corollary 1.The function m(x, y) satisfies the integro-differential equation for (x, y) ∈ C, subject to the boundary condition Proof.It follows from the expansion of M(x, y; α) into an infinite series (see Cox and Miller [10]): Notice that in our case, E[T n (x, y)] will exist for any n ∈ {1, 2, . ..} because of Equation ( 18) below.
Corollary 2. The probability p(x, y) is a solution of the integro-differential equation for (x, y) ∈ C.Moreover, the boundary condition is where ∂D := ∂C \ ∂C 0 .
In this paper, we consider the special case when the random variable Z 1 is such that the process (X(t), Y(t)) will leave the continuation region C at the latest at the moment τ 1 of the first event of the Poisson process.Let T 0 (x, y) be the random variable that corresponds to T(x, y) when λ = 0. We can write that where τ 1 has an exponential distribution with parameter λ.Furthermore, the sum in Equation ( 9) can be replaced by Z 1 1 {N(t)>0} , where 1 {N(t)>0} is the indicator variable of the event {N(t) > 0}, and the equation is valid for t ∈ [0, T(x, y)].We can say that the process (X(t), Y(t)) is killed at time T(x, y).
An application of the above problem is the following: as mentioned in Section 1, a more realistic model for the wear of a machine is the degenerate two-dimensional process (X(t), Y(t)) defined in Equations ( 1) and (2), when ρ(•, •) is a strictly increasing function.Rishel [22] proposed this model (in n dimensions) in the context of an optimal control problem.If X(t) denotes the remaining amount of deterioration that a device can undergo before it must be repaired or replaced, then ρ(•, •) should be strictly negative instead.Moreover, the remaining lifetime of the device is the first-passage time to zero or to a level at which it is considered worn out.
Presently, many electronic devices, especially mobile phones, are often replaced as soon as they break down, rather than being repaired.A mobile phone failure can be observed as a jump from the current value of X(t) to zero, so that the device is killed at the time of the jump.It is also possible that the device will be replaced before a failure occurs, due to normal wear and tear or because it has become obsolete.Thus, deterioration could also include the age of the device.
Similarly, in the case of humans, the downward jump to zero could occur during a massive heart attack or stroke.
Because we assume that (x, y + z) ∈ ∂C for any possible value z of the random variable Z, the integro-differential Equations ( 10) and ( 12) become, respectively, the partial differential equations and In the case of Equation ( 16), if (x, y + z) ∈ ∂C 0 ∀z, then whereas, we have if (x, y + z) ∈ ∂D ∀z.If (x, y + z) belongs to ∂C 0 for some values of z, and to ∂D for other values of z, then the integral in Equation ( 16) is replaced by Solving integro-differential equations explicitly and exactly is not an easy task.In Section 4, an example of a problem that we can indeed solve analytically will be presented.As above, the integro-differential equations will be reduced to PDE's, and the method of similarity solutions will be used to transform these PDE's into ordinary differential equations.

Optimal Control
In this section, we consider a controlled version of the two-dimensional process (X(t), Y(t)): where u(•, •) is the control variable, which is assumed to be a continuous function, and b is a non-zero constant.The aim is to find the value of the control that minimizes the expected value of the cost function where q > 0 and θ are constants.If the parameter θ is positive (respectively negative), the optimizer must try to minimize (respectively maximize) the time spent by the controlled process in the continuation C, taking the quadratic control costs into account.This type of problem is known as a homing problem; see Whittle [23] and/or [24].To solve the above problem, we can make use of dynamic programming.We define the value function That is, F(x, y) is the expected cost (or reward, if it is negative) obtained by choosing the optimal value of the control variable in the interval [0, T(x, y)).
Proposition 2. The value function F(x, y) satisfies the second-order, non-linear partial integrodifferential equation Moreover, we have the boundary condition Proof.First, thanks to Bellman's principle of optimality, we can write that Next, let ξ := x + ρ(x, y) ∆t (32) We have Since N(∆t) has a Poisson distribution with parameter λ ∆t, we can write that and Hence, Now, assuming that F(x, y) is twice the differentiable with respect to x and to y, making use of Taylor's formula for the functions of two variables, we obtain that Similarly, we find that Indeed, by independence, we have Let w := y + z.We compute so that Thus, where r := y − x.This is an application of the method of similarity solutions, and r is called the similarity variable.For the method to work, both the equation and the boundary conditions must be expressed in terms of r (after simplification).Here, we find that Equation (60) reduces to the ordinary differential equation (ODE) subject to the boundary conditions N(k i ) = 1, for i = 1, 2. With the help of the mathematical software program Maple, we find that the general solution of the above equation can be written as where M(•, •, •) and U(•, •, •) are Kummer functions.The constants c 1 and c 2 are uniquely determined from the boundary conditions Since, as noted in Section 2 (see Equation ( 18)), T(x, y) = min{T 0 , τ 1 }, when λ is large, the function M(x, y; α) should be close to the moment-generating function of an exponential random variable with parameter λ, namely In Figure 1, we present the functions M 0 (α) and M(x, y; α) for α ∈ (0, 10), when λ = 1, k 1 = 0, k 2 = 1 and y − x = 0.5.We observe that the two functions differ significantly.However, the two functions are very similar when λ = 20, as can be observe in Figure 2. When λ = 100, M 0 (α) and M(x, y; α) practically coincide for α ∈ (0, 10).Next, the function m(x, y) = E[T(x, y)] satisfies the PDE (see Equation ( 20)) subject to m(x, y) = 0 if y − x = k 1 or k 2 .Setting m(x, y) = n(r), we obtain the ODE with n(k i ) = 0, for i = 1, 2. We find that The particular solution that satisfies the boundary conditions n(0) = n(1) = 0 is presented in Figure 3.
Assuming that p(x, y) = q(r), we obtain the ODE whose general solution is The solution that satisfies the boundary conditions q(0) = 1 and q(1) = 0 is shown in Figure 4, when λ = 1 and p 0 = 1/2.To conclude this section, we will try to find the optimal control of the two-dimensional process (X u (t), Y u (t)) defined by To do so, we must solve the non-linear second-order PDE As above, we make use of the method of similarity solutions.We look for a solution of the form F(x, y) = G(r = y − x).Equation (74) becomes If λ = 0, Maple is able to obtain the general solution of the preceding equation: where and The constants c 1 and c 2 are determined by making use of the boundary conditions When λ > 0, Maple and Mathematica are unable to provide an analytical expression for the solution of Equation (75).It is, however, not difficult to obtain a numerical solution for any choice of the parameters.For instance, if we choose b = q = θ = λ = 1, k 1 = 1 and k 2 = 2, we obtain the function G(r), as shown in Figure 5, together with the function obtained when λ = 0. Finally, in Figure 6, we present the corresponding optimal controls.

Conclusions
In this paper, we have considered degenerate two-dimensional jump-diffusion processes, defined in such a way that the first component of the vector (X(t), Y(t)) is a strictly increasing or decreasing function with respect to time.This kind of process is more realistic than a one-dimensional diffusion or jump-diffusion process in many applications, especially when X(t) represents the age or wear of a certain device.We could generalize the model by incorporating more than one diffusion process Y(t).The diffusion processes could model the various variables that influence X(t).For example, in the case of wear, important factors to consider are temperature, speed of use, etc.
In Section 2, we obtained equations for functions defined in terms of a first-passage time for the processes (X(t), Y(t)).Moreover, we treated an optimal control problem for these processes in Section 3. Finally, a particular problem was solved explicitly in Section 4.
As mentioned in Section 1, there are few first-passage problems for one-dimensional jump-diffusion processes that have been solved exactly and explicitly so far.Here, we were able to find exact analytical expressions for quantities defined in terms of a firstpassage time for a (degenerate) two-dimensional jump-diffusion process.Furthermore, in Section 2, we saw that the processes considered in this paper could serve as models in real-life applications, such as the remaining amount of deterioration that a given device can undergo before it needs to be repaired or replaced.
In general, to solve this type of problem, it is necessary to find the solution of an integrodifferential equation with partial derivatives.We considered the case when the process leaves the continuation region at the latest when the first event of the Poisson process occurs.In this case, the equation to be solved becomes a partial differential equation.Using the method of similarity solutions, it is sometimes possible to reduce this PDE to an ODE.It should also be possible to find a numerical solution to any particular problem.
As a follow-up to this work, we would like to find exact analytical solutions to problems where the equations to be solved are integro-differential equations; for example, by trying to transform the integro-differential equations into differential equations.