Asymptotic Duration for Optimal Multiple Stopping Problems

Abstract

We study the asymptotic duration of optimal stopping problems involving a sequence of independent random variables that are drawn from a known continuous distribution. These variables are observed as a sequence, where no recall of previous observations is permitted, and the objective is to form an optimal strategy to maximise the expected reward. In our previous work, we presented a methodology, borrowing techniques from applied mathematics, for obtaining asymptotic expressions for the expectation duration of the optimal stopping time where one stop is permitted. In this study, we generalise further to the case where more than one stop is permitted, with an updated objective function of maximising the expected sum of the variables chosen. We formulate a complete generalisation for an exponential family as well as the uniform distribution by utilising an inductive approach in the formulation of the stopping rule. Explicit examples are shown for common probability functions as well as simulations to verify the asymptotic calculations.

1. Introduction

Optimal stopping problems are formulated in terms of observing random variables and determining the stopping time(s) in order to maximise an objective function (which can be thought of as some reward). The discrete single-stopping problem involves observing a sequence of random variables $y_1,y_2,\cdots ,y_N$, and making the choice to stop on a particular observation $y_m$, where $1\le m\le N$, which is based only on the variables that have been previously observed. After stopping, the “player” receives some pay-off which is a function of the variables observed $y_1,\dots ,y_m$. In [1], we used the pay-off of $y_m$ (i.e., the value of the variable stopped on). To extend this to the multiple optimal stopping problem we stop on $k>1$ variables and, after stopping on variables $y_{m_1},y_{m_2},\dots ,y_{m_k}$ at times $m_1,m_2,\dots ,m_k$, we then receive a gain which is a function of the variables observed. This problem is a subset of a general class of other optimal stopping problems that all aim to find a sequential procedure to maximise the expected reward (see Section 13.4 of [2] for a more extensive discussion of this class of problem). The secretary problem is arguably the most well known (see [3,4]), and it has a wide range of variations (see [5,6]), but there is also a rich literature of other examples (see [7]). By the ‘duration’ (sometimes referred to as ‘time’) of the stopping problem, we refer to how many observations the statistician observes before optimally stopping. It should be emphasised that knowing how long you would be waiting, on average, for potentially large sequences of observations is also useful to understand (see [8]), which highlights the need for asymptotic analysis to address this question.

Less focus has been placed on understanding the asymptotic behaviour of the stopping duration, with most pre-existing results focusing on secretary-type, so called “no-information”, problems where the distribution of the observations is unknown. The asymptotic expectation and variance of the stopping time for the secretary problem were studied in [9]. Similar asymptotic analyses for other variants of no-information problems can be found in [10,11,12,13], where the techniques/formulations used differ depending on the particular variation or structure of the problem.

There is substantially less literature describing the asymptotic behaviour for “full-information” problems, when the distribution of the variables is known a priori. A smaller subset addresses the multiple stopping problem, which we focus on in this study.

Gilbert and Mosteller [11] studied the optimal stopping strategy for the full-information problem in which the objective is to maximise the probability of attaining the best observation, known as the full-information best-choice problem [14]. The optimal rule was shown to be a threshold strategy wherein the player would stop on $y_m$ if it was the best observed so far after all other observations and its value exceeded a threshold that depended on m. The asymptotic behaviour of this rule was also derived.

In the full-information case, the pay-off can instead be in terms of the actual values of the variables stopped on. A special case of this is the uniform game (see Section 5a of [11]), which is closely related to Cayley’s problem (see [3,15]). In [11], the authors showed an asymptotic expression for the expected reward of a sequence of n independent and identically distributed (iid) random variables having the standard uniform distribution (see also [15]). In [9], Mazalov and Peshkov found the asymptotic behaviour of the expected value and variance of the stopping time as $N/3$ and $N^2/18$, respectively, when the variables are from the uniform distribution. However, the techniques applied were specific to the structure of the distribution, and thus, it is difficult to easily extend or generalise this to other distribution functions. In [16,17], using the extreme value theory, Kennedy and Kertz proved limit theorems for threshold-stopped random variables and derived the asymptotic distribution of the reward sequence of the optimal stopping (iid) random variables. The asymptotic pay-off for the multiple case is briefly analysed in (Section 5c of [11]).

In [1], we outlined a novel approach for a general asymptotic technique for calculating the asymptotic behaviour of the pay-off as well as $\mathbb{E}\left({\tau }_N\right)$ and $\mathrm{Var}\left({\tau }_N\right)$ in the single-stopping case, where ${\tau }_N$ is the single-stopping time, as $N\to \infty$ for general classes of probability distributions in the full-information problem where we wish to maximise the expected reward $y_m$. The techniques in our previous paper, which are extended in this study, employ the asymptotic analysis of difference and differential equations in order to establish and solve asymptotic differential equations for the quantities of interest. Differential equations were also used in [18,19]. In this study, we extend some of our results in [1] to the multiple stopping case through some inductive arguments and verify these results with simulations. For simplicity, we only analyse continuous distributions and reserve the notation $f\left(x\right),F\left(x\right)$, and $h\left(y\right)=1-F\left(x\right)$ for the continuous probability density, cumulative distribution, and “survivor” function, respectively. We use the notation $f\left(x\right)$∼$g\left(x\right)$ to establish the asymptotic relation that ${\displaystyle \underset{x\to \infty }{lim}{\displaystyle \frac{f\left(x\right)}{g\left(x\right)}}=1}$.

2. Formulation of the Multiple Optimal Stopping Problem

As in the single-stopping problem, we still sequentially observe the sequence $y_1,y_2,\dots ,y_N$ of independent, identically distributed (iid) random variables from a known distribution, we must decide which k of these variables to stop on. After k, $k\ge 2$ stoppings at times $m_1,m_2,\dots ,m_k$, where $1\le m_1<m_2<\dots <m_k\le N$, we receive the gain $Z_{m_1,m_2,\dots ,m_k}=y_{m_1}+y_{m_2}+\cdots +y_{m_k}.$ The random variable $y_m$ can be interpreted as the value of some asset, such as a house, at time m. The problem of selling k identical objects in a finite time (or horizon) N with one offer per time period and no recall of previous offers is analogous to the multiple optimal stopping problem described. If we stop at $m_1$ after observations ($y_1$,$y_2$,…, $y_{m_1}$), then we proceed to observe another sequence $y_{m_1+1},y_{m_1+2},\dots ,y_N$ (whose length depends on $m_1$) and must solve the new optimal stopping problem on this sequence. From [20], we have the following theorem.

Theorem 1.

Let $y_1,y_2,\dots ,y_N$ be a sequence of independent random variables with known cumulative distribution functions (cdfs) $F_1,F_2,\dots ,F_N$. Let $v^{L,l}$ be the value, which is the optimal expected reward, of a game with l, $l\le k$ stoppings and L, $L\le N$ steps. If there exist $\mathbb{E}\left(y_1\right),\mathbb{E}\left(y_2\right),\dots ,\mathbb{E}\left(y_N\right)$, then the value v of the ‘game’ is $v^{N,k}$, where

$$\begin{array}{cc}\hfill v^{n,1}& =\mathbb{E}\left(\max \{y_{N-n+1},v^{n-1,1}\}\right),1\le n\le N\hfill \\ \hfill v^{0,1}& =-\infty \hfill \\ \hfill v^{n,k-i+1}& =\mathbb{E}\left(\max \{y_{N-n+1}+v^{n-1,k-i},v^{n-1,k-i+1}\}\right),k-i+1\le n\le N\hfill \\ \hfill v^{k-i,k-i+1}& =-\infty ,i=k-1,\dots ,1.\hfill \end{array}$$

(1)

We put

$$\begin{array}{cc}\hfill {\tau }_1^{*}& =\min \{m_1:1\le m_1\le N-k+1,y_{m_1}\ge v^{N-m_1,k}-v^{N-m_1,k-1}\}\hfill \\ \hfill {\tau }_k^{*}& =\min \{m_k:{\tau }_{k-1}^{*}<m_k\le N,y_{m_k}\ge v^{N-m_k,1}\}\hfill \\ \hfill {\tau }_i^{*}& =\min \{m_i:{\tau }_{i-1}^{*}<m_i\le N-k+i,y_{m_i}\ge v^{N-m_i,k-i+1}-v^{N-m_i,k-i}\},\hfill \\ \hfill i& =2,\dots ,k-1\hfill \end{array}$$

then, ${\tau }^{*}=({\tau }_1^{*},{\tau }_2^{*},\dots ,{\tau }_k^{*})$ is the optimal stopping rule.

If we define $v_{i,j}=v^{N-j,k-i+1}-v^{N-j,k-i}$, then we may notice that the stopping rules now take the form

$$\begin{array}{cc}\hfill {\tau }_1^{*}& =\min \{m_1:1\le m_1\le N-k+1,y_{m_1}\ge v_{1,m_1}\}\hfill \\ \hfill {\tau }_k^{*}& =\min \{m_k:{\tau }_{k-1}^{*}<m_k\le N,y_{m_k}\ge v^{N-m_k,1}\}\hfill \\ \hfill {\tau }_i^{*}& =\min \{m_i:{\tau }_{i-1}^{*}<m_i\le N-k+i,y_{m_i}\ge v_{i,m_i}\},i=2,\dots ,k-1\hfill \end{array}$$

(2)

so we may interpret $v_{i,j}$ as the appropriate threshold value that needs to be satisfied for stopping for the ith occasion at the jth term in the sequence of N observations. We can then define $w_{i,j}=P(y<v_{i,j})$, which can now be interpreted as the probability that in the above situation we do not stop.

Example 1 (Multiple Stopping on the Uniform (0, 1) Distribution with $k=2$ stops).

Let $y_1,y_2,\dots y_N$ be a sequence of independent, identically distributed random variables that follow the uniform $U(0,1)$ distribution.

We derived the equations for $v^{n,1}$ in the original paper (for details, see [1]):

$v^{n,1}=\frac{1+{\left(v^{n-1,1}\right)}^2}{2}$ where $v^{0,1}=0$. For $v^{n,2}$ we have that

$$\begin{array}{cc}\hfill v^{n,2}& =\mathbb{E}\left(\max \{v^{n-1,1}+y_{N-n+1},v^{n-1,2}\}\right)\hfill \\ \hfill & =\mathbb{E}\left(\max \{y_{N-n+1},v^{n-1,2}-v^{n-1,1}\}\right)+v^{n-1,1}\hfill \\ \hfill & =\frac{{(v^{n-1,2}-v^{n-1,1})}^2+1}{2}+v^{n-1,1}\hfill \end{array}$$

which can then be used to numerically determine the values of the ‘game’.

For the example of $N=6$, we may produce the values for $v^{n,1}$ and $v^{n,2}$ for each value of n from $n=1$ to $n=6$, as displayed in Table 1. For example, consider this sequence of simulated variables:

$$0.1081,0.6987,0.1483,0.4123,0.8968,0.7242.$$

For the first stop, we would arrive on $y_2=0.6987$, since $y_3$ is the first variable to satisfy $y_{m_1}\ge v^{N-m_1,2}-v^{N-m_1,1}$ and for the second, we arrive on $y_5=0.8968$, as this is the first subsequent variable for which $y_{m_2}\ge v^{N-m_2,1}$. This particular example would have resulted in the reward $0.6987+0.8968=1.5955.$

Table 1. Values for the standard uniform distribution.

n	1	2	3	4	5	6
$v^{n,1}$	0.5000	0.6250	0.6953	0.7417	0.7751	0.8004
$v^{n,2}$		1.0000	1.1953	1.3203	1.4091	1.4761
$v^{n,2}-v^{n,1}$		0.3750	0.5000	0.5786	0.6340	0.6757

3. Computing $v^{n,k}$ Behaviour

We establish a similar recurrence result as we did for the single-stopping case. We note that this relation for $v^{n,k}$ will now be a second-order relation. For a convenient evaluation of the expectation, we use the fact that for any continuous integrable random variable X with cdf $F\left(x\right)$, the expectation can be given by

$$E\left(X\right)={\int }_0^{\infty }(1-F\left(x\right))\mathrm{d}x-{\int }_{-\infty }^{0}F\left(x\right)\mathrm{d}x$$

(3)

Theorem 2.

Let Y be an integrable random variable whose expectation exists, and which is drawn from a continuous probability distribution function (pdf) $f\left(y\right)$ with survivor function $h\left(y\right)=1-F\left(y\right)$. The value of a sequence with $n+1$ steps and k stops remaining is given by

$$v^{n+1,k}=v^{n,k}+{\int }_{v^{n,k}-v^{n,k-1}}^{\infty }h\left(y\right)\mathrm{d}y.$$

(4)

Proof. 

For ease of notation, we let $v=v^{n-1,k}-v^{n-1,k-1}$. Then, by definition, we have

$$\begin{array}{cc}\hfill v^{n+1,k}& =\mathbb{E}\left(\max \{y_{N-n}+v^{n,k-1},v^{n,k}\}\right)\hfill \\ \hfill & =v^{n,k-1}+\mathbb{E}\left(\max \{y_{N-n},v^{n,k}-v^{n,k-1}\}\right)\hfill \\ \hfill & =v^{n,k-1}+\mathbb{E}\left(\max \{y_{N-n},v\}\right)\hfill \end{array}$$

(5)

where the last expectation, using (3), is given by

$$\begin{array}{cc}\hfill & -{\int }_{-\infty }^{0}P(\max (Y,v\le y)\mathrm{d}y+{\int }_0^{v}P(\max (Y,v)>y)\mathrm{d}y+{\int }_v^{\infty }P(\max (Y,v)>y)\mathrm{d}y\hfill \\ & =0+{\int }_0^{v}1\mathrm{d}y+{\int }_v^{\infty }P(Y>y)\mathrm{d}y\hfill \\ & =v+{\int }_v^{\infty }h\left(y\right)\mathrm{d}y\hfill \\ \hfill & =v^{n,k}-v^{n,k-1}+{\int }_{v^{n,k}-v^{n,k-1}}^{\infty }h\left(y\right)\mathrm{d}y.\hfill \end{array}$$

Substituting this result, noting that the $v^{n,k-1}$ terms cancel, we obtain the required result. □

We note that if $f\left(y\right)$ has bounded support in the positive direction such that $f\left(y\right)=0$ for $y>y_{\max }$, it follows from above that

$$v^{n+1,k}=v^{n,k}+{\int }_{v^{n,k}-v^{n,k-1}}^{y_{\max }}h\left(y\right)\mathrm{d}y.$$

(6)

By the controlling factor method, we also have that $v^{n+1,k}-v^{n,k}\sim {\left(v^{n,k}\right)}^{\prime }$, which can be combined with the previous integral to establish the following:

$${\left(v^{n,k}\right)}^{\prime }\sim {\int }_{v^{n,k}-v^{n,k-1}}^{\infty }h\left(y\right)\mathrm{d}y.$$

(7)

This may be differentiated on both sides to obtain the asymptotic relation for the second derivative:

$${\left(v^{n,k}\right)}^{″}\sim -h(v^{n,k}-v^{n,k-1})·{(v^{n,k}-v^{n,k-1})}^{\prime }.$$

(8)

This expression can now be rearranged for $h(v^{n,k}-v^{n,k})$ to give

$$h(v^{n,k}-v^{n,k-1})\sim -\frac{{\left(v^{n,k}\right)}^{″}}{{(v^{n,k}-v^{n,k-1})}^{\prime }}.$$

(9)

Depending on the asymptotic nature of $v^{n,k},v^{n,k-1}$ and their derivatives, this result can be used through direct substitution. In other scenarios, the derivative expressions may not yield useful expressions and the behaviour of $h(v^{n,k}-v^{n,k-1})$ can be analysed directly without this result. We will provide variations of this in the subsequent example calculations.

Example Calculations

We illustrate the application of these ideas to some common distributions, such as the uniform and exponential distributions. However, the nature of the differential equations that arise from the multiple stopping problem are much harder to solve—some of them have no solution.

Example 2.

The uniform distribution is given by $f\left(y\right)=\frac{1}{b-a}$ on $y\in [a,b]$, where $b>a$.

For the differential equation for the large asymptotic behaviour, we have that $h\left(y\right)=\frac{b-y}{b-a}$. Defining $v:=v^{n,2}-v^{n,1}$ and rearranging in the asymptotic relation for ${\left(v^{n,2}\right)}^{\prime }$ we obtain

$$v^{\prime }\sim {\int }_v^b\frac{b-y}{b-a}\mathrm{d}y+{\left(v^{n,1}\right)}^{\prime }.$$

From [1], we have the asymptotic relation $v^{n,1}=v_n\sim b-\frac{2(b-a)}{n}$, which can be directly substituted into the above equation:

$$\begin{array}{c}\hfill v^{\prime }\sim \frac{{(b-v)}^2}{2(b-a)}-\frac{2(b-a)}{n^2}.\end{array}$$

We can solve this formal differential equation to obtain

$$v\sim b+(a-b)\left(\frac{1+\sqrt{5}}{n}-\frac{2c}{n(n^{\sqrt{5}}+c)}\right)\sim b+\frac{(a-b)(1+\sqrt{5})}{n}$$

where c arises as a constant of integration but may be dropped since it is part of a sub-dominant term. We now replace v with $v^{n,2}-v^{n,1}$, substitute our asymptotic relation for $v^{n,1}$, and rearrange for $v^{n,2}$ to obtain

$$v^{n,2}\sim 2b-\frac{(3+\sqrt{5})(b-a)}{n}\mathrm{as}n\to \infty .$$

We may notice, in general, that whenever v satisfies the relation

$$v^{\prime }\sim \frac{{(b-v)}^2}{2(b-a)}-\frac{\Delta (b-a)}{n^2}$$

(10)

where $\Delta$ is some positive constant, that we obtain the asymptotic relation

$$v\sim b+\frac{(a-b)(1+\sqrt{1+2\Delta })}{n}.$$

(11)

This can be used to generalise the asymptotic behaviour for $v^{n,k}$ for the uniform distribution.

Theorem 3.

Consider $X_1,X_2,\dots ,X_N$ to be independent identically distributed uniform variables on $[a,b]$, $b>a$. The reward sequence $v^{n,k}$ follows the asymptotic relation

$$v^{n,k}\sim kb-\frac{{\Delta }_k(b-a)}{n}\mathrm{as}n\to \infty ,$$

(12)

where ${\Delta }_{k+1}=1+{\Delta }_k+\sqrt{1+2{\Delta }_k}$ for $k\ge 0$ and ${\Delta }_0=0$.

Proof. 

We have shown this to be true for $k=1$ and $k=2$. Now, assume that

$$v^{n,k}\sim kb-\frac{{\Delta }_k(b-a)}{n}\mathrm{as}n\to \infty .$$

Now define $v=v^{n,k+1}-v^{n,k}$, we have

$$v^{\prime }\sim \frac{{(b-v)}^2}{2(b-a)}-\frac{{\Delta }_k(b-a)}{n^2}$$

which, from (10) and (11), has the asymptotic solution

$$v\sim b+\frac{(a-b)(1+\sqrt{1+2{\Delta }_k})}{n}.$$

To conclude the proof, we rearrange for $v^{n,k+1}$ to obtain

$$v^{n,k+1}\sim b(k+1)-\frac{(1+{\Delta }_k+\sqrt{1+2{\Delta }_k})(b-a)}{n}=b(k+1)-\frac{{\Delta }_{k+1}(b-a)}{n}.$$

□

We noted in [1] that the behaviour of $v^{n,1}$ was identical for the family of distributions with exponential tails. The next example will seek to unify such distributions that corresponded to $\alpha =1$ in the multiple stopping scenario.

Example 3.

A continuous probability density function $f\left(y\right)$ is given with a survival function that, for sufficiently large y, satisfies

$$\left|h\left(y\right)-\gamma e^{-(y/\beta )}\right|<\frac{e^{-(y/\beta )}}{y^{\Delta }}$$

(13)

for positive Δ, and where β and γ are positive constants. Assume each of the terms in the sequence of reward values $v^{n,1},\dots ,v^{n,k}$ increases without bound.

We obtain the ordinary differential equation

$$\frac{d{v}^{n,k}}{dn}\sim {\int }_{v^{n,k}-v^{n,k-1}}^{\infty }h\left(y\right)\mathrm{d}y=\gamma \beta \mathsf{\Gamma }\left(1,\frac{v^{n,k}-v^{n,k-1}}{\beta }\right)+g^{*}\left(n\right)$$

(14)

where $\mathsf{\Gamma }$ represents the upper incomplete gamma function, and

$$|g^{*}\left(n\right)|<{\int }_{v^{n,k}-v^{n,k-1}}^{\infty }\frac{e^{-y/\beta }}{y^{\Delta }}\mathrm{d}y<\frac{1}{{(v^{n,k}-v^{n,k-1})}^{\Delta }}{\int }_{v^{n,k}-v^{n,k-1}}^{\infty }{e}^{-y/\beta }\mathrm{d}y$$

which is sub-dominant in the asymptotic differential equation. The solution to the differential equation is thus approximated by

$$\frac{d{v}^{n,k}}{dn}\sim \gamma \beta \mathsf{\Gamma }\left(1,\frac{v^{n,k}-v^{n,k-1}}{\beta }\right).$$

This gives

$$\frac{d{v}^{n,k}}{dn}\sim \gamma e^{-(v^{n,k}-v^{n,k-1})/\beta }\mathrm{as}n\to \infty .$$

(15)

We have, from [1], $v^{n,1}\sim \beta log\left(n\right)$, and so the general case for $k>2$ may be presented by mathematical induction.

Theorem 4.

Let $X_1,X_2,\dots ,X_N$ be random variables from a distribution $f\left(y\right)$ that, for sufficiently large y, satisfies

$$\left|h\left(y\right)-\gamma e^{-(y/\beta )}\right|<\frac{e^{-(y/\beta )}}{y^{\Delta }}$$

(16)

for positive Δ, and where β and γ are positive constants. Assume each of the terms in the sequence of reward values $v^{n,1},\dots ,v^{n,k}$ increases without bound. Then, the asymptotic behaviour of $v^{n,k}$ is given by

$$v^{n,k}\sim \beta log\left(n^k\right)\mathrm{as}n\to \infty .$$

(17)

Proof. 

From (15) we have that the behaviour of $v^{n,k}$ is related by

$$\frac{d{v}^{n,k}}{dn}\sim \gamma e^{-v^{n,k}/\beta }{e}^{v^{n,k-1}/\beta }$$

We have verified the claim for $n=1$ in [1]. We now assume that $v^{n,k}\sim \beta log\left(n^k\right)$ as $n\to \infty$ and use this to prove the same for $v^{n,k+1}$. We write the asymptotic differential equation ${\left(v^{n,k+1}\right)}^{\prime }\sim \gamma e^{-(v^{n,k+1}-v^{n,k})/\beta }$ and substitute for our assumed asymptotic for $v^{n,k}$ to obtain

$${\left(v^{n,k+1}\right)}^{\prime }\sim \beta e^{-\left(v^{n,k+1}-\beta log\left(n^k\right)\right)/\beta }=\beta n^k{e}^{-v^{n,k+1}/\beta }$$

which is a separable differential equation with solution

$$v^{n,k+1}\sim \beta log\left(n^{k+1}\right)$$

as required. □

4. Calculating the Optimal Expectation

In this section, we continue the ideas from the single-stopping calculation to calculate the expectation for the multiple stopping rules. We proceed to find the asymptotic for the expected value ${\tau }_1^{*}$, the first stopping time. We can then find the rest of the expectations in an inductive fashion under certain conditions. We now extend some of the previous notation to reflect the multiple reward sequences, as well as k stopping variables. Let ${\tau }_1^{*},{\tau }_2^{*},\dots ,{\tau }_k^{*}$ denote the 1st, 2nd, …, kth stopping times, respectively, and let $w_{i,j}=P(y<v_{i,j}),v_{i,j}=v^{N-j,k-i+1}-v^{N-j,k-i},i=1,\dots ,k,j=1,\dots ,N$.

5. An Asymptotic Equation for $\mathbb{E}\left({\mathbf{\tau }}_{\mathbf{1}}^{*}\right)$

By recalling the notation from the beginning of this section, we obtain $\mathbb{E}\left({\tau }_1^{*}\right)$ through

$$\begin{array}{cc}\hfill \mathbb{E}\left({\tau }_1^{*}\right)& =\sum _{n=1}^{N-k+1}nP(y_1<v_{1,1},\dots ,y_{n-1}<v_{1,n-1},y_n\ge v_{1,n})\hfill \\ \hfill & =(1-w_{1,1})+2w_{1,1}(1-w_{1,2})+\cdots +N{w}_{1,1}{w}_{1,2}\cdots w_{1,N-k}\hfill \\ \hfill & =1+\sum _{n=1}^{N-k}\prod _{j=n}^{N-k}{w}_{1,N-k+1-j},\hfill \end{array}$$

We split the summation for $\mathbb{E}\left({\tau }_1^{*}\right)$ on a value $k^{*}$, where $0\ll k^{*}\ll N$:

$$\begin{array}{cc}\hfill \mathbb{E}\left({\tau }_1^{*}\right)& =1+\sum _{n=1}^{k^{*}-1}\prod _{j=n}^{N-k}{w}_{1,N-1-j}+\sum _{n=k^{*}}^{N-k}\prod _{j=n}^{N-k}{w}_{1,N-1-j}\hfill \end{array}$$

where we apply the fact that $0<w_{1,N-1-j}<1$ to obtain a bound for the first summation term:

$$0<\sum _{n=1}^{k^{*}-1}\prod _{j=n}^{N-k}{w}_{1,N-1-j}<k^{*}-1.$$

For the second summation term, as $k^{*}$ is large in the limit as $N\to \infty$, we may use the asymptotic approximations for $v^{n,k}$ obtained in the previous section.

$$\begin{array}{cc}\hfill \sum _{n=k^{*}}^{N-k}\prod _{j=n}^{N-k}{w}_{1,N-1-j}& =\sum _{n=k^{*}}^{N-k}\prod _{j=n}^{N-k}\left(1-h\left(v_{1,N-1-j}\right)\right)\sim \sum _{n=k^{*}}^{N-k}\prod _{j=n}^{N-k}\left(1+\frac{{\left(v^{j+1,k}\right)}^{″}}{{(v^{j+1,k}-v^{j+1,k-1})}^{\prime }}\right).\hfill \end{array}$$

For many distributions, this can be simplified through using the large-n asymptotic for $v^{n,k}$ and its derivatives, or obtaining an asymptotic expression for $h(v^{n,k}-v^{n,k-1})$ through other means. In the case where $h\left(v_{1,N-1-k}\right)\sim \frac{\lambda }{j}$, we have from [1] that

$$\mathbb{E}\left({\tau }_1^{*}\right)\sim \frac{N}{\lambda +1}\mathrm{as}N\to \infty .$$

Example 4.

Uniform Distribution for k stops.

For simplicity, we first consider the double stopping problem ($k=2$). From Example 2, we have that

$$v^{n,1}\sim b-\frac{2(b-a)}{n}\mathrm{and}{v}^{n,2}\sim 2b-\frac{(3+\sqrt{5})(b-a)}{n}\mathrm{as}n\to \infty .$$

This obtains $v^{n,2}-v^{n,1}\sim b-\frac{(b-a)(1+\sqrt{5})}{n}$, and thus, $h(v^{n,2}-v^{n,1})\sim \frac{1+\sqrt{5}}{n}.$

We then have that

$$\mathbb{E}\left({\tau }_1^{*}\right)\sim \sum _{n=k^{*}}^{N-2}\prod _{j=n}^{N-2}\left(1-h\left(v_{1,N-1-j}\right)\right)\sim \sum _{n=k^{*}}^{N-2}\prod _{j=n}^{N-2}\left(1-\frac{1+\sqrt{5}}{j}\right)\sim \frac{N}{2+\sqrt{5}}.$$

For the general result with $k>2$ stops for the uniform distribution, we apply the asymptotic behaviour of $h(v^{n,k}-v^{n,k-1})$ to obtain

$$\mathbb{E}\left({\tau }_1^{*}\right)\sim \sum _{n=k^{*}}^{N-k}\prod _{j=n}^{N-k}\left(1-\frac{1+\sqrt{1+2{\Delta }_{k-1}}}{j}\right)\sim \frac{N}{2+\sqrt{1+2{\Delta }_{k-1}}}.$$

For $k=1$, conveniently ${\Delta }_0=0$, this retrieves $\frac{N}{3}$. For $k=2$, we have that ${\Delta }_1=2$, and so, this retrieves $\frac{N}{2+\sqrt{5}}$, consistent with our previous results.

Example 5 (Distributions with an Exponential Tail (k stops)).

We once again consider a probability distribution $f\left(y\right)$ is given with a survival function that, for sufficiently large y, satisfies

$$\left|h\left(y\right)-\gamma e^{-(y/\beta )}\right|<\frac{e^{-(y/\beta )}}{y^{\Delta }}$$

(18)

where the additional conditions are described in Example 3.

We found that the sequences $v^{n,k}$ satisfy the asymptotic relation $v^{n,k}\sim \beta log\left(n^k\right)$ and so

$$v^{n,k}-v^{n,k-1}\sim \beta log\left(n^k\right)-\beta log\left(n^{k-1}\right)=\beta log\left(n\right).$$

From this we obtain asymptotic relations for the derivatives:

$${(v^{n,k}-v^{n,k-1})}^{″}\sim \frac{\beta }{n}\mathrm{and}{\left(v^{n,k}\right)}^{″}\sim -\frac{\beta k}{n^2}\mathrm{as}n\to \infty .$$

Hence, we obtain

$$\mathbb{E}\left({\tau }_1^{*}\right)\sim \sum _{n=k^{*}}^{N-k}\prod _{j=n}^{N-k}\left(1+\frac{{\left(v^{j+1,k}\right)}^{″}}{{(v^{j+1,k}-v^{j+1,k-1})}^{\prime }}\right)\sim \sum _{n=k^{*}}^{N-k}\prod _{j=n}^{N-k}\left(1-\frac{k}{j}\right)\sim \frac{N}{k+1}.$$

Provided that $\mathbb{E}\left({\tau }_1^{*}\right)$ is asymptotically of this form, we may make use of linearity of expectation to obtain convenient conditional formulae that are not as complicated as those encountered in the previous section.

6. An Inductive Approach for $\mathbb{E}\left({\tau }_j^{*}\right),j>\mathbf{1}$

Due to the independence of observations, it starts to make physical sense to view the expectation of the $(j+1)$th stopping time as a function of only the previous stopping time. We, thus, investigate the properties of the $(j+1)$th stopping time when conditioned on the jth. We would expect then to only need to add the additional expected number of observations to stop one more time out of a ‘reduced’ optimal stopping problem. We now introduce the notation ${\tau }_{N,j,k}^{*}$ to allow for more flexible interactions between stopping times.

Definition 1.

Let ${\tau }_{N,j,k}^{*}$ denote the jth stopping time (out of k) in the optimal stopping problem with N observations.

Here, ${\tau }_{j,N,k}^{*}$ denotes the ${\tau }_j^{*}$ used in the previous section. ${\tau }_{1,N,1}^{*}$ corresponds to the ${\tau }^{*}$ in the single-stopping problem.

Theorem 5.

Let $X_1,X_2,\dots ,X_N$ be independent and identically distributed random variables where the expectations of the optimal stopping times $\left\{{\tau }_{j,N,k}^{*}\right\}$ exist. Suppose further that the first stopping time out of k stops has asymptotic expectation $\mathbb{E}\left({\tau }_{1,N,k}^{*}\right)\sim \frac{N}{{\lambda }_{1,k}}$ for some constant ${\lambda }_{1,k}$. Then, the following relation holds:

$$\mathbb{E}\left({\tau }_{j+1,N,k}^{*}\right)\sim \mathbb{E}\left({\tau }_{j,N,k}^{*}\right)+\frac{N-\mathbb{E}\left({\tau }_{j,N,k}^{*}\right)}{{\lambda }_{1,k-j}}$$

(19)

where ${\lambda }_{1,k-j}$ is some other positive constant.

Proof. 

We first show that the conditional expectation, $\mathbb{E}\left({\tau }_{j+1,N,k}^{*}\right|{\tau }_{j,N,k}^{*}=m)$, is given by

$$\begin{array}{cc}\hfill & \mathbb{E}\left(\min \{m_{j+1}:m<m_{j+1}\le N-k+j+1,y_{m_{j+1}}\ge v^{N-m_{j+1},k-j}-v^{N-m_{j+1},k-j-1}\}\right)\hfill \\ \hfill & =m+\hfill \\ \hfill & \mathbb{E}\left(\min \{m_1:1\le m_1\le (N-m)-(k-j)+1:y_{m_1}\ge v^{N-m,k-j}-v^{N-m,(k-j)-1}\}\right)\hfill \\ \hfill & =m+\mathbb{E}\left({\tau }_{1,N-m,k-j}\right),\hfill \end{array}$$

where the last equality follows directly from the definition of ${\tau }_1^{*}$ in the previous section. By the assumption of the form of the expectation, we thus have that

$$\mathbb{E}\left({\tau }_{j+1,N,k}^{*}\right|{\tau }_{j,N,k}^{*})\sim {\tau }_{j,N,k}^{*}+\frac{N-{\tau }_{j,N,k}^{*}}{{\lambda }_{1,k-j}}$$

(20)

and so through linearity properties of expectation, as well as the law of total expectation, we finally obtain

$$\mathbb{E}\left({\tau }_{j+1,N,k}^{*}\right)\sim \mathbb{E}\left({\tau }_{j,N,k}^{*}\right)+\frac{N-\mathbb{E}\left({\tau }_{j,N,k}^{*}\right)}{{\lambda }_{1,k-j}}.$$

(21)

□

This has a reasonable physical interpretation as we would expect that the expectation of the $(j+1)$th stopping time will be the expectation of the jth stopping time as well as the additional time it would take to stop one more time in the revised stopping problem to stop at the next observation. This revised stopping problem has a reduced $N-\mathbb{E}\left({\tau }_{j,N,k}^{*}\right)$ number of observations on average, as well as $k-j$ stops remaining since we have already stopped j times out of the original k times. A consequence of this theorem is that the asymptotic expectations may all be considered linear.

7. Asymptotic Equations for $\mathbb{E}\left({\tau }_{j,N,k}^{*}\right)$

We now demonstrate how the ideas discussed in the previous section can be applied to obtain all of the multiple stopping times for some classes of distribution. Simulations were also conducted to support the asymptotic calculations in this study. Figure 1 compares the large-N asymptotic predictions for the expected duration for the standard uniform distribution under the optimal double stopping rule and Figure 2 shows a similar comparison under a triple stopping rule for the exponential and the gamma distribution. In this section, we will now adopt the previous notation for the jth stopping time out of k, for N observations as ${\tau }_{j,N,k}^{*}$.

Figure 1. Comparison of large-N asymptotic predictions for the expectation of the optimal double stopping rule for the standard uniform distribution ($a=0,b=1$).

Figure 2. Comparison of large-N asymptotic predictions for the expectation of the optimal triple stopping rule $(k=3)$ for the standard exponential distribution ($\beta =1$) and the gamma distribution. ($\alpha =3,\beta =2$). The rows correspond to the first, second, and third stops, respectively. The first column is for the exponential distribution, and the second for the gamma distribution.

These results were verified by simulation (see Figure 1) which show the realised expectation converging to the asymptotic prediction. This procedure can be extended to calculate further results for more stopping times permitted. Simulation results (see Figure 2) for the exponential and gamma distribution further support these asymptotic calculations.

Example 6 (Double Stopping on the Uniform Distribution).

For the uniform distribution, we obtain an asymptotic expression for the first stopping time out of k stops. It is given by

$$\mathbb{E}\left({\tau }_{1,N,k}^{*}\right)\sim \frac{N}{2+\sqrt{1+2{\Delta }_{k-1}}},$$

where ${\Delta }_k$ is defined through the recursive scheme

$$\left\{\begin{array}{cc}{\Delta }_{k+1}=1+{\Delta }_k+\sqrt{1+2{\Delta }_k}\hfill & k=0,1,2,3,\dots \hfill \\ {\Delta }_0=0\hfill \end{array}\right.$$

We can then apply Equation (21), for the example of the double stopping case, recursively to obtain $\mathbb{E}\left({\tau }_{1,N,2}\right)\sim \frac{1}{2+\sqrt{5}}N$ and $\mathbb{E}\left({\tau }_{2,N,2}\right)\sim \frac{2\sqrt{5}-3}{3}N$.

Example 7 (Multiple Stopping on Distributions with Exponential Tails).

We saw that the expressions can get somewhat out of hand for the uniform distribution. However, the structure can sometimes behave nicely to lead to a closed form result that need not be recursively evaluated. This is true for the class of distributions outlined in Example 3. We prove the following theorem for the general asymptotic expectation:

Theorem 6.

Let $X_1,X_2,\dots ,X_N$ be independent and identically distributed random variables. Define ${\tau }_{j,N,k}^{*}$ to be the jth stopping time out of k stops for the sequence of N observations. If $\mathbb{E}\left({\tau }_{1,N,k}^{*}\right)\sim \frac{N}{k+1}$, then we have that

$$\mathbb{E}\left({\tau }_{j,N,k}^{*}\right)\sim \frac{jN}{k+1}\mathrm{for}j=1,2\dots ,k.$$

(22)

Proof. 

We see that the base case is in agreement; we now proceed with the inductive argument that

$$\begin{array}{cc}\hfill \mathbb{E}\left({\tau }_{j+1,N,k}^{*}\right)& \sim \mathbb{E}\left({\tau }_{j,N,k}^{*}\right)+\frac{N-\mathbb{E}\left({\tau }_{j,N,k}^{*}\right)}{k-j+1}(\mathrm{from}\mathrm{Theorem}5)\hfill \\ \hfill & =\frac{jN}{k+1}+\frac{N-\frac{jN}{k+1}}{k-j+1}\hfill \\ \hfill & =\frac{(j+1)N}{k+1}.\hfill \end{array}$$

□

8. Conclusions

In this paper, we have derived asymptotics of multiple optimal stopping times for sequences of independent identically distributed continuous random variables through extending the pre-existing methodology that we developed for the single-stopping case. It is anticipated that a similar class of results can be established for other classes of distribution, but it is not clear if the resulting differential equations are easily solvable.

Asymptotic calculations were performed on a number of probability distributions, on both bounded and unbounded domains. The asymptotic properties for $k\ge 2$ were then inductively obtained. Numerical simulations were subsequently performed to calculate the expectation of the optimal stopping rule for a range of values of N, ranging from 10 to 1000. In each case, the simulated results tended towards the asymptotic prediction in the large-N limit, validating the asymptotic and inductive approach.