Appendix A. Convergence in Distribution °
Let be a metric space and be the -algebra on generated by the open balls , , . We refer to as open-ball σ-algebra. If is separable, then coincides with the Borel -algebra . If is not separable, then might be strictly smaller than and thus a continuous real-valued function on is not necessarily -measurable. Let be the set of all bounded, continuous and -measurable real-valued functions on , and be the set of all probability measures on .
Let
be an
-valued random variable on some probability space
for every
. Then, referring to
Billingsley (
1999, sct. 1.6), the sequence
is said to
converge in distribution to
if
In this case, we write
. This is the same as saying that the sequence
converges to
in the weak
topology on
; for details see Appendix A of
Beutner and Zähle (
2016). It is worth mentioning that two probability measures
coincide if
for some separable
and
for all uniformly continuous
(see, for instance, (
Billingsley 1999, Theorem 6.2)).
In Appendices A–C in
Beutner and Zähle (
2016), several properties of convergence in distribution
(and weak
convergence) have been discussed. The following two subsections complement this discussion.
Appendix A.1. Slutsky-Type Results for the Open-Ball σ-Algebra
For a sequence
of
-valued random variables that are all defined on the same probability space
, the sequence
is said to
converge in probability to
if the mappings
,
, are
-measurable and satisfy
In this case, we write
. The superscript
points to the fact that measurability of the mapping
is a requirement of the definition (and not automatically valid). Note, however, that in the specific situation where
for some
, measurability of the mapping
does hold (see Lemma B.3 in
Beutner and Zähle (
2016)). In addition, note that the measurability always hold when
is separable; in this case, we also write
instead of
.
Theorem A1. Let and be two sequences of -valued random variables on a common probability space , and assume that the mapping is -measurable for every . Let be an -valued random variable on some probability space with for some separable . Then, and together imply .
Proof. In view of
, we obtain for every fixed
Since
f lies in
and we assumed
, we also have
for every
. Thus,
which together with the Portmanteau theorem (in the form of (
Beutner and Zähle 2016, Theorem A.4)) implies the claim. ☐
Set
and let
be the
-algebra on
generated by the open balls with respect to the metric
Recall that
, where the inclusion may be strict.
Corollary A1. Let and be two sequences of -valued random variables on a common probability space . Let be an -valued random variable on some probability space with for some separable . Let . Let be a metric space equipped with the corresponding open-ball σ-algebra . Then, and together imply:
- (i)
.
- (ii)
for every continuous and -measurable .
Proof. Assertion (ii) is an immediate consequence of Assertion (i) and the Continuous Mapping theorem in the form of (
Billingsley 1999, Theorem 6.4); take into account that
takes values only in
and that
is separable with respect to
. Thus, it suffices to show Assertion (i). First note that we have
Indeed, for every
(with
the set of all bounded, continuous and
-measurable real-valued functions on
) we have
by the assumption
and the fact that the mapping
lies in
(the latter was shown in the proof of Theorem 3.1 in
Beutner and Zähle (
2016)).
Second, the distance
is
-measurable for every
, because
is
-measurable and
is
-measurable (due to Lemma B.3 in
Beutner and Zähle (
2016)). Along with
, we obtain in particular that
. Together with Equation (
A2) and Theorem A1 (applied to
,
,
), this implies
; take into account again that
takes values only in
and that
is separable with respect to
. ☐
Corollary A2. Let be a normed vector space and d be the induced metric defined by . Let and be two sequences of -valued random variables on a common probability space . Let be an -valued random variable on some probability space with for some separable . Let . Assume that the map defined by is -measurable. Then, and together imply .
Proof. The assertion is an immediate consequence of Corollary A1 and the fact that h is clearly continuous. ☐
Appendix A.2. Delta-Method and Chain Rule for Uniformly Quasi-Hadamard Differentiable Maps
Now, assume that is a subspace of a vector space . Let be a norm on and assume that the metric d is induced by . Let be another vector space and be any subspace. Let be a norm on and be the corresponding open-ball -algebra on . Let denote the null in . Moreover, let and be the -algebra on generated by the open balls with respect to the metric .
Let be a probability space and be any map for every . Recall that and refer to convergence in distribution and convergence in probability, respectively. Moreover, recall Definition A1 of quasi-Hadamard differentiability.
Theorem A2. Let be a map defined on some . Let be some -separable subset of . Let and define the singleton set . Let be a sequence of positive real numbers tending to ∞, and consider the following conditions:
- (a)
takes values only in .
- (b)
takes values only in , is -measurable and satisfiesfor some -valued random variable ξ on some probability space with . - (c)
takes values only in and is -measurable.
- (d)
The map H is uniformly quasi-Hadamard differentiable with respect to tangentially to with trace and uniform quasi-Hadamard derivative .
- (e)
for all .
- (f)
The uniform quasi-Hadamard derivative can be extended to such that the extension is continuous at every point of and -measurable.
- (g)
The map defined by is -measurable.
Then, the following two assertions hold:
- (i)
If Conditions (a)–(d) hold true, then is -measurable and - (ii)
If Conditions (a)–(g) hold true, then
Proof. (i): For every
, let
and define the map
by
Moreover, define the map
by
Now, the claim would follow by the extended Continuous Mapping theorem in the form of Theorem C.1 in
Beutner and Zähle (
2016) applied to the functions
,
, and the random variables
,
, and
if we can show that the assumptions of Theorem C.1 in
Beutner and Zähle (
2016) are satisfied. First, by Assumption (a) and the last part of Assumption (b), we have
and
. Second, by Assumption (c), we have that
is
-measurable. Third, the map
is continuous by the definition of the quasi-Hadamard derivative. Thus,
is
-measurable, because the trace
-algebra
coincides with the Borel
-algebra on
(recall that
is separable). In particular,
is
-measurable. Fourth, Condition (a) of Theorem C.1 in
Beutner and Zähle (
2016) holds by Assumption (b). Fifth, Condition (b) of Theorem C.1 in
Beutner and Zähle (
2016) is ensured by Assumption (d).
(ii): For every
, let
and
be as above and define the map
by
Moreover, define the map
by
For Equation (
A5), it suffices to show that the assumption of the extended Continuous Mapping theorem in the form of Theorem C.1 in
Beutner and Zähle (
2016) applied to the functions
and
(as defined above) are satisfied. The claim then follows by Theorem C.1 in
Beutner and Zähle (
2016). First, we have already observed that
and
. Second, we have seen in the proof of Part (i) that
is
-measurable,
. By Assumption (f), the extended map
is
-measurable, which implies that
is
-measurable. Thus,
is
-measurable (to see this note that, in view of
for the coordinate projections
on
, Theorem 7.4 of
Bauer (
2001) shows that the map
is
-measurable if and only if the maps
and
are
-measurable). In particular, the map
is
-measurable,
. Third, we have seen in the proof of Part (i) that the map
is
-measurable. Thus, the map
is
-measurable (one can argue as above) and in particular
-measurable. Fourth, Condition (a) of Theorem C.1 in
Beutner and Zähle (
2016) holds by Assumption (b). Fifth, Condition (b) of Theorem C.1 in
Beutner and Zähle (
2016) is ensured by Assumption (d) and the continuity of the extended map
at every point of
(recall Assumption (f)). Hence, Equation (
A5) holds.
By Assumption (g) and the ordinary Continuous Mapping theorem (see (
Billingsley 1999, Theorem 6.4)) applied to Equation (
A5) and the map
,
, we now have
i.e.,
The following lemma provides a chain rule for uniformly quasi-Hadamard differentiable maps (a similar chain rule with different
was found in
Varron (
2015)). To formulate the chain rule, let
be a further vector space and
be a subspace equipped with a norm
.
Lemma A1. Let and be maps defined on subsets and such that . Let and be subsets of and , respectively. Let and be sets of sequences in and , respectively, and assume that the following three assertions hold.
- (a)
For every , we have .
- (b)
H is uniformly quasi-Hadamard differentiable with respect to tangentially to with trace and uniform quasi-Hadamard derivative , and we have .
- (c)
is uniformly quasi-Hadamard differentiable with respect to tangentially to with trace and uniform quasi-Hadamard derivative .
Then, the map is uniformly quasi-Hadamard differentiable with respect to tangentially to with trace , and the uniform quasi-Hadamard derivative is given by .
Proof. Obviously, since and is associated with trace , the map can also be associated with trace .
Now, let
be a quadruple with
,
,
satisfying
as well as
, and
satisfying
. Then,
Note that by assumption,
and in particular
. By the uniform quasi-Hadamard differentiability of
H with respect to
tangentially to
with trace
,
Moreover,
and
, because
H is associated with trace
and
. Hence, by the uniform quasi-Hadamard differentiability of
with respect to
tangentially to
, we obtain
This completes the proof. ☐
Appendix B. Delta-Method for the Bootstrap
The functional delta-method is a widely used technique to derive bootstrap consistency for a sequence of plug-in estimators with respect to a map
H from bootstrap consistency of the underlying sequence of estimators. An essential limitation of the classical functional delta-method for proving bootstrap consistency in probability (or outer probability) is the condition of Hadamard differentiability on
H (see Theorem 3.9.11 of
van der Vaart Wellner (
1996)). It is commonly acknowledged that Hadamard differentiability fails for many relevant maps
H. Recently, it was demonstrated in
Beutner and Zähle (
2016) that a functional delta-method for the bootstrap
in probability can also be proved for
quasi-Hadamard differentiable maps
H. Quasi-Hadamard differentiability is a weaker notion of “differentiability” than Hadamard differentiability and can be obtained for many relevant statistical functionals
H (see, e.g.,
Beutner et al. 2012;
Beutner and Zähle 2010,
2012;
Krätschmer et al. 2013;
Krätschmer and Zähle 2017). Using the classical functional delta-method to prove almost sure (or outer almost sure) bootstrap consistency for a sequence of plug-in estimators with respect to a map
H from almost sure (or outer almost sure) bootstrap consistency of the underlying sequence of estimators requires
uniform Hadamard differentiability on
H (see Theorem 3.9.11 of
van der Vaart Wellner (
1996)). In this section, we introduce the notion of
uniform quasi-Hadamard differentiability and demonstrate that one can even obtain a functional delta-method for the
almost sure bootstrap and
uniformly quasi-Hadamard differentiable maps
H.
To explain the background and the contribution of this section more precisely, assume that we are given an estimator for a parameter in a vector space, with n denoting the sample size, and that we are actually interested in the aspect of . Here, H is any map taking values in a vector space. Then, is often a reasonable estimator for . One of the main objects in statistical inference is the distribution of the error , because the error distribution can theoretically be used to derive confidence regions for . However, in applications, the exact specification of the error distribution is often hardly possible or even impossible. A widely used way out is to derive the asymptotic error distribution, i.e., the weak limit of for suitable normalizing constants tending to infinity, and to use as an approximation for for large n. Since usually still depends on the unknown parameter , one should use the notation instead of . In particular, one actually uses as an approximation for for large n.
Not least because of the estimation of the parameter
of
, the approximation of
by
is typically only moderate. An often more efficient alternative technique to approximate
is the bootstrap. The bootstrap has been introduced by
Efron (
1979) and many variants of his method have been introduced since then. One may refer to
Davison and Hinkley (
1997);
Efron (
1994);
Lahiri (
2003);
Shao and Tu (
1995) for general accounts on this topic. The basic idea of the bootstrap is the following. Re-sampling the original sample according to a certain re-sampling mechanism (depending on the particular bootstrap method) one can sometimes construct a so-called bootstrap version
of
for which the conditional law of
“given the sample” has the same weak limit
as the law of
has. The latter is referred to as bootstrap consistency. Since
depends only on the sample and the re-sampling mechanism, one can at least numerically determine the conditional law of
“given the sample” by means of a Monte Carlo simulation based on
repetitions. The resulting law
can then be used as an approximation of
, at least for large
n.
In applications, the roles of
and
are often played by a distribution function
F and the empirical distribution function
of
n random variables that are identically distributed according to
F, respectively. Not least for this particular setting several results on bootstrap consistency for
are known (see also
Appendix B.2). The functional delta-method then ensures that bootstrap consistency also holds for
when
H is suitably differentiable at
. Technically speaking, as indicated above, one has to distinguish between two types of bootstrap consistency. First bootstrap consistency
in probability for
can be associated with
where
represents the sample,
denotes the conditional law of
given the sample
,
is the bounded Lipschitz distance, and the superscript
refers to outer probability. At this point, it is worth pointing out that we consider weak convergence (respectively, convergence in distribution) with respect to the open-ball
-algebra, in symbols
(respectively,
), as defined in (
Billingsley 1999, sct. 6) (see also
Dudley 1966,
1967;
Pollard 1984;
Shorack and Wellner 1986) and that by the Portmanteau theorem A.3 in
Beutner and Zähle (
2016) weak convergence
holds if and only if
. Second bootstrap consistency
almost surely for
means that
In
Beutner and Zähle (
2016), it has been shown that Equation (
A6) follows from the respective analogue for
when
H is suitably quasi-Hadamard differentiable at
. This extends Theorem 3.9.11 of
van der Vaart Wellner (
1996) which covers only Hadamard differentiable maps. In this section, we show that Equation (
A7) follows from the respective analogue for
when
H is suitably
uniformly quasi-Hadamard differentiable at
; the notion of uniform quasi-Hadamard differentiable is introduced in Definition A1 below. This extends Theorem 3.9.13 of
van der Vaart Wellner (
1996) which covers only Hadamard differentiable maps.
Appendix B.1. Abstract Delta-Method for the Bootstrap
Theorem A4 provides an abstract delta-method for the almost sure bootstrap. It is based on the notion of uniform quasi-Hadamard differentiability which we introduce first. This sort of differentiability extends the notion of quasi-Hadamard differentiability as introduced in
Beutner and Zähle (
2010,
2016). The latter corresponds to the differentiability concept in (i) of Definition A1 ahead with
and
as in (iii) and (v) of this definition. Let
and
be vector spaces. Let
and
be subspaces equipped with norms
and
, respectively. Let
be any map defined on some subset
.
Definition A1. Let be a subset of , and be a set of sequences in .
(i) The map H is said to be uniformly quasi-Hadamard differentiable with respect to tangentially to with trace if for all and there is some continuous map such thatholds for each quadruple , with , , satisfying as well as , and satisfying . In this case, the map is called uniform quasi-Hadamard derivative of H with respect to tangentially to . (ii) If consists of all sequences with , , and for some fixed , then we replace the phrase “ with respect to ” by “at θ” and “” by “”.
(iii) If consists only of the constant sequence , , then we skip the phrase “uniformly” and replace the phrase “ with respect to ” by “at θ” and “” by “”. In this case, we may also replace “ for all ” by “ for all ”.
(iv) If , then we skip the phrase “quasi-”.
(v) If , then we skip the phrase “with trace ”.
The conventional notion of uniform Hadamard differentiability as used in Theorem 3.9.11 of
van der Vaart Wellner (
1996) corresponds to the differentiability concept in (i) with
as in (ii),
as in (iv), and
as in (v). Proposition 1 shows that it is beneficial to refrain from insisting on
as in (iv). It was recently discussed in
Belloni et al. (
2017) that it can be also beneficial to refrain from insisting on the assumption of (ii). For
(“non-quasi” case), uniform Hadamard differentiability in the sense of Definition B.1 in
Belloni et al. (
2017) corresponds to uniform Hadamard differentiability in the sense of our Definition A1 (Parts (i) and (iv)) when
is chosen as the set of all sequences
in a compact metric space
with
for which
. In Comment B.3 of
Belloni et al. (
2017), it is illustrated by means of the quantile functional that this notion of differentiability (subject to a suitable choice of
) is strictly weaker than the notion of uniform Hadamard differentiability that was used in the classical delta-method for the almost sure bootstrap, Theorem 3.9.11 in
van der Vaart Wellner (
1996). Although this shows that the flexibility with respect to
in our Definition A1 can be beneficial, it is somehow even more important that we allow for the “quasi” case.
Of course, the smaller the family the weaker the condition of uniform quasi-Hadamard differentiability with respect to . On the other hand, if the set is too small, then Condition (e) in Theorem A4 ahead may fail. That is, for an application of the functional delta-method in the form of Theorem A4 the set should be large enough for Condition (e) to be fulfilled and small enough for being able to establish uniform quasi-Hadamard differentiability with respect to of the map H.
We now turn to the abstract delta-method. As mentioned in
Section 1, convergence in distribution will always be considered for the open-ball
-algebra. We use the terminology
convergence in distribution (symbolically
) for this sort of convergence; for details see
Appendix A and Appendices A–C of
Beutner and Zähle (
2016). In a separable metric space the notion of convergence in distribution
boils down to the conventional notion of convergence in distribution for the Borel
-algebra. In this case, we use the symbol ⇝ instead of
.
Let
be a probability space, and
be a sequence of maps
Regard
as a sample drawn from
, and
as a statistic derived from
. Somewhat unconventionally, we do not (need to) require at this point that
is measurable with respect to any
-algebra on
. Let
be another probability space and set
The probability measure
represents a random experiment that is run independently of the random sample mechanism
. In the sequel,
will frequently be regarded as a map defined on the extension
of
. Let
be any map. Since
depends on both the original sample
and the outcome
of the additional independent random experiment, we may regard
as a bootstrapped version of
. Moreover, let
be any map. As with
, we often regard
as a map defined on the extension
of
. We use
together with a scaling sequence to get weak convergence results for
. The role of
is often played by
itself (see Example A1), but sometimes also by a different map (see Example A2). Assume that
,
, and
take values only in
.
Let and be the open-ball -algebras on and with respect to the norms and , respectively. Note that coincides with the Borel -algebra on when is separable. The same is true for . Set and let be the -algebra on generated by the open balls with respect to the metric . Recall that , because any -open ball in is the product of two -open balls in .
Theorem A3 is a consequence of Theorem A2 in
Appendix A.2 as we assume that
takes values only in
. The proof of the measurability statement of Theorem A3 is given in the proof of Theorem A4. Theorem A3 is stated here because, together with Theorem A4, it implies almost sure bootstrap consistency whenever the limit
is the same in Theorem A3 and Theorem A4.
Theorem A3. Let be a sequence in and . Let be a separable subspace and assume that . Let be a sequence of positive real numbers with , and assume that the following assertions hold:
- (a)
takes values only in , is -measurable, and satisfiesfor some -valued random variable ξ on some probability space with . - (b)
takes values only in and is -measurable.
- (c)
H is uniformly quasi-Hadamard differentiable with respect to tangentially to with trace and uniform quasi-Hadamard derivative .
Then, is -measurable and Theorem A4. Let be any set of sequences in . Let be a separable subspace and assume that . Let be a sequence of positive real numbers with , and assume that the following assertions hold:
- (a)
takes values only in , is -measurable, and satisfiesfor some -valued random variable ξ on some probability space with . - (b)
takes values only in and is -measurable.
- (c)
H is uniformly quasi-Hadamard differentiable with respect to tangentially to with trace and uniform quasi-Hadamard derivative .
- (d)
The uniform quasi-Hadamard derivative can be extended from to such that the extension is -measurable and continuous at every point of .
- (e)
for -a.e. ω.
- (f)
The map defined by is -measurable.
Then, is -measurable and Remark A1. In Condition (a) of Theorem A4, it is assumed that is -measurable for . Thus, the mapping is -measurable for every fixed . That is, can be seen as an -valued random variable on for every fixed , so that assertion (A10) makes sense. By the same line of reasoning one can regard as an -valued random variable on for every fixed , so that also assertion (A11) makes sense. Remark A2. Condition (c) in Theorem A3 (respectively, Theorem A4) assumes that the trace is given by , which implies that the first part of Condition (b) in Theorem A3 (respectively, Theorem A4) is automatically satisfied.
Remark A3. Condition (f) of Theorem A4 is automatically fulfilled when is separable. Indeed, in this case we have so that every continuous map (such as ) is -measurable.
Proof. Proof. Proof of Theorem A4. First note that by the assumption imposed on
(see Assumption (a)) and Assumption (c) the map
is
-measurable. Next, note that
By Equation (
A10) in Assumption (a) and the Continuous Mapping theorem in the form of (
Billingsley 1999, Theorem 6.4) (along with
and the continuity of
), we have that
for
-a.e.
. Moreover, for every fixed
we have that
is
-measurable by Assumption (f), and for
-a.e.
we have
by Part (ii) of Theorem A2 (recall that
was assumed to take values only in
), where
refers to convergence in probability
(see
Appendix A.1) and
,
,
play the roles of
,
,
, respectively. Hence, from Corollary A2, we get that Equation (
A11) holds. ☐
Appendix B.2. Application to Plug-In Estimators of Statistical Functionals
Let
,
,
be as introduced at the beginning of
Section 3. Let
be a
-separable subspace and assume
. Moreover, let
be a map defined on a set
of distribution functions of finite (not necessarily probability) Borel measures on
, where
is any vector space. In particular,
. In the following,
,
,
, and
play the roles of
,
,
, and
, respectively. As before, we let
be a normed subspace of
equipped with the corresponding open-ball
-algebra
.
Let
be a probability space. Let
be any sequence and
be a sequence of real-valued random variables on
. Moreover, let
be the empirical distribution function of
, which will play the role of
. It is defined by
Assume that takes values only in . Let be another probability space and set . Moreover, let be any map. Assume that take values only in . Furthermore, let be any map that takes values only in . In the present setting Theorems A3 and A4 can be reformulated as follows, where we recall from Remark A3 that Condition (f) of Theorem A4 is automatically fulfilled when is separable.
Corollary A3. Let be a sequence in and . Let be a sequence of positive real numbers with , and assume that the following assertions hold:
- (a)
takes values only in and satisfiesfor some -valued random variable B on some probability space with . - (b)
takes values only in and is -measurable.
- (c)
H is uniformly quasi-Hadamard differentiable with respect to tangentially to with trace and uniform quasi-Hadamard derivative .
Then, is -measurable and Note that the measurability assumption in Condition (a) of Theorem A3 is automatically satisfied in the present setting (and is therefore omitted in Condition (a) of Corollary A3). Indeed, is easily seen to be -measurable, because coincides with the trace -algebra of .
Corollary A4. Let be any set of sequences in . Let be a sequence of positive real numbers with , and assume that the following assertions hold:
- (a)
takes values only in , is -measurable, andfor some -valued random variable B on some probability space with . - (b)
takes values only in and is -measurable.
- (c)
H is uniformly quasi-Hadamard differentiable with respect to tangentially to with trace and uniform quasi-Hadamard derivative .
- (d)
The uniform quasi-Hadamard derivative can be extended from to such that the extension is -measurable, and continuous at every point of .
- (e)
for -a.e. ω.
- (f)
The map defined by is -measurable.
Then, is -measurable and The following examples illustrate
and
. In Example A1, we have
, and in Example A2
may differ from
. Examples for uniformly quasi-Hadamard differentiable functionals
H can be found in
Section 3. In the examples in
Section 3.1 and
Section 3.3 we have
, and in the Example in
Section 3.2 we have
and
for some
.
Example A1. Let be a sequence of i.i.d. real-valued random variables on with distribution function F satisfying , and be given by Equation (A12). Let be a triangular array of nonnegative real-valued random variables on such that Setting S1. or Setting S2. of Section 2.1 is met. Define the map by . Recall that Setting S1. is nothing but Efron’s boostrap
(Efron (1979)), and that Setting S2. is in line with the Bayesian bootstrap
of Rubin (1981) if is exponentially distribution with parameter 1. In Section 5.1 in Beutner and Zähle (2016), it was proved with the help of results of Shorack and Wellner (1986) and van der Vaart Wellner (1996) that respectively Condition (a) of Corollary A3 (with ) and Condition (a) of Corollary A4 (with ) hold for and , where is an F-Brownian bridge. Here, can be chosen to be the set of all whose discontinuities are also discontinuities of F. In addition, note that, in view of , Condition (e) holds if is (any subset of) the set of all sequences of distribution functions on satisfying , , and (see, for instance, Theorem 2.1 in Zähle (2014)). Example A2. Let be a strictly stationary sequence of β-mixing random variables on with distribution function F, and be given by Equation (A12). Let be a sequence of integers such that as , and for all . Set for all . Let be a triangular array of random variables on such that are i.i.d. according to the uniform distribution on for every . Define the map by with given by Equation (8), and recall from Section 2.2 that this is the blockwise bootstrap.
Similar as in Lemma 5.3 in Beutner and Zähle (2016) it follows that , with , takes values only in and is -measurable. That is, the first part of Condition (a) of Corollary A4 holds true for . Now, assume that Assumptions A1.–A3. of Section 2.2 hold true. Then, as discussed in Example 4.4 and Section 5.2 of Beutner and Zähle (2016), it can be derived from a result in Arcones and Yu (1994) that under Assumptions A1. and A2. we have that Condition (a) of Corollary A3 holds for , , and , where is a centered Gaussian process with covariance function . Here, can be chosen to be the set of all whose discontinuities are also discontinuities of F. Moreover, Theorem A5 below shows that under the assumptions A1.–A3. the second part of Condition (a) (i.e., Equation (A14)) and Condition (e) of Corollary A4 hold for with (see also Equation (9)) and the same choice of , B, and , when is the set of all sequences with , , and . Theorem A5. In the setting of Example A2 assume that assertions A1.–A3. of Section 2.2 hold, and let be the set of all sequences with , , and . Then, the second part of assertion (a) (i.e., Equation (A14)) and assertion (e) in Corollary A4 hold. Proof. Proof of second part of (a): It is enough to show that under assumptions A1.–A3. the Assumptions (A1)–(A4) of Theorem 1 in
Bühlmann (
1995) hold when the class of functions is
. Here,
and
with
for
and
for
. Due to A2. and A3. we only have to verify Assumptions (A3) and (A4) of Theorem 1 in
Bühlmann (
1995). That is, we show that the following two assertions hold.
- (1)
There exist constants such that for all .
- (2)
for the envelope function .
Here, the bracketing number is the minimal number of -brackets with respect to (-norm with respect to ) to cover , where an -bracket with respect to is the set, , of all functions f with for some Borel measurable functions with pointwise and .
(1) We only show that (1) with
replaced by
holds true. Analogously, one can show that the same holds true for
(and therefore for
). On the one hand, since
by Assumption (a), we can find for every
a finite partition
such that
and
. On the other hand, using integration by parts we obtain
so that we can find a finite partition
such that
and
.
Now, let
be the partition consisting of all points
and
, and set
Then,
. Moreover,
where we used Minkovski’s inequality and Equation (
A15), and that
is non-increasing on
and
. Since
F is at least
on
, we have
due to Equation (
A16). Thus,
, so that
provides an
-bracket with respect to
. It is moreover obvious that the
-brackets
,
, cover
. Thus,
for a suitable constant
and all
.
(2) The envelope function is given by for and by (recall that is continuous) for . Then, under Assumption (a) the integrability condition 2) holds.
Proof of (e): We have to show that
-a.s. We only show that
because the analogue for the positive real line can be shown in the same way. Let
and
be as defined in Equation (
A17). By assumption A1. we have
, so that similar as above we can find a finite partition
such that
,
, are
-brackets with respect to
(
-norm with respect to
F) covering the class
introduced above. We proceed in two steps.
Step 1. First we show that
holds true for every
. Since
, for Equation (
A19) it suffices to show
To prove Equation (
A20), we note that for every
there is some
such that
(see Step 1). Therefore, since
is an
-bracket with respect to
,
That is, Equation (
A19) holds true.
Step 2. Because of Equation (
A19), for Equation (
A18) to be true, it suffices to show that
for every
. We only show the second convergence in Equation (
A21), the first convergence can be shown even easier. We have
The first summand on the right-hand side of
converges
-a.s. to 0 by Theorem 1 (ii) (and Application 5, p. 924) in
Rio (
1995) and our assumption A1. The second summand converges
-a.s. to 0 too, which can be seen as follows. From Equation (
9), we obtain for
n sufficiently large
so that for
n sufficiently large
We have seen above that converges -a.s. to the constant . Since converges to ∞ at a slower rate than n (by assumption A3.), it follows that converges -a.s. to 0. Using the same arguments we obtain that converges -a.s. to 0. Hence, converges -a.s. to 0. Analogously, one can show that converges -a.s. to 0. ☐