Estimation of an Entropy-based Functional

Abstract

Given a function f from $[0,1]$ to the real line, we consider the (nonlinear) functional $\mathtt{\text{h}}$ obtained by evaluating the continuous entropy of the “density function” of f. Motivated by an application in signal processing, we wish to estimate $\mathtt{\text{h}}(f)$. Our main tool is a decomposition of $\mathtt{\text{h}}$ into two terms, which each have favorable scaling properties. We show that, if functions f and g satisfy a regularity condition, then the smallness of ${\parallel f-g\parallel }_{\infty }$ and $\parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }$, along with some basic control on derivatives of f and g, is sufficient to imply that $\mathtt{\text{h}}(f)$ and $\mathtt{\text{h}}(g)$ are close.

1. Introduction

We define the continuous entropy of a probability density ρ to be

$$\mathtt{\text{h}}(\rho )=-{\int }_{u\in \mathbb{R}}\rho (u)\log \left[\rho (u)\right]du$$

(where the base of the logarithm is $e=2.71828...$). Stripped of its interpretation, continuous entropy is just a particular way of measuring how “spread out” a probability density is. In this paper, we are motivated by a novel application of this measurement in signal processing.

An ultrasound probe generates a short acoustic pulse which travels through the medium of interest (in medical applications, this medium is the tissue of the patient). As the pulse travels through the medium, features within the medium cause some of the pulse to be reflected back towards the probe, and the strength of the reflection contains information about the feature at that location. The signal of interest is the intensity of this reflected pulse that arrives back at the probe over time. This signal can be divided into many short windows of time; after re-scaling the time axis, one of these time windows can be represented by the interval $[0,1]$, and the signal over that time window can be represented by a real-valued function f on the interval $[0,1]$. So, $f(t)$ represents the intensity of the reflected pulse that arrives back at the probe at the time $t\in [0,1]$.

The ultrasound probe can only measure $f(t)$ at a discrete set of values of t, and this measurement can be corrupted in various ways. Therefore, one step in the processing of an ultrasound signal is to reconstruct f from the measurements of the ultrasound probe. Then, a functional is applied to f to obtain a single number, the idea being that this number contains all the relevant information about the reflected signal over that particular window in time. A standard functional in the industry is the “energy” in the signal, ${\int }_0^1{|f|}^2$, or (more often) its logarithm. However, there is a series of papers by Hughes and others ([1,2,3,4,5,6,7]) demonstrating the utility of using the continuous entropy of the “density function” of f as the functional, instead of the log-energy.

Suppose that f is a real-valued function on $[0,1]$. By “density function” of f, we mean the following: Define the measure ${\mu }_f$ on $\mathbb{R}$ by ${\mu }_f(E)=|f^{-1}(E)|$, where $|A|$ denotes the Lebesgue measure of the set $A\subset [0,1]$. Suppose that ${\mu }_f$ is given by the density ${\rho }_f$, i.e.,

$${\mu }_f(E)={\int }_E{\rho }_f$$

This density ${\rho }_f$ is the “density function” of f. We can think of $[0,1]$ as a probability space and f as a function on that probability space (i.e., a random variable) and then ${\rho }_f$ is literally the density of the “random variable” f.

Another way to think of ${\rho }_f$ is as follows: Suppose that we consider the uniform distribution $U(0,1)$. Suppose we choose a point t at random from $[0,1]$ according to $U(0,1)$, and we calculate $f(t)$. The density ${\rho }_f$ gives the relative frequency of different outcomes of this experiment. Also, intuitively we can say that

$${\rho }_f(u)=\sum _{\{t:f(t)=u\}}\frac{1}{|f^{\prime }(t)|}$$

(although we will not attempt to formalize this definition for general f). Let us abuse notation and write

$$\mathtt{\text{h}}(f)=\mathtt{\text{h}}({\rho }_f)$$

The functional of interest is $f\to \mathtt{\text{h}}(f)$.

The effectiveness of using the $\mathtt{\text{h}}$ functional has been shown in several different settings in ultrasound signal processing, ranging from defect detection in plexiglass [1] to detection of contrast agents in the tissue of live animals ([5,7]). In some situations, the $\mathtt{\text{h}}$ functional produced images of objects which, because the material was engineered by the experimenter, were known to be present but were not detected by the log-energy functional.

The interpretation of what is being measured by $\mathtt{\text{h}}(f)$ in this context is not yet clear. However, given the utility of the technique, we believe the area merits investigation. We would like to answer the question

$$“\text{How}\text{well}\text{can}\text{we}\text{estimate}\mathtt{\text{h}}(f)?”$$

In a real-world setting we will perhaps receive some samples of f corrupted by noise, and we would like to estimate $\mathtt{\text{h}}(f)$ (as we described, the processing of an ultrasound signal is an example of this). Therefore, we seek a solution to the following problem, where the “regularity conditions” and the form of the “partial information” is not specified:

Estimation Problem 1. 

Let f be a function on $[0,1]$ that satisfies some regularity conditions, and suppose we are given some “partial information” on f, such as samples corrupted by noise. We wish to estimate $\mathtt{\text{h}}(f)$ with an estimator $\widehat{\mathtt{\text{h}}}$, and give a quantitative bound on the error $|\mathtt{\text{h}}(f)-\widehat{\mathtt{\text{h}}}|$, such that the bound is invariant under the scaling $f\to \lambda f$ for $\lambda \ne 0$.

We seek this scale invariance because, if $\widehat{\mathtt{\text{h}}}$ is any reasonable estimator of $\mathtt{\text{h}}(f)$, the true difference $\mathtt{\text{h}}(f)-\widehat{\mathtt{\text{h}}}$ does not change under the scaling $f\to \lambda f$ for $\lambda \ne 0$, so we do not want our bound on that difference to change under this scaling either. (This scaling issue is discussed more in section 5.).

Our main result will not give a specific prescription for solving this problem. Instead, we reduce it to a much more tractable problem, namely the problem of taking noisy samples from f and producing an approximating function g which is “close” to f in some quantifiable way.

However, our concern at the moment is to relate problem 1 to the existing literature, and we notice that problem 1 is similar to the standard entropy estimation problem (for an overview of this problem, see Beirlant et al. [8]):

Estimation Problem 2 (Entropy Estimation).

Let X be a random variable with unknown density function ρ. We make N independent observations of X. From these, we wish to estimate the continuous entropy of ρ.

There is a large literature concerning the entropy estimation problem, so one might hope to solve problem 1 by applying some of the standard methods in the field to the density ${\rho }_f$. However, this is not what we will do. Since we are departing from traditional methods, we would like to provide some indication that new methods may in fact be necessary; before working to develop new methods to solve a problem, one would like to know whether the problem is obviously solved with readily available standard methods. In section 2., we review the current literature with regard to how it applies to densities such as ${\rho }_f$. This review is not comprehensive; the purpose is only to show that the density ${\rho }_f$ presents some difficulty for current methods of estimation of $\mathtt{\text{h}}(\rho )$.

Some of the difficulties that we find, stated in general terms, are the following. First, the methods we have seen often involve intermediate parameters that truncate the tails of integrals, truncate the unbounded pieces of ρ, or perform other types of operations that enable a splitting into “good” and “bad” regions. In view of the scale-invariance that we seek, these intermediate parameters will all become involved and their rates of growth will need to be related in a way that produces the desired result and remains scale-invariant. This task, for a density such as ${\rho }_f$, seems to be non-trivial. Second, many of the results which might apply to ${\rho }_f$ use methods which are non-quantitative: theorems from measure theory such as Lebesgue Differentiation, the Lebesgue Dominated Convergence Theorem, Egoroff’s Theorem, and the Borel-Cantelli Lemma. Replacing these theorems with quantitative estimates will introduce more difficulties. Finally, and perhaps most importantly from an intuitive point of view, there seems to be a general progression from “easy” to “hard” as the assumptions on ρ become less stringent, moving from differentiable to continuous to bounded to $L^2$, and so on. The density ${\rho }_f$ seems to fall on the “hard” end of this continuum. This does not correspond to the progression of difficulty that a practitioner of signal processing would expect to see when they attempt to solve problem 1. Even an extremely well behaved function (such as $f(t)=\sin (2\pi t)$) will produce a density ${\rho }_f$ that will be ranked as “badly behaved.” A signal processor would expect that the problem would be “easy” for well behaved functions f, and get harder as the function f becomes more badly behaved.

In short, the readily available methods that we are aware of do not seem appropriate for the problem we are presented with. Fortunately, we believe that estimating $\mathtt{\text{h}}(f)$ is easier than solving the entropy estimation problem for ${\rho }_f$, for the simple reason that we are able to take advantage of the “good behavior” of f in the time domain, instead of only having access to the “bad behavior” of the density ${\rho }_f$. For example, suppose that in a real-world setting we receive sampled values of f at specific values of t, say $\{t_j\}$ (not to be confused with the values of t where f actually has critical points, which later will be referred to as $\{t_i\}$). Neglecting noise (purely for the simplicity of this explanation), we receive the data $\{(t_j,f(t_j))\}$. We intend to use all the information available to us, i.e. the time series $\{(t_j,f(t_j))\}$, not just the (unordered) values $\{f(t_j)\}$. For instance, we may want to estimate $f^{\prime }$ or $f^{″}$, which will certainly make use of the indicated time series. We will not attempt to apply a standard entropy estimation technique directly to the values $\{f(t_j)\}$.

One strategy would be to first construct an approximating function g (using the observations $\{(t_j,f(t_j))\}$), and then calculate $\mathtt{\text{h}}(g)$ and use this as our estimate of $\mathtt{\text{h}}(f)$. In Hughes et al. [7], this method is applied to real-time imaging calculations in a laboratory setting; instead of using just the values $\{f(t_j)\}$ directly, they first construct an approximating function g and then calculate the desired entropy value using g. In [7], they are estimating a variant of the Rényi entropy of ${\rho }_f$, not $\mathtt{\text{h}}(f)$ as we have defined it, but the same authors have demonstrated the utility of $\mathtt{\text{h}}(f)$. Incidentally, the quantity that is eventually computed in [7] is

$$\sum _i\frac{1}{|g^{″}(s_i)|}$$

where the $\{s_i\}$ are the critical points of g, and the authors comment [7] that “while this involves use of the second derivative of $f(t)$ at its critical points, which can be expected to increase noise in the processing chain output, surprisingly the resulting signal processing scheme does not sacrifice sensitivity.” This gives some assurance that our result, which requires control on multiple derivatives of f, is not without practical application.

Our goal in this paper is to estimate the difference $\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)$ in terms of quantities involving f and g (not ${\rho }_f$ or ${\rho }_g$). As we mentioned, this reduces the problem at hand, problem 1, to a much more tractable question of function approximation given noisy data. We note, for completeness, that a specific prescription for solving problem 1, using methods similar to our methods here, was given in Maurizi [9], but we believe that the approach we present now will be clearer than the approach given in [9].

As mentioned, in Section 2., we review the current literature with regard to how it applies to densities such as ${\rho }_f$. After some definitions and notation in Section 3., we will highlight a useful identity in Section 4. which may be of independent interest, and we discuss some issues regarding the scaling $f\to \lambda f$ for $\lambda \ne 0$ in Section 5.. In Section 6. we present a proposition showing why the regularity assumption (Definition 6) we introduce is necessary at least in some form, if not the specific form given here. Our regularity assumption on f and g, defined precisely in Section 7, essentially prevents f and g from becoming “too flat” by preventing their first and second derivatives from simultaneously vanishing, this is quantified with a parameter δ.

Section 7. contains our main result, Theorem 1, which states that if f and g satisfy the regularity assumption with parameter δ, then the difference $\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)$ can be bounded by the metrics ${\parallel f-g\parallel }_{\infty },{\parallel f^{\prime }-g^{\prime }\parallel }_{\infty }$, and the value δ, along with factors that control the overall sizes of f and g and their derivatives.

The proof of the main result is outlined in Section 8. and carried out in the subsequent sections.

2. Background

We now look to the current literature, seeking a method to estimate $\mathtt{\text{h}}(f)$ with quantitative bounds on the error, in a manner that is invariant under the scaling $f\to \lambda f$ for $\lambda \ne 0$.

The first sign that new methods may be necessary is that, from the viewpoint of entropy estimation, the densities ${\rho }_f$ are very badly behaved. A non-degenerate critical point in f will induce an asymptote of the form $1/\sqrt{x}$ in ${\rho }_f$, so we are in general dealing with density functions that are unbounded, not $L^2$, and have discontinuities in the interior of their support. These properties, and other features of the density ${\rho }_f$, are discussed in [6,7].

Many of the standard results in entropy estimation do not apply to such badly behaved densities. Several theorems in entropy estimation (such as Goria et al. [10], van Es [11], Joe [12], Levit [13], and more recently Leonenko et al. [14]) use the assumption that ρ is bounded. In Tsybakov and van der Meulen [15] and Eggermont and LaRiccia [16], the assumption that ρ is twice differentiable was needed. In Dmitriev and Tarasenko [17] and Ahmad and Lin [18], ρ was assumed to have a bounded derivative. In Hall and Morton [19] ρ was assumed to have a continuous derivative. In Mokkadem [20], a “distributional” equivalent of ${\rho }^{″}\in L^1$ was required.

There are methods of solving problem 2 that could shed light on densities such as ${\rho }_f$. In Kozachenko and Leonenko [21], Vasicek [22], and Györfi and van der Meulen [23], methods are developed which solve the entropy estimation problem 2 defined above and impose only mild restrictions on ρ.

In Kozachenko and Leonenko [21], they consider densities on ${\mathbb{R}}^m$, for simplicity we will consider the case $m=1$. Note that they use different notation; we will continue to use ρ to stand for the density function, whereas they use f. A nearest-neighbor estimator is used, and only fairly mild conditions are imposed on the density ρ: For some $ϵ>0$, the following two equations must hold:

$$\begin{array}{cc}& {\int |\log \rho (x)|}^{1+ϵ}\rho (x)dx<\infty \hfill \end{array}$$

(1)

$$\begin{array}{cc}& \int |\log |x-{y||}^{1+ϵ}\rho (x)\rho (y)dxdy<\infty \hfill \end{array}$$

(2)

(these are equations numbered (3) and (4) in [21]). Densities such as our ${\rho }_f$ will “typically” satisfy these constraints (for instance, the function $f(t)=t^2$ on $[0,1]$ has density ${\rho }_f(u)=(1/2)u^{-1/2},u\in (0,1]$ and this density satisfies these constraints). The authors then prove that the estimator ${\mathtt{\text{h}}}_N$ computed from N independent samples of ρ satisfies $\mathbb{E}({\mathtt{\text{h}}}_N)\to \mathtt{\text{h}}(\rho )$ as $N\to \infty$. We would like to discuss some features of their proof, so for convenience of our reader we attempt to sketch it here: They first show that

$${\mathtt{\text{h}}}_N=(1/N)\sum _{i=1}^N{\zeta }_i$$

Due to their dependence on N, the ${\zeta }_i$ could also be written ${\zeta }_{i,N}$. The ${\zeta }_i$ are identically distributed random variables, so $\mathbb{E}({\mathtt{\text{h}}}_N)=\mathbb{E}({\zeta }_i)$ and we focus just on ${\zeta }_i$. They consider the cumulative distribution function $F_{N,x}(u)$ of $e^{{\zeta }_i}$ conditioned on the case when the i-th sample equals x, i.e., $X_i=x$. Let $\nu (y,r)=\{x\in \mathbb{R}:|x-y|<r\}$; one computes that

$$F_{N,x}(u)=1-{\left(1-{\int }_{\nu (x,\frac{u}{2\gamma (N-1)})}\rho (y)dy\right)}^{N-1}$$

where $\log \gamma =.5772\dots$ is the Euler constant (this is their equation (8) ). We have

$$\left|\nu \left(x,\frac{u}{2\gamma (N-1)}\right)\right|=\frac{u}{\gamma (N-1)}$$

so we can write

$${\int }_{\nu (x,\frac{u}{2\gamma (N-1)})}\rho (y)dy=\left(\frac{u}{\gamma (N-1)}\right)\frac{1}{|\nu (x,\frac{u}{2\gamma (N-1)})|}{\int }_{\nu (x,\frac{u}{2\gamma (N-1)})}\rho (y)dy$$

This means that, by the Lebesgue Differentiation Theorem, since $\nu (x,\frac{u}{2\gamma (N-1)})$ shrinks to x as $N\to \infty$, we have

$$\frac{1}{|\nu (x,\frac{u}{2\gamma (N-1)})|}{\int }_{\nu (x,\frac{u}{2\gamma (N-1)})}\rho (y)dy⟶\rho (x)$$

and so $F_{N,x}(u)\to 1-e^{-\rho (x)u/\gamma }$ (this is their equation (8) ). They define

$$F_x(u)=1-e^{-\rho (x)u/\gamma }$$

Let the random variable ${\xi }_{N,x}$ have the cumulative distribution function $F_{N,x}$ and the random variable ${\xi }_x$ have the cumulative distribution function $F_x$. We can compute

$$\mathbb{E}(\log {\xi }_x)=-\log \rho (x)$$

and so one might hope that, since $F_{N,x}\to F_x$, we have $\mathrm{E}(\log {\xi }_{N,x})\to \mathbb{E}(\log {\xi }_x)=-\log \rho (x)\$$ because that would mean that $\mathbb{E}({\zeta }_i^{(N)}|X_i=x)\to -\log \rho (x)$ (this is their equation (9) ). The bulk of their proof is to show that this is in fact true. The proof is then completed by taking the pointwise result

$$\mathbb{E}({\zeta }_i^{(N)}|X_i=x)\to -\log \rho (x)$$

and extending it to a convergence of integrals,

$$\mathbb{E}{\mathtt{\text{h}}}_N=\mathbb{E}({\zeta }_i^{(N)})=\int \mathbb{E}({\zeta }_i^{(N)}|X_i=x)\rho (x)dx\to \int (-\log \rho (x))\rho (x)dx=\mathtt{\text{h}}(\rho )\$$$

This is done in their equation (21).

There are several rates of convergence that would have to be quantified: First, a quantitative bound on

$$\left|\frac{1}{|\nu (x,\frac{u}{2\gamma (N-1)})|}{\int }_{\nu (x,\frac{u}{2\gamma (N-1)})}\rho (y)dy-\rho (x)\right|$$

would be needed. Next, we would need to quantify how the rate of convergence of $F_{N,x}\to F_x$ translates into the convergence $\mathbb{E}(\log {\xi }_{N,x})\to \mathbb{E}(\log {\xi }_x)$. This rate would need to be somehow uniform in x; their constants $C_1$ (from their equation (20) ) and therefore $C_2$ (at the bottom of [21, p. 99]) depend on x in an unspecified way. Some sort of uniformity is needed because the convergence $\mathbb{E}(\log {\xi }_{N,x})\to \mathbb{E}(\log {\xi }_x)$ must be translated into the convergence

$$\int \mathbb{E}(\log {\xi }_{N,x})\rho (x)dx\to \int \mathbb{E}(\log {\xi }_x)\rho (x)dx$$

Quantifying these various rates of convergence appears to be nontrivial. We also note that subsequent research which has worked with estimators analogous to the one used in [21], such as [15], [10], and [14], has required more assumptions on the density in question.

In Györfi and van der Meulen [23], histogram-based density estimators ${\rho }_N$ (computed from the sampled values $\{X_i\}$) are used. The stated results make no assumptions on ρ other than the finiteness of $\mathtt{\text{h}}(\rho )$. They do not give a quantitative rate of convergence, and non-quantitative methods in the proof (such as the Borel-Cantelli Lemma) suggest that a quantitative result would not necessarily follow trivially from their methods. Also, choosing the proper bin size and grid placement for a histogram is a continual problem. For unbounded densities such as ${\rho }_f$, any bin size which can capture the very tight grouping of sample points near an asymptote of ${\rho }_f$ will then be too narrow and cause a “choppy” estimate in areas where ${\rho }_f$ is not particularly large. One way of addressing this general drawback of histograms has been to adjust the “coarseness” of the estimate depending on how tightly the sample points are grouped, such as nearest-neighbor or sample-spacing-type estimators. We could also accept inefficiency near the critical points, but since we seek quantitative bounds this is not desirable. In practice, we believe that estimating the entropy of a density such as ${\rho }_f$ with a histogram estimator will mean very large inefficiencies near the critical points.

In Vasicek [22], no assumptions are made on ρ other than $\int u^2\rho (u)<\infty$, which is satisfied by ${\rho }_f$. Let $\{X_{(i)}\}$ be the order statistics of the sample $\{X_i\}$, and let F be the cumulative distribution function of ρ. They use a sample-spacing estimator with a parameter m specifying the number of “neighbors” that will be considered when estimating the value of the density ρ near $X_{(i)}$. Their conclusion is that their estimator ${\mathtt{\text{h}}}_{m,N}^{\prime }$ converges in probability to $\mathtt{\text{h}}(\rho )$ as $N\to \infty$, as long as $m\to \infty ,m/N\to 0$. One key question, of course, is how m is chosen. They state [22] that “an optimal choice of m for a given [N], however, depends on the (unknown) [ρ]. In general, the smoother the density [ρ], the larger is the optimal value of m.” Therefore, one would expect that we would need to choose relatively small values of m since ${\rho }_f$ is very far from smooth, while still having $m\to \infty$. A key step in the proof is that

$$\frac{F(X_{(i+m)})-F(X_{(i-m)})}{X_{(i+m)}-X_{(i-m)}}\left(\text{which}\text{is}\text{equal}\text{to}\frac{1}{X_{(i+m)}-X_{(i-m)}}{\int }_{X_{(i-m)}}^{X_{(i+m)}}\rho \right)$$

will equal $\rho (x^{\prime })$ for some $x^{\prime }\in (X_{(i-m)},X_{(i+m)})$, whenever ρ is positive and continuous over the interval $(X_{(i-m)},X_{(i+m)})$. This step is essentially using the Lebesgue Differentiation Theorem, so the accuracy of

$$\frac{1}{X_{(i+m)}-X_{(i-m)}}{\int }_{X_{(i-m)}}^{X_{(i+m)}}\approx \rho (x^{\prime })$$

simultaneously over the entire domain of ρ, will need to be quantified. The tradeoff between needing m to be small in order to capture the “fine-scale” behavior of ${\rho }_f$ near an asymptote, while needing m to be large in order to carry through the rest of the proof, appears non-trivial. A discussion of results which use a fixed m, versus $m\to \infty$ as in [22], is in Tsybakov and van der Meulen [15]. We note also that results using similar techniques, which have achieved conclusions stronger than the “convergence in probability” shown here (such as van Es [11]) have required more restrictions on ρ.

There are other results, such as those in Godavarti and Hero [24], Csiszár [25], and Rényi [26], which do not specifically solve problem 2, but which give insight into how one might produce a value that is “close” to $\mathtt{\text{h}}(\rho )$ given only some information about ρ. These results are not “stochastic”; the approximations for $\mathtt{\text{h}}(\rho )$ that will be considered are not constructed by sampling from ρ.

In Csiszár [25], the only assumption is that the entropy integral exists, there are no other assumptions on the density ρ. Their results are explained in terms of a general measurable space X, and in their presentation the density ρ is not the central object, it simply arises as the Radon-Nikodym derivative of a probability measure on X (in their notation this probability measure is μ), with respect to a σ-finite measure on X (in their notation this σ-finite measure is λ). We will assume that the measurable space is $\mathbb{R}$, the σ-finite measure is Lebesgue measure on $\mathbb{R}$, and the probability measure is given by the density ρ. In their Theorem 1, they show that $\mathtt{\text{h}}(\rho )$ equals the infimum value of a set of “approximate entropies,” where the density ρ is replaced with its average value on each set in a partition $\{A_i\}$ of $\mathbb{R}$, and then the entropy of the resulting “approximate density” is calculated. If we define the characteristic function of a set S, ${\chi }_S$, by

$${\chi }_S(u)=\left\{\begin{array}{cc}1\hfill & \text{if}u\in S\hfill \\ 0\hfill & \text{if}u\notin S\hfill \end{array}\right.$$

then the “approximate density” is:

$${\rho }_{\{A_i\}}(u)=\sum _i\frac{1}{|A_i|}\left({\int }_{A_i}\rho \right){\chi }_{A_i}(u)(\text{this}\text{is}\approx \rho )$$

and the “approximate entropy” is:

$$\begin{array}{cc}\hfill \mathtt{\text{h}}({\rho }_{\{A_i\}})& =-\int {\rho }_{\{A_i\}}\log {\rho }_{\{A_i\}}\hfill \\ & =-\sum _i\left({\int }_{A_i}\rho \right)\log \left(\frac{1}{|A_i|}{\int }_{A_i}\rho \right)(\text{this}\text{is}\approx \mathtt{\text{h}}(\rho ))\hfill \end{array}$$

They prove this result by choosing a specific set of approximate entropy values ${\mathtt{\text{h}}}_{ϵ}$, where in fact ${\mathtt{\text{h}}}_{ϵ}$ is easily seen to be within ϵ of the true value $\mathtt{\text{h}}(\rho )$. This is certainly quantitative in the sense that the approximating value, ${\mathtt{\text{h}}}_{ϵ}$, is known to be within a certain explicit amount of the true value. The question then becomes whether we can calculate the value ${\mathtt{\text{h}}}_{ϵ}$. The approximating values ${\mathtt{\text{h}}}_{ϵ}$ are obtained, essentially, by the definition of the Lebesgue integral of $\log \rho$ (with respect to the measure given by ρ) as a limit of integrals of simple functions (for this definition, see for example [27]. For a probability measure μ on $\mathbb{R}$ and a function $g\in L^1(\mu )$, if we choose ϵ and we let

$$A_i=\{x\in \mathbb{R}:g(x)\in [ϵi,ϵ(i+1))\},i\in \mathbb{Z}$$

then it is obvious that ${\sum }_i(ϵi)\mu (A_i)$ is within ϵ of $\int gd\mu$. In [25], $\log \rho$ plays the role of g, the measure μ is given by ρ, and we have

$${\mathtt{\text{h}}}_{ϵ}=\sum _i(ϵi)\mu (A_i)\text{is}\text{within}ϵ\text{of}\int \log \rho d\mu ={\int }_{\mathbb{R}}\rho \log \rho$$

In order to calculate the value of ${\sum }_i(ϵi)\mu (A_i)$, we need to calculate $\mu (A_i)$, which is

$$\mathbb{P}[\log \rho \in [ϵi,ϵ(i+1))]={\int }_{\{\log \rho \in [ϵi,ϵ(i+1))\}}\rho$$

However, we do not know ρ, so we do not know the exact locations on $\mathbb{R}$ when $\log \rho$ is going to fall in a certain range. Of course, as we mentioned, the purpose of the ${\mathtt{\text{h}}}_{ϵ}$ construction is an existence proof, not necessarily as a calculational or estimation tool.

We turn now to Rényi [26]. Rényi considers density functions ρ, the only requirement on ρ being that the discrete entropy of the “integer compression” of ρ is finite, by which we mean (suppose X is a random variable having density ρ):

$$-\sum _{j\in \mathbb{Z}}\mathbb{P}[X\in [j,j+1]]\log \mathbb{P}[X\in [j,j+1]<\infty$$

(3)

This is not equivalent to the existence of the entropy integral, as he shows in [26]. This admits a wide class of densities, including ${\rho }_f$ for the functions f we will consider. The key tool will be to consider successively finer step-function approximations of ρ, the partition set being the grid with “step size” $1/N$. Let us define ${\rho }_{[1/N]}$ by

$${\rho }_{[1/N]}=\sum _{j\in \mathbb{Z}}{\chi }_{[j/N,(j+1)/N)}\left(\frac{1}{1/N}{\int }_{j/N}^{(j+1)/N}\rho \right)$$

Note that ${\rho }_{[1/N]}$ is essentially the “true” histogram with the grid ${\{j/N\}}_{j=-\infty }^{\infty }$ (an example of the approximations ${\rho }_{\{A_i\}}$ considered in Csiszár [25]). Rényi proves that, if the “integer compression” of ρ has finite discrete entropy, then

$$\underset{N\to \infty }{\lim }\mathtt{\text{h}}({\rho }_{[1/N]})=-\int \rho \log \rho$$

One sign that this result will not be optimal for a density such as ${\rho }_f$ is that Rényi first proves this convergence for bounded ρ. He introduces an intermediate parameter L (which specifies the point at which the tails of the sums and integrals will be cut off), proves the pointwise convergence result

$${\rho }_{[1/N]}(x)\log {\rho }_{[1/N]}(x)\to \rho (x)\log \rho (x)\text{a.e.}x$$

(see his equation (35) ) and then applies the “Theorem of Lebesgue” [26] (we believe he is referring to the Dominated Convergence Theorem) over the interval $[-L,L]$ to obtain

$${\int }_{-L}^L{\rho }_{[1/N]}\log {\rho }_{[1/N]}\to {\int }_{-L}^L\rho \log \rho \text{as}N\to \infty$$

The tails of the integral are bounded using our numbered equation (3), which completes the proof in the case of bounded ρ. For unbounded ρ he introduces another intermediate parameter A and considers the truncated function ${\rho }_A$ (which is bounded):

$${\rho }_A(u)=\left\{\begin{array}{cc}\rho (u)\hfill & \text{if}\rho (u)\le A\hfill \\ 0\hfill & \text{if}\rho (u)>A\hfill \end{array}\right.$$

He applies the result for bounded ρ to ${\rho }_A$, and then he bounds the difference

$$\int {\rho }_{[1/N]}\log {\rho }_{[1/N]}-\int {({\rho }_A)}_{[1/N]}\log {({\rho }_A)}_{[1/N]}$$

using expressions involving A and L (but, importantly, not involving N, see his equations (48), (50) and (53) ). Therefore, his equations (56), (57), (58) and (59) show how the parameters L, A and N interact to produce the result. To make this result quantitative would entail balancing these various rates of convergence, which seems to be non-trivial. Finally, being essentially a histogram-type method (the bin sizes depend only on N), it would face the difficulties that come along with histograms that we covered in our discussion of Györfi and van der Meulen [23].

We turn now to Godavarti and Hero [24]. In Theorem 4 of [24], the convergence $\mathtt{\text{h}}({\rho }_N)\to \mathtt{\text{h}}(\rho )$ is proved under fairly general conditions. The only assumptions are that ${\rho }_N\to \rho$ pointwise and that there exist a constant L and some $\kappa >1$ such that the following statements are true:

$${\int \rho |\log \rho |}^{\kappa }\le L$$

(4)

and for all N,

$$\int {\rho }_N{|\log {\rho }_N|}^{\kappa }\le L$$

(5)

Densities such as ${\rho }_f$ typically satisfy equation (4). Furthermore, if we require f to satisfy the regularity assumption we introduce later (Definition 6), and if an approximation method is used to produce $g_N$ given N samples from f, it is reasonable to expect that ${\rho }_f$ and $\{{\rho }_{g_N}\}$ would satisfy equations (4) and (5) for some L. The necessary pointwise convergence is assured by any reasonable method of producing $g_N$, and then Theorem 4 of [24] in fact proves the result $\mathtt{\text{h}}(g_N)\to \mathtt{\text{h}}(f)$ (although we will not attempt to formalize the above argument). This is essentially what we prove in our Theorem 1, with the key difference that our result is quantitative. This certainly makes one wonder if the result in Theorem 4 of [24] could in fact be easily adapted to obtain a quantitative proof of our Theorem 1, thus not needing our methods. Currently, we believe the answer is no. Or, looked at another way, we believe that a quantitative version of their result for the problem we are examining would end up confronting the same issues we faced and might end up resembling our methods.

Since [24] does not present a quantitative result, we look to their proof to see if such a result can be found in their methods. Note that in [24], “N” has a meaning different from how we have been using it, and “i” is the variable which corresponds to our “N.” We will use the notation from [24] for the moment, for the convenience of the reader who might want to refer back to [24] during our short discussion below; we hope confusion can be avoided. The main ingredients of the proof in [24] are several variables $(N,K,i)$ which must become “sufficiently large” and the application of theorems such as Egoroff’s Theorem and Lebesgue Dominated Convergence. One of the chief advantages of these powerful theorems from measure theory is that they can bootstrap pointwise convergence into stronger conclusions and bypass potentially forbidding quantitative relationships. Also, the “essence of the proof” (as they state in [24]) is to find a set $A_{ϵ}$ such that everything happening on $A_{ϵ}^c$ (the “bad” set) is negligible, and the desired convergence does in fact take place on $A_{ϵ}$. The set $A_{ϵ}$, by construction, is a set on which ρ and all but finitely many of the ${\rho }_i$ are bounded above by N and bounded away from zero by $1/N$. There is a tradeoff between the size of N (larger N means a smaller “bad” set, but less control on the “good” set) and the other parameters K and i. Therefore, to obtain a quantitative result, the rates of growth of these parameters must be balanced, and the use of the measure-theoretic theorems mentioned above must be replaced with concrete estimates; this would appear to be non-trivial.

There is another issue of concern in [24], a scaling issue. As mentioned in problem 1, the scaling $f\to \lambda f$ for $\lambda \ne 0$ should essentially have no effect on the problem. Note that this scaling induces the standard $L^1$ scaling

$$\rho (u)\to {|\lambda |}^{-1}\rho (u/\lambda )$$

so a method which proves a bound analogous to ours should respect this scaling. The assumption by Godavarti and Hero (in equation (4) of the present article) involves a constant, L, and the specific value of L is used in the proof (for example, we need $N,K$ large enough that (essentially) $L/{|\log K|}^{\kappa -1}$ and $L/{|\log N|}^{\kappa -1}$ are sufficiently small). The scaling $\rho (u)\to {|\lambda |}^{-1}\rho (u/\lambda )$ transforms the integral in (4) into

$${\int \rho |\log |\lambda |}^{-1}+\log \rho {|}^{\kappa }$$

which means in turn that a different value of L (and therefore different values of K and N) will be needed. Therefore, in order to prove a result similar to our Theorem 1, this scaling issue will need to be resolved as well.

3. Definitions and Notation

We will use the following notation:

Definition 1. 

For real numbers $a,b$, we define

$$\langle a,b\rangle =[\min \{a,b\},\max \{a,b\}]=\left\{\begin{array}{cc}[a,b]\hfill & ifa\le b\hfill \\ [b,a]\hfill & ifa>b\hfill \end{array}\right.$$

For subsets $S,T\subset \mathbb{R}$, we define

$$S+T=\{s+t:s\in S,t\in T\}$$

So, for example,

$$\langle a,b\rangle +[-2,3]=[\min \{a,b\}-2,\max \{a,b\}+3]$$

As we defined in the Introduction,

Definition 2. 

For a measurable function $h:[0,1]\to \mathbb{R}$, we define the measure ${\mu }_h$ on $\mathbb{R}$ by

$${\mu }_h(E)=|h^{-1}(E)|$$

We will have occasion to refer to the “monotone pieces” of a function, so we make the following definition. We do not want to consider functions which have critical points at either 0 or 1, for technical reasons and to avoid more cumbersome notation, so we define:

Definition 3. 

For a differentiable function h with a finite set of critical points $t_1<\cdots <t_k$, and with $h^{\prime }(0),h^{\prime }(1)\ne 0$, we define $t_0=0,t_{k+1}=1$ and for $j=0,\dots ,k$ we define $h_j{=h|}_{[t_j,t_{j+1}]}$.

For any function h, we will abbreviate the domain and range of h:

Definition 4. 

We denote the domain of h by $Dom(h)$, and the range of h by $Range(h)$

Recall the definition of the (discrete) Shannon entropy, H, of a finite probability mass distribution $\overrightarrow{p}={\{p_i\}}_{i=1}^n$:

$$H(\overrightarrow{p})=-\sum _i{p}_i\log p_i$$

Let h be a differentiable function with finitely many critical points and $h^{\prime }(0),h^{\prime }(1)\ne 0$. Suppose that ${\mu }_h$ is absolutely continuous. Then we know that the measure given by ${\mu }_{h_i}(E)=|h_i^{-1}(E)|$ is absolutely continuous; let its density be ${\rho }_{h_i}$. For a fixed u in the range of h, note that u is in the range of at least one, possibly several, of the $h_i$. For those i such that u is in the range of $h_i$ we see that ${\rho }_{h_i}$ is non-zero, and we know that

$$\sum _i{\rho }_{h_i}(u)={\rho }_h(u)$$

So, we define the probability mass distribution

$${\overrightarrow{p}}_h(u)={\left\{\frac{{\rho }_{h_i}(u)}{{\rho }_h(u)}\right\}}_{i=0}^k$$

(6)

Note that the number of non-zero entries of ${\overrightarrow{p}}_h(u)$ equals the number of i such that u is in the range of $h_i$.

4. A Useful Identity

Assume that f is a differentiable function with a finite set of critical points $t_1<\cdots <t_k$ and $f^{\prime }(0),f^{\prime }(1)\ne 0$. The following identity is the centerpiece of our method for proving our main result (a helpful comment [28] led us to this identity):

$$\mathtt{\text{h}}(f)={\int }_{t=0}^1\log |f^{\prime }(t)|dt-{\int }_{t=0}^{1}H\left({\overrightarrow{p}}_f(f(t))\right)dt$$

(7)

In the subsequent sections of this article, to simplify a discussion we will sometimes refer to the terms on the right hand side by $(L)$ and $(E)$, corresponding to the “logarithm” term and the “entropy” term. To prove (7), note the following:

$$\begin{array}{cc}\hfill \sum [{\rho }_{f_i}\log {\rho }_{f_i}]-{\rho }_f\log {\rho }_f& =\sum [{\rho }_{f_i}\log {\rho }_{f_i}-{\rho }_{f_i}\log {\rho }_f]\hfill \\ & =\sum {\rho }_{f_i}\log ({\rho }_{f_i}/{\rho }_f)\hfill \\ & ={\rho }_f\sum ({\rho }_{f_i}/{\rho }_f)\log ({\rho }_{f_i}/{\rho }_f)\hfill \\ & =-{\rho }_{f}H({\overrightarrow{p}}_f(u))\hfill \end{array}$$

Integrating both sides, we have

$$\int \sum [{\rho }_{f_i}\log {\rho }_{f_i}]-\int {\rho }_f\log {\rho }_f=-\int {\rho }_{f}H({\overrightarrow{p}}_f(u))$$

or

$$\sum _i{\int }_{u\in Range(f_i)}{\rho }_{f_i}(u)\log {\rho }_{f_i}(u)du+\mathtt{\text{h}}(f)=-\int {\rho }_{f}H({\overrightarrow{p}}_f(u))$$

The identity ${\int }_u{\rho }_{f_i}(u)g(u)du={\int }_{t\in Dom(f_i)}g(f_i(t))dt$, and the formula

$${\rho }_{f_i}(u)=\frac{1}{|f_i^{\prime }\left(f_i^{-1}(u)\right)|}$$

are used on the first term to obtain

$$\sum _i{\int }_{u\in Range(f_i)}{\rho }_{f_i}(u)\log {\rho }_{f_i}(u)du=\sum _i{\int }_{t\in Dom(f_i)}\log {\rho }_{f_i}(f_i(t))dt={\int }_{t=0}^1\log \left(\frac{1}{|f_i^{\prime }(t)|}\right)dt$$

The identity ${\int }_u{\rho }_f(u)g(u)du={\int }_{t=0}^{1}g(f(t))dt$ is used on the last term to obtain

$$-{\int }_u{\rho }_{f}H({\overrightarrow{p}}_f(u))du=-{\int }_{t=0}^{1}H\left({\overrightarrow{p}}_f(f(t))\right)dt$$

and so (7) is proved.

This identity is equivalent to a standard identity for Shannon entropy. Suppose we have a probability space Ω, a finite set S, and a random variable $X:\Omega \to S$. Furthermore, suppose we have an “indicator” function

$$\Delta :\Omega ⟶\mathbb{N}$$

which takes a finite number of values on Ω. We think of Δ as specifying a partition of Ω, i.e.,

$$\Omega ={\Delta }^{-1}(1)\cup {\Delta }^{-1}(2)\cup \dots ={\Omega }_1\cup {\Omega }_2\cup \dots$$

If X maps the ${\Omega }_i$ to disjoints parts of S, i.e.,

$$X({\Omega }_i)\cap X({\Omega }_j)=\varnothing \text{for}i\ne j$$

then we would have

$$\begin{array}{cc}\hfill H(X)& =-\sum _{s\in S}\mathbb{P}[X=s]\log \mathbb{P}[X=s]\hfill \\ & =-\sum _i\sum _{s\in X({\Omega }_i)}\mathbb{P}[X=s]\log \mathbb{P}[X=s]\hfill \\ & =-\sum _i\sum _{s\in X({\Omega }_i)}(\mathbb{P}({\Omega }_i)\mathbb{P}[X=s|{\Omega }_i])\log (\mathbb{P}({\Omega }_i)\mathbb{P}[X=s|{\Omega }_i])\hfill \\ & =-\sum _i\mathbb{P}({\Omega }_i)\sum _{s\in X({\Omega }_i)}\mathbb{P}[X=s|{\Omega }_i]\left(\log (\mathbb{P}({\Omega }_i)+\log \mathbb{P}[X=s|{\Omega }_i])\right)\hfill \\ & =-\sum _i\mathbb{P}({\Omega }_i)\log (\mathbb{P}({\Omega }_i)-\sum _i\mathbb{P}({\Omega }_i)\sum _{s\in X({\Omega }_i)}\mathbb{P}[X=s|{\Omega }_i]\log \mathbb{P}[X=s|{\Omega }_i]\hfill \\ & =H(\Delta )+H(X|\Delta )\hfill \end{array}$$

where $H(X|\Delta )$ is the conditional entropy of X given Δ (see [29]). In other words, $H(\Delta )+H(X|\Delta )$ would be the Shannon entropy of X if the images of the ${\Omega }_i$ under X did not overlap. If they did overlap, this would be an overestimation of the entropy of X, and in general the true value is given by subtracting a term, $H(\Delta |X)$, which represents the amount of “overlap”:

$$H(X)=H(\Delta )+H(X|\Delta )-H(\Delta |X)$$

(8)

This identity follows from the “chain rule” for entropy [29].

In our case, the probability space is $[0,1]$, the set is $\mathbb{R}$, the random variable is $f:[0,1]\to \mathbb{R}$, and the indicator function on the probability space is the function

$$\Delta (t)=i\text{if}t\in [t_i,t_{i+1}]$$

which (essentially) partitions $[0,1]$ : $[0,1]=[0,t_1]\cup [t_1,t_2]\cup \dots \cup [t_k,1]$. As above,

$$H(\Delta )+\mathtt{\text{h}}(f|\Delta )={\int }_{t=0}^1\log |f^{\prime }(t)|dt$$

measures the entropy of f as if the different “monotone laps” of f did not overlap. If they do overlap then this is an overestimate, and the true value is given by subtracting the term representing the amount of “overlap”:

$$\mathtt{\text{h}}(\Delta |f)={\int }_{t=0}^{1}H\left({\overrightarrow{p}}_f(f(t))\right)dt$$

In this way, (7) is analogous to (8).

5. Scaling

For simplicity, we consider $\lambda >0$ (the case $\lambda <0$ is no different). For $\lambda >0$, we have ${\rho }_{\lambda f}(u)={\lambda }^{-1}{\rho }_f(u/\lambda )$, and so $\mathtt{\text{h}}(\lambda f)=\mathtt{\text{h}}(f)+\log \lambda$. This means

$$\mathtt{\text{h}}(\lambda f)-\mathtt{\text{h}}(\lambda g)=\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)$$

So, the difference $\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)$ is invariant under the scaling $f\to \lambda f$. However, if we take $[a,b]\subset \mathbb{R}$, and we consider

$${\int }_a^b{\rho }_f\log {\rho }_f-{\rho }_g\log {\rho }_g$$

then, scaling by the factor λ, we have

$${\int }_{\lambda a}^{\lambda b}{\rho }_{\lambda f}\log {\rho }_{\lambda f}-{\rho }_{\lambda g}\log {\rho }_{\lambda g}=\left[{\int }_a^b{\rho }_f\log {\rho }_f-{\rho }_g\log {\rho }_g\right]+(\log \lambda )\left[{\int }_a^b{\rho }_f-{\rho }_g\right]$$

Since the second term on the right hand side of this equation is not necessarily zero, we see that an integral over just a part of the domain may not be scale-invariant.

Turning our attention to the identity (7), we see that for $\lambda >0$ we have:

$$\begin{array}{cc}\hfill {\int }_{t=0}^1\log |{(\lambda f)}^{\prime }(t)|dt& ={\int }_{t=0}^1\log |f^{\prime }(t)|dt+\log \lambda \hfill \\ \hfill {\int }_{t=0}^{1}H\left({\overrightarrow{p}}_{\lambda f}\left((\lambda f)(t)\right)\right)dt& ={\int }_{t=0}^{1}H({\overrightarrow{p}}_f(f(t)))dt\hfill \end{array}$$

Perhaps more importantly, each term can be divided along its respective axis, and each individual piece scales similarly:

$$\begin{array}{cc}\hfill {\int }_{t_a}^{t_b}\log |{(\lambda f)}^{\prime }(t)|dt& ={\int }_{t_a}^{t_b}\log |f^{\prime }(t)|dt+(t_b-t_a)\log \lambda \hfill \\ \hfill {\int }_{t_a}^{t_b}H\left({\overrightarrow{p}}_{\lambda f}\left((\lambda f)(t)\right)\right)dt& ={\int }_{t_a}^{t_b}H({\overrightarrow{p}}_f(f(t)))dt\hfill \end{array}$$

Our main strategy takes advantage of precisely this. Recall the definitions for (L) and (E) that followed (7); both (L) and (E) will be subdivided along the t axis.

6. A Counter-Example

We will denote the $i^{th}$ derivative of f (or g) by $f^{(i)}$ (or $g^{(i)}$). One might hope that a theorem along the following lines could be proved, C being an absolute constant:

$$|\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)|\le C\sum _{i=0}^K{\parallel f^{(i)}-g^{(i)}\parallel }_{\infty }$$

This would show that the $\mathtt{\text{h}}$-functional is a continuous map in the norm topology of some Banach space of differentiable functions. One can see immediately that this result is not possible, just due to scaling: If we let $f_n=(1/n)f,g_n=(1/n)g$ then $|\mathtt{\text{h}}(f_n)-\mathtt{\text{h}}(g_n)|=|\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)|$ while the right hand side converges to zero as $n\to \infty$. However, we might hope that we could prove a result such as

$$|\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)|\le \left[C\sum _{i=0}^K{\parallel f^{(i)}-g^{(i)}\parallel }_{\infty }\right]/\min \left\{\{\parallel f^{(i)}{\parallel }_{\infty },\parallel g^{(i)}{{\parallel }_{\infty }\}}_{i=0}^K\right\}$$

(9)

In other words, perhaps this scaling issue is the only problem and the $\mathtt{\text{h}}$-functional is continuous when restricted to points f and g which are bounded away from zero in this Banach space. This is not the case. In fact, suppose that we ask even less of the result we seek. Suppose that we seek some $r>0$, $c>0$, and some function Γ of $2K+2$ variables which is bounded on compact subsets of the open positive orthant ${({\mathbb{R}}_{+})}^{2K+2}$, and we just want the following bound to hold: Whenever

$$\left[\sum _{i=0}^K{\parallel f^{(i)}-g^{(i)}\parallel }_{\infty }^r\right]\Gamma (\parallel f^{(0)}{\parallel }_{\infty },\dots ,\parallel f^{(K)}{\parallel }_{\infty },\parallel g^{(0)}{\parallel }_{\infty },\dots ,\parallel g^{(K)}{\parallel }_{\infty })\le c$$

is true, we have

$$|\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)|\le \left[\sum _{i=0}^K{\parallel f^{(i)}-g^{(i)}\parallel }_{\infty }^r\right]\Gamma (\parallel f^{(0)}{\parallel }_{\infty },\dots ,\parallel f^{(K)}{\parallel }_{\infty },\parallel g^{(0)}{\parallel }_{\infty },\dots ,\parallel g^{(K)}{\parallel }_{\infty })$$

(10)

In other words, whenever the differences $\{\parallel f^{(i)}-g^{(i)}{{\parallel }_{\infty }^r\}}_{i=0}^K$, after being multiplied by the (large) value Γ, are sufficiently small, then we have a quantitative estimate on $\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)$.

This is still not possible. Note that (9) is a special case of (10). Note also that our main result, Theorem 1, is almost a special case of (10) with $K=3$, the only additional element in Theorem 1 being the regularity condition (Definition 6) and the involvement of the parameter from that condition, δ.

The impossibility of a result such as (10) can be seen just looking at monotone functions, in which case the $\mathtt{\text{h}}$ functional is identical to the functional $f\to \int \log |f^{\prime }|$. The reason is quite straightforward: The functional $f\to \int \log |f^{\prime }|$ will become large if a function is extremely flat even for a short distance, whereas any ${\parallel ·\parallel }_{\infty }$-type norm will not necessarily measure a large difference between a function that is extremely flat and a function that is merely somewhat flat. We formalize this in the following proposition:

Proposition 5. 

Fix K and ϵ. There exists a constant $C_K$ depending only on K, and monotone functions $f,g$, such that

$$\begin{array}{cc}& \forall i\le K,f^{(i)},g^{(i)}arecontinuous\hfill \\ & \forall i\le K,\parallel f^{(i)}-g^{(i)}{\parallel }_{\infty }\le ϵ\hfill \\ & \forall i\le K,\parallel f^{(i)}{\parallel }_{\infty },{\parallel g^{(i)}\parallel }_{\infty }\ge 1\hfill \\ & \forall i\le K,\parallel f^{(i)}{\parallel }_{\infty },{\parallel g^{(i)}\parallel }_{\infty }\le C_K\hfill \end{array}$$

but

$$\left|\int \log |f^{\prime }|-\log |g^{\prime }|\right|\ge 1$$

Proof. 

Let $f_0(t)=t^N$ and $g_0(t)=t^M$; we are free to choose M and N independently, and the idea is to let $M>>N>>K$. Define

$$f(t)=f_0(t)\text{on}[0,1/2],f(t)=2^{K+1}{(t-1/2)}^{K+1}+\sum _{i=0}^K\frac{f^{(i)}(1/2)}{i!}{(t-1/2)}^i\text{on}[1/2,1]$$

$$g(t)=g_0(t)\text{on}[0,1/2],g(t)=2^{K+1}{(t-1/2)}^{K+1}+\sum _{i=0}^K\frac{g^{(i)}(1/2)}{i!}{(t-1/2)}^i\text{on}[1/2,1]$$

We see that f and g were chosen to have $f^{(i)},g^{(i)}$ continuous for $i\le K$. If we choose N and M so that $K<N/2,N<M/2$, we see that $\forall i\le K$, $\alpha \le 1/2$,

$$g_0^{(i)}(\alpha )\le M^K{\alpha }^{M/2}\text{and}{f}_0^{(i)}(\alpha )\ge {\alpha }^N$$

so once we choose sufficiently large M so that $M^K{\alpha }^{M/2}\le {\alpha }^N$ for all $\alpha \le 1/2$, we have $g_0^{(i)}(\alpha )\le f_0^{(i)}(\alpha )\forall i\le K,\alpha \le 1/2$ and therefore $g^{(i)}(t)\le f^{(i)}(t)\forall i\le K,t\in [0,1]$.

Note that, on $[0,1/2]$ we have $f^{(i)}(t)\le f^{(i)}(1/2)\le N^K{(1/2)}^{N/2}$. Choose sufficiently large N so that

$$N^K{(1/2)}^{N/2}\le ϵ/\left[(K+1)K^K\right]$$

This certainly means $|f^{(i)}-g^{(i)}|\le ϵ$ on $[0,1/2]$. On $[1/2,1]$ we just need ${\sum }_{i=0}^K\frac{f^{(i)}(1/2)}{i!}{(t-1/2)}^i$ and all of its derivatives to be smaller than ϵ and then we will have $\parallel f^{(i)}-g^{(i)}{\parallel }_{\infty }\le ϵ\forall i\le K$. We see that

$${\left[\sum _{i=0}^K\frac{f^{(i)}(1/2)}{i!}{(t-1/2)}^i\right]}^{(j)}(t)\le (K+1)\left[\underset{i\le K}{\max }{f}^{(i)}(1/2)\right]K^K\le ϵ$$

and so we have $\parallel f^{(i)}-g^{(i)}{\parallel }_{\infty }\le ϵ\forall i\le K$.

By the above calculation, we have the immediate bound

$$f^{(i)}(t)\le f^{(i)}(1)\le 2^{K+1}(K+1)!+(K+1)K^K\left[\underset{i\le K}{\max }{f}^{(i)}(1/2)\right]$$

and we see that the following is true, with a constant $C_K^{\prime }$ depending only on K:

$$\underset{i\le K}{\max }{f}^{(i)}(1/2)\le N^K{2}^{-(N-K)}\le C_K^{\prime }$$

which means $\parallel f^{(i)}{\parallel }_{\infty }\le C_K$ (and the same is true for $g^{(i)}$ since $g^{(i)}\le f^{(i)}$).

Finally, observing that $f^{(i)}(1)\ge 2^{K+1}{(1/2)}^{K+1}+\left(\sum \cdots \right)\ge 1$, and noting that the same is true for g, we have $\forall i\le K,\parallel f^{(i)}{\parallel }_{\infty },{\parallel g^{(i)}\parallel }_{\infty }\ge 1$.

So, f and g meet the criteria of the proposition. At this point, let us “fix” the value of N that we have arrived at. Since $f^{\prime },g^{\prime }>0$ and $g^{\prime }\le f^{\prime }$, we have

$$\begin{array}{cc}\hfill \left|\int \log |f^{\prime }|-\log |g^{\prime }|\right|& ={\int }_0^1\log f^{\prime }-\log g^{\prime }\hfill \\ & \ge {\int }_0^{1/2}\log f^{\prime }-\log g^{\prime }\hfill \\ & =(1/2)[\log N-\log M]+[(N-1)-(M-1)]\left[{\int }_0^{1/2}\log t\right]\hfill \end{array}$$

and as $M\to \infty$ this expression grows without bound, proving the proposition. ☐

7. Main Result

Suppose that we have functions $f,g:[0,1]\to \mathbb{R}$. We will assume that each function has three continuous derivatives; we do not attempt to determine the precise smoothness required. The quantities $\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }$ will be used frequently, and the reader might be worried, glancing at Theorem 1, that the estimate becomes “worse” as $\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }$ become small, certainly a puzzling property. In fact we only need upper bounds on $|f^{‴}|,|g^{‴}|$, not least upper bounds, so the reader may substitute any upper bounds on $|f^{‴}|,|g^{‴}|$ in place of $\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }$ as long as it is done consistently. Notice that this is still scale invariant (if A is an upper bound on $|f^{‴}|$ then $\lambda A$ is an upper bound on ${|(\lambda f)}^{‴}|$). We will use $\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }$ for simplicity.

The quantities which actually bring $\mathtt{\text{h}}(f)$ and $\mathtt{\text{h}}(g)$ close together are ${\parallel f-g\parallel }_{\infty }$ and $\parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }$; the “smallness” of these quantities, relative to the size of the third derivatives of f and g, will be quantified by a parameter ϵ.

However, Proposition 5 means that the “smallness” of ${\parallel f-g\parallel }_{\infty }$ and $\parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }$ alone cannot imply the result we are seeking. We will impose an additional regularity assumption.

Definition 6 

(The Set $A(\delta )$). Suppose that $h:[0,1]⟶\mathbb{R}$. For $0<\delta \le 1$, we say $h\in A(\delta )$ if h, $h^{\prime }$, $h^{″}$ and $h^{(3)}$ are continuous, and the following two conditions hold:

$$\begin{array}{cc}\hfill |h^{\prime }(t)|+|h^{″}(t)|& \ge \delta \parallel h^{(3)}{\parallel }_{\infty }\forall t\in [0,1]\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill |h^{\prime }(t)|& \ge \delta \parallel h^{(3)}{\parallel }_{\infty }\forall t\in [0,\delta ]\cup [1-\delta ,1]\hfill \end{array}$$

(12)

We will require that f and g are in the set $A(\delta )$. Intuitively, the requirement for a function to be in $A(\delta )$ is a quantitative statement that its first and second derivatives do not simultaneously vanish, i.e., it does not have any “flat spots”. The second half of the condition, equation (12), is not as natural; its purpose is simply to eliminate some uninteresting technical difficulties at the endpoints of $[0,1]$.

Lastly, we will need bounds on the ratios

$$\frac{\max \parallel f^{\prime }{\parallel }_{\infty },{\parallel g^{\prime }\parallel }_{\infty }}{\min \left\{\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }\right\}}\text{and}\frac{\max \parallel f^{″}{\parallel }_{\infty },{\parallel g^{″}\parallel }_{\infty }}{\min \left\{\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }\right\}}$$

With these assumptions, we can bound the difference between $\mathtt{\text{h}}(f)$ and $\mathtt{\text{h}}(g)$:

Theorem 1 (Main Result).

Suppose that $f,g:[0,1]\to \mathbb{R}$ are each members of $A(\delta )$, and for some $ϵ\le 2^{-20}{\delta }^8$ we have

$$\begin{array}{cc}& {\parallel f-g\parallel }_{\infty }\le ϵ\min \left\{\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }\right\}\hfill \end{array}$$

(13)

$$\begin{array}{cc}& \parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }\le {ϵ}^{3/4}\min \left\{\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }\right\}\hfill \end{array}$$

(14)

Furthermore, let $C_1,C_2$ be constants such that $C_1,C_2\ge 3$ and

$$C_1^{-1}\le \frac{\max \parallel f^{\prime }{\parallel }_{\infty },{\parallel g^{\prime }\parallel }_{\infty }}{\min \left\{\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }\right\}}\le C_1,\frac{\max \parallel f^{″}{\parallel }_{\infty },{\parallel g^{″}\parallel }_{\infty }}{\min \left\{\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }\right\}}\le C_2$$

Then

$$|\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)|\le C{C}_1{C}_2\log (C_1{\delta }^{-1}{ϵ}^{-1/4}){ϵ}^{1/4}/{\delta }^4$$

where C is an absolute constant.

Our main result can be thought of as a converse to Proposition 5: If the functions f and g do not have any extreme “flat spots” then (essentially) the smallness of ${\parallel f-g\parallel }_{\infty }$ and $\parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }$ are sufficient to imply that $\mathtt{\text{h}}(f)$ and $\mathtt{\text{h}}(g)$ are close.

Notice that the result is scale invariant: the assumptions and conclusion are not altered in the least if we multiply both f and g by the same non-zero constant.

8. Overview of Proof

The proof begins by using the “useful identity” from Section 4. to split the difference $\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)$ into two terms:

$$\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)={\int }_{t=0}^1\log |f^{\prime }(t)|-\log |g^{\prime }(t)|-{\int }_{t=0}^{1}H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)$$

As in Section 4., we abbreviate the first term by $(L)$ and the second term by $(E)$. Next, we wish to split the domain of integration into a “good” set and a “bad” set. However, this is done differently for $(L)$ and $(E)$.

The regions of $[0,1]$ that are causing the problems are the places where the derivatives of f and g are very small (or zero). Let us denote this region by $B^{*}$. When we are dealing with just $\log |f^{\prime }|,\log |g^{\prime }|$ , it is just the set $B^{*}$ that will constitute the “bad” set. Therefore, we write

$$\int \log |f^{\prime }(t)|-\log |g^{\prime }(t)|={\int }_{B^{*}}\log |f^{\prime }(t)|-\log |g^{\prime }(t)|+{\int }_{{(B^{*})}^c}\log |f^{\prime }(t)|-\log |g^{\prime }(t)|$$

or $(L)=(LB)+(LG)$.

When dealing with the expression $H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)$, it is of course the same set $B^{*}$ that causes the trouble, but the affected areas spread beyond just those values of t which are in $B^{*}$. If we have a value of t such that the function value $f(t)$ is in the image (by f) of some other point $t^{\prime }$ with f having a small derivative at $t^{\prime }$, then the point t will be a “problem” for $H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)$, since the finite probability mass distribution ${\overrightarrow{p}}_f\left(f(t)\right)$ involves all points $t^{\prime }$ such that $f(t^{\prime })=f(t)$. In other words, the behavior of $H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)$ is not determined by just a neighborhood of t. This makes sense if we recall that this term is trying to capture the “overlap” in function values that might happen between widely separated regions of the domain of f.

We can think of this in the following way: the “bad” set $B^{*}$ is going to “infect” a region of the u-axis (the set $\mathcal{I}$ from Definition 16) and then any points in $[0,1]$ which map to that “infected” area will also be “infected”. The set $G^{*}$ (defined precisely in Definition 16) is the portion of $[0,1]$ that is not “infected” at all, i.e.,. it is “good”. If $t\in G^{*}$, this means that there is no point $t^{\prime }\in B^{*}$ for which $f(t^{\prime })=f(t)$ or $g(t^{\prime })=g(t)$ (the statement is slightly stronger than this, but that is the idea).

To summarize: We divide $[0,1]$ into a “bad” region $B^{*}$, a “good” region $G^{*}$, and a “neutral” region ${(B^{*})}^c\cap {(G^{*})}^c$. For $(L)$, it is $B^{*}$ that is “bad” and everything else is “good,” while for $(E)$ it is only $G^{*}$ that is “good” and everything else is “bad”.

Therefore, for purposes of $H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)$ the bad set will be ${(G^{*})}^c$ and so we write

$$\begin{array}{cc}\hfill \int H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)=& {\int }_{{(G^{*})}^c}H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)\hfill \\ & +{\int }_{G^{*}}H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)\hfill \end{array}$$

or $(E)=(EB)+(EG)$.

Note that each piece $(LG),(LB),(EG),(EB)$ by itself is scale invariant. As discussed in section 5., some care must be taken to split $\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)$ into scale invariant pieces.

In preparation for estimating the size of each of the four pieces, in Section 9. and Section 10. we develop some results pursuing the implications of the regularity assumption. These implications are both quantitative and qualitative. The quantitative implications are pursued in Section 9.. In Section 10., we define the space $E(\tau )$ (see Definition 11), which is meant to abstract just some “qualitative” or “incidence” properties which follow from the quantitative results, and we prove some results about $E(\tau )$.

In section 11., we apply the results from Section 9. and Section 10. to f and g. In brief, the bound on each piece is as follows.

Looking first at $(L)$, the bound for $(LG)$ is immediate, so we concentrate on $(LB)$. The regularity assumption (Definition 6) will imply that critical points of f and g roughly coincide; we consider the portion of $B^{*}$ near just one critical point, call this $B_j^{*}$. As $ϵ\to 0$, certainly $|B_j^{*}|\to 0$, but $B_j^{*}$ is where $|f^{\prime }|$ or $|g^{\prime }|$ are small (or zero), so $\log |f^{\prime }|,\log |g^{\prime }|$ can be unbounded and we cannot say that $(LB)$ is negligible just because $|B_j^{*}|$ is small. However, we observe that f can be approximated by a parabola $p_f$ near a critical point, and the same is true for g. We have

$${\int }_{B_j^{*}}\log |f^{\prime }|={\int }_{B_j^{*}}\log |p_f^{\prime }|+{\int }_{B_j^{*}}\log |f^{\prime }/p_f^{\prime }|$$

and it is only $\log |p_f^{\prime }|$ which is unbounded; the estimate of $(LB)$ is completed by proving a bound on

$${\int }_{B_j^{*}}\log |p_f^{\prime }|-{\int }_{B_j^{*}}\log |p_g^{\prime }|$$

We turn next to $(EB)$. This piece is bounded by “crude” size estimates, the main point being that the set ${(G^{*})}^c$ is small. The integrand,

$$H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)$$

is bounded in size by the logarithm of one plus the number of critical points of f (the regularity assumption will imply that f and g have the same number of critical points). The size of the set ${(G^{*})}^c$ is bounded just by the number of components of $B^{*}$, the size of $B^{*}$, and some basic data about the functions f and g.

Finally, we have $(EG)$. We in fact only need some “incidence” data about the functions f and g to prove the result here (this data is summarized in Definition 11). The bound on this piece proceeds essentially as follows: Once we consider only $t\in G^{*}$, we can say

$${\overrightarrow{p}}_f\left(f(t)\right)={\{p_i\}}_{i=0}^k,{\overrightarrow{p}}_g\left(g(t)\right)={\{q_i\}}_{i=0}^k$$

and we need to bound $|{\sum }_{i=0}^k{p}_i\log p_i-q_i\log q_i|$. On $G^{*}$, we will see that $p_i,q_i$ are bounded below, so we use the inequality

$$|p_i\log p_i-q_i\log q_i|\le [1+\max |\log p_i|,|\log q_i|]|p_i-q_i|$$

and the proof is completed by estimating the difference

$$|p_i-q_i|=\left|\frac{{\rho }_{f_i}}{{\rho }_f}-\frac{{\rho }_{g_i}}{{\rho }_g}\right|$$

which ultimately reduces to the fact that $f^{\prime }\left(f_j^{-1}\left(f(t)\right)\right)$ and $g^{\prime }\left(g_j^{-1}\left(g(t)\right)\right)$ are close together because $|f(t)-g(t)|\le {\parallel f-g\parallel }_{\infty }$, and $f_j^{-1},g_j^{-1}$ are going to be close together when evaluated on points that are close, and finally $f^{\prime },g^{\prime }$ differ by at most $\parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }$ when evaluated on the same point, so they are going to be close together when evaluated on points that are close.

In the following sections, the functions “f” and “g” will always refer to the two specific functions assumed by Theorem 1. We will use h and l to refer to generic functions.

9. The Space $A(\delta )$

In the following, we assume $h,l\in A(\delta )$.

Lemma 7. 

If $|h^{″}(t_0)|\ge A>0$ and $|t-t_0|\le A/(2\parallel h^{(3)}{\parallel }_{\infty })$, then $|h^{″}(t)|\ge A/2$.

$|h^{″}(t)-h^{″}(t_0)|\le \parallel h^{(3)}{\parallel }_{\infty }|t-t_0|$.

Lemma 8. 

Distinct zeros of $h^{\prime }$ are separated by a distance of at least $\delta /2$. This also means $h^{\prime }$ has at most $2/\delta$ zeros on$[0,1]$.

$h^{\prime }(t_0)=0$ implies that $|h^{″}(t_0)|\ge \delta \parallel h^{(3)}{\parallel }_{\infty }$, and so

$$|t-t_0|\le \delta /2\Rightarrow |t-t_0|\le (\delta \parallel h^{(3)}{\parallel }_{\infty })/(2\parallel h^{(3)}{\parallel }_{\infty })$$

and then Lemma 7 implies $|h^{″}(t)|>0$. Since $h^{″}$ is continuous, $h^{″}$ does not change sign between t and $t_0$, so a zero of $h^{\prime }$ must be further than $\delta /2$ away from $t_0$.

 

Lemma 9. If $\gamma \le {\delta }^2/16$ and $|h^{\prime }(t_0)|\le \gamma \parallel h^{(3)}{\parallel }_{\infty }$ then $\exists t,|t-t_0|\le 4\gamma /\delta$ with $h^{\prime }(t)=0$.

By definition of $A(\delta )$, we know that $t_0\in (\delta ,1-\delta )$ and $|h^{″}(t_0)|\ge (\delta /2){\parallel h^{(3)}\parallel }_{\infty }$. By Lemma 7 we know that $h^{″}$ has the same sign on

$$[t_0-\delta /4,t_0+\delta /4]$$

and $|h^{″}|$ is greater than $(\delta /4)\parallel h^{(3)}{\parallel }_{\infty }$ on that interval. Without loss of generality, suppose that

$$0\le h^{\prime }(t_0)\le \gamma {\parallel h^{(3)}\parallel }_{\infty },h^{″}(t_0)<0$$

Let $t=t_0+4\gamma /\delta$. We see that $4\gamma /\delta \le \delta /4$ which means $t\in [0,1]$ and $t\in [t_0-\delta /4,t_0+\delta /4]$. So, on $[t_0,t]$, we have $h^{″}\le -(\delta /4){\parallel h^{(3)}\parallel }_{\infty }$. Therefore,

$$\begin{array}{cc}\hfill h^{\prime }(t)& =h^{\prime }(t_0)+{\int }_{t_0}^t{h}^{″}\hfill \\ & \le \gamma \parallel h^{(3)}{\parallel }_{\infty }+{\int }_{t_0}^t-(\delta /4){\parallel h^{(3)}\parallel }_{\infty }\hfill \\ & =\gamma \parallel h^{(3)}{\parallel }_{\infty }-(4\gamma /\delta )(\delta /4){\parallel h^{(3)}\parallel }_{\infty }\hfill \\ & =0\hfill \end{array}$$

So $h^{\prime }(t_0)\ge 0$ and $h^{\prime }(t)\le 0$ and it is proved.

Lemma 10. 

If

$$\parallel h^{\prime }-l^{\prime }{\parallel }_{\infty }<\left({\delta }^2/16\right)\min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}$$

then

$$\begin{array}{cc}\hfill h^{\prime }(t_0)=0& \Rightarrow \exists t^{\prime }with{l}^{\prime }(t^{\prime })=0,|t^{\prime }-t_0|\le 4\frac{\parallel h^{\prime }-l^{\prime }{\parallel }_{\infty }}{\delta \min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}}\hfill \end{array}$$

(15)

$$\begin{array}{c}\hfill nonumber\\ \hfill l^{\prime }(s_0)=0& \Rightarrow \exists s^{\prime }with{h}^{\prime }(s^{\prime })=0,|s^{\prime }-s_0|\le 4\frac{\parallel h^{\prime }-l^{\prime }{\parallel }_{\infty }}{\delta \min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}}\hfill \end{array}$$

(16)

This is a corollary of Lemma 9; if $h^{\prime }(t_0)=0$ then

$$|l^{\prime }(t_0)|<\left({\delta }^2/16\right)\min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}$$

and we apply Lemma 9 to l. We proceed analogously if we assume $l^{\prime }(t_0)=0$.

10. The Space $E(\tau )$

Definition 11. 

We say that $h:[0,1]⟶\mathbb{R}$ is a member of $E(\tau )$ if $h,h^{\prime },h^{″},h^{(3)}$ are continuous and

$$\begin{array}{cc}\hfill \left[t_1\ne t_{2}and{h}^{\prime }(t_1)=h^{\prime }(t_2)=0\right]& \Rightarrow |t_1-t_2|>2\tau \hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill t\in [0,\tau ]\cup [1-\tau ,1]& \Rightarrow h^{\prime }(t)\ne 0\hfill \end{array}$$

(18)

Definition 12. 

For $h,l\in E(\tau )$, we say $h\sim l$ if the following two statements are true:

$$\begin{array}{cc}& \left[h^{\prime }(t)=0\Rightarrow \exists t^{\prime },|t-t^{\prime }|<\tau ,l^{\prime }(t^{\prime })=0\right]\hfill \\ & \left[l^{\prime }(s)=0\Rightarrow \exists s^{\prime },|s-s^{\prime }|<\tau ,h^{\prime }(s^{\prime })=0\right]\hfill \end{array}$$

The set $E(\tau )$ just isolates some of the “incidence” properties of the critical points of h and l; we will make use of it because of the following two lemmas which connect it to $A(\delta )$:

Lemma 13. 

If $h\in A(\delta )$ then there exists τ such that $h\in E(\tau )$.

This follows directly from the definition of $A(\delta )$ and Lemma 8.

Lemma 14. 

If $h,l\in A(\delta )$ and

$$\parallel h^{\prime }-l^{\prime }{\parallel }_{\infty }<\left({\delta }^2/16\right)\min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}$$

then there exists τ such that $h,l\in E(\tau )$ and $h\sim l$ in $E(\tau )$.

This follows directly from Lemma 10 and Lemma 14.

Lemma 15. 

If $h,l\in E(\tau )$ and $h\sim l$, then h and l have the same number of critical points, and if $h^{\prime }=0$ at $t_1<\cdots <t_k$ and $l^{\prime }=0$ at $s_1<\cdots <s_k$ then

$$\forall i=1,\dots ,k-1,\max \{t_i,s_i\}<\min \{t_{i+1},s_{i+1}\}$$

Define the function Φ,

$$\Phi :\{\text{critical}\text{points}\text{of}h\}⟶\{\text{critical}\text{points}\text{of}l\}$$

as follows: For a critical point t of h, let

$$\Phi (t)=t^{\prime }\text{where}|t^{\prime }-t|<\tau ,l^{\prime }(t^{\prime })=0$$

It is easy to check that Φ is well defined and is, in fact, a bijection. It follows that $s_i=\Phi (t_i)$, and so the second conclusion follows from the fact that $|t_{i+1}-t_i|>2\tau$, $|t_i-s_i|<\tau$, $|t_{i+1}-s_{i+1}|<\tau$.

Definition 16. 

For functions h and l, and $B\subset [0,1]$, let

$$\begin{array}{cc}& {\mathcal{I}}_{h,l}(B)=\bigcup _{t\in B}{\langle h(t),l(t)\rangle +[-\parallel h-l\parallel }_{\infty }{,\parallel h-l\parallel }_{\infty }]\hfill \\ & G_{h,l}(B)=[0,1]\setminus \left(h^{-1}(\mathcal{I})\cup l^{-1}(\mathcal{I})\right)\hfill \end{array}$$

(recall the notation from Definition 1).

Lemma 17. 

Suppose $h,l\in E(\tau )$, and $h\sim l$. By Lemma 15 suppose they have critical points $t_1<\cdots <t_k$ and $s_1<\cdots <s_k$ respectively, and define $h_i,l_i,t_0,s_0,t_{k+1},s_{k+1}$ as in Definition 3. Suppose we have a set B such that

$$B\supset \{0\}\cup \{1\}\cup \bigcup _i\langle t_i,s_i\rangle$$

Let $\mathcal{I}={\mathcal{I}}_{h,l}(B)$ and $G=G_{h,l}(B)$ be defined as in Definition 16. Fix $i\in \{0,\dots ,k\}$ and $t\in G$. Then the following four statements are equivalent:

$$\begin{array}{cc}& 1)h(t)\in Range(h_i)\hfill \\ & 2)h(t)\in Range(l_i)\hfill \\ & 3)l(t)\in Range(h_i)\hfill \\ & 4)l(t)\in Range(l_i)\hfill \end{array}$$

Suppose that $1)$ holds and $h(t)\in \text{Range}(h_i)$. Without loss of generality, suppose that $h_i^{\prime }>0$. We know

$$h(t)\notin \left[\underset{\langle t_i,s_i\rangle }{\min }{\{h,l\}-\parallel h-l\parallel }_{\infty },\underset{\langle t_i,s_i\rangle }{\max }\{h,l\}+{\parallel h-l\parallel }_{\infty }\right]$$

since that set is in $\mathcal{I}$ and $t\in G$. Since $h(t)\ge h(t_i)$, we know $h(t)\ge {\max }_{\langle t_i,s_i\rangle }\{h,l\}+{\parallel h-l\parallel }_{\infty }$. Also,

$$l(t)\notin \left[\underset{\langle t_i,s_i\rangle }{\min }{\{h,l\}-\parallel h-l\parallel }_{\infty },\underset{\langle t_i,s_i\rangle }{\max }\{h,l\}+{\parallel h-l\parallel }_{\infty }\right]$$

for the same reason, and we cannot have $l(t)<{\min }_{\langle t_i,s_i\rangle }\{h,l\}-{\parallel h-l\parallel }_{\infty }$ since that would imply

$${|h(t)-l(t)|>\parallel h-l\parallel }_{\infty }$$

So, $l(t)\ge {\max }_{\langle t_i,s_i\rangle }\{h,l\}+{\parallel h-l\parallel }_{\infty }$. By similar logic,

$$h(t),l(t)\le \underset{\langle t_{i+1},s_{i+1}\rangle }{\min }\{h,l\}-{\parallel h-l\parallel }_{\infty }$$

So, we see that

$$\underset{\langle t_i,s_i\rangle }{\max }{\{h,l\}+\parallel h-l\parallel }_{\infty }\le h(t),l(t)\le \underset{\langle t_{i+1},s_{i+1}\rangle }{\min }\{h,l\}-{\parallel h-l\parallel }_{\infty }$$

So, the left hand side is less than or equal to the right hand side and we have $h(t),l(t)\in \text{Range}(h_i),\text{Range}(l_i)$. This proves

$$1)\Rightarrow 2),3),4)$$

The other implications are seen in the same manner.

Lemma 18. 

Under the same assumptions as Lemma 17, for $t\in G$ we have

$$\langle h(t),l(t)\rangle \subset {\mathcal{I}}^c$$

and so

$$h^{-1}\left(\langle h(t),l(t)\rangle \right),l^{-1}\left(\langle h(t),l(t)\rangle \right)\subset B^c$$

This follows from the observation that $\mathcal{I}$ contains no intervals of length less than ${\parallel h-l\parallel }_{\infty }$, and $h(t),l(t)\notin \mathcal{I}$.

Lemma 19. 

Under the same assumptions as Lemma 17, if we have $t\in G$, $u\in \langle h(t),l(t)\rangle$ and $h(t)\in Range(h_j)$ then

$$\langle h_j^{-1}(u),l_j^{-1}(u)\rangle \subset Dom(h_j)\cap Dom(l_j)\cap B^c$$

and therefore neither $h^{\prime }$ nor $l^{\prime }$ changes sign on $\langle h_j^{-1}(u),l_j^{-1}(u)\rangle$.

We know by Lemma 17 that $u\in \text{Range}(h_j)\cap \text{Range}(l_j)$. Suppose $h_j(t_1)=u$. Then $t_1\notin B$ by Lemma 18. From Lemma 15 we see that $\text{Dom}(h_j)\cap B^c=\text{Dom}(l_j)\cap B^c$. So,

$$\begin{array}{cc}& t_1\in \text{Dom}(h_j)\cap B^c\hfill \\ \hfill \Rightarrow & t_1\in \text{Dom}(l_j)\hfill \\ \hfill \Rightarrow & h_j^{-1}(u)\in \text{Dom}(h_j)\cap \text{Dom}(l_j)\cap B^c\hfill \end{array}$$

The same is true for $l_j^{-1}(u)$, and so $\langle h_j^{-1}(u),l_j^{-1}(u)\rangle \subset \text{Dom}(h_j)\cap \text{Dom}(l_j)\cap B^c$.

11. Applying the Results to f and g

We now return to the two specific functions f and g, which have the properties stated in the assumptions of Theorem 1. We introduce a variable, μ, which will allow us to divide $[0,1]$ into areas where either $f^{\prime }$ or $g^{\prime }$ are “small” according to μ, specifically

$$|f^{\prime }(t)|\le \mu \parallel f^{(3)}{\parallel }_{\infty }\text{or}|g^{\prime }(t)|\le \mu \parallel g^{(3)}{\parallel }_{\infty }$$

and areas where both $f^{\prime }$ and $g^{\prime }$ are “large” according to μ, specifically

$$|f^{\prime }(t)|\ge \mu \parallel f^{(3)}{\parallel }_{\infty }\text{and}|g^{\prime }(t)|\ge \mu \parallel g^{(3)}{\parallel }_{\infty }$$

At the end of the proof, we will optimize μ; it will turn out that $\mu ={ϵ}^{1/4}$ is optimal.

We state here some restrictions on the values of μ that we will consider:

$${ϵ}^{3/4}<\mu \le (1/32){\delta }^2$$

(19)

Notice that, since we will set $\mu ={ϵ}^{1/4}$, we will require $ϵ\le 2^{-20}{\delta }^8$.

We know that $f,g\in A(\delta )$, and we also know that $\parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }<{\delta }^2/16$ by (14) and (19). This means that we can apply the results of Section 9. to f and g, and with Lemmas 13 and 14 we see that we can also apply the results of section 10..

With this in mind, by Lemma 15 let us suppose that $f^{\prime }=0$ at $t_1^{*}<\cdots <t_k^{*}$ and $g^{\prime }=0$ at $s_1^{*}<\cdots <s_k^{*}$. Next, we want to construct a region covering the critical points of f and g, and extending beyond the critical points far enough to cover the entire area where $f^{\prime }$ or $g^{\prime }$ is “small” (according to μ).

Definition 20. 

For $i=1,\dots ,k$, let $J_i^{*}=[\min \{t_i^{*},s_i^{*}\}-4\mu /\delta ,\max \{t_i^{*},s_i^{*}\}+4\mu /\delta ]$.

We see that the critical points of f and g are contained in the $J_i^{*}$. Furthermore, suppose that $|f^{\prime }(t_0)|\le \mu \parallel f^{(3)}{\parallel }_{\infty }$. By Lemma 9, we see that the distance from $t_0$ to a critical point of f is no more than $4\mu /\delta$, which means that $t_0$ is contained in some $J_i^{*}$. The same is true for g, and therefore we have

$$\left\{t:|f^{\prime }(t)|\le \mu \parallel f^{(3)}{\parallel }_{\infty }\text{or}|g^{\prime }(t)|\le \mu \parallel g^{(3)}{\parallel }_{\infty }\right\}\subset \bigcup _{i=1}^k{J}_i^{*}$$

(20)

For purposes of the expression $H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)$, the “bad” set will also include the endpoints of the interval, so we define the sets $B^{*}$ and $G^{*}$ as follows.

Definition 21. 

$$\begin{array}{cc}\hfill B^{*}& =\{0\}\cup \{1\}\cup \bigcup _{i=1}^k{J}_i^{*}\hfill \\ \hfill G^{*}& =G_{f,g}(B^{*})\hfill \end{array}$$

Notice that $G^{*}\cap B^{*}=\varnothing$, but we do not necessarily have $G^{*}\cup B^{*}=[0,1]$.

We are now ready to state how we will decompose the expression $\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)$ in order to estimate it. First, recall by equation (7) that

$$\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)={\int }_t\log |f^{\prime }(t)|-\log |g^{\prime }(t)|-\left({\int }_{t}H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)\right)$$

We will divide $[0,1]$ in different ways to estimate the different integrals:

$$\begin{array}{cc}& \text{For}{\int }_t\log |f^{\prime }(t)|-\log |g^{\prime }(t)|\text{we}\text{write}[0,1]=\bigcup J_i^{*}\cup {\left(\bigcup J_i^{*}\right)}^c\hfill \\ & \text{For}{\int }_{t}H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)\text{we}\text{write}[0,1]=G^{*}\cup {\left(G^{*}\right)}^c\hfill \end{array}$$

This allows us to decompose our main expression, $\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)$, into the following four pieces:

Definition 22. 

$$\begin{array}{cc}& (LB)={\int }_{\bigcup J_i^{*}}\log |f^{\prime }(t)|-\log |g^{\prime }(t)|\hfill \\ & (LG)={\int }_{{\left(\bigcup J_i^{*}\right)}^c}\log |f^{\prime }(t)|-\log |g^{\prime }(t)|\hfill \\ & (EB)={\int }_{{\left(G^{*}\right)}^c}H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)\hfill \\ & (EG)={\int }_{G^{*}}H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)\hfill \end{array}$$

Next, we have four propositions, each estimating one of the above expressions in a certain setting. Each proposition will be applicable to the functions f and g, and the sets $J_i^{*},B^{*}$ and $G^{*}$, but it will be helpful to state and prove the propositions using only the relevant information in each case.

In order to not interrupt our exposition, we have placed the proofs of these propositions in the appendix.

Proposition 23 

(Used to bound $(LB)$). Suppose that $h,l\in A(\delta )$ and

$$\parallel h^{\prime }-l^{\prime }{\parallel }_{\infty }<\left({\delta }^2/16\right)\min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}$$

By Lemma 15, let their critical points be $t_1<\cdots <t_k$ and $s_1<\cdots <s_k$ respectively. Let

$$J_i=[\min \{t_i,s_i\}-a,\max \{t_i,s_i\}+a]$$

and suppose that we know $\forall i,|t_i-s_i|+a\le \delta /4$. Then

$$\left|{\int }_{\bigcup J_i}\log |h^{\prime }(t)|-\log |l^{\prime }(t)|\right|\le \sum _i\left[(|t_i-s_i|+2a)\frac{\parallel h^{″}-l^{″}{\parallel }_{\infty }}{\delta \min \parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }}+(8/\delta )(|t_i-s_i{|+2a)}^2\right]$$

Proposition 24 

(Used to bound $(LG)$). Suppose that h and l have three continuous derivatives on $[0,1]$. Let

$$X=\left\{t:|h^{\prime }(t)|\ge \mu \parallel h^{(3)}{\parallel }_{\infty }and|l^{\prime }(t)|\ge \mu \parallel l^{(3)}{\parallel }_{\infty }\right\}$$

Then

$$\left|{\int }_X\log |h^{\prime }(t)|-\log |l^{\prime }(t)|\right|\le \frac{\parallel h^{\prime }-l^{\prime }{\parallel }_{\infty }}{\mu \min \parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }}$$

This follows immediately from Lemma 27.

Proposition 25 

(Used to bound $(EB)$). Suppose that $h,l$ have three continuous derivatives and have finite numbers, $k_h$ and $k_l$, of critical points respectively, and $h^{\prime }(0),l^{\prime }(0),h^{\prime }(1),l^{\prime }(1)\ne 0$. Suppose that we have closed intervals ${\{L_{\alpha }\}}_{\alpha \in \mathcal{A}}$ and we define $B={\bigcup }_{\alpha \in \mathcal{A}}{L}_{\alpha }$ and $G=G_{h,l}(B)$ as in Definition 16. Then

$$|G^c|\le (k_h+k_l+2)\left({4|\mathcal{A}|\parallel h-l\parallel }_{\infty }+\sum _{\alpha }\left(\underset{L_{\alpha }}{\max }|h^{\prime }|,|l^{\prime }|\right)|L_{\alpha }|\right){\left(\underset{B^c}{\min }|h^{\prime }|,|l^{\prime }|\right)}^{-1}+|B|$$

Proposition 26 

(Used to bound $(EG)$). Suppose that $h,l\in E(\tau )$ and $h\sim l$ in $E(\tau )$. By Lemma 15, let their critical points be $t_1<\cdots <t_k$ and $s_1<\cdots <s_k$ respectively. Suppose we have a set B such that

$$B\supset \{0\}\cup \{1\}\cup \bigcup _i\langle t_i,s_i\rangle$$

and define $G=G_{h,l}(B)$ as in Definition 16. Then

$$\begin{array}{cc}& \left|{\int }_{G}H\left({\overrightarrow{p}}_h\left(h(t)\right)\right)-H\left({\overrightarrow{p}}_l\left(l(t)\right)\right)\right|\hfill \\ & \le (k+1)(k+2)\left[1+\left|\log \frac{{\min }_{B^c}|h^{\prime }|,|l^{\prime }|}{(k+1)\max \parallel h^{\prime }{\parallel }_{\infty },{\parallel l^{\prime }\parallel }_{\infty }}\right|\right](\max \parallel h^{\prime }{\parallel }_{\infty },{\parallel l^{\prime }\parallel }_{\infty }){\left(\underset{B^c}{\min }|h^{\prime }|,|l^{\prime }|\right)}^{-2}\hfill \\ & \times \left\{\parallel h^{\prime }-l^{\prime }{\parallel }_{\infty }+2{\parallel h-l\parallel }_{\infty }{\left(\underset{B^c}{\min }|h^{\prime }|,|l^{\prime }|\right)}^{-1}\frac{\parallel h^{″}{\parallel }_{\infty }+{\parallel l^{″}\parallel }_{\infty }}{2}\right\}\hfill \end{array}$$

Now, we apply these propositions to f and g. We will abbreviate

$${\Gamma }_1=\frac{\max \parallel f^{\prime }{\parallel }_{\infty },{\parallel g^{\prime }\parallel }_{\infty }}{\min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}},{\Gamma }_2=\frac{\max \parallel f^{″}{\parallel }_{\infty },{\parallel g^{″}\parallel }_{\infty }}{\min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}}$$

The assumptions of Theorem 1 mean that

$$C_1^{-1}\le {\Gamma }_1\le C_1,{\Gamma }_2\le C_2$$

and without loss of generality we may assume $C_1,C_2\ge 3$.

We know that $\parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }<{\delta }^2/16$ by (14) and (19), and therefore by Lemma 10 and equation (14) we have

$$\forall i,|t_i^{*}-s_i^{*}|\le 4{ϵ}^{3/4}/\delta$$

(21)

To apply Proposition 23 (with $a=4\mu /\delta$), we must have

$$4{ϵ}^{3/4}/\delta +4\mu /\delta \le \delta /4$$

and this is implied by (19). So, with some simplifying to arrive at the second equation, we have

$$\begin{array}{cc}\hfill |(LB)|& \le \sum _i\left[(|t_i-s_i|+2·4\mu /\delta )\frac{\parallel f^{″}-g^{″}{\parallel }_{\infty }}{\delta \min \parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }}+(8/\delta )(|t_i-s_i{|+2·4\mu /\delta )}^2\right]\hfill \\ & \le k(4{ϵ}^{3/4}+8\mu ){\delta }^{-2}\left[\frac{\parallel f^{″}-g^{″}{\parallel }_{\infty }}{\min \parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }}+(8/\delta )(4{ϵ}^{3/4}+8\mu )\right]\hfill \\ & \le 12k\mu /{\delta }^2\left[{\Gamma }_2+96\mu /\delta \right]\hfill \\ & \le 12k{C}_2·97\mu /{\delta }^2\hfill \end{array}$$

(22)

Next, looking to Proposition 24, recalling equation (20) we have

$${\left(\bigcup _{i=1}^k{J}_i^{*}\right)}^c\subset \left\{t:|f^{\prime }(t)|\ge \mu \parallel f^{(3)}{\parallel }_{\infty }\text{and}|g^{\prime }(t)|\ge \mu \parallel g^{(3)}{\parallel }_{\infty }\right\}$$

and so we can apply Proposition 24 to f and g, which gives

$$\begin{array}{cc}\hfill |(LG)|& \le \frac{\parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }}{\mu \min \parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }}\hfill \\ & \le {ϵ}^{3/4}/\mu \hfill \end{array}$$

(23)

Looking to Proposition 25, we have $B^{*}=\{0\}\cup \{1\}\cup {\bigcup }_{i=1}^k{J}_i^{*}=\bigcup L_{\alpha }$. Equation (20) means that

$$B^{*}\supset \left\{t:|h^{\prime }(t)|\le \mu \parallel h^{(3)}{\parallel }_{\infty }\text{or}|l^{\prime }(t)|\le \mu \parallel l^{(3)}{\parallel }_{\infty }\right\}$$

which means

$${\left(\underset{{(B^{*})}^c}{\min }|h^{\prime }|,|l^{\prime }|\right)}^{-1}\le {\left[\mu \min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}\right]}^{-1}$$

In the notation of Proposition 25, $k_f,k_g=k$ and $|\mathcal{A}|=k+2$, and aside from $\{0\}$ and $\{1\}$ which have measure zero, $|J_i|\le (4{ϵ}^{3/4}/\delta )+4\mu /\delta \le 8\mu /\delta$. Also, note that

$$(\underset{J_i}{\max }|h^{\prime }|,|l^{\prime }|)\le (\max \parallel f^{″}{\parallel }_{\infty },{\parallel g^{″}\parallel }_{\infty })·|J_i|\le 8\mu /\delta (\max \parallel f^{″}{\parallel }_{\infty },{\parallel g^{″}\parallel }_{\infty })$$

which means

$$\begin{array}{cc}\hfill |G^c|& \le (k_h+k_l+2)\left({4|\mathcal{A}|\parallel h-l\parallel }_{\infty }+\sum _{\alpha }\left(\underset{L_{\alpha }}{\max }|h^{\prime }|,|l^{\prime }|\right)|L_{\alpha }|\right){\left(\underset{B^c}{\min }|h^{\prime }|,|l^{\prime }|\right)}^{-1}+|B|\hfill \\ & \le 2(k+1)\frac{4(k+2)ϵ\min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}+k{(8\mu /\delta )}^2\max \parallel f^{″}{\parallel }_{\infty },{\parallel g^{″}\parallel }_{\infty }}{\mu \min \left\{\parallel h^{(3)}{\parallel }_{\infty },{\parallel l^{(3)}\parallel }_{\infty }\right\}}+k(8\mu /\delta )\hfill \\ & \le 8{(k+2)}^2ϵ/\mu +2^{7}k(k+1)\mu /{\delta }^2\left({\Gamma }_2\right)+8k\mu /\delta \hfill \\ & \le 2^7{(k+2)}^2{C}_2(ϵ/\mu +\mu /{\delta }^2+\mu /\delta )\hfill \\ & \le 2^8{(k+2)}^2{C}_2(ϵ/\mu +\mu /\delta )\hfill \end{array}$$

This means

$$(EB)\le 2^8{(k+2)}^2{C}_2(ϵ/\mu +\mu /\delta )\log (k+1)$$

(24)

Looking to Proposition 26, by Lemma 14 and Definition 21 we can apply Proposition 26 to f and g, so we have

$$\begin{array}{cc}& \left|{\int }_{G^{*}}H\left({\overrightarrow{p}}_f\left(f(t)\right)\right)-H\left({\overrightarrow{p}}_g\left(g(t)\right)\right)\right|\hfill \\ & \le (k+1)(k+2)\left[1+\left|\log \frac{{\min }_{B^c}|f^{\prime }|,|g^{\prime }|}{(k+1)\max \parallel f^{\prime }{\parallel }_{\infty },{\parallel g^{\prime }\parallel }_{\infty }}\right|\right](\max \parallel f^{\prime }{\parallel }_{\infty },{\parallel g^{\prime }\parallel }_{\infty }){\left(\underset{B^c}{\min }|f^{\prime }|,|g^{\prime }|\right)}^{-2}\hfill \\ & \times \left\{\parallel f^{\prime }-g^{\prime }{\parallel }_{\infty }+2{\parallel f-g\parallel }_{\infty }{\left(\underset{B^c}{\min }|f^{\prime }|,|g^{\prime }|\right)}^{-1}\frac{\parallel f^{″}{\parallel }_{\infty }+{\parallel g^{″}\parallel }_{\infty }}{2}\right\}\hfill \\ & \le {(k+2)}^2\left[1+\left|\log \frac{\mu \min \left\{\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }\right\}}{(k+1)\max \parallel f^{\prime }{\parallel }_{\infty },{\parallel g^{\prime }\parallel }_{\infty }}\right|\right](\max \parallel f^{\prime }{\parallel }_{\infty },{\parallel g^{\prime }\parallel }_{\infty }){\left(\mu \min \left\{\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }\right\}\right)}^{-2}\hfill \\ & \times \left\{{ϵ}^{3/4}\min \left\{\parallel f^{(3)}{\parallel }_{\infty },{\parallel g^{(3)}\parallel }_{\infty }\right\}+2ϵ\max \parallel f^{″}{\parallel }_{\infty },{\parallel g^{″}\parallel }_{\infty }\right\}\hfill \\ & \le {(k+2)}^2{\Gamma }_1\left({ϵ}^{3/4}/{\mu }^2+2ϵ{\Gamma }_2\right)\left[1+\left|\log \frac{\mu }{(k+1){\Gamma }_1}\right|\right]\hfill \\ & \le 6C_1{C}_2{(k+2)}^2\log (C_1(k+1){\mu }^{-1}){ϵ}^{3/4}/{\mu }^2\hfill \end{array}$$

(25)

Simplifying the logarithm term in the last line was the reason for assuming $C_1\ge 3$.

Combining equations (22), (23), (24), (25), and letting C denote an absolute constant, we have

$$|\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)|\le C{C}_1{C}_2{(k+2)}^2\log (C_1(k+1){\mu }^{-1})\left[\mu /{\delta }^2+{ϵ}^{3/4}/\mu +ϵ/\mu +\mu /\delta +{ϵ}^{3/4}/{\mu }^2\right]$$

Examining just the last expression in the product, again letting C be an absolute constant, we have

$$\mu /{\delta }^2+{ϵ}^{3/4}/\mu +ϵ/\mu +\mu /\delta +{ϵ}^{3/4}/{\mu }^2\le C(\mu /{\delta }^2+{ϵ}^{3/4}/{\mu }^2)$$

Treating δ as a fixed parameter, this expression is optimized if $\mu ={ϵ}^{1/4}$, and using Lemma 8 we see that $k\le 2/\delta$. Therefore, with C being another absolute constant, we have

$$|\mathtt{\text{h}}(f)-\mathtt{\text{h}}(g)|\le C{C}_1{C}_2\log (C_1{\delta }^{-1}{ϵ}^{-1/4}){ϵ}^{1/4}/{\delta }^4$$