Entropy vs. Energy Waveform Processing: A Comparison Based on the Heat Equation

Abstract

Virtually all modern imaging devices collect electromagnetic or acoustic waves and use the energy carried by these waves to determine pixel values to create what is basically an “energy” picture. However, waves also carry “information”, as quantified by some form of entropy, and this may also be used to produce an “information” image. Numerous published studies have demonstrated the advantages of entropy, or “information imaging”, over conventional methods. The most sensitive information measure appears to be the joint entropy of the collected wave and a reference signal. The sensitivity of repeated experimental observations of a slowly-changing quantity may be defined as the mean variation (i.e., observed change) divided by mean variance (i.e., noise). Wiener integration permits computation of the required mean values and variances as solutions to the heat equation, permitting estimation of their relative magnitudes. There always exists a reference, such that joint entropy has larger variation and smaller variance than the corresponding quantities for signal energy, matching observations of several studies. Moreover, a general prescription for finding an “optimal” reference for the joint entropy emerges, which also has been validated in several studies.

1. Introduction

All applications of imaging technology begin with collection of a flux, either electromagnetic [1] or acoustic [2], and nearly all images produced by current technology, e.g., for remote sensing or medical imaging, are representations based on some type of scattered energy. The reasons for this are mainly historical, since the detectors are typically developed by physicists and electrical engineers who are intimately acquainted with the wave equation and transmission line theory where conservation of energy is a central concept. On a more practical level, transduction elements are frequently characterized in terms of energy conversion efficiencies, making it natural to think of the subsequent image formation process in terms of either electromagnetic or acoustic field energy. From this perspective, if the received energy arriving during some small interval of time must be reduced to a single number in order to compute an image pixel value, energy is the obvious choice.

For instance, in the field of medical ultrasonics, tumor detection and tissue classification with ultrasound remain a highly useful and clinically relevant approach (liver, kidney, prostate, breast, heart, eye, thyroid, pancreas, gall bladder, etc.). A number of teams, including those of Insana [3–5], Forsberg [6,7], Deng [8], and others, have pursued novel data acquisition and reduction schemes (spectral, cepstral, wavelet, elastographic, harmonic, etc.) to augment diagnostic power from traditional radio frequency data, and progress continues apace. Nevertheless, although hardware improvements in clinical imaging systems over the last 50 years have dramatically improved the ability of ultrasound to display tissue features, resident signal processing algorithms have not evolved much beyond fundamental presentation of the energy of backscattered compressional waves.

Since all detectors have a finite response time, their output is essentially an integral of the incoming flux that they measure taken over a very short time interval. As such, it is natural to ask if other integrals or functionals of the incident flux might also have utility if represented as images [9–14].

In fact, numerous experimental studies have demonstrated the utility of information theoretic quantities for this type of analysis of experimentally-measured waveforms. In the standard application of information theory, as initiated by Shannon, the random variables (i.e., waveforms, which we will denote f and g defined on [0, 1]) are assumed to have the same underlying distribution [15]. However, additional assumptions, such as differentiability, permit computation of distributions, both individual and joint, directly from the measured waveforms. Operationally, the ability to differentiate experimentally-measured waveforms, which contain noise, requires regularization. For our studies, this is accomplished using optimal smoothing splines [16]. Differentiability combined with regularization then permits computation of the distributions for individual waveforms, or pairs of waveforms in the case of joint distributions. Information-theoretic quantities may then be computed from these distributions in an approach that corresponds more closely to that initiated by Kolmogorov and Chaitan, where entropy is a measure of the intrinsic complexity of individual mathematical objects [17,18]. The underlying assumption in experimental applications of these quantities is that the waveforms, usually acquired in scattering measurements, faithfully capture the complexity of the interrogated scattering architectures. The merit of this strategy has been demonstrated in several experimental studies, which have investigated the sensitivities of several entropies for the detection of small changes in waveforms acquired in acoustic experiments [19–24].

To date, the most sensitive of these is a joint entropy of two waveforms, one acquired in an acoustic backscatter (i.e., an echo) measurement from an experimental specimen, f(t), and the other a reference, g(t), which may be obtained separately by experiment or by theoretical modeling [25]. To compute this entropy for differentiable waveforms f(t), g(t), we must calculate their joint distribution w_f,g, which is not a function, but a tempered distribution [25]. Thus, calculations based on w_f,g require “coarse-graining” on a uniform grid of C × C squares covering the x, y-plane to obtain a discrete joint probability distribution p^C(j, k) from w_f,g by integrating its product with smoothed versions of the characteristic functions of these squares (Equations 40 to 44 of [25]). This is followed by a limiting process where the grid size, C, is taken to zero. For instance, the calculation of the joint entropy begins with the following computation:

$$H_{f,g}=\underset{C\to 0}{\lim }{H}^C\left(f,g\right)=\underset{C\to 0}{\lim }{\displaystyle \sum _{j,k}{p}^C\left(j,k\right)}\log \left[p^C\left(j,k\right)\right]$$

(1)

In the limit where C → 0 and f, g are piecewise differentiable functions on [0,1] (without any intervals of constancy), Equation (1) becomes [25]:

$$H_{f,g}=-\frac{1}{2}{\displaystyle {\int }_0^1\frac{\min \left[|f^{\prime }\left(t\right)|,|g^{\prime }\left(t\right)|\right]}{\max \left[|f^{\prime }\left(t\right)|,|g^{\prime }\left(t\right)|\right]}}dt-{\displaystyle {\int }_0^1\log }\left[\max \left[|f^{\prime }\left(t\right)|,|g^{\prime }\left(t\right)|\right]\right]dt$$

(2)

In this note we determine conditions on g(t) that maximize the sensitivity of H_f,g to small changes in f(t). This requires a lengthy calculation.

We will be comparing this sensitivity to that of the signal energy, E_f, of f(t) defined by:

$$E_f={\displaystyle {\int }_0^{1}f{\left(t\right)}^{2}dt}$$

(3)

Numerous studies have shown that entropy signal receivers are always at least as sensitive as E_f is to small changes in f (see Equation (5)). In fact, in many cases, it is actually much more sensitive [19–25].

A typical result for materials characterization is shown in Figure 1 [26], which shows images of a graphite/epoxy composite laminate scanned using a 2.5-MHz transducer on a 101 × 101 point grid. The backscattered ultrasound was digitized for off-line analysis. The peak-to-peak image was produced using the peak-to-peak amplitudes of the waveforms. The rest were produced using a moving window (128 points long) analysis to produce a stack of images corresponding to different depths whose minima were then projected onto a single image-plane to permit rapid analysis of the entire image set. This projection scheme is frequently used to reduce the amount of data that must be inspected or in the case where the defect is not confined to a narrow range of depths. The E_f image was produced using Equation (3). The H_f [20], I_f,∞ [27], H_f,g ([25] or Equation (2)) and H_f,gO are entropy images.

Similar results have also been obtained in medical imaging [27–38].

2. The Main Result

All of our studies have been based on the (fairly typical) situation where an experimentalist acquires waveforms of a few microseconds duration over a much greater period of time spanning minutes or longer. Thus, there are two time scales: the long experimental time scale and a much shorter measurement time scale (which we have parametrized on the interval [0, 1], the domain of f(t) and g(t)). If we denote a measurement at time t by M_t, then the sensitivity is defined by [39]:

$$c:=\frac{\text{mean}\left(\left\{{\mathrm{M}}_i-{\mathrm{M}}_0:i=1,\dots ,n\right\}\right)}{\text{standard}\text{error}\left(\left\{{\mathrm{\Delta }}_i:i=1,\dots ,n\right\}\right)}$$

(4)

It is essentially the noise-normalized change in a physical quantity at different measurement times. The situation also describes measurements where data (i.e., f(t), g(t)) are acquired at different times between which the measurement device has been moved; for instance, Figure 1, where the experimental times map to different spatial locations in the experimental specimen.

For evaluation of receiver sensitivities in theoretical studies, we will maximize the similar quantity:

$$\frac{\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle }{\sqrt{\text{Var}\left[\delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\right]}}$$

(5)

where the product σς(t) captures the impact of Gaussian noise on the experimental measurement (ς(t) is a functional of a Brownian path (see Equation (15)) and σ is a scalar representing the signal-to-noise ratio). On the other hand, the perturbing function εη(t), where ε is small and η ∈ C¹[0, 1], models the effect of the variation of the scattering architecture, as captured in Figure 1. Together, these account for the specific form of the measured function, which we write as (f + εη + σς) (t); all of these functions will be discussed fully in the next section. The experimental change referred to prior to Equation (4) is then quantified mathematically by the directional derivative:

$${\delta H_{f+\sigma \varsigma ,g}\left[\eta \right]:=\frac{d{H}_{f+\epsilon \eta +\sigma \varsigma ,g}}{d\epsilon }|}_{\epsilon =0}$$

(6)

The measure of noise is:

$$\text{Var}\left[\delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\right]:=\langle {\left(\delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\right)}^2\rangle -{\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle }^2$$

(7)

where all of the means are computed as Wiener integrals as discussed below. Similar definitions hold for the signal energy E_f.

After carefully-defining the quantities, f(t), η(t), σ, ς(t), in terms of a physical measurement model, having finite time resolution width Δ ≪ 1, we calculate the variations δH_f+σς,g[η] and δE_f+σς[η] in Section 4. The mean values of these are then calculated in Section 5 as Wiener integrals. The joint entropy is considered first in Section 4.1. After deriving the mean variation as a Wiener integral, we rewrite it in Section 5.1.1 as a Lebesgue integral. This integral represents a family of solutions to the heat equation with initial conditions defined on the real line. However, the noise-level plays the role usually held by time.

The results of the calculations may be summarized as:

Theorem 1. Suppose that f is in C³[0, 1] and there exists t₀ ∈ (0, 1), such that f′(t₀) = 0 and f″(t₀) ≠ 0. Then, there exists C₁ > 0, such that for all C₂ > 0, there exists a piecewise differentiable function g, independent of η, such that:

$$\underset{\sigma \to 0}{\lim }|\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle |\ge C_2|{\eta }^{\prime }\left(t_0\right)|-C_1{\Vert {\eta }^{\prime }\Vert }_{\infty }$$

In particular, provided η′(t₀) ≠ 0, one can make$\underset{\sigma \to 0}{\lim }\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle$ arbitrarily large.

In nearly all practical situations, such a t₀ exists. We prove this theorem by constructing an example reference at the end of Section 5.1.1.

There are actually many such g(t). A good way to construct examples is, roughly, to pick a zero, t₀, of f′(t) and have g′(t) be a non-zero constant on one side of t₀, while on the other side, it differs from f′(t) by only a small constant 1 ≫ ω > 0. In the limit as ω → 0, |δH_f+σς,g[η]| > 2|η′ (t₀)| log[1/ω]/f″(t₀). Exact details are provided in the proof, where a bound on the size of g(t) is also provided.

On the other hand, the variation for the signal energy, provided by Equation (81), is calculated in Section 4.2. If f and η are bounded, then in the noise-free limit, it is (see Equation (51)):

$$\delta E_{f+\sigma \varsigma }\left[\eta \right]=2{\displaystyle {\int }_0^{1}f\left(t\right)}\eta \left(t\right)dt$$

(8)

which is bounded.

Thus, if η′(t₀) ≠ 0, there exists g, such that:

$$\underset{\sigma \to 0}{\lim }|\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle |\ge \underset{\sigma \to 0}{\lim }|\langle \delta H_{f+\sigma \varsigma }\left[\eta \right]\rangle |$$

(9)

There are many theorems describing the behavior of solutions to the heat equation for small time discussed in Section 5.1.2. These results are used in Section 6 to prove:

Theorem 2. Suppose K₀ > 0 is a constant, f, η are in C¹[0, 1] and g is piecewise C¹[0, 1]. Let 0 < α < 1/6. Suppose that, except at finitely many points, |g′(t)| ≠|f′(t)| and |g′(t)| ≥ K₀Δ^α.

• Then, for Δ small, there exists a constant K₁ that depends on ║η′║_∞, but not on g or σ, such that:

  $$Var\left[\delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\right]\le K_1{\mathrm{\Delta }}^{1-6\alpha }{\sigma }^2+O\left[\mathrm{\Delta }{\sigma }^2+{\sigma }^4\right]$$

  (10)
• There exists K₂ > 0, that depends on ║η║_∞, but not on Δ or σ, such that:

  $$Var\left[\delta E_{f+\sigma \varsigma }\left[\eta \right]\right]=K_2{\mathrm{\Delta }}^{1/2}{\sigma }^2+O\left[{\sigma }^3\right]$$

  (11)

Consequently, given the freedom to choose g(t), the sensitivity of joint entropy analysis can always be made greater than that of signal energy analysis. For instance, choosing α = 1/16 and ω = Δ^1/32 and setting g′(t) = f′ (t) − ω for t < t₀ and g′(t) = 1 for t > t₀ produces a reference that satisfies the requirement Δ^1/32 ∼ |g′(t)| ≫ Δ^α ∼ Δ^1/16 and leads to δH_f+σς,g[η] ∼ η′(t₀) log[Δ]/(4f″ (t₀)); so that the confidence for H_f,g is O[σ⁻¹Δ^−5/16 log[Δ]], whereas the signal energy will have confidence $O\left[\sqrt{\sigma }{\mathrm{\Delta }}^{-1/4}\right]$, since δE_f+σς[η] = O[1].

3. Approach

3.1. The Physical Setup

We assume that we have an experimental system that is measured by some means, e.g., interrogated by waves, π(t), of some sort, such as microwave or ultrasonic radiation, and that we collect some portion of the waves scattered in this system to obtain both perturbed, f_pert.(t), and unperturbed functions, f_unpert.(t), for times normalized by the appropriate choice of units to lie between zero and one. The difference |f_unpert.(t) − f_pert.(t)| is assumed to be small and to be caused physically by a change in the system as measured at different spatial locations (as in Figure 1) and/or different times if the system is evolving. We also assume that there is Gaussian distributed noise in the system and that the noise functions for all measurements are represented by Brownian paths, which we will always denote by x(t) or, when there are multiple Brownian paths that must be distinguished, by x_i(t). Moreover, we assume that these functions, which all have a variance of one, are scaled by an experimentally-determined signal-to-noise ratio, σ, which will typically have positive values much less than one-tenth.

If the system is linear, with initial impulse-response function Λ(t) and final perturbed impulse-response function Λ(t) + ελ(t), these functions may be represented mathematically as:

$$\begin{array}{l}{f}_{\text{unpert}.}\left(t\right)={\displaystyle {\int }_0^1\mathrm{\Lambda }\left(t-t_1\right)}\pi \left(t_1\right)d{t}_1+\sigma x_1\left(t\right)\\ f_{\text{pert}.}\left(t\right)={\displaystyle {\int }_0^1[\mathrm{\Lambda }(t-t_1)}+\epsilon \lambda (t-t_1)]\pi (t_1)d{t}_1+\sigma x_2\left(t\right)\end{array}$$

(12)

If the system is non-linear, these equations become approximate representations of f_unpert. (t) and f_pert. (t) in the case where the perturbation to the system is small.

All experimental systems have finite time resolution. In addition, an experimentalist may deliberately signal average successive measurements to cancel noise. Both of these facts may be expressed using convolution of ideal experimental measurements, such those in Equation (12), with a finite measurement function, µ_Δ(t) of limited time duration, i.e., supp(µ_Δ)⊊ [0, 1]. This requires us to replace Equation (12) by:

$$\begin{array}{l}{f}_{\text{unpert}.}\left(t\right)={\displaystyle {\int }_0^1{\mu }_{\mathrm{\Delta }}}\left(t-t_2\right)\left[{\displaystyle {\int }_0^1\mathrm{\Lambda }\left(t_2-t_1\right)}\pi \left(t_1\right)d{t}_1+\sigma x_1\left(t_2\right)\right]d{t}_2\\ =f\left(t\right)+\sigma \varsigma \left(t\right)\end{array}$$

(13)

and:

$$\begin{array}{l}{f}_{\text{pert}.}\left(t\right)={\displaystyle {\int }_0^1{\mu }_{\mathrm{\Delta }}}\left(t-t_2\right)\left[{\displaystyle {\int }_0^1[\mathrm{\Lambda }(t_2-t_1)}+\epsilon \lambda (t_2-t_1)]\pi (t_1)d{t}_1+\sigma x_2\left(t_2\right)\right]d{t}_2\\ f_{\text{pert}.}\left(t\right)=f(t)+\epsilon \eta (t)+\sigma \varsigma (t)\end{array}$$

(14)

where,

$$\begin{array}{l}f(t)={\displaystyle {\int }_0^1{\displaystyle {\int }_0^1{\mu }_{\mathrm{\Delta }}\left(t-t_2\right)}\mathrm{\Lambda }(t_2-t_1)\pi (t_1)d{t}_{1}d{t}_2}\\ \eta (t)={\displaystyle {\int }_0^1{\mu }_{\mathrm{\Delta }}\left(t-t_2\right)}{\displaystyle {\int }_0^1\lambda (t_2-t_1)\pi (t_1)d{t}_{1}d{t}_2}\\ \varsigma (t)={\displaystyle {\int }_0^1{\mu }_{\mathrm{\Delta }}\left(t-t_2\right)}{x}_2(t_2)d{t}_2={\displaystyle {\int }_0^1{x}_2(t_2)d{M}_t}(t_2)\end{array}$$

(15)

with:

$$\frac{d}{d{t}_2}{M}_t(t_2)={\mu }_{\mathrm{\Delta }}(t-t_2)$$

(16)

Physically, f(t) is the “well-behaved”, i.e., noise-free and smooth, part of the linear response of the experimental system (as described by Λ(t)) to the probing “waveform”, π(t₁), while η(t) is the noise-free and smooth change in the system, resulting from the perturbation of the system, as represented by the function λ(t).

We will additionally assume that,

$${\Vert {\eta }^{\prime }\Vert }_{\infty }<\infty$$

(17)

for later use.

Later, we will also need ς′(t):

$$\begin{array}{l}{\varsigma }^{\prime }(t)={\displaystyle {\int }_0^1\frac{d}{dt}{\mu }_{\mathrm{\Delta }}}(t-t_2)x_2(t_2)d{t}_2,\\ =-{\displaystyle {\int }_0^1{x}_2(t_2)d{m}_t}(t_2)\end{array}$$

(18)

with:

$$m_t(t_2):={\mu }_{\mathrm{\Delta }}(t-t_2)$$

(19)

We will also assume that µ_Δ(t) is non-negative; typically, a unit step function of width less than one whose derivatives we will treat in the sense of distributions. We have from Equation (14):

$${f^{\prime }}_{\text{pert}.}(t)=f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)$$

(20)

which we will need for the evaluation of Equation (2).

3.2. Characteristics of the Measurement Window µ_Δ(t) Relevant to the Variation and Variance of H_{f+εη+σς,g}

Up to this point, we have not specified the experimental window function µ_Δ(t). While many different choices for this function are possible, the most common is a unit height step function of width much smaller that the total time of the experimental measurement, which we have scaled to be one. Experimental conditions usually provide for at least sixteen sample points in this window. This is motivated by the experimental goal of selecting digitizer equipment and settings so that the experimental window function is very small compared to the scale over which the input waveform exhibits significant change. Consequently, for typical measurements, we have values of ║m_t║ ≪ 1/16. To make these comments more precise, we will define:

$${\mu }_{\mathrm{\Delta }}(t)=\frac{1}{\sqrt{\mathrm{\Delta }}}{\chi }_{(0,\mathrm{\Delta }]}(t){\chi }_{[0,1]}(t)$$

(21)

where χ_(0,Δ](t) is the characteristic function of (0, Δ]. This function has finite total variation and m_t(1) = µ_Δ(t − 1) = 0, for all t ∈ [0, 1], which we must always have for technical reasons (see the Appendix on Wiener integrals Equations (128) and (146)). Later, we will also need, for the calculation of the variance of joint entropy and its variation, the relation (jointly continuous in s and t):

$$\langle m_{t,}{m}_s\rangle ={\displaystyle {\int }_0^1{m}_t(t_2)}{m}_s(t_2)d{t}_2=\{\begin{array}{ll}0,\hfill & \mathrm{if}|t-s|\ge \mathrm{\Delta }\hfill \\ \frac{\mathrm{\Delta }-|t-s|}{\mathrm{\Delta }}\hfill & \mathrm{if}|t-s|<\mathrm{\Delta }\text{and}\{t,s\}\not\subset [1-\mathrm{\Delta },1]\hfill \\ \frac{1-\max (s,t)}{\mathrm{\Delta }},\hfill & \mathrm{if}|t-s|<\mathrm{\Delta }\text{and}\{t,s\}\subset [1-\mathrm{\Delta },1]\hfill \end{array}$$

(22)

and using ║m_t║ to denote the L²-norm on [0, 1]:

$$\Vert m_t\Vert =\{\begin{array}{ll}1,\hfill & \mathrm{if}t\in [0,1-\mathrm{\Delta }]\hfill \\ \sqrt{\frac{1-t}{\mathrm{\Delta }}},\hfill & \mathrm{if}t\in [1-\mathrm{\Delta },1]\hfill \end{array}$$

(23)

3.3. Characteristics of the Measurement Window µ_Δ(t) Relevant to the Variation and Variance of E_f+εη+σς

If we define:

$$M_t(t_2)={\displaystyle {\int }_{t_2}^1{\mu }_{\mathrm{\Delta }}}(t-s)ds=\{\begin{array}{ll}\sqrt{\mathrm{\Delta }},\hfill & \mathrm{if}{t}_2<t-\mathrm{\Delta }\hfill \\ -\frac{1}{\sqrt{\mathrm{\Delta }}}{t}_2+\frac{1}{\sqrt{\mathrm{\Delta }}}t,\hfill & \mathrm{if}t-\mathrm{\Delta }<t_2<t\hfill \\ 0,\hfill & \mathrm{if}t<t_2\hfill \end{array}$$

(24)

and:

$$\Vert M_t\Vert =\sqrt{t\mathrm{\Delta }-\frac{2}{3}{\mathrm{\Delta }}^2}$$

(25)

Then, the last term in Equation (13) may be written as:

$$\varsigma (t)={\displaystyle {\int }_0^1{\mu }_{\mathrm{\Delta }}}(t-t_2)x_1(t_2)d{t}_2={\displaystyle {\int }_0^1{x}_1(t_2)d{M}_t(t_2)}$$

(26)

We also note for later use:

$$\langle M_{t_1,}{M}_{t_2}\rangle ={\displaystyle {\int }_0^1{M}_{t_1}}(s)M_{t_2}(s)ds=\min (t_1,t_2)\mathrm{\Delta }+O[{\mathrm{\Delta }}^2]$$

(27)

In particular, if 0 ≤ t₁, t₂ ≤ 1, then we have the order relation:

$$\frac{\langle M_{t_1,}{M}_{t_2}\rangle }{\Vert M_{t_1}\Vert }=\frac{\min (t_1,t_2)}{\sqrt{t_1}}\sqrt{\mathrm{\Delta }}+O[{\mathrm{\Delta }}^2]$$

(28)

4. Calculation of the Variations of Joint Entropy and Signal Energy

4.1. Calculation of the Variation, δH_f+σς,g(η)

We now calculate the average change in Equation (2) when f(t) is perturbed by the function εη(t) + σς(t) as shown in Equation (20). The calculation is broken into two parts.

4.1.1. The First Term

To begin with, we will focus on the first term in Equation (2). Using the terms defined above this is:

$$-\frac{1}{2}\left[{\displaystyle {\int }_0^1\frac{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|}{\max \left[|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|,|g^{\prime }(t)|\right]}dt-1}\right]$$

(29)

Let A(ε) denote the set of points where $|\mathrm{g}\prime \left(\mathrm{t}\right)|>|{f^{\prime }}_{\text{pert}}.\left(t\right)|$. Similarly, denote the corresponding set of points where $|{f^{\prime }}_{\text{pert}.}(t)|>|g^{\prime }(t)|$. by B(ε). We will additionally assume that the set of critical points of f(t) and g(t) are disjoint, so that χ_A(ε)(t) ≠ 0 ⇒ g′(t) ≠ 0 and χ_B(ε)(t) ≠ 0 ⇒ f′(t) ≠ 0. The indicator functions may be expressed in terms of Heaviside functions, H(t), as:

$$\begin{array}{l}{\chi }_{A(\epsilon )}(t)=H(|g^{\prime }(t)|-|{f^{\prime }}_{\text{pert}.}(t)|)\\ {\chi }_{B(\epsilon )}(t)=H(|{f^{\prime }}_{\text{pert}.}(t)|-|g^{\prime }(t)|)\end{array}$$

(30)

Using these conventions, the first term in Equation (29) becomes:

$$-\frac{1}{2}{\displaystyle {\int }_{A(\epsilon )}\frac{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|}{|g^{\prime }(t)|}}dt-\frac{1}{2}{\displaystyle {\int }_{B(\epsilon )}\frac{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|}{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}}dt+\frac{1}{2}$$

(31)

Now,

$$\frac{d|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}{d\epsilon }={\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))$$

(32)

where we have used the operational relation for Dirac delta functions: xδ(x) = 0. In addition, if f′(t) + εη′(t) + σς′(t) ≠ 0 (which is true if t ∈ B(ε), since the set of critical points of f(t) and g(t) are disjoint), we similarly have:

$$\frac{d}{d\epsilon }\frac{1}{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}=\frac{{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}{{(f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}^2}$$

(33)

Using these relations, we differentiate Equation (31) with respect to ε as the first step in obtaining the variation, V_I, of the first term:

$$\begin{array}{l}-\frac{1}{2}{\displaystyle {\int }_{A(\epsilon )}\frac{{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}{|g^{\prime }(t)|}}dt-\frac{1}{2}{\displaystyle {\int }_{B(\epsilon )}\frac{{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}dt}\\ -\frac{1}{2}{\displaystyle {\int }_{B(\epsilon )}(|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|)}\left[-\frac{{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}{{(f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}^2}\right]dt+\\ {-\frac{1}{2}\frac{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|}{|g^{\prime }(t)|}|}_{\partial A(\epsilon )}-\frac{1}{2}{\frac{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|}{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}|}_{\partial B(\epsilon )}\end{array}$$

(34)

Now, we let ε → 0, so that Expression (34) becomes the variation for V_I:

$$\begin{array}{l}{V}_I=-\frac{1}{2}{\displaystyle {\int }_{A(0)}\frac{{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}{|g^{\prime }(t)|}}dt-\frac{1}{2}{\displaystyle {\int }_{B(0)}\frac{{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}{|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}}dt\\ -\frac{1}{2}{\displaystyle {\int }_{B(0)}(|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|)}\left[-\frac{{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}{{(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}^2}\right]dt+\\ -\frac{1}{2}{\frac{|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|}{|g^{\prime }(t)|}|}_{\partial A(0)}-\frac{1}{2}{\frac{|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|}{|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}|}_{\partial B(0)}\end{array}$$

(35)

Now, it is always possible to pick a reference g(t), so that at ∂A(0) and ∂B(0), we have:

$${-\frac{1}{2}\frac{|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(t)|}{|g^{\prime }(t)|}|}_{\partial A(0)}{-\frac{1}{2}\frac{|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|+|g^{\prime }(g)|}{|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}|}_{\partial B(0)}\ll 1$$

(36)

since the boundary points will either come in canceling pairs, with a contribution of ±1 at each of the boundary points (top panel of Figure 2); or there will be exact cancellations at all boundary points, but one, and the contribution from this point may be made arbitrarily small (bottom panel of Figure 2). The examples shown in Figure 2 have been chosen also to match the description of an “optimal” reference given in the Introduction after Theorem 4.

Thus, V_I (Equation (35)) becomes:

$$\frac{1}{2}{\displaystyle {\int }_0^1{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}\left[-{\chi }_A(t)\frac{1}{|g^{\prime }(t)|}+{\chi }_B(t)\frac{|g^{\prime }(t)|}{{(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}^2}\right]dt$$

(37)

where we have written χ_A(t) and χ_B(t) for χ_A(0)(t) and χ_B(0)(t), respectively. Finally, we rewrite Equation (30):

$$\begin{array}{l}{\chi }_{A(\epsilon )}(t)=H\left(|g^{\prime }(t)|-|{f^{\prime }}_{\text{pert}.}(t)|\right)\\ {\chi }_{B(\epsilon )}(t)=H\left(|{f^{\prime }}_{\text{pert}.}(t)|-|g^{\prime }(t)|\right)\end{array}$$

in terms of f′(t), g′(t), η′(t) and ς′(t), using Equation (20), and set ε = 0 to find:

$$\begin{array}{l}{\chi }_A(t)=H\left(|g^{\prime }(t)|-|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|\right)\\ {\chi }_B(t)=H\left(|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|-|g^{\prime }(t)|\right)\end{array}$$

(38)

This permits us to use Expression (37) to write:

$$\begin{array}{l}{V}_1=\frac{1}{2}{\displaystyle {\int }_0^1{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))\times }\\ \times [-H(|g^{\prime }(t)|-|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|)\frac{1}{|g^{\prime }(t)|}+H(|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|-|g^{\prime }(t)|)\frac{|g^{\prime }(t)|}{{(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}^2}]dt\end{array}$$

(39)

4.1.2. Logarithmic Term

Similarly, V_II, the variation for the logarithmic part of Equation (2), is obtained by first computing the derivative:

$$-\frac{d}{d\epsilon }{\displaystyle {\int }_0^1\log [\max \left[|{f^{\prime }}_{\text{pert}}(t)|,|g^{\prime }\left(t\right)|\right]]dt}$$

(40)

As in the case for the first term, the variations at ∂A(ε) and ∂B(ε) cancel in pairs, since |f′(t) + εη′(t) + σς′(t)| = |g′(t)| at the boundary points; hence, the derivative is:

$$-{\displaystyle {\int }_{B(\epsilon )}\frac{1}{|f^{\prime }(t)+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)}+\epsilon {\eta }^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))dt$$

(41)

Now, we set ε = 0 to obtain the variation, V_II, for the logarithmic part:

$$-{\displaystyle {\int }_0^{1}H(|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|-|g^{\prime }(t)|)\frac{1}{|f^{\prime }(t)+\sigma {\varsigma }^{\prime }|(t)|}{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }}(t)+\sigma {\varsigma }^{\prime }(t))dt$$

(42)

4.1.3. Total Variation

The total variation, V, is now:

$$\begin{array}{l}\delta H_{f+\sigma \varsigma ,g}[\eta ]=V_1+V_{11}\\ =\frac{1}{2}{\displaystyle {\int }_0^1{\eta }^{\prime }(t)\mathrm{sgn}(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}[-\frac{H(|g^{\prime }(t)|-|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|)}{|g^{\prime }(t)|}+\frac{H(|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|-|g^{\prime }(t)|)}{{(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t))}^2}|g^{\prime }(t)|-2\frac{H(|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|-|g^{\prime }(t)|)}{|f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)|}]dt\\ ={\displaystyle {\int }_0^1{\eta }^{\prime }(t)G(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)},|g^{\prime }(t)|)dt\end{array}$$

(43)

where:

$$F(u,v):=\mathrm{sgn}(u)\left[-\frac{H|v|-|u|}{2|v|}+\frac{H|v|-|u|}{2u^2}|v|-\frac{H|u|-|v|}{|u|}\right]$$

(44)

is continuous in u for fixed v and continuous in v, except for a discontinuity across v = 0. Figure 3 contains a plot of a typical G(u, v). Moreover, for fixed v, G(u, v) is an odd function of u, a fact that will be significant later.

For later use, we record the alternate forms of G(u, v):

$$\begin{array}{l}G(u,v)=-\frac{H(|v|-|u|)+H(|v|-|u|)-2H(u)}{2|u|}\\ -H(-u-|v|)\left(\frac{1}{u}+\frac{|v|}{2}\frac{1}{u^2}\right)-H(u-|v|)\left(\frac{1}{u}-\frac{|v|}{2}\frac{1}{u^2}\right)\end{array}$$

(45)

and:

$$G(u,v)=\{\begin{array}{ll}-\frac{1}{u}-\frac{|v|}{2u^2},\hfill & \mathrm{if}u\le -|v|\le 0\hfill \\ \frac{1}{2|v|},\hfill & \mathrm{if}-|v|\le u\le 0\hfill \\ -\frac{1}{2|v|},\hfill & \mathrm{if}0\le u\le |v|\hfill \\ -\frac{1}{u}+\frac{|v|}{2u^2},\hfill & \mathrm{if}0\le |v|\le u\hfill \end{array}$$

(46)

We also record for later use the first and second partial derivatives of G(u, v) with respect to u,

$$\frac{\partial }{\partial u}G(u,v)=G(u,v)=\{\begin{array}{ll}\frac{1}{u^2}-\frac{|v|}{u^3},\hfill & \mathrm{if}u\le -|v|\le 0\hfill \\ 0,\hfill & \mathrm{if}-|v|\le u\le 0\hfill \\ 0,\hfill & \mathrm{if}0\le u\le |v|\hfill \\ \frac{1}{u^2}-\frac{|v|}{u^3},\hfill & \mathrm{if}0\le |v|\le u\hfill \end{array}$$

(47)

which is continuous in u and, therefore, yields upon further differentiation with respect to u the distribution-free second partial derivative,

$$\frac{{\partial }^2}{\partial u^2}G(u,v)=\{\begin{array}{ll}-\frac{2}{u^3}-\frac{3|v|}{u^4},\hfill & \mathrm{if}u\le -|v|\le 0\hfill \\ 0,\hfill & \mathrm{if}-|v|\le u\le 0\hfill \\ 0,\hfill & \mathrm{if}0\le u\le |v|\hfill \\ -\frac{1}{u^3}-\frac{3|v|}{u^4},\hfill & \mathrm{if}0\le |v|\le u\hfill \end{array}$$

(48)

4.2. Calculation of the Variation, δE_f+σς(η), for E_f

We now wish to characterize the signal receiver for energy, which is given by Equation (3). The perturbed signal receiver value is:

$$E_{f+\epsilon \eta +\sigma \varsigma }(\epsilon )={\displaystyle {\int }_0^1{[f(t)+\epsilon \eta (t)+\sigma \varsigma (t)]}^{2}dt}$$

(49)

Following the usual steps, we differentiate with respect to ε:

$$\frac{d{E}_{f+\epsilon \eta +\sigma \varsigma }(\epsilon )}{d\epsilon }=2{\displaystyle {\int }_0^1[f(t)+\epsilon \eta (t)+\sigma \varsigma (t)]\eta (t)dt.}$$

(50)

The variation δE_f+σς[η] is now given by:

$$\delta E_{f+\sigma \varsigma }[\eta ]=\underset{\epsilon \to 0}{\lim }\frac{d{E}_{f+\epsilon \eta +\sigma \varsigma }(\epsilon )}{d\epsilon }=2{\displaystyle {\int }_0^1[f(t)+\sigma \varsigma (t)]\eta (t)dt.}$$

(51)

5. Calculation of the Average Variations of Joint Entropy and Signal Energy

5.1. Calculation of the Average Variation, ⟨δH_f+σς,g(η)⟩, by Wiener Integration

We now compute the average variation or expectation value of the variation over the space of noise functions, i.e., we may average Equation (43) over the Brownian paths. This is obtained from the Wiener integral of $\frac{\delta H_{f,g}}{\delta f(t)}$ and is:

$$\begin{array}{l}\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle :={\displaystyle {\int }_{C_0[0,1]}\delta H_{f+\sigma \varsigma ,g}\left[\eta \right]d_{W}x}\\ ={\displaystyle {\int }_{C_0[0,1]}{\displaystyle {\int }_0^1{\eta }^{\prime }(t)G(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t)},|g^{\prime }(t)|])dt{d}_{W}x}\\ ={\displaystyle {\int }_{C_0[0,1]}{\displaystyle {\int }_0^1{\eta }^{\prime }(t)G\left(f^{\prime }(t)-\sigma {\displaystyle {\int }_0^{1}x(t_2)d{m}_t(t_2),|g^{\prime }(t)|}\right)}}dt{d}_{W}x\end{array}$$

(52)

The function G(u, v) is continuous in u for all v; hence (Theorem 29.6 on page 443 of [40]), we can interchange the order of evaluation of integration, so:

$$\begin{array}{l}{\displaystyle {\int }_{C_0[0,1]}\delta H_{f+\sigma \varsigma ,g}[\eta ]d_{W}x}\\ ={\displaystyle {\int }_0^1{\eta }^{\prime }(t)\times {\displaystyle {\int }_{C_0[0,1]}G\left(f^{\prime }(t)-\sigma {\displaystyle {\int }_0^{1}x(t_2)d{m}_t(t_2),|g^{\prime }(t)|}\right)}}{d}_{W}xdt\end{array}$$

(53)

From Equation (128) in the Appendix, this is equal to the Lebesgue integral:

$$\begin{array}{l}{\displaystyle {\int }_{C_0[0,1]}\delta H_{f+\sigma \varsigma ,g}[\eta ]d_{W}x}\\ ={\displaystyle {\int }_0^1\frac{1}{\sqrt{2\pi }}}{\eta }^{\prime }(t){\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{u^2}{2}}}G(f^{\prime }(t)-u\sigma \Vert m_t\Vert ,|g^{\prime }(t)|)dudt\end{array}$$

(54)

where:

$$\Vert m_t\Vert :=\sqrt{{\displaystyle {\int }_0^1{m}_t{(t_2)}^{2}d{t}_2}}$$

(55)

Finally,

$$\begin{array}{l}\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle ={\displaystyle {\int }_{C_0[0,1]}\frac{\delta H_{f,g}}{\delta f(t)}{d}_W}\\ =\frac{1}{\sqrt{\pi }}{\displaystyle {\int }_0^1{\eta }^{\prime }(t)}{\displaystyle {\int }_{\infty }^{-\infty }{e}^{-\frac{{(z-f^{\prime }(t))}^2}{4s}}}G\left(z,|g^{\prime }(t)|\right)\frac{-1}{\sqrt{4s}}dzdt\end{array}$$

(56)

where the last integral is obtained by the change of variables:

$$z=f^{\prime }(t)-(u/\sqrt{2})\sqrt{4s},\text{where}\Vert m_t\Vert \sigma =\sqrt{2s}$$

(57)

This can be further simplified as:

$$\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle ={\displaystyle {\int }_0^1{\eta }^{\prime }(t)}{\displaystyle {\int }_{-\infty }^{\infty }\left[\frac{1}{\sqrt{4\pi s}}{e}^{-\frac{{(z-f^{\prime }(t))}^2}{4s}}\right]}G\left(z,|g^{\prime }(t)|\right)dzdt$$

(58)

where, based on the discussion above, typical experimental conditions will lead to σ being on the order of 10⁻⁶.

5.1.1. The Average Variation, ⟨δH_f+σς,g[η]⟩, and the Heat Equation

We observe that the bracketed term in Equation (58) is the Green’s function for the heat equation [41,42], defined on ℝ, i.e.,

$$\left[\frac{1}{\sqrt{4\pi s}}{e}^{-\frac{x^2}{4s}}\right]$$

(59)

Thus, we have:

$$\begin{array}{l}\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle ={\displaystyle {\int }_0^1{\eta }^{\prime }(t)}{\displaystyle {\int }_{-\infty }^{\infty }\left[\frac{1}{\sqrt{4\pi s}}{e}^{-\frac{{(z-f^{\prime }(t))}^2}{4s}}\right]}G\left(z,|g^{\prime }(t)|\right)dzdt,\\ ={\displaystyle {\int }_0^1{\eta }^{\prime }(t)}{u}_{|g^{\prime }(t)|}(f^{\prime }(t),s)\end{array}$$

(60)

where:

$$u_{|g^{\prime }(t)|}(f^{\prime }(t),s):={\displaystyle {\int }_{-\infty }^{\infty }\left[\frac{1}{\sqrt{4\pi s}}{e}^{-\frac{{(z-f^{\prime }(t))}^2}{4s}}\right]}G\left(z,|g^{\prime }(t)|\right)dz$$

(61)

is a point on one member of a family of heat surfaces that are determined by the heat equations,

$$\frac{{\partial }^2}{\partial x^2}{u}_{|g^{\prime }(t)|}\left(x,s\right)-\frac{\partial }{\partial s}{u}_{|g^{\prime }(t)|}\left(x,s\right)=0$$

(62)

with different (in fact, |g′(t)|-dependent) “initial conditions” (i.e., at s = 0) defined by G (z, |g′(t)|). A typical example is shown in Figure 4.

Since for small noise, s, the heat kernel is approximated well by a Dirac delta function, we might expect that in the limit s → 0⁺:

$$u_{|g^{\prime }(t)|}\left(f^{\prime }(t),s\right)\sim G(f^{\prime }(t),|g^{\prime }(t)|)$$

(63)

in which case, Equation (60) becomes:

$$\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle \sim {\displaystyle {\int }_0^1{\eta }^{\prime }(t)G(f^{\prime }(t)},|g^{\prime }(t)|)dt$$

(64)

which we will shortly show is unbounded for certain choices of g(t). A good way to construct such g(t) is to pick a zero of f′(t) and have g′(t) differ only slightly from f′(t) on one side of the zero and differ greatly from it on the other side. Referring back to Equation (78), if we assume that |g′(t)| ≩ 0, then G(z, |g′(t)|) ∈ L² [ℝ], and there are a large number of theorems describing the behavior of u_|g′(t)|(x, s). In particular, a limiting form of Theorem 5 in Chapter 5 on page 67 of [42], where the vertical sides are pushed out to ±∞, guarantees that as s → 0, u_|g′(t)|(z, s) → G(z, |g′(t)|).

5.1.2. The Structure of the Solution-Surfaces Associated with ⟨δH_f+σς,g[η]⟩

Figure 4 also illustrates the useful inequality:

$$|G\left(z,|g^{\prime }\left(t\right)|\right)|\le \frac{1}{2|g^{\prime }\left(t\right)|}$$

(65)

With this result, we are now ready to prove:

Theorem 3. Suppose that f is in C³[0, 1] and there exists t₀ ∈ (0, 1), such that f′(t₀) = 0 and f^′′(t₀) ≠ 0. Then, there exists C₁ > 0, such that for all C₂ > 0, there exists a piecewise differentiable function g, independent of η, such that:

$$\underset{\sigma \to 0}{\lim }|\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle |\ge C_2|{\eta }^{\prime }\left(t_0\right)|-C_1{\Vert {\eta }^{\prime }\Vert }_{\infty }$$

(66)

In particular, provided η′(t₀) ≠ 0, one can make$\underset{\sigma \to 0}{\lim }\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle$ arbitrarily large.

Proof. We shall consider the case f″(t₀) > 0; the other case is similar.

Choose ξ > 0 and small enough that f′∈ C²[0, 1] and strictly increasing on [t₀ − ξ, t₀ + ξ], so that:

$$|f^{\prime }\left(t_0+h\right)-h{f}^{″}\left(t_0\right)|<\frac{f^{″}(t_0)}{2}|h|,\text{for}-\xi \le h\le \xi$$

(67)

We define:

$$g(t)=\{\begin{array}{ll}f(t)-\omega t,\hfill & \mathrm{if}{t}_0-\xi <t<t_0\hfill \\ f(t_0)-\omega t_0+t-t_0,\hfill & \mathrm{if}{t}_0\le t<t_0+\xi \hfill \end{array}$$

(68)

where ω > 0 is chosen small.

We extend g to be piecewise C³[0, 1] off(t₀ − ξ, t₀ + ξ) and such that g′(t) ≥ 1 for all t ∈ [0, 1]\[t₀ − ξ, t₀ + ξ), except possibly finitely many points where it does not exist. The exact details of the extension do not matter, since contributions from the extension will be negligible compared to those from (t₀ − ξ, t₀ + ξ), as long as we do not choose a reference g(t) that does not violate the conditions discussed after Equation (36). Figure 2 illustrates how the extension might be chosen in one case.

We break the integral in Equation (64) into an integral over [t₀ − ξ, t₀ + ξ] and its compliment. The latter piece is bounded by:

$$C_1{\Vert {\eta }^{\prime }\Vert }_{\infty }$$

(69)

where:

$$C_1:=\sup \left\{G\left(u,v\right):v\ge 1\right\}=1$$

(70)

The integral over [t₀ − ξ, t₀ + ξ] becomes:

$$\begin{array}{l}{\eta }^{\prime }(t_0){\displaystyle {\int }_{t_0-\xi }^{t_0}G\left(f^{\prime }(t),|g^{\prime }(t)|\right)}dt+{\eta }^{\prime }(t_0){\displaystyle {\int }_{t_0}^{t_0+\xi }G(f^{\prime }(t),|g^{\prime }(t)|)dt}\\ ={\eta }^{\prime }(t_0){\displaystyle {\int }_{t_{0-\xi }}^{t_0}\frac{1}{f^{\prime }(t)-\omega }dt-{\eta }^{\prime }(t_0){\displaystyle {\int }_{t_0}^{t_0+\xi }dt}}\\ ={\eta }^{\prime }(t_0){\displaystyle {\int }_{t_0-\xi }^{t_0}\frac{1}{f^{\prime }(t)-\omega }dt-\xi {\eta }^{\prime }(t_0)}\end{array}$$

(71)

The term ξη′(t₀) may be incorporated into C₁║η′║_∞ by increasing C₁ slightly. By Equation (67), we have:

$$\begin{array}{l}|{\eta }^{\prime }\left(t_0\right)||{\displaystyle {\int }_{t_{0-\xi }}^{t_0}\frac{1}{f^{\prime }\left(t\right)-\omega }}|\ge |{\eta }^{\prime }\left(t_0\right)|{\displaystyle {\int }_{t_0-\xi }^{t_0}\frac{1}{\left(t_0-t\right)\frac{f^{″}\left(t_0\right)}{2}+\omega }dt}\\ =|{\eta }^{\prime }\left(t_0\right)|\frac{2}{f^{″}\left(t_0\right)}\log \left[1+\frac{\xi f^{″}\left(t_0\right)}{2\omega }\right]\\ \sim |{\eta }^{\prime }\left(t_0\right)|\frac{2}{f^{″}\left(t_0\right)}\log \frac{1}{\omega }\mathrm{as}\omega \to 0\end{array}$$

(72)

If we define:

$$C_2:=\frac{2}{f^{″}\left(t_0\right)}\log \frac{1}{\omega }$$

(73)

then by Equation (64):

$$\underset{\sigma \to 0}{\lim }\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle \ge C_2|{\eta }^{\prime }\left(t_0\right)|-C_1{\Vert {\eta }^{\prime }\Vert }_{\infty }$$

(74)

Finally, g does not depend on η. This completes the proof. □

We note, however, that in order to keep the variance of the variation small, so that we maintain high sensitivity (Equation (5)), we must keep $\omega \gg \sqrt{\mathrm{\Delta }}$ as indicated in Equation (109). This provides a practical bound on the magnitude of the variation.

We also record for later use the calculation of the maximum magnitudes of the first and second partial derivatives of G(u, v) with respect to u as:

$$\frac{\partial G\left(z,|g^{\prime }\left(t\right)|\right)}{\partial z}\le \frac{4}{27}\frac{1}{|g^{\prime }(t){|}^2}$$

(75)

and:

$$\frac{{\partial }^{2}G\left(z,|g^{\prime }\left(t\right)|\right)}{\partial z^2}\le \frac{1}{|g^{\prime }(t){|}^3}$$

(76)

Given the structure of the integral in Equation (61), we see that u_|g′(t)|(f′(t), s) → 0 as s → ∞. Moreover,

$$\mathrm{sgn}(u_{|g^{\prime }(t)|}(f^{\prime }(t),s))=-\mathrm{sgn}(f^{\prime }(t))$$

(77)

A later analysis of the signal energy E_f will reveal a different “initial condition” (these “initial conditions” refer to the noise-free case), which characterizes that receiver.

The structure of G (z, |g′(t)|) is more clearly seen in Equation (46), which we recall:

$$G(z,|g^{\prime }(t)|)=\{\begin{array}{ll}-\frac{1}{z}-\frac{|g^{\prime }(t)|}{2z^2},\hfill & \mathrm{if}z\le -|g^{\prime }(t)|\le 0\hfill \\ \frac{1}{2|g^{\prime }(t)|},\hfill & \mathrm{if}-|g^{\prime }(t)|\le z\le 0\hfill \\ -\frac{1}{2|g^{\prime }(t)|},\hfill & \mathrm{if}0\le z\le |g^{\prime }(t)|\hfill \\ -\frac{1}{2}+\frac{|g^{\prime }(t)|}{2z^2}\hfill & \mathrm{if}|g^{\prime }(t)|\le z\hfill \end{array}$$

(78)

We note that G (z, |g′(t)|) is determined by the mathematical form of the signal receiver, in this case H_f,g, as well as the reference waveform g′(t).

As shown in Figure 4, the supremum of the initial conditions may be made arbitrarily large by making g′(t) smaller. The effect of this change is reflected in the structure of the solution (heat) surface, as shown in Figure 5. The figure also shows that for all z and s:

$$|u_{|g^{\prime }(t)|}\left(z,s\right)|\le \left|\frac{1}{2g^{\prime }(t)}\right|$$

(79)

which is consistent with the maximum property for solutions to the heat equation (discussed in Chapter 2, Section 3 of [42]).

5.2. Calculation of the Average Variation ⟨δE_f+σς(η)⟩

From Equation (51), the average variation is the Wiener integral:

$$\langle \delta E_{f+\sigma \varsigma }\left[\eta \right]\rangle =2{\displaystyle {\int }_{C_0[0,1]}{\displaystyle {\int }_0^1\left[f(t)+\sigma \varsigma \left(t\right)\right]}\eta \left(t\right)}dt{d}_{W}x$$

(80)

which becomes:

$$\begin{array}{l}\langle \delta E_{f+\sigma \varsigma }\left[\eta \right]\rangle =2\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_{-\infty }^{\infty }{\displaystyle {\int }_0^1\left[f(t)+\sigma \Vert M_t\Vert u\right]}}{e}^{-\frac{u^2}{2}}\eta \left(t\right)dtdz\\ =2\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_{-\infty }^{\infty }{\displaystyle {\int }_0^{1}f(t)\eta (t)e^{-\frac{u^2}{2}}}}dtdz\\ =2{\displaystyle {\int }_0^{1}f(t)\eta (t)dt}\end{array}$$

(81)

which has no dependence on the noise level σ and, thus, no heat surface.

We also observe that the integral in Equation (81) is the variation for the signal energy receiver in the noise-free analysis.

6. The Variances of the Variations

In this section, we perform the calculations required to prove:

Theorem 4. Suppose K₀ > 0 is a constant, f, η are in C¹[0, 1] and g is piecewise C¹[0, 1]. Let 0 < α < 1/6. Suppose that, except at finitely many points, |g′(t)| ≠ |f′(t)| and |g′(t)| ≥ K₀Δ^α.

• Then, there exists a constant K₁ that depends on ║η′║_∞, but not on g or σ, such that:

  $$Var[\delta H_{f+\sigma \varsigma ,g}[\eta ]]\le K_1{\mathrm{\Delta }}^{1-6\alpha }{\sigma }^2+O[\mathrm{\Delta }{\sigma }^2+{\sigma }^4]$$

  (82)
• There exists K₂ > 0, that depends on ║η║_∞, but not on Δ or σ, such that:

$$Var[\delta H_{f+\sigma \varsigma }[\eta ]]=K_2{\mathrm{\Delta }}^{1/2}{\sigma }^2+O[{\mathrm{\Delta }}^{5/2}{\sigma }^2]$$

(83)

We break the proof into two parts.

Proof. Part I: Calculation of the variance of δH_f+σς,g[η]

$$\begin{array}{l}Var[\delta H_{f+\sigma \varsigma ,g}[\eta ]]={\displaystyle {\int }_{C_0[0,1]}{\left[\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle -\delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\right]}^2{d}_{W}x}\\ ={\displaystyle {\int }_{C_0[0,1]}{[\delta H_{f+\sigma \varsigma ,g}[\eta ]]}^2{d}_{W}x-{\langle \delta H_{f+\sigma \varsigma ,g}\left[\eta \right]\rangle }^2}\end{array}$$

(84)

Recalling Equation (43), the Wiener integral on the right-hand side of Equation (84) becomes:

$${\displaystyle {\int }_{C_0[0,1]}{\left[{\displaystyle {\int }_0^1{\eta }^{\prime }(t)G(f^{\prime }(t)+\sigma {\varsigma }^{\prime }(t),|g^{\prime }(t)|)dt}\right]}^2{d}_{W}x}$$

(85)

Expanding the square as a double integral (and using Equation (18) for ς′(t) and Equation (19) for m_t), the Wiener integral may be written as:

$$\begin{array}{l}{\displaystyle {\int }_0^1{\displaystyle {\int }_0^1{\eta }^{\prime }(t_1){\eta }^{\prime }(t_2)}}\times {\displaystyle {\int }_{C_0[0,1]}G\left(f^{\prime }(t_1)-\sigma {\displaystyle {\int }_0^{1}x\left(t_3\right)d{m}_{t_1}}(t_3),|g^{\prime }(t_1)|\right)}\\ \times G\left(f^{\prime }(t_2)-\sigma {\displaystyle {\int }_0^{1}x(t_3)d{m}_{t_2}}(t_3),|g^{\prime }(t_2)|\right)d_{W}xd{t}_{1}d{t}_2\end{array}$$

(86)

where, since G(u, v) is jointly continuous in u, v, as before, we can interchange the order of integrations. The inner Wiener integral may be replaced by a Lebesgue integral, using Equation (146). It becomes:

$$\begin{array}{l}\frac{1}{2\pi }{\displaystyle {\int }_{-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{u_1^2}{2}-\frac{u_2^2}{2}}}G(f^{\prime }(t_1)}-\sigma \Vert m_{t_1}\Vert u_1,|g^{\prime }(t_1)|)\\ \times G\left(f^{\prime }\left(t_2\right)-\sigma \left(u_1\langle m_{t_1},m_{t_2}\rangle +u_2{n}_2(t_1,t_2\right)\right),|g^{\prime }\left(t_2\right)|)d{u}_{1}d{u}_2\end{array}$$

(87)

where (from Equation (143)):

$$n_2(t_1,t_2)=\sqrt{\langle m_{t_2},m_{t_2}\rangle -\frac{{\langle m_{t_1},m_{t_2}\rangle }^2}{\langle m_{t_1},m_{t_1}\rangle }}$$

(88)

We also point out for later use that calculation of the maximum of n₂(t₁, t₂)² shows:

$$n_2{\left(t_1,t_2\right)}^2\le 1$$

(89)

Changing variables using:

$$\begin{array}{l}{z}_1=f^{\prime }\left(t_1\right)-\sqrt{2s_1}{u}_1,\text{where}\sigma \Vert m_{t_1}\Vert =\sqrt{2s_1}\\ z_2=f^{\prime }\left(t_2\right)-\sqrt{2s_2}{u}_2,\text{where}\sigma n_2\left(t_1,t_2\right)=\sqrt{2s_2}\end{array}$$

(90)

permits the integral in Expression (87) to be rewritten as:

$$\begin{array}{l}{\displaystyle {\int }_{-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }\frac{1}{\sqrt{4\pi s_1}}\frac{1}{\sqrt{4\pi s_2}}{e}^{-\frac{{(z_1-f^{\prime }(t_1))}^2}{4s_1}-\frac{{(z_2-f^{\prime }(t_2))}^2}{4s_2}}}}\\ \times G(z_1,|g^{\prime }(t_1)|)G\left(z_2+(z_1-f^{\prime }(t_1))\frac{\langle m_{t_1},m_{t_2}\rangle }{\Vert m_{t_1}\Vert },|g^{\prime }(t_2)|\right)d{z}_{1}d{z}_2\end{array}$$

(91)

Substituting this into Equation (84), we obtain:

$$\begin{array}{l}\text{Var}[\delta H_{f+\sigma \varsigma ,g}[\eta ]]\\ ={\displaystyle {\int }_0^1{\displaystyle {\int }_0^1{\eta }^{\prime }\left(t_1\right){\eta }^{\prime }\left(t_2\right)}}\\ \times {\displaystyle {\int }_{-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }\frac{1}{\sqrt{4\pi s_1}}\frac{1}{\sqrt{4\pi s_2}}{e}^{-\frac{{(z_1-f^{\prime }(t_1))}^2}{4s_1}-\frac{{(z_2-f^{\prime }(t_2))}^2}{4s_2}}}}\\ \times G(z_1,|g^{\prime }(t_1)|)G\left(z_2+z_1-f^{\prime }(t_1))\frac{\langle m_{t_1},m_{t_2}\rangle }{\Vert m_{t_1}\Vert },|g^{\prime }(t_2)|\right)\\ \times d{z}_{1}d{z}_{2}d{t}_{1}d{t}_2\\ -{\left[{\displaystyle {\int }_0^1{\eta }^{\prime }(t){\displaystyle {\int }_{-\infty }^{\infty }\left[\frac{1}{\sqrt{4\pi s}}{e}^{-\frac{{(z-f^{\prime }(t))}^2}{4s}}\right]}}G(z,|g^{\prime }(t)|)dzdt\right]}^2\end{array}$$

(92)

As a double check of the result, we observe that as s₁, s₂→0⁺, the heat-kernels in the first and second terms approach Dirac delta functions centered at f′(t), f′(t₁) and f′(t₂), i.e., δ(z−f′(t)), δ(z₁− f′(t₁)) and δ(z₂− f′(t₂)), respectively, and:

$$\underset{s_1,s_2\to 0^{+}}{\lim }\text{Var}[\delta H_{f+\sigma \varsigma ,g}[\eta ]]=0$$

(93)

as expected.

We now rewrite the right-hand side of Equation (92) in terms of solutions of the heat equation, as was done following Equation (60), and find that Equation (92) equals:

$${\displaystyle {\int }_0^{1}d{t}_1{\eta }^{\prime }(t_1)}{{\displaystyle {\int }_0^{1}d{t}_2{\eta }^{\prime }(t_2)\mathrm{\Lambda }(t_1,t_2,s_1,s_2)-\left[{\displaystyle {\int }_0^{1}d{t}_1{\eta }^{\prime }(t)u_{|g^{\prime }(t)|}}(f^{\prime }(t),s)\right]}}^2$$

(94)

where:

$$\begin{array}{l}\mathrm{\Lambda }\left(t_1,t_2,s_1,s_2\right):={\displaystyle {\int }_{-\infty }^{\infty }d{z}_1\frac{1}{\sqrt{4\pi s_1}}}{e}^{-\frac{{(z_1-f^{\prime }(t_1))}^2}{4s_1}}G\left(z_1,|g^{\prime }(t_1)|\right)\\ \times {\displaystyle {\int }_{-\infty }^{\infty }d{z}_2}\frac{1}{\sqrt{4\pi s_2}}{e}^{-\frac{{(z_2-f^{\prime }(t_2))}^2}{4s_2}}\\ \times G\left(z_2+(z_1-f^{\prime }(t_1))\frac{\langle m_{t_1},m_{t_2}\rangle }{\Vert m_{t_1}\Vert },|g^{\prime }(t_2)|\right)\\ ={\displaystyle {\int }_{-\infty }^{\infty }d{z}_1}\frac{1}{\sqrt{4\pi s_1}}{e}^{-\frac{{(z_1-f^{\prime }(t_1))}^2}{4s_1}}G\left(z_1,|g^{\prime }(t_1)|\right)\\ \times u_{|g^{\prime }(t_2)|}\left(f^{\prime }(t_2)+(z_1-f^{\prime }(t_1))\frac{\langle m_{t_1},m_{t_2}\rangle }{\Vert m_{t_1}\Vert },s_2\right)\end{array}$$

(95)

where we have used the change of variables:

$$\xi =z_2+(z_1-f^{\prime }(t_1))\frac{\langle m_{t_1},m_{t_2}\rangle }{\Vert m_{t_1}\Vert }$$

(96)

and Equation (61) to go from the first to the second equations.

We now observe that, by Equation (22), $\langle m_{t_1},m_{t_2}\rangle =0$ when |t₁ − t₂| > Δ, in which case:

$$\begin{array}{l}\mathrm{\Lambda }\left(t_1,t_2,s_1,s_2\right)={\displaystyle {\int }_{-\infty }^{\infty }d{z}_1\frac{1}{\sqrt{4\pi s_1}}}{e}^{-\frac{{(z_1-f^{\prime }(t_1))}^2}{4s_1}}G\left(z_1,|g^{\prime }(t_1)|\right)\\ \times {\displaystyle {\int }_{-\infty }^{\infty }d{z}_2}\frac{1}{\sqrt{4\pi s_2}}{e}^{-\frac{{(z_2-f^{\prime }(t_2))}^2}{4s_2}}G(z_2,|g^{\prime }(t_2)|)\\ =u_{|g^{\prime }(t_1)|}(f^{\prime }(t_1),s_1)u_{|g^{\prime }(t_2)|}(f^{\prime }(t_2),s_2)\end{array}$$

(97)

and so, Equation (94) may be written as:

$$\begin{array}{l}\text{Var}[\delta H_{f+\sigma \varsigma ,g}[\eta ]]\\ ={\displaystyle {\int }_0^{1}d{t}_1{\eta }^{\prime }(t_1)}{\displaystyle {\int }_0^{1}d{t}_2{\eta }^{\prime }(t_2)u_{|g^{\prime }(t_1)|}}(f^{\prime }(t_1),s_1)u_{|g^{\prime }(t_2)|}(f^{\prime }(t_2),s_2)\\ -{\displaystyle {\int }_0^{1}d{t}_1{\eta }^{\prime }(t_1)}{\displaystyle {\int }_{t_1-\mathrm{\Delta }}^{t_1+\mathrm{\Delta }}d{t}_2{\eta }^{\prime }(t_2)[u_{|g^{\prime }(t_1)|}}(f^{\prime }(t_1),s_1)u_{|g^{\prime }(t_2)|}(f^{\prime }(t_2),s_2)]\\ +{\displaystyle {\int }_0^{1}d{t}_1{\eta }^{\prime }(t_1)}{\displaystyle {\int }_{t_1-\mathrm{\Delta }}^{t_1+\mathrm{\Delta }}d{t}_2{\eta }^{\prime }(t_2)\mathrm{\Lambda }(t_1,t_2,s_1,s_2)}.\\ -{\left[{\displaystyle {\int }_0^{1}d{t}_1{\eta }^{\prime }(t)u_{|g^{\prime }(t)|}}(f^{\prime }(t),s)\right]}^2\end{array}$$

(98)

The functions appearing in Equation (98) are based on integrals that have the form:

$$\frac{1}{\sqrt{4\pi s_i}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z-a)}^2}{4s_i}}}h\left(z\right)dz\sim h\left(a\right)+s_i{h}^{″}(a)+O[s_i^2]h^{(4)}(a)$$

(99)

where limiting behavior as s_i → 0 for i = 1, 2 may be found using a Laplace expansion, as described in Equation 6.4.35 of Bender and Orszag [43], and Equation (99) holds, provided h is C⁵ in a neighborhood of a.

Consequently, we see that, after rewriting the first integral as a product of a t₁ and a t₂ integral and using Equations (90), the difference of the first and the last terms in Equation (98) is of order σ⁴ in the limit s₁, s₂ → 0 and may therefore be dropped.

Consequently, we focus attention on the remaining integrals. Equation (99) enables this refinement of the limiting form of Equation (63):

$$\begin{array}{l}{u}_{|g^{\prime }(t)|}(f^{\prime }(t),s)={\displaystyle {\int }_{-\infty }^{\infty }\frac{1}{\sqrt{4\pi s}}}{e}^{-\frac{{(z-f^{\prime }(t))}^2}{4s}}G\left(z,|g^{\prime }(t)|\right)dz\\ \sim G(f^{\prime }(t),|g^{\prime }(t)|)+s{G}^{(2)}(f^{\prime }(t),|g^{\prime }(t)|)+O\left[s_i^2\right]G^{(4)}(f^{\prime }(t),|g^{\prime }(t)|)\end{array}$$

(100)

where the n-th-partial derivative of G(u, v) with respect to the first argument is denoted by G⁽ⁿ⁾(u, v) and is guaranteed to exist for all n at all but finitely many points by the assumption that |g′(t)| ≠ |f′(t)| at all but finitely many points. Moreover, since |g′(t)| is bounded away from zero by hypothesis, there is a uniform bound on G⁽⁴⁾ (f′(t), |g′(t)|) offthe set of points where |g′(t)| = |f′(t)|.

Consider the second integral in Equation (98). The term in brackets becomes (ignoring those finitely many points where G(f′(t), |g′(t)|) might be non-smooth, because |f′(t)| = |g′(t)|), and using Equation (90) to replace s₁, s₂:

$$\begin{array}{l}[u_{|g^{\prime }(t_1)|}(f^{\prime }(t_1),s_1)u_{|g^{\prime }(t_2)|}(f^{\prime }(t_2),s_2)]\sim \\ G(f^{\prime }(t_1),|g^{\prime }(t_1)|)G(f^{\prime }(t_2),|g^{\prime }(t_2)|)+\frac{{\sigma }^2}{2}G(f^{\prime }(t_1),|g^{\prime }(t_1)|)n_2{(t_1,t_2)}^2{G}^{(2)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)\\ +\frac{{\sigma }^2}{2}{\Vert m_{t_1}\Vert }^2{G}^{(2)}(f^{\prime }(t_1),|g^{\prime }(t_1)|)G(f^{\prime }(t_2),|g^{\prime }(t_2)|)+O[{\sigma }^4]\end{array}$$

(101)

We next consider the third integral in Equation (98), by Equations (95) and (100) as s₂ → 0,

$$\mathrm{\Lambda }(t_1,t_2,s_1,s_2)\sim {\displaystyle {\int }_{-\infty }^{\infty }d{z}_1}\frac{1}{\sqrt{4\pi s_1}}{e}^{-\frac{{(z_1-f^{\prime }(t_1))}^2}{4s_1}}G(z_1,|g^{\prime }(t_1)|)[{\mathrm{\Phi }}_1(z_1)+s_2{\mathrm{\Phi }}_2(z_1)]$$

(102)

with:

$$\begin{array}{l}{\mathrm{\Phi }}_1(z_1)+s_2{\mathrm{\Phi }}_2(z_1):=G\left(f^{\prime }(t_2)+(z_1-f^{\prime }(t_1))\frac{\langle m_{t_1},m_{t_2}\rangle }{\Vert m_{t_1}\Vert },|g^{\prime }(t_2)|\right)\\ +s_2{G}^{(2)}\left(f^{\prime }(t_2)+(z_1-f^{\prime }(t_1))\frac{\langle m_{t_1},m_{t_2}\rangle }{\Vert m_{t_1}\Vert },|g^{\prime }(t_2)|\right)\end{array}$$

(103)

defined as indicated to simplify notation. Applying Equation (99) to Equation (102), we find that Λ(t₁, t₂, s₁, s₂) is asymptotic to:

$$\begin{array}{l}G(f^{\prime }(t_1),|g^{\prime }(t_1)|)[{\mathrm{\Phi }}_1(f^{\prime }(t_1))+s_2{\mathrm{\Phi }}_2(f^{\prime }(t_2))]\\ +s_1{G}^{(2)}(f^{\prime }(t_1),|g^{\prime }(t_1)|)[{\mathrm{\Phi }}_1(f^{\prime }(t_1))+s_2{\mathrm{\Phi }}_2(f^{\prime }(t_1))]\\ +2s_1{G}^{(1)}(f^{\prime }(t_1),|g^{\prime }(t_1)|)\left[{\mathrm{\Phi }}_1^{(1)}(f^{\prime }(t_1))+s_2{\mathrm{\Phi }}_2^{(1)}(f^{\prime }(t_1))\right]\\ +s_{1}G(f^{\prime }(t_1),|g^{\prime }(t_1)|)\left[{\mathrm{\Phi }}_1^{(2)}(f^{\prime }(t_1))+s_2{\mathrm{\Phi }}_2^{(1)}(f^{\prime }(t_1))\right]+O\left[{\sigma }^4\right]\end{array}$$

(104)

Since ${\mathrm{\Phi }}_1^{(n)}(f^{\prime }(t_1))=G^{(n)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)$ and ${\mathrm{\Phi }}_2^{(n)}(f^{\prime }(t_1))=G^{(n+2)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)$, Λ(t₁, t₂, s₁, s₂) becomes:

$$\begin{array}{l}\mathrm{\Lambda }(t_1,t_2,s_1,s_2)\sim G(f^{\prime }(t_1),|g^{\prime }(t_1)|)G(f^{\prime }(t_2),|g^{\prime }(t_2)|)\\ +s_{2}G(f^{\prime }(t_1),|g^{\prime }(t_1)|)G^{(2)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)\\ +s_2{G}^{(2)}(f^{\prime }(t_1),|g^{\prime }(t_1)|)G(f^{\prime }(t_2),|g^{\prime }(t_2)|)\\ +2s_1{G}^{(1)}(f^{\prime }(t_1),|g^{\prime }(t_1)|)G^{(1)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)\\ +s_{2}G(f^{\prime }(t_1),|g^{\prime }(t_1)|)G^{(2)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)+O[{\sigma }^4]\end{array}$$

(105)

where we used Equation (90) to write the error term as a function of σ.

Using (105) and (90), the difference between the second and third integrals appearing in Equation (98) gives:

$$\begin{array}{l}\text{Var}[\delta H_{f+\sigma \varsigma }[\eta ]]=\frac{{\sigma }^2}{2}{\displaystyle {\int }_0^{1}d{t}_1{\eta }^{\prime }(t_1)}{\displaystyle {\int }_{t_1-\mathrm{\Delta }}^{t_1+\mathrm{\Delta }}d{t}_2{\eta }^{\prime }(t_2)}\\ \{G(f^{\prime }(t_1),|g^{\prime }(t_1)|)n{(m_{t_1},m_{t_2})}^2{G}^{(2)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)\\ +2{\Vert m_{t_1}\Vert }^2{G}^{(1)}(f^{\prime }(t_1),|g^{\prime }(t_1)|)G^{(1)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)\\ +{\Vert m_{t_1}\Vert }^2{\eta }^{\prime }(t_1)G(f^{\prime }(t_1),|g^{\prime }(t_1)|)G^{(2)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)\\ -G(f^{\prime }(t_1),|g^{\prime }(t_1)|){\Vert m_{t_2}\Vert }^2{G}^{(2)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)\}\end{array}$$

(106)

to accuracy O[σ⁴]. This may be further simplified, introducing errors of O[2Δ] by truncating the upper bound of integration for t₁ in the last two integrals from 1 to 1−2Δ, so that by Equation (23), ║m_t║ = 1, and the last two pieces cancel, leaving:

$$\begin{array}{l}\text{Var}[\delta H_{f+\sigma \varsigma ,g}[\eta ]]=\\ +\frac{{\sigma }^2}{2}{\displaystyle {\int }_0^{1}d{t}_1{\eta }^{\prime }(t_1)G(f^{\prime }(t_1),|g^{\prime }(t_1)|){\displaystyle {\int }_{t_1-\mathrm{\Delta }}^{t_1+\mathrm{\Delta }}d{t}_2{\eta }^{\prime }(t_2)n{(m_{t_1},m_{t_2})}^2{G}^{(2)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)}}\\ +{\sigma }^2{\displaystyle {\int }_0^{1}d{t}_1{\Vert m_{t_1}\Vert }^2{\eta }^{\prime }(t_1)G^{(1)}(f^{\prime }(t_1),|g^{\prime }(t_1)|){\displaystyle {\int }_{t_1-\mathrm{\Delta }}^{t_1+\mathrm{\Delta }}d{t}_2{\eta }^{\prime }(t_2)G^{(1)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)}}+O\left[{\sigma }^4\right]\end{array}$$

Focusing on the product of integrals in the second row, we see that at the cost of an additional error term of O[Δ], we may replace ║m_t║ by 1 in the t₁ integral and then use Schwartz’s inequality to bound the integral by the norms shown. To bound the t₂ integral, we use Equations (17) and (75) to obtain:

$$\begin{array}{l}\text{Var}[\delta H_{f+\sigma \varsigma ,g}[\eta ]]\\ \le +\frac{{\sigma }^2}{2}{\displaystyle {\int }_0^{1}d{t}_1{\eta }^{\prime }(t_1)G(f^{\prime }(t_1),|g^{\prime }(t_1)|){\displaystyle {\int }_{t_1-\mathrm{\Delta }}^{t_1+\mathrm{\Delta }}d{t}_2{\eta }^{\prime }(t_2)n{(m_{t_1},m_{t_2})}^2{G}^{(2)}(f^{\prime }(t_2),|g^{\prime }(t_2)|)}}\\ +{\sigma }^2{\Vert {\eta }^{\prime }\Vert }^2{\Vert G^{(1)}\Vert }^2{\displaystyle {\int }_{t_1-\mathrm{\Delta }}^{t_1+\mathrm{\Delta }}d{t}_2{\Vert {\eta }^{\prime }\Vert }_{\infty }\frac{4}{27}\frac{1}{|g^{\prime }(t_2){|}^2}}+O\left[{\sigma }^4\right]\end{array}$$

(107)

Focusing next on the integrals in the first row of the inequality, we use Schwartz’s inequality to bound the t₁ integral and Equations (17), (76) and (89) to bound the t₂ integral to obtain:

$$\begin{array}{l}\text{Var}[\delta H_{f+\sigma \varsigma ,g}[\eta ]]\\ \le \frac{{\sigma }^2}{2}{\Vert {\eta }^{\prime }\Vert }^2{\Vert G\Vert }^2{\Vert {\eta }^{\prime }\Vert }_{\infty }{\displaystyle {\int }_{t_1-\mathrm{\Delta }}^{t_1+\mathrm{\Delta }}d{t}_2\frac{1}{|g^{\prime }(t_2){|}^3}}\\ +{\sigma }^2{\Vert {\eta }^{\prime }\Vert }^2{\Vert G^{(1)}\Vert }^2{\Vert {\eta }^{\prime }\Vert }_{\infty }\frac{4}{27}{\displaystyle {\int }_{t_1-\mathrm{\Delta }}^{t_1+\mathrm{\Delta }}d{t}_2\frac{1}{|g^{\prime }(t_2){|}^2}}+O\left[{\sigma }^2\right]O\left[\mathrm{\Delta }\right]+\left[{\sigma }^4\right].\end{array}$$

(108)

Since the reference satisfies the constraint:

$$|g^{\prime }(t)|\ge K_0{\mathrm{\Delta }}^{\alpha }$$

(109)

where α > 0, then we have:

$$\begin{array}{l}\text{Var}[\delta H_{f+\sigma \varsigma ,g}[\eta ]]\le \frac{{\sigma }^2}{2}{\Vert {\eta }^{\prime }\Vert }_{\infty }^3{\left(K_0{\mathrm{\Delta }}^{-\alpha }\right)}^2\mathrm{\Delta }{K}_0^3{\mathrm{\Delta }}^{-3\alpha }\\ +{\sigma }^2{\Vert {\eta }^{\prime }\Vert }_{\infty }^3{\left(\frac{4}{27}{K}_0^2{\mathrm{\Delta }}^{-2\alpha }\right)}^2\frac{4}{27}\mathrm{\Delta }{K}_0^2{\mathrm{\Delta }}^{-2\alpha }\\ +O\left[{\sigma }^2\right]O\left[\mathrm{\Delta }\right]+O\left[{\sigma }^4\right]\end{array}$$

(110)

This simplifies to our final bound:

$$0<\text{Var}\left[\delta H_{f+\sigma \varsigma }{}_{,g}\left[\eta \right]\right]\le K_3{\Vert {\eta }^{\prime }\Vert }_{\infty }^3{\sigma }^2{\mathrm{\Delta }}^{1-6\alpha }+O\left[{\sigma }^4\right]$$

(111)

where $K_3=K_0^6{(4/27)}^3$ is a constant independent of g, η, Δ and σ. Let $K_1:=K_3{\Vert {\eta }^{\prime }\Vert }_{\infty }^3$, and we are done.

Part II: Calculation of the variance of δE_f+σς[η] The variance of the variation, δE_f+σς[η], is:

$$\text{Var}[\delta E_{f+\sigma \varsigma }[\eta ]]={\displaystyle {\int }_{C_0[0,1]}{[\delta E_{f+\sigma \varsigma }[\eta ]]}^2}{d}_{W}x-{\langle \delta E_{f+\sigma \varsigma }[\eta ]\rangle }^2$$

(112)

Recalling Equations (81) and (51) and using Equation (13) for ς(t) and Equation (16) for M_t, the Wiener integral on the right-hand side of Equation (112) becomes:

$$\begin{array}{l}{\displaystyle {\int }_{C_0[0,1]}{\left[2{\displaystyle {\int }_0^1[f(t)+\sigma \varsigma (t)]\eta (t)}dt\right]}^2{d}_W}x\\ =2{\displaystyle {\int }_0^{1}d{u}_{1}2{\displaystyle {\int }_0^{1}d{u}_1\eta (t_1)}}\eta (t_2)\\ \times {\displaystyle {\int }_{C_0[0,1]}{d}_{W}x}[\left(f(t_1)+\sigma {\displaystyle {\int }_0^1{x}_2(t_3)d{M}_{t_1}}(t_3)\right)\\ \times \left(f(t_2)+\sigma {\displaystyle {\int }_0^1{x}_2(t_3)d{M}_{t_2}}(t_3)\right)]\end{array}$$

(113)

Using Equation (146), the right-hand side may be written in terms of Lebesgue integrals as:

$$\begin{array}{l}2{\displaystyle {\int }_0^{1}d{t}_{1}2{\displaystyle {\int }_0^{1}d{t}_1\eta (t_1)}\eta (t_2)\frac{1}{2\pi }}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{u_2^2}{2}}}d{u}_2\\ \times \left[(f(t_1)+\sigma \Vert M_{t_1}\Vert u_1)(f(t_2)+\sigma \left(\langle M_{t_1},M_{t_2}\rangle u_1+N_2(t_1,t_2)u_2)\right)\right]\end{array}$$

(114)

where (from Equation (143)):

$$N_2(t_1,t_2)=\sqrt{\langle M_{t_2},M_{t_2}\rangle -\frac{{\langle M_{t_1},M_{t_2}\rangle }^2}{\langle M_{t_1},M_{t_1}\rangle }}$$

(115)

Changing variables using:

$$\begin{array}{l}f\left(t_1\right)-z_1=-\sqrt{2s_1}{u}_1,\text{where}\sigma \Vert M_{t_1}\Vert =\sqrt{2s_1}\\ f\left(t_2\right)-z_2=-\sqrt{2s_2}{u}_2,\text{where}\sigma N_2(t_1,t_2)=\sqrt{2s_2}\end{array}$$

(116)

permits Expression (114) to be rewritten as:

$$\begin{array}{l}2{\displaystyle {\int }_0^{1}2}{\displaystyle {\int }_0^1\eta (t_1)}\eta (t_2)\frac{1}{\sqrt{4\pi s_1}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_1-f(t_1))}^2}{4s_1}}}\frac{1}{\sqrt{4\pi s_2}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_2-f(t_2))}^2}{4s_2}}}\\ \times \left[z_1\left(z_2+(z_1-f(t_1))\frac{\langle M_{t_1},M_{t_2}\rangle }{\Vert M_{t_1}\Vert }\right)\right]d{z}_{2}d{z}_{1}d{t}_{2}d{t}_1\end{array}$$

Using Equation (28), this becomes:

$$\begin{array}{l}4{\displaystyle {\int }_0^1{\displaystyle {\int }_0^1\eta (t_1)}\eta (t_2)\frac{1}{\sqrt{4\pi s_1}}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_1-f(t_1))}^2}{4s_1}}}\\ \times \frac{1}{\sqrt{4\pi s_2}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_2-f(t_2))}^2}{4s_2}}}{z}_1{z}_{2}d{z}_{2}d{z}_{1}d{t}_{2}d{t}_1+\sqrt{\mathrm{\Delta }}\mathrm{\Omega }\end{array}$$

(117)

where:

$$\begin{array}{l}\mathrm{\Omega }=4{\displaystyle {\int }_0^1{\displaystyle {\int }_0^1\eta (t_1)}\eta (t_2)\frac{1}{\sqrt{4\pi s_1}}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_1-f(t_1))}^2}{4s_1}}}\frac{1}{\sqrt{4\pi s_2}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_2-f(t_2))}^2}{4s_2}}}\\ \times z_1(z_1-f(t_1))\frac{\min (t_1,t_2)}{\sqrt{t_1}}d{z}_{2}d{z}_{1}d{t}_{2}d{t}_1+O\left[{\mathrm{\Delta }}^2\right]{\sigma }^2\end{array}$$

(118)

Note that the first term in Equation (118) may be calculated exactly as:

$$4{\displaystyle {\int }_0^1{\displaystyle {\int }_0^1\eta (t_1)\eta (t_2)f}(t_1)f(t_2)}d{t}_{1}d{t}_2$$

(119)

which, by Equation (81), cancels the −⟨δE_f+σς[η]⟩² appearing in Equation (112).

We now rewrite Ω as:

$$\begin{array}{l}\mathrm{\Omega }=4{\displaystyle {\int }_0^1{\displaystyle {\int }_0^1\eta (t_1)}\eta (t_2)\frac{1}{\sqrt{4\pi s_1}}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_1-f(t_1))}^2}{4s_1}}}\frac{1}{\sqrt{4\pi s_2}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_2-f(t_2))}^2}{4s_2}}}\\ \times {(z_1-f(t_1))}^2\frac{\min (t_1,t_2)}{\sqrt{t_1}}d{z}_{2}d{z}_{1}d{t}_{2}d{t}_1\\ +4{\displaystyle {\int }_0^1{\displaystyle {\int }_0^1\eta (t_1)}\eta (t_2)\frac{1}{\sqrt{4\pi s_1}}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_1-f(t_1))}^2}{4s_1}}}\frac{1}{\sqrt{4\pi s_2}}{\displaystyle {\int }_{-\infty }^{\infty }{e}^{-\frac{{(z_2-f(t_2))}^2}{4s_2}}}\\ \times f(t_1)(z_1-f(t_1))\frac{\min (t_1,t_2)}{\sqrt{t_1}}d{z}_{2}d{z}_{1}d{t}_{2}d{t}_1+O\left[{\mathrm{\Delta }}^2\right]{\sigma }^2\end{array}$$

(120)

In the second integral the z₁ integration is of an odd integrand over a symmetric interval and, hence, vanishes, so that we obtain after some rearrangement and simplification of the remaining Gaussian integrals:

$$\mathrm{\Omega }=4{\displaystyle {\int }_0^1{\displaystyle {\int }_0^1\frac{\min (t_1,t_2)}{\sqrt{t_1}}}}\eta (t_1)\eta (t_2)s_{2}d{t}_{2}d{t}_1+O\left[{\mathrm{\Delta }}^2\right]{\sigma }^2$$

(121)

$$=2\sqrt{2}{\sigma }^2{\displaystyle {\int }_0^1{\displaystyle {\int }_0^1\frac{\min (t_1,t_2)}{\sqrt{t_1}}}}\eta (t_1)\eta (t_2)N_2(t_1,t_2)d{t}_{2}d{t}_1+O\left[{\mathrm{\Delta }}^2\right]{\sigma }^2$$

(122)

where we have used Equation (116) to obtain the last equation.

Inserting the limiting form of the integral into Expression (117) of small positive s₁, s₂ into Equation (112) and using Equation (119), we obtain:

$$\text{Var}[\delta E_{f+\sigma \varsigma }[\eta ]]=K_2{\mathrm{\Delta }}^{1/2}{\sigma }^2+O({\mathrm{\Delta }}^{5/2}{\sigma }^2)$$

(123)

where K₂ is a constant independent of Δ and σ, but depending on η.

This completes the proof of the theorem. □

7. A Short List of Wiener Integrals

7.1. The First Wiener Integral

The first type of Wiener integral is described by the following:

Theorem 5. Let ρ(t) be real and of bounded variation on [0, 1] and the ρ be normalized, such that:

$$\rho (1)=0.$$

(124)

Let:

$$A=\sqrt{{\displaystyle {\int }_0^1\rho {(t)}^{2}dt}}$$

(125)

and let F (u) be a (real or complex) measurable function defined on −∞ < u < ∞. Then, a necessary and sufficient condition that:

$$F\left[{\displaystyle {\int }_0^{1}x(t)d\rho (t)}\right]$$

(126)

be a Wiener measurable function of x(•) over C₀[0, 1] is that:

$$e^{-\frac{u^2}{2}}F(Au)$$

(127)

be of class L₁ on −∞ < u < ∞. Moreover, if this condition is satisfied,

$${\displaystyle {\int }_{C_0[0,1]}F}\left[{\displaystyle {\int }_0^{1}x(t)d\rho }(t)\right]d_{W}x=\frac{1}{\sqrt{2\pi }}{\displaystyle {\int }_{-\infty }^{\infty }F(Au)e^{-\frac{u^2}{2}}}du$$

(128)

We derive this equation below. Published results of a similar form may be found in [40], Theorem 29.7 (n = 1 case and assuming that ρ is normalized to unity). Other references are either Koval’chick [44] (page 106, Equation (13)) or Cameron and Martin [45] (page 393, Equation (6.3)). The same result is derived by an argument that will be familiar to many physicists in Paley, Wiener and Zygmund (see [46] Equation (2.11)), although the result derived there also assumes that the ρ is normalized to one. There is a difference of $\sqrt{2}$ between the result contained in Equation (128) and the older references [44,46], due to use of different definitions of the normal distribution used to define the Brownian motion on which the Wiener measure is based.

7.1.1. Random Variables Derivation

We want to calculate:

$$E_W(F({\varsigma }^{\prime }(t)))={\displaystyle {\int }_{C_0[0,1]}F}({\varsigma }^{\prime }(t))d_{W}x$$

(129)

ς′(t) is a Gaussian random variable with zero mean, so ${\varsigma }^{\prime }(t)~\text{N}(0,{\sigma }^2)$, where σ² = E_W (ς′(t)²). From this:

$$\begin{array}{l}{E}_W(F({\varsigma }^{\prime }(t)))={\displaystyle {\int }_{C_0[0,1]}F}({\varsigma }^{\prime }(t))d_{W}x\\ ={\displaystyle {\int }_{-\infty }^{\infty }\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{u^2}{2{\sigma }^2}}}F(u)du\end{array}$$

(130)

7.2. The Second Wiener Integral

The second type of Wiener integral we need is:

$$\begin{array}{l}{\displaystyle {\int }_{C_0[0,1]}{d}_{W}xF}\left[{\displaystyle {\int }_0^{1}x}(t)d{\rho }_1(t),\dots ,{\displaystyle {\int }_0^{1}x}(t)d{\rho }_n(t)\right]\\ ={(2\pi )}^{-n/2}{\displaystyle {\int }_{-\infty }^{\infty }d{u}_1\dots }{\displaystyle {\int }_{-\infty }^{\infty }d{u}_n}F(u_1,\dots ,u_n)e^{-{\sum }_{k=1}^n\frac{u_k^2}{2}}\end{array}$$

(131)

where ρ_i(1) = 0 for each index i and the ρ_i are orthonormal.

Similar versions of this integral appear in many sources, for instance in Koval’chick [44] (page 107, Equation (14)), which contains (after transcription into the modern conventions):

$$\begin{array}{l}{\displaystyle {\int }_{C_0[0,1]}{d}_{W}xF}\left[{\displaystyle {\int }_0^1{\rho }_1}(t)dx(t),\dots ,{\displaystyle {\int }_0^1{\rho }_n}(t)dx(t)\right]\\ ={(2\pi )}^{-n/2}{\displaystyle {\int }_{-\infty }^{\infty }d{u}_1\dots }{\displaystyle {\int }_{-\infty }^{\infty }d{u}_n}F(u_1,\dots ,u_n)e^{-{\sum }_{k=1}^n\frac{u_k^2}{2}}\end{array}$$

(132)

This form appears to be derivable from Equation (131) using integration by parts. Specifically,

$$\begin{array}{l}{\displaystyle {\int }_{C_0[0,1]}{d}_{W}xF}\left[{\displaystyle {\int }_0^1{\rho }_1}(t)dx(t),\dots ,{\displaystyle {\int }_0^1{\rho }_n}(t)dx(t)\right]\\ ={(2\pi )}^{-n/2}{\displaystyle {\int }_{-\infty }^{\infty }d{u}_1\dots }{\displaystyle {\int }_{-\infty }^{\infty }d{u}_n}F(u_1,\dots ,u_n)e^{-{\sum }_{k=1}^n\frac{u_k^2}{2}}\end{array}$$

(133)

We would formally transform this into the form of Equation (131) by the following steps:

First observe that the integral in Equation (133) is equal to:

$$\begin{array}{l}{\displaystyle {\int }_{C_0[0,1]}{d}_{W}xF}\left[-{\displaystyle {\int }_0^1{\rho }_1}(t)dx(t),\dots ,-{\displaystyle {\int }_0^1{\rho }_n}(t)dx(t)\right]\\ ={(2\pi )}^{-n/2}{\displaystyle {\int }_{-\infty }^{\infty }d{u}_1\dots }{\displaystyle {\int }_{-\infty }^{\infty }d{u}_n}F(-u_1,\dots ,-u_n)e^{-{\sum }_{k=1}^n\frac{{(-u_k^2)}^2}{2}}\\ ={(2\pi )}^{-n/2}{\displaystyle {\int }_{-\infty }^{\infty }d{u}_1\dots }{\displaystyle {\int }_{-\infty }^{\infty }d{u}_n}F(u_1,\dots ,u_n)e^{-{\sum }_{k=1}^n\frac{u_k^2}{2}}\end{array}$$

(134)

Next, use the facts that we have assumed that ρ_i(1) = 0 and, additionally, that the Brownian paths are normalized according to x(0) = 0, so that (classical) integration by parts of the integrals in the first line of Equation (134) would yield:

$$\begin{array}{l}{\displaystyle {\int }_{C_0[0,1]}{d}_{W}xF}\left[{\displaystyle {\int }_0^{1}x}(t)d{\rho }_1(t),\dots ,{\displaystyle {\int }_0^{1}x}(t)d{\rho }_n(t)\right]\\ ={(2\pi )}^{-n/2}{\displaystyle {\int }_{-\infty }^{\infty }d{u}_1\dots }{\displaystyle {\int }_{-\infty }^{\infty }d{u}_n}F(u_1,\dots ,u_n)e^{-{\sum }_{k=1}^n\frac{u_k^2}{2}}\end{array}$$

(135)

Although this “derivation” shows that the two forms are equivalent, as desired, it overlooks the fact that the integrals in Equation (132) cannot be classical integrals. In fact, Wiener [47] (p. 68) states that integrals of the form:

$${\displaystyle {\int }_0^1{\rho }_i}(t)dx(t)$$

(136)

are actually Itō integrals. The correct integration-by-parts formula in this case is:

$$X_t{Y}_t=X_0{Y}_0+{\displaystyle {\int }_0^t{X}_{s-}d{Y}_s}+{\displaystyle {\int }_0^t{Y}_{s-}d{X}_s}+{[X,Y]}_t$$

(137)

where [X, Y]_t is the quadratic covariation process with:

$${[X,Y]}_t:=\underset{\Vert P\Vert \to 0}{\lim }{\displaystyle \sum _{k=1}^n|X_{t_k}-X_{t_{k-1}}||}{Y}_{t_k}-Y_{t_{k-1}}|$$

(138)

where P ranges over partitions of the interval [0, t] and the norm, ║P║, of the partition, t₀ <⋯< t_n, is the mesh, i.e., max{|t_i − t_i−1| : i = 1,…, n}.

However, in the case where Y is of bounded variation:

$$\begin{array}{l}{[X,Y]}_t\le \max \{|X_{t_k}-X_{T_{k-1}}|,k=1,\dots ,n\}\underset{\Vert P\Vert \to 0}{\lim }{\displaystyle \sum _{k=1}^n|Y_{t_k}-Y_{T_{k-1}}}|\\ \le \underset{\Vert P\Vert \to 0}{\lim }\max \{|X_{t_k}-X_{t_{k-1}}|,k=1,\dots ,n\}{V}_0^t[Y]\\ =0\end{array}$$

(139)

so that Equation (137) reduces to the classical integration by parts formula. Thus, the derivation above is (accidentally) correct.

The number of sources containing detailed derivations of these equations in English appears to be limited. The only source we have been able to locate is Paley, Wiener and Zygmund, which is completely self-contained and contains the equivalent of our Equation (131) (see Equation (2.14) in [46]), although the result derived there assumes that the measure is normalized to one and uses a slightly different notation for the Brownian paths.

We need to compute Wiener integrals like those on the left-hand side of Equation (131) in the case where the ρ_k(t) are not orthonormal. Moreover, we only need to consider the special form:

$${\displaystyle {\int }_{c_0[0,1]}{F}_1}\left[{\displaystyle {\int }_0^{1}x(t)d{\rho }_1}(t)\right]F_2\left[{\displaystyle {\int }_0^{1}x(t)d{\rho }_2(t)}\right]d_{W}x$$

(140)

To apply Equation (131), we use the Gram–Schmidt process to obtain an orthonormal family ν₁(t), ν₂(t) from the original ρ₁(t), ρ₂(t). Using the short-hand notation:

$$\langle f\left(t\right),g\left(t\right)\rangle ={\displaystyle {\int }_0^{1}f\left(t\right)g\left(t\right)dt}$$

(141)

The Gram–Schmidt process is:

$$\begin{array}{l}{v}_1(t)=\frac{{\rho }_1(t)}{\sqrt{\langle {\rho }_1,{\rho }_1\rangle }}=\frac{{\rho }_1(t)}{N_1}\\ v_2(t)=\frac{{\rho }_2(t)-v_1(t)\langle {\rho }_2,v_1\rangle }{\sqrt{\langle {\rho }_2-v_1\langle {\rho }_2,v_1\rangle ,{\rho }_2-v_1\langle {\rho }_2,v_1\rangle \rangle }}\\ =\frac{1}{N_2}{\rho }_2(t)-\frac{\langle {\rho }_2,v_1\rangle }{N_2}\frac{1}{N_1}{\rho }_1(t)\end{array}$$

(142)

where:

$$\begin{array}{l}{N}_1:=\sqrt{\langle {\rho }_1,{\rho }_1\rangle }=\Vert {\rho }_1\Vert \\ N_2:=\sqrt{\langle {\rho }_2-v_1\rangle \langle {\rho }_2-v_1\rangle ,{\rho }_2-v_1\langle {\rho }_2,v_1\rangle \rangle }\\ =\sqrt{\langle {\rho }_2,{\rho }_2\rangle -2\langle {\rho }_2,v_1\rangle \langle {\rho }_2,v_1\rangle +{\langle {\rho }_2,v_1\rangle }^2\langle v_1,v_1\rangle }\\ =\sqrt{\langle {\rho }_2,{\rho }_2\rangle -2{\langle {\rho }_2,v_1\rangle }^2+{\langle {\rho }_2,v_1\rangle }^2}\\ =\sqrt{\langle {\rho }_2,{\rho }_2\rangle -\frac{1}{N_1^2}{\langle {\rho }_1,{\rho }_2\rangle }^2}\end{array}$$

(143)

which is expressed in matrix form:

$$\left[\begin{array}{c}{v}_1(t)\\ v_2(t)\end{array}\right]=\left[\begin{array}{cc}\frac{1}{N_1}& 0\\ \frac{-\langle {\rho }_1,{\rho }_2\rangle }{N_1{N}_2}& \frac{1}{N_2}\end{array}\right]\left[\begin{array}{c}{\rho }_1(t)\\ {\rho }_2(t)\end{array}\right]$$

(144)

or:

$$\left[\begin{array}{c}{\rho }_1(t)\\ {\rho }_2(t)\end{array}\right]=\left[\begin{array}{cc}{N}_1& 0\\ \langle {\rho }_1,{\rho }_2\rangle & N_2\end{array}\right]\left[\begin{array}{c}{v}_1(t)\\ v_2(t)\end{array}\right]$$

(145)

This permits us to rewrite Equation (140) as:

$${\displaystyle {\int }_{c_0[0,1]}{F}_1}\left[N_1{\displaystyle {\int }_0^{1}x(t)d{v}_1}(t)\right]F_2\left[\langle {\rho }_1,{\rho }_2\rangle {\displaystyle {\int }_0^{1}x(t)d{v}_1(t)+N_2}{\displaystyle {\int }_0^{1}x(t)d{v}_2}(t)\right]d_{W}x$$

to which (since the ν_k(1) = 0, k = 1, 2) we may now apply Equation (131) to obtain:

$$\begin{array}{l}{\displaystyle {\int }_{c_0[0,1]}{F}_1}\left[N_1{\displaystyle {\int }_0^{1}x(t)d{v}_1(t)}\right]\times F_2\left[\langle p_1,p_2\rangle {\displaystyle {\int }_0^{1}x(t)d{v}_1(t)+N_2}{\displaystyle {\int }_0^{1}x(t)d{v}_2}(t)\right]d_{W}x\\ =\frac{1}{2\pi }{\displaystyle {\int }_{-\infty }^{\infty }d{u}_1}{\displaystyle {\int }_{-\infty }^{\infty }d{u}_2{e}^{-\frac{u_1^2}{2}-\frac{u_2^2}{2}}}{F}_1[N_1{u}_1]F_2\left[\langle {\rho }_1,{\rho }_2\rangle u_1+N_2{u}_2\right]\\ =\frac{1}{2\pi }{\displaystyle {\int }_{-\infty }^{\infty }d{u}_1}{\displaystyle {\int }_{-\infty }^{\infty }d{u}_2{e}^{-\frac{u_1^2}{2}-\frac{u_2^2}{2}}}{F}_1\left[\Vert {\rho }_1\Vert u_1\right]F_2\left[\langle {\rho }_1,{\rho }_2\rangle u_1+N_2{u}_2\right]\end{array}$$

(146)