Assessing, Testing and Estimating the Amount of Fine-Tuning by Means of Active Information

Abstract

A general framework is introduced to estimate how much external information has been infused into a search algorithm, the so-called active information. This is rephrased as a test of fine-tuning, where tuning corresponds to the amount of pre-specified knowledge that the algorithm makes use of in order to reach a certain target. A function f quantifies specificity for each possible outcome x of a search, so that the target of the algorithm is a set of highly specified states, whereas fine-tuning occurs if it is much more likely for the algorithm to reach the target as intended than by chance. The distribution of a random outcome X of the algorithm involves a parameter $\theta$ that quantifies how much background information has been infused. A simple choice of this parameter is to use $\theta f$ in order to exponentially tilt the distribution of the outcome of the search algorithm under the null distribution of no tuning, so that an exponential family of distributions is obtained. Such algorithms are obtained by iterating a Metropolis–Hastings type of Markov chain, which makes it possible to compute their active information under the equilibrium and non-equilibrium of the Markov chain, with or without stopping when the targeted set of fine-tuned states has been reached. Other choices of tuning parameters $\theta$ are discussed as well. Nonparametric and parametric estimators of active information and tests of fine-tuning are developed when repeated and independent outcomes of the algorithm are available. The theory is illustrated with examples from cosmology, student learning, reinforcement learning, a Moran type model of population genetics, and evolutionary programming.

1. Introduction

When Gödel published their incompleteness theorems [1], the mathematical world was shaken such that to date it has neither recovered nor fully assimilated the consequences [2]. Hilbert’s program to base mathematics on a finite set of axioms had previously been pursued by Alfred North Whitehead and Bertrand Russell [3]. However, this approach turned out to be wrong when Gödel proved that no finite set of axioms in a formal system can prove all its true statements, including its own consistency. At a similar but lesser scale, when David Wolpert and William MacReady published their No Free Lunch Theorems (NFLTs, [4,5]), there was disquiet in the community because these results imply that there is no one-size-fit-all algorithm that can do well in all searches [6], and thus that a “theory of everything” is not possible in machine learning. Wolpert and MacReady concluded that it was necessary to incorporate “problem-specific knowledge into the behavior of the algorithm” [5]. Thus, active information (actinfo) was introduced in order to measure the amount of information carried out by such problem-specific knowledge [7,8]. More specifically, the NFLTs say that no search works better on average than a blind search, i.e., a search according to a uniform distribution. Accordingly, actinfo is defined as

$$\begin{array}{c}\hfill I^{+}=log\frac{P\left(A\right)}{P_0\left(A\right)},\end{array}$$

(1)

where $A\subset \mathrm{\Omega }$ is the non-empty target of the search algorithm, a subset of the finite sample space $\mathrm{\Omega }$, and $P_0$ is a uniform probability measure ($P_0\left(A\right)=\left|A\right|/\left|\mathrm{\Omega }\right|$). P must be seen here as the probability measure induced by the problem-specific knowledge of the researcher, whereas $P_0$ is the underlying distribution assumed in the NFLTs. This corresponds to absence of problem-specific knowledge, in accordance with Bernoulli’s Principle of Insufficient Reason (PoIR). An equivalent characterization of actinfo is the reduction in functional information

$$I^{+}=I_{f0}-I_f=-log{P}_0\left(A\right)-(-logP\left(A\right))$$

(2)

between algorithms that do not and do make use background knowledge. The name functional information was introduced by Szostak and collaborators [9,10]. It refers to applications wherein A corresponds to all outcomes of an algorithm that are functional according to some criterion. Then, $I_{f0}$ and $I_f$ are the self-information (measured in nats) of the event that an algorithm X produces a functional outcome, given that it was generated under $P_0$ and P, respectively.

Suppose we do not know whether problem-specific knowledge has been used or not when the random search $X\in \mathrm{\Omega }$ was generated. This corresponds to a hypothesis testing problem

$$\begin{array}{c}{H}_0:X\sim P_0,\hfill \\ H_1:X\sim P,\hfill \end{array}$$

(3)

where data are generated from distributions $P_0$ and P under the null and alternative hypotheses $H_0$ and $H_1$, respectively. It follows from (1) that $I^{+}$ is the log likelihood ratio when testing $H_0$ against $H_1$, if data are censored so that only $X\in A$ is known.

When the sample space $\mathrm{\Omega }$ is finite or a continuous, bounded subset of a Euclidean space, the PoIR can be motivated by the fact that the uniform distribution maximizes the Shannon entropy, since it thereby maximizes ignorance about the outcome of X. However, the uniform distribution is neither a feasible choice of $P_0$ for infinite, countable sample spaces nor for continuous, unbounded samples spaces. For this reason, actinfo was generalized to deal with unbounded spaces [11], by choosing $P_0$ to maximize Shannon entropy under side constraints $\xi$, such as the existence of various moments. This gives rise to a family of null distributions $P_0=P_{0\xi }$, with a $\xi$ a nuisance parameter that has to be estimated or controlled for in order to estimate or give bounds for the active information.

Actinfo has also been used for mode detection in unsupervised learning, among other applications [12,13]. Based on previous works by Montañez [14,15], actinfo has been used in the past for testing hypotheses [16]. More specifically, P is regarded as a random measure in [16], so that $I^{+}$ is random as well and expressions for the tail probability of $I^{+}$ can be found.

1.1. Fine-Tuning

Fine-tuning (FT) was introduced by Carter in physics and cosmology [17]. According to FT, the constants in the laws of nature and/or the boundary conditions in the standard models of physics must belong to intervals of low probability in order for life to exist. Since its inception, FT has generated a great deal of fascination, seen in multiple divulgation books (e.g., [18,19,20,21]) and scientific articles (e.g., [22,23,24,25]). For a given constant of nature X, the connection between FT and active information can be described in three steps:

(i)

Establishing the life-permitting interval (LPI) A that allows the existence of life for the constant, with $\mathrm{\Omega }=(0,\infty )={\mathbb{R}}^{+}$ or $\mathrm{\Omega }=\mathbb{R}$ denoting the range of values that this constant could possibly take, including those that do not permit life.

(ii)

Determining the probability $P_0\left(A\right)$ of such a LPI. If $P_0=P_{0\xi }$ contains unknown parameters $\xi$, find an upper bound

$$P_{0\max }\left(A\right)=\underset{\xi }{max}{P}_{0\xi }\left(A\right)$$

(4)

of $P_0\left(A\right)$.

(iii)

Suppose that $H_1$ corresponds to an agent who uses background knowledge of what is required for life to exist in order to bring about a constant of nature X that definitely permits life ($P\left(A\right)=1$). The active information $I^{+}=I_{f0}=-log{P}_0\left(A\right)$ is then a measure of how much background knowledge this agent has infused. Following [26,27], we conclude that X is finely tuned when the lower bound $-log{P}_{0\max }\left(A\right)$ of $I^{+}=I_{f0}$ is large enough. That is, FT corresponds to infusing a high degree of background knowledge into a problem.

Fine-tuning has also been used in biology. Dingjan and Futerman explored it for cell membranes [28,29], whereas Thorvaldsen and Hössjer [30] formalized it for a large class of biological models. According to [30], a system is fine-tuned if it satisfies the two following requirements:

(a)

It has an independent specification;

(b)

It is very unlikely to occur by chance.

1.2. The Present Article

In this article, actinfo will not only be used in the algorithmic sense. It will also be employed for testing the presence of and estimating the degree of fine-tuning (FT) of a search algorithm or agent who brings about X. Our definition of FT relies on (a) and (b), and in order to formalize these two concepts, we introduce a specificity function f, which quantifies, in terms of $f\left(x\right)$, how specified an outcome $x\in \mathrm{\Omega }$ is. The target A, on the other hand, is a set of highly specified states, that is, all states with a degree of specificity that exceed a given threshold $f_0$. Then, $I^{+}$ in (1) is a test statistic for testing whether an algorithm has a much larger probability of reaching the set of highly specified states compared to a random search. This is a test of FT, since reaching the target corresponds to specificity (a), whereas reaching it with a much higher probability than expected by chance corresponds to (b).

To calculate $I^{+}$, the distributions $P_0$ and P of the random search algorithm under $H_0$ and $H_1$, respectively, need to be defined. As mentioned above, the null distribution $P_0$ is typically chosen according to some criterion, such as a maximizer of entropy, possibly with some extra constraints on moments for unbounded $\mathrm{\Omega }$, which was the strategy implemented in [26,27]. Another possibility is to choose $P_0$ as the equilibrium distribution of a Markov chain that models the dynamics of the system under the null hypothesis of no external input. In general, $P_0=P_{0\xi t}$ involves a number of nuisance parameters $\xi$, and sometimes, also the time point t when an algorithm that does not make use of external information stops. The choice of $P=P_{\theta \xi t}$ is problem specific, and it possibly involves the nuisance parameters $\xi$ of the null distribution, the time point t when the algorithm stops, as well as the tuning parameters $\theta$ that correspond to infusing the background knowledge into the search problem. Therefore, in its most general form, the actinfo (1) is a function $I^{+}=I^{+}(\theta ,\xi ,t)$ of the tuning parameters $\theta$, the nuisance parameters $\xi$, and the time point t.

This general framework has many applications based on different choices of f, A, $P_0$, and P. For some models, f is a binary function that quantifies functionality, so that A is the set of objects of a certain type (e.g., universes, proteins, protein complexes, or cellular networks) that are functional or permit life, among the set $\mathrm{\Omega }$ of all such objects.

Another possibility is to choose A as the set of populations x whose (expected) fitness $f\left(x\right)$ exceeds a given threshold. In this setting, $P_{0\xi t}\left(A\right)$ corresponds to the probability that a randomly chosen population would evolve and reach target A of high fitness at time t, given that no background knowledge of the specificity function f is used to generate X, so that natural selection does not occur. The functional information $I_{f0}=-log{P}_{0\xi t}\left(A\right)$ corresponds to the amount of external information that an evolutionary algorithm infuses under $H_1$, given that it brings about X so that A happens with certainty ($P\left(A\right)=1$) within time t. In this case, the population is finely tuned when $I_{f0}$ is large enough. More generally, we say that an evolutionary algorithm that generates $X\sim P=P_{\theta \xi t}$ after t time steps is finely tuned when $I^{+}(\theta ,\xi ,t)$ is large enough. Typically, $\theta$ involves the selection parameters that determine to which extent a population evolves towards higher fitness.

A third possibility is to choose $f\left(X\right)=X$ as the test score of a randomly chosen student, whereas $A=[f_0,\infty )$ is the set of results of those students who pass the test with a score of at least $f_0$. Assume that $f\left(X\right)\sim N(\xi ,1)$ for a randomly chosen student who did not prepare for the test ($H_0$), whereas $f\left(X\right)\sim N(\xi +\theta t,1)$ for a randomly chosen student who prepared for the test for a period of length t ($H_1$). Then, $P_0\left(A\right)=P_{0\xi }\left(A\right)=1-\mathrm{\Phi }(f_0-\xi )$, whereas $P\left(A\right)=P_{\theta \xi t}\left(A\right)=1-\mathrm{\Phi }(f_0-\xi -\theta t)$, where $\mathrm{\Phi }$ is the cumulative distribution function of a standard normal distribution. In particular, the tuning parameter $\theta >0$ corresponds to the amount of knowledge that a student is expected to generate per unit time of study.

The unified treatment of search problems and FT of this paper, is organized as follows: Section 2 introduces the specification function f and the set A of highly specified states. Section 3 introduces a class of probability distributions $P=P_{\theta }$ for which the specificity function f is used to exponentially tilt the null distribution $P_0$, so that outcomes with high specificity are more likely to occur, and with a scalar tuning parameter $\theta$ of $P_{\theta }$ that corresponds to the amount of exponential tilting. A proof is presented that it is possible to obtain a Metropolis–Hastings type Markov chain in discrete time $t=0,1,2,\dots$, whose outcome $X=X_t$ at time t has the aforementioned exponentially tilted distribution under equilibrium, that is, when t is large. The corresponding actinfo $I^{+}(\theta ,t)$ is shown to increase monotonically with t towards an equilibrium limit. The actinfo of a search algorithm $X=X_{t\wedge T}$ that stops at time T, when the targeted set A of highly specified states has been reached, is also shown to increase more rapidly. Section 4 introduces various nonparametric and parametric estimators of actinfo, and corresponding tests of FT, when n repeated and independent outputs of the search algorithm are available. In particular, large deviations theory is used to prove that the significance levels of these tests, i.e., the probability of detecting FT under $H_0$, goes to zero at an exponential rate when the sample size n increases. Section 5 presents a number of examples from cosmology, student learning, reinforcement learning, and population genetics, that illustrate our approach. A discussion in Section 6 follows, whereas the proof and further details about the models are presented in Section 7.

2. Specificity and Target

Consider a function $f:\mathrm{\Omega }\to \mathbb{R}$ and assume that the objective of the search algorithm, or the agent that brings about X, is to find regions in $\mathrm{\Omega }$ where f is large. The rationale for this is an independent specification, where a more specified state $x\in \mathrm{\Omega }$ corresponds to a larger $f\left(x\right)$. It is further assumed that the target set in (1) is given by

$$\begin{array}{c}\hfill A=\{x\in \mathrm{\Omega };f\left(x\right)\ge f\left(x_0\right)\}\end{array}$$

(5)

for some $x_0\in \mathrm{\Omega }$. This implies that the purpose of the search algorithm or the agent is to bring about an X that is at least as specified as $x_0$. We will refer to f as a specificity function of the agent or an objective function of the search algorithm.

Several examples of specificity functions are provided in Section 5. For instance, Example 2 deals with student learning. For a special case of this model, $f\left(x\right)=x$ represents the test score of a student, whereas $x_0$ is a reference value that corresponds to the minimum score needed to pass the test.

For cosmological FT (Example 1), x is the value of a particular constant of nature and the specificity function equals

$$f\left(x\right)=1_{\{x\in A\}},$$

(6)

where $1_{\{·\}}$ is the indicator function. That is, f has a binary range, with $f\left(x\right)=1$ and 0 corresponding to whether x permits a universe with life, and in particular, $x_0$ is a universe that permits life. From this, A is the LPI of this constant. Moreover, X is the value of this constant of nature for a randomly generated universe, with a distribution that either incorporates external information ($H_1$) or not ($H_0$).

In the context of proteins, x is taken to be an amino acid sequence, whereas $f\left(x\right)$ in (6) quantifies whether the protein that the amino acid corresponds to is functional (1) or not (0). For instance, X could be the outcome of a random evolutionary process, the goal of which is to generate a functioning protein, and this process either makes use of external information ($H_1$) or not ($H_0$). In a more refined example (Example 4), x is a molecular machine that consists of a possibly large number of proteins (or parts), and $f\left(x\right)$ is (a monotone function of) the fitness of x.

Interpretation of Target

There are at least two ways of interpreting $x_0$, and hence also the target set A. According to the first interpretation, $x_0$ is the outcome of random variable $X^{\prime }\in \mathrm{\Omega }$; that is, the outcome of a first search. Suppose that X is another random variable that represents a second (possibly future) search, independent of $X^{\prime }$. Then, if we condition the outcome $x_0$ of the first search, the actinfo $I^{+}$ in (1) is the log likelihood ratio for the event that the second search variable X is at least as specified as the observed value $f\left(x_0\right)$ of the first search.

There is, however, no need to associate $x_0$ in (5) with a first search variable $X^{\prime }$. Instead, some a priori information may be used to define which values of f represent a high amount of specificity. This gives rise to the second interpretation of $x_0$, according to which $x_0$ is used for defining outcomes with a high and low degree of specificity, using $f_0=f\left(x_0\right)$ as a cutoff. According to this interpretation, the two sets A in (5) and its complement

$$\begin{array}{c}\hfill A^c=\mathrm{\Omega }\backslash A=\{x;f\left(x\right)<f\left(x_0\right)\}\end{array}$$

represent a dichotomization of specificity, so that A and $A^c$ consist of all states with high and low specificity, respectively. With this interpretation of x, $I^{+}$ is the log likelihood ratio for testing FT based on the search variable X. In particular, suppose that the specificity function f is bounded, i.e.,

$$\begin{array}{c}\hfill f_{\max }=\underset{x\in \mathrm{\Omega }}{max}f\left(x\right)<\infty .\end{array}$$

(7)

Then, the most stringent definition of high specificity,

$$\begin{array}{c}\hfill f_0=f_{\max },\end{array}$$

(8)

only regards outcomes with a maximal value of f as highly specified, so that

$$\begin{array}{c}\hfill A={\mathrm{\Omega }}_{\max }=\{x\in \mathrm{\Omega };f\left(x\right)=f_{\max }\}.\end{array}$$

(9)

Note that (6) is a special case of (9).

3. Active Information for Exponentially Tilted Systems

Throughout Section 3, $\xi$ is assumed to be known and the null distribution does not involve any time index t. Therefore, $P_0$ is known, whereas $P=P_{\theta t}$ involves the tuning parameters $\theta$ and the time index t. It will be further assumed in Section 3.1 and Section 3.2 that the system is in equilibrium, or that the time index t is fixed, so that t can also be dropped under $H_1$ ($P=P_{\theta }$).

3.1. Exponential Tilting

Let $P_{\theta }$ be an exponentially tilted version of $P_0$ for some scalar tuning parameter $\theta >0$, which will also be called a tilting parameter. Exponential tilting is often used for rare events simulation [31,32]. Here, f is used to define the tilted version of $P_0$ as

$$\begin{array}{c}\hfill P_{\theta }\left(x\right)=\frac{e^{\theta f\left(x\right)}}{M\left(\theta \right)}{P}_0\left(x\right),\end{array}$$

(10)

with

$$\begin{array}{c}\hfill M\left(\theta \right)=\sum _{x\in \mathrm{\Omega }}{e}^{\theta f\left(x\right)}{P}_0\left(x\right)\end{array}$$

(11)

a normalizing constant assuring that $P_{\theta }$ is a probability measure. For countable sample spaces $\mathrm{\Omega }$, we interpret $P_0\left(x\right)$ and $P_{\theta }\left(x\right)$ as the probability masses, whereas for continuous sample spaces, they are probability densities and the sum in (11) is replaced by an integral. The larger the tilting parameter $\theta >0$, the more the probability mass of $P_{\theta }$ concentrates on regions of large f. In particular, $P_{\infty }$, the weak limit of $P_{\theta }$ as $\theta \to \infty$, is supported on (9) whenever (7) holds.

The parametric family

$$\begin{array}{c}\hfill \mathcal{P}=\{P_{\theta };\theta \ge 0\}\end{array}$$

(12)

of distributions is an exponential family [33] (Section 1.5), and each $P_{\theta }\in \mathcal{P}$ gives rise to a separate version of actinfo. This is summarized in the following proposition (cf. Section 7 for a proof):

Proposition 1.

Suppose the target set A is defined as in (5) for some $x_0\in \mathrm{\Omega }$ such that $P_0\left(A\right)>0$. Then, $P_{\theta }\left(A\right)$ is a strictly increasing function of $\theta \ge 0$ with $P_{\infty }\left(A\right)=1$. Consequently, the actinfo

$$\begin{array}{c}\hfill I^{+}\left(\theta \right)=log\frac{P_{\theta }\left(A\right)}{P_0\left(A\right)}\end{array}$$

(13)

is a strictly increasing function of $\theta \ge 0$, with $I^{+}\left(0\right)=0$ and $I^{+}(\infty )=I_{f0}=-log{P}_0\left(A\right)$.

The intuitive interpretation of Proposition 1 is that the larger $\theta$ is, the more problem-specific knowledge is infused into $P_{\theta }$ in terms of shifting probability mass towards regions in $\mathrm{\Omega }$ where f, the specificity function, is large.

A simple instance of exponential tilting is the student learning example of Section 1.2. Recall that $f\left(x\right)=x$ is the test score of a student, with $X\sim N(\xi ,1)$ for a randomly chosen student who did not prepare for the test ($H_0$), whereas $X\sim N(\xi +\theta ,1)$ is the test score of a randomly chosen student who prepared for the test during $t=1$ units of time ($H_1$). It is clear that

$$\begin{array}{ccc}\hfill P_0\left(x\right)& =& e^{-{(x-\xi )}^2/2}/\sqrt{2\pi },\hfill \\ \hfill P_{\theta }\left(x\right)& =& e^{-{(x-\xi -\theta )}^2/2}/\sqrt{2\pi }=P_0\left(x\right)e^{\theta x}/M\left(\theta \right).\hfill \end{array}$$

3.2. Metropolis–Hastings Systems with Exponential Tilting Equilibrium

Inspired by Markov Chain Monte Carlo methods [34], consider a Markov chain $X_0,X_1,\dots \in \mathrm{\Omega }$ for which $P_{\theta }$ is the equilibrium distribution. Consequently, if $P=P_{\theta }$ (that is, under the alternative hypothesis $H_1$ in (3) when $\theta >0$), $X=X_t$ may be interpreted as the outcome of an algorithm after t iterations, provided that t is so large that the equilibrium has been reached. The assumption is made that this algorithm knows f and tries to explore the whole state space $\mathrm{\Omega }$. If the Markov chain has an equilibrium distribution (10), this corresponds to an algorithm that favors jumps towards the regions of large f when $\theta >0$, an effect which is accentuated the higher the value of $\theta$ is. In further detail, the transition kernel of the chain is an instance of the well-known Metropolis–Hastings (MH) algorithm [35,36], which is closely related to simulated annealing [37]. This kernel has a probability or density

$$\begin{array}{c}\hfill {\pi }_{\theta }(x,y)=r_{\theta }\left(x\right)\delta (x,y)+{\alpha }_{\theta }(x,y)q(x,y)\end{array}$$

(14)

for jumps from x to y, where $\delta (x,·)$ is a point mass at $x\in \mathrm{\Omega }$, $q(x,·)$ is a proposal distribution of jumps from a current position x of the Markov chain,

$$\begin{array}{c}\hfill {\alpha }_{\theta }(x,y)=min\left[1,\frac{e^{\theta f\left(y\right)}{P}_0\left(y\right)q(y,x)}{e^{\theta f\left(x\right)}{P}_0\left(x\right)q(x,y)}\right]\end{array}$$

(15)

is the probability of accepting a proposed move from x to y, whereas

$$\begin{array}{c}\hfill r_{\theta }\left(x\right)=1-\sum _{y\in \mathrm{\Omega }}{\alpha }_{\theta }(x,y)q(x,y)\end{array}$$

(16)

is the probability that the Markov chain rejects a proposed move away from x (for continuous sample spaces $q(x,·)$ is a probability density and then the sum in (16) is replaced by an integral). The transition of the Markov chain from $X_t=x$ to the next state $X_{t+1}$ is described in two steps as follows. First, a candidate $Y\sim q(x,·)$ is proposed. Then, in the second step, this candidate is either accepted with a probability of ${\alpha }_{\theta }(x,Y)$, so that $X_{t+1}=Y$, or it is rejected with probability $1-{\alpha }_{\theta }(x,Y)$, so that $X_{t+1}=X_t$. It is well known that $P_{\theta }$ is the equilibrium distribution of this Markov chain whenever it is irreducible; that is, provided the proposal distribution q is defined in such a way that moving between any pair of states in $\mathrm{\Omega }$ in a finite number of steps is possible [38], pp. 243–245.

In particular, if q is symmetric and $P_0$ is uniform, then a proposed upward move with $f\left(Y\right)>f\left(x\right)$ and $P_{\theta }\left(Y\right)>P_{\theta }\left(x\right)$ is always accepted, whereas a proposed downward move with $f\left(Y\right)<f\left(x\right)$ is accepted with a probability of $P_{\theta }\left(Y\right)/P_{\theta }\left(x\right)$. The Markov chain only makes local jumps if $q(x,·)$ puts all its probability mass in a small neighborhood of x, for any $x\in \mathrm{\Omega }$. At the other extreme is a chain with the global proposal distribution $q(x,·)\sim P_{\theta }$ for any $x\in \mathrm{\Omega }$; all proposed jumps of this chain are then accepted ($\alpha (x,y)=1$), and ${\left\{X_t\right\}}_{t=1}^{\infty }$ is a sequence of independent and identically distributed (i.i.d.) random variables with $X_t\sim P_{\theta }$.

The choice of proposal distribution q is problem specific. In this section, we defined q for the Metropolis–Hastings type algorithms that require knowledge of the specificity function f, since the acceptance probability (15) is a function of f. Proposed moves also occur for evolutionary algorithms (Examples 4 and 5 of Section 5). These algorithms are typically the result of many small changes, with specificity corresponding to functionality or fitness. The proposed moves are local mutations that either survive (are accepted) or do not. Other algorithms (such as reinforcement learning in Example 3 of Section 5) only make use of estimates of the specificity function. However, it is still meaningful for these algorithms to talk about proposed moves that are initially large (exploration phase) followed by a subsequent period of small or no moves (exploitation phase). In the context of Metropolis–Hastings algorithms, this is the strategy of simulated annealing, where large moves are initially proposed (corresponding to high temperatures), followed by subsequent small proposed moves (corresponding to low temperatures).

3.3. Active Information for Metropolis–Hastings Systems in Non-Equilibrium

Suppose, for simplicity, that the sample space $\mathrm{\Omega }$ is finite, and that the states in $\mathrm{\Omega }$ are listed in some order. Let

$$\begin{array}{c}\hfill {\mathbf{P}}_0=(P_0\left(x\right);x\in \mathrm{\Omega })\end{array}$$

(17)

be a row vector of length $\left|\mathrm{\Omega }\right|$ with all the null distribution probabilities, and let

$$\begin{array}{c}\hfill {\mathrm{\Pi }}_{\theta }=\left({\pi }_{\theta }(x,y);x,y\in \mathrm{\Omega }\right)\end{array}$$

(18)

be a square matrix of order $\left|\mathrm{\Omega }\right|$ that defines the transition kernel of the Markov chain ${\left\{X_t\right\}}_{t=0}^{\infty }$ of Section 3.2. If $X_0\sim P_0$, then by the Kolmogorov–Chapman equation $X_t\sim P_{\theta t}$, where

$$\begin{array}{c}\hfill (P_{\theta t}\left(x\right);x\in \mathrm{\Omega })={\mathbf{P}}_{\theta t}={\mathbf{P}}_0{\mathrm{\Pi }}_{\theta }^t.\end{array}$$

(19)

Hence, if $P=P_{\theta t}$, then $X=X_t$ corresponds to observing the Markov chain at time t, under the alternative hypothesis $H_1$ in (3). Some basic properties of the corresponding actinfo are summarized in the following proposition, which is proved in Section 7:

Proposition 2.

Suppose that $X=X_t$ is obtained by iterating t times a Markov chain with initial distribution (17) and transition kernel (18). The actinfo then equals

$$I^{+}(\theta ,t)=log\frac{P_{\theta t}\left(A\right)}{P_0\left(A\right)}=log\frac{{\mathbf{P}}_0{\mathrm{\Pi }}_{\theta }^t\mathbf{v}}{{\mathbf{P}}_0\mathbf{v}},$$

(20)

where $\mathbf{v}$ is a column vector of length $\left|\mathrm{\Omega }\right|$ with ones in positions $x\in A$ and zeros in positions $x\in A^c$. In particular, $I^{+}(\theta ,0)=0$ and

$$\underset{t\to \infty }{lim}{I}^{+}(\theta ,t)=I^{+}\left(\theta \right).$$

(21)

Therefore, $I^{+}(\theta ,t)>0$ corresponds to knowledge of f being used to generate t jumps of the Markov chain, under the alternative hypothesis $H_1$ in (3).

3.4. Active Information for Metropolis–Hastings Systems with Stopping

In Section 3.3, $P\sim P_{\theta t}$ was obtained by starting a random search with null distribution $P_0$, and then iterating the Markov chain of Section 3.2t times. However, knowledge of f can be utilized even more and stop the Markov chain if the target A in (5) is reached before time t. This can be formalized by introducing the stopping time

$$\begin{array}{c}\hfill T=min\{t\ge 0;X_t\in A\}\end{array}$$

(22)

and letting

$$\begin{array}{c}\hfill P_{\theta ts}\left(x\right)=P(X_{t\wedge T}=x)\end{array}$$

(23)

be the probability distribution of the stopped Markov chain $X_{t\wedge T}$, with the last index s in (23) being an acronym for stopping. In particular,

$$\begin{array}{c}\hfill P_{\theta ts}\left(A\right)=\sum _{x\in A}{P}_{\theta ts}\left(x\right)=P(T\le t)\end{array}$$

(24)

is the probability of reaching the target A for the first time after t iterations or earlier. The theory of phase-type distributions can then be used to compute the target probability $P_{\theta ts}\left(A\right)$ in (23) [39,40]. To this end, clump all states $x\in A$ into one absorbing state, and decompose the transition kernel in (18) according to

$$\begin{array}{c}\hfill {\mathrm{\Pi }}_{\theta }=\left(\begin{array}{cc}{\mathrm{\Pi }}_{\theta }^{\mathrm{na}}& {\mathrm{\Pi }}_{\theta }^{\mathrm{na},\mathrm{a}}\\ \mathbf{0}& 1\end{array}\right),\end{array}$$

(25)

where ${\mathrm{\Pi }}_{\theta }^{\mathrm{na}}$ is a square matrix of order $|A^c|$ containing the transition probabilities between all non-absorbing states in $A^c$, whereas ${\mathrm{\Pi }}_{\theta }^{\mathrm{na},\mathrm{a}}$ is a column vector of length $|A^c|$ with transition probabilities $\pi (x,A)$ from all the non-absorbing states $x\in A^c$ into the absorbing state A. Moreover, ${\mathbf{P}}_0^{\mathrm{na}}=\left(P_0\left(x\right);x\in A^c\right)$ is a row vector of length $|A^c|$ that is the restriction of the start-distribution ${\mathbf{P}}_0$ in (17) to all non-absorbing states. Then

$$\begin{array}{c}\hfill P_{\theta ts}\left(A\right)=1-{\mathbf{P}}_0^{\mathrm{na}}{\left({\mathrm{\Pi }}_{\theta }^{\mathrm{na}}\right)}^t\mathbf{1},\end{array}$$

(26)

where $\mathbf{1}$ is a column vector of $|A^c|$ ones.

The actinfo $I_s^{+}$ of a search procedure with stopping is thus defined:

Proposition 3.

Suppose that $X=X_t$ is obtained by iterating a Markov chain with an initial distribution (17) and a transition kernel (18) (for some $\theta \ge 0$) at most t times, and stopping whenever the set A is reached. Then, the actinfo is given by

$$\begin{array}{c}\hfill I_s^{+}(\theta ,t)=log\frac{P_{\theta ts}\left(A\right)}{P_0\left(A\right)}=log\frac{1-{\mathbf{P}}_0^{\mathrm{na}}{\left({\mathrm{\Pi }}_{\theta }^{\mathrm{na}}\right)}^t\mathbf{1}}{{\mathbf{P}}_0\mathbf{v}},\end{array}$$

(27)

with ${\mathbf{P}}_0$ and $\mathbf{v}$ as in Proposition 2, whereas ${\mathbf{P}}_0^{\mathrm{na}}$, ${\mathrm{\Pi }}_{\theta }^{\mathrm{na}}$, and $\mathbf{1}$ are defined below (25) and (26). This actinfo satisfies

$$\begin{array}{c}\hfill I_s^{+}(\theta ,t)\ge I^{+}(\theta ,t)\end{array}$$

(28)

and $I_s^{+}(\theta ,t)$ is a non-decreasing function of t such that

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}{I}_s^{+}(\theta ,t)=I_{f0}\end{array}$$

(29)

and

$$\begin{array}{c}\hfill \sum _{t=0}^{\infty }\left(1-P_0\left(A\right)e^{I_s^{+}(\theta ,t)}\right)=E\left(T\right).\end{array}$$

(30)

Proposition 3 is proven in Section 7. Inequality (28) states that, for a search procedure with t iterations, knowledge about f that is used for stopping the Markov chain in (18) will increase the actinfo, regardless of whether knowledge about f was used ($\theta >0$) or not ($\theta =0$) when iterating the Markov chain. Equation (29) is a consequence of the fact that target A is eventually reached with probability 1, so that the actinfo of a search procedure with stopping equals the functional information $I_{f0}=-log{P}_0\left(A\right)$ after many iterations of the Markov chain. Moreover, Equation (30) tells that the rate at which $P_0\left(A\right)e^{I_s^{+}(\theta ,t)}$ approaches 1 is determined by the expected waiting time $E\left(T\right)$ of reaching the target.

From Proposition 3, actinfo for a system with stopping is closely related to the phase-type distribution of the waiting time T until the target is reached. This has been studied in [41], in the context of the expression of a number of genes, with x being the collection of the regulatory regions of all these genes.

4. Estimating Active Information and Testing Fine-Tuning

In Section 3, we gave explicit expressions of the actinfo, for Metropolis–Hastings algorithms with a scalar tuning parameter $\theta$. In general, however, it might be infeasible to calculate $I^{+}$, either because the sample space is very large, or the nuisance parameters $\xi$ and/or the tuning parameters $\theta$ are unknown. If is of interest then to consider ways of estimating $I^{+}$ from data, for instance through Monte Carlo-based methods. To this end, we will assume that the random search algorithm is repeated independently, under the same conditions, n times. For instance, suppose that ${\left\{X_{it}\right\}}_{t=0}^{\infty }$ corresponds to independent realizations $i=1,\dots ,n$ of a search algorithm. If these independent realizations are recorded or stopped at one single time point, the outcome is either $X_i=X_{it}$ for $i=1,\dots ,n$, or $X_i=X_{i,t\wedge T_i}$, for $i=1,\dots ,n$, depending on whether the search algorithm is stopped at a fixed time point t or at random time points ${\left\{T_i\right\}}_{i=1}^n$. In either case, an output of i.i.d. random variables

$$X_1,\dots ,X_n\sim Q$$

(31)

is obtained. These repeated outcomes of the search algorithm will be used to test for and estimate the degree of fine-tuning. The methodology depends on whether the null distribution $P_0$ is known or involves unknown nuisance parameters.

4.1. Null Distribution Known

Suppose the null distribution $P_0$ is known. The sample in (31) is then used for testing between the two hypotheses

$$\begin{array}{c}\hfill \begin{array}{cc}{H}_0:\hfill & Q=P_0,\hfill \\ H_1:\hfill & Q\in {\mathcal{P}}_1,\hfill \end{array}\end{array}$$

(32)

with

$${\mathcal{P}}_1=\{P;P\left(A\right)\ge p_{\min }\}$$

(33)

the set of distributions that correspond to fine-tuning. Suppose an estimate $\widehat{Q}\left(A\right)$ of the probability that $X\in A$ is computed from data (31), with an associated empirical actinfo

$$\begin{array}{c}\hfill {\widehat{I}}^{+}={\widehat{I}}_n^{+}=log\frac{\widehat{Q}\left(A\right)}{P_0\left(A\right)}.\end{array}$$

(34)

If $\widehat{Q}\left(A\right)$ is a consistent estimator of $Q\left(A\right)$, then for large sample sizes, ${\widehat{I}}^{+}$ will be close to

$$\begin{array}{c}\hfill I_Q^{+}=log\frac{Q\left(A\right)}{P_0\left(A\right)},\end{array}$$

(35)

which equals 0 under $H_0$ and $I^{+}=I_P^{+}$ under $H_1$, for some particular $P\in {\mathcal{P}}_1$. To test $H_0$ against $H_1$,

$$\begin{array}{c}\hfill \mathrm{reject}{H}_0\mathrm{when}{\widehat{I}}^{+}\ge I_{\min },\end{array}$$

(36)

where $I_{\min }$ is a pre-specified lower bound on the range of values of the actinfo that corresponds to FT.

4.1.1. Nonparametric Estimator and Test

In Section 3, $P=P_{\theta }$, $P=P_{\theta t}$, or $P=P_{\theta ts}$ were used for distributions that make use of pre-specified knowledge. These distributions involve the tilting parameter $\theta$, and possibly also the number of iterations t of the algorithm and a stopping time T. In this section, however, no other assumption than $P\in {\mathcal{P}}_1$ is made on P, and a nonparametric version of the empirical actinfo is used. The fraction

$$\begin{array}{c}\hfill \widehat{Q}\left(A\right)=\frac{1}{n}\sum _{i=1}^n{1}_{\{X_i\in A\}}\end{array}$$

(37)

of random searches that fall into A is used as an estimate of $Q\left(A\right)$. Therefore, (37) only requires the knowledge of the set A, not of the function f.

The following result establishes the asymptotic normality of the nonparametric version of the estimator ${\widehat{I}}^{+}$ in (34). Moreover, large deviations [42] are used to show that the significance level of the nonparametric version of the FT test (36) goes to zero exponentially fast with n (see Section 7 for more details of the proof).

Proposition 4.

Suppose the empirical actinfo ${\widehat{I}}_n^{+}$ in (34) is computed nonparametrically, using (37) as an estimate of the target probability $Q\left(A\right)$. Then, ${\widehat{I}}_n^{+}$ is an asymptotically normal estimator of $I_Q^{+}$ in (35), in the sense that

$$\begin{array}{c}\hfill \sqrt{n}({\widehat{I}}_n^{+}-I_Q^{+})\stackrel{\mathcal{L}}{⟶}N(0,V)\mathrm{as}n\to \infty ,\end{array}$$

(38)

where $\stackrel{\mathcal{L}}{⟶}$ refers to convergence in distribution, and

$$\begin{array}{c}\hfill V=\frac{1-Q\left(A\right)}{Q\left(A\right)}\end{array}$$

(39)

is the variance of the limiting normal distribution. The significance level of the test (36) for fine-tuning, with threshold $I_{\min }$, satisfies

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{log\left(P_{H_0}({\widehat{I}}_n^{+}\ge I_{\min })\right)}{n}=C,\end{array}$$

(40)

where

$$\begin{array}{c}\hfill C=p_{\min }log\frac{p_{\min }}{P_0\left(A\right)}+(1-p_{\min })log\frac{1-p_{\min }}{1-P_0\left(A\right)}\end{array}$$

(41)

is the Kullback–Leibler divergence between Bernoulli distributions with success probabilities $p_{\min }=P_0\left(A\right)exp\left(I_{\min }\right)$ and $P_0\left(A\right)$, respectively.

Remark 1.

The conclusion of Proposition 4 is that the probability of observing actinfo that corresponds to fine-tuning by chance decays at rate $e^{-Cn}$ when the sample size n becomes large.

4.1.2. Parametric Estimator and Test

Suppose that there is a priori knowledge that P is close to the parametric exponential family $\mathcal{P}$ of distributions in (10)–(12) for some value $\theta >0$ of the tilting parameter. A parametric test of actinfo is naturally defined. For this, first compute the maximum likelihood estimate

$$\begin{array}{c}\hfill \widehat{\theta }={\widehat{\theta }}_n=arg\underset{\theta \ge 0}{max}\sum _{i=1}^{n}log{P}_{\theta }\left(X_i\right)\end{array}$$

(42)

of $\theta$, and use it to define a parametric estimate

$$\begin{array}{c}\hfill \widehat{Q}\left(A\right)=P_{\widehat{\theta }}\left(A\right)\end{array}$$

(43)

of the target probability $Q\left(A\right)$ that is inserted into (34) to define a parametric version of the empirical actinfo ${\widehat{I}}^{+}$. As opposed to (37), the estimate (43) requires the full knowledge of f.

To analyze the properties of the estimator (34) and test (36), introduce

$$\begin{array}{c}\hfill {\theta }^{*}=arg\underset{\theta \ge 0}{min}{D}_{KL}(Q\parallel P_{\theta }),\end{array}$$

(44)

where

$$\begin{array}{c}\hfill D_{KL}(Q\parallel P_{\theta })=\sum _{x\in \mathrm{\Omega }}Q\left(x\right)log\frac{Q\left(x\right)}{P_{\theta }\left(x\right)}\end{array}$$

(45)

is the Kullback–Leibler divergence between Q and $P_{\theta }$. From (44), $P_{{\theta }^{*}}$ is the distribution in $\mathcal{P}$ that best approximates Q. In particular, ${\theta }^{*}=\theta$ if $Q\in \mathcal{P}$ and $Q=P_{\theta }$ for some $\theta \ge 0$.

The following proposition shows that ${\widehat{I}}^{+}$ is an asymptotically normal estimator of $I^{+}\left({\theta }^{*}\right)$ in (13), which differs from $I_Q^{+}$ in (35) whenever $Q\notin \mathcal{P}$. Moreover, the proposition also provides large sample properties of the significance level of the test for actinfo (cf. Section 7 for details of the proof):

Proposition 5.

Suppose the empirical actinfo ${\widehat{I}}_n^{+}$ in (34) is computed parametrically, using an estimate (43) of the target probability $Q\left(A\right)$. Then, ${\widehat{I}}_n^{+}$ is an asymptotically normal estimator of $I^{+}\left({\theta }^{*}\right)$, in the sense that

$$\begin{array}{c}\hfill \sqrt{n}\left({\widehat{I}}_n^{+}-I^{+}\left({\theta }^{*}\right)\right)\stackrel{\mathcal{L}}{⟶}N(0,V)\mathrm{as}n\to \infty ,\end{array}$$

(46)

where the variance of the limiting normal distribution is given by

$$\begin{array}{c}\hfill V=\frac{{\mathit{Cov}}_{P_{{\theta }^{*}}}^2\left[f\left(X\right)I(f\left(X\right)\ge f_0)\right]{\mathit{Var}}_Q\left[f\left(X\right)\right]}{P_{{\theta }^{*}}^2\left(A\right){\mathit{Var}}_{P_{{\theta }^{*}}}^2\left[f\left(X\right)\right]}.\end{array}$$

(47)

Moreover, the significance level of the parametric test for fine-tuning, based on (36) and (43), satisfies

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{log\left[P_{H_0}\left({\widehat{I}}_n^{+}\ge I_{\min }\right)\right]}{n}=C,\end{array}$$

(48)

for

$$\begin{array}{c}\hfill C=\underset{\varphi >0}{sup}\left\{\varphi E_{P_{\min }}\left[f\left(X\right)\right]-logM\left(\varphi \right)\right\},\end{array}$$

(49)

where $P_{\min }=P_{{\theta }_{\min }}$, ${\theta }_{\min }<{\theta }^{*}$ is the solution of $P_{{\theta }_{\min }}\left(A\right)=p_{\min }=P_0\left(A\right)exp\left(I_{\min }\right)$, $M\left(\varphi \right)$ is given by (11), whereas $p_{\min }$ is defined in (33).

4.1.3. Comparison between Nonparametric and Parametric Estimates of Actinfo

The two versions of empirical actinfo are complementary. The nonparametric version is preferable in the sense that it makes less assumptions about the distribution P of the random algorithm under $H_1$, and in particular, it is a consistent estimator of $I_Q^{+}$ in (35). The parametric version of ${\widehat{I}}^{+}$, on the other hand, is preferable when $nQ\left(A\right)$ is small, since it makes use of all data in order to estimate $Q\left(A\right)$, although it is not a consistent estimator of $I_Q^{+}$ when $Q\notin \mathcal{P}$. The asymptotic variances in (39) and (47), as well as the rates of exponential significance level decrease in (41) and (49), agree when $Q=P_{{\theta }^{*}}$ and $f\left(x\right)=f_0{1}_{\{x\in A\}}$, which is a special case of (8).

4.2. Null Distribution Unknown

Suppose that the null distribution $P_0=P_{0\xi }$ involves an unknown nuisance parameter $\xi \in \mathrm{\Xi }$. The objective is then to test the two hypotheses

$$\begin{array}{c}\hfill \begin{array}{cc}{H}_0:\hfill & Q\in {\mathcal{P}}_0,\hfill \\ H_1:\hfill & Q\in {\mathcal{P}}_1,\hfill \end{array}\end{array}$$

(50)

where the set of distribution under the null and alternative hypotheses equals

$${\mathcal{P}}_0=\{P_{0\xi };\xi \in \mathrm{\Xi }\}$$

(51)

and (33), respectively.

4.2.1. One Sample Available

The actinfo

$$I_Q^{+}=I_Q^{+}\left(\xi \right)=log\frac{Q\left(A\right)}{P_{0\xi }\left(A\right)}$$

(52)

cannot be consistently estimated if only one sample (31) is available. The best course of action is thus to estimate a lower bound

$${\widehat{I}}^{+}={\widehat{I}}_n^{+}=log\frac{\widehat{Q}\left(A\right)}{P_{0\max }\left(A\right)}$$

(53)

of $I^{+}$, with $P_{0\max }\left(A\right)$ defined in (4) and $\widehat{Q}\left(A\right)$ an estimate of $Q\left(A\right)$. This estimator will have an asymptotic bias

$$B=I_Q^{+}\left({\xi }^{*}\right)-I_Q^{+}=log\frac{P_{0\xi }\left(A\right)}{P_{0\max }\left(A\right)}\le 0,$$

(54)

where ${\xi }^{*}$ is the nuisance parameter that maximizes $P_{0\xi }\left(A\right)$[43]. For the numerator of (53), either the nonparametric estimate of $Q\left(A\right)$ in (37) can be used, or a parametric class

$$\mathcal{P}=\{P_{\theta \xi };\theta \in \mathrm{\Theta },\xi \in \mathrm{\Xi }\}$$

of distributions can be used that involves a tuning parameter vector $\theta$ and a vector of nuisance parameters $\xi$. If Q is thought to be close to $\mathcal{P}$, the parametric estimate

$$\widehat{Q}\left(A\right)=P_{\widehat{\theta }\widehat{\xi }}\left(A\right)$$

(55)

of $Q\left(A\right)$ is used, which generalizes (43), with

$$(\widehat{\theta },\widehat{\xi })=arg\underset{\theta ,\xi }{max}\sum _{i=1}^{n}log{P}_{\theta \xi }\left(X_i\right).$$

(56)

When the sample size n tends towards infinity, the estimator (56) will converge to

$$({\theta }^{*},{\xi }^{*})=arg\underset{\theta ,\xi }{min}{D}_{KL}(Q\parallel P_{\theta \xi }).$$

(57)

The following result is an extension of Propositions 4 and 5, when nuisance parameters $\xi$ are added and a general type of tuning parameter $\theta$ (not necessarily a scalar tilting parameter) is used. A short proof of the proposition is offered in Section 7.

Proposition 6.

Suppose that the null distribution $P_0=P_{0\xi }$ involves an unknown parameter ξ and the actinfo $I_Q^{+}$ in (52) is estimated by ${\widehat{I}}_n^{+}$ in (53), using an estimator $\widehat{Q}\left(A\right)$ of the target probability $Q\left(A\right)$ that is either nonparametric (37) or parametric (55). Given these assumptions, ${\widehat{I}}_n^{+}$ is an asymptotically normal estimator, in the sense that

$$\sqrt{n}({\widehat{I}}_n^{+}-I_Q^{+}-B)\stackrel{\mathcal{L}}{⟶}N(0,V)\mathrm{as}n\to \infty .$$

(58)

The asymptotic bias B in (58) is defined in (54) whereas the asymptotic variance V is defined in (39) for the nonparametric estimator of $I_Q^{+}$, whereas

$$\begin{array}{ccc}\hfill V& =& E\left[{\psi }_{{\theta }^{*}{\xi }^{*}}\left(X\right)\right|X\in A]E{\left[{\psi }_{{\theta }^{*}{\xi }^{*}}^{\prime }\left(X\right)\right]}^{-1}E\left[{\psi }_{{\theta }^{*}{\xi }^{*}}^T\left(X\right){\psi }_{{\theta }^{*}{\xi }^{*}}\left(X\right)\right]\hfill \\ \hfill & ·& E{\left[{\left({\psi }_{{\theta }^{*}{\xi }^{*}}^{\prime }\right)}^T\left(X\right)\right]}^{-1}E{\left[{\psi }_{{\theta }^{*}{\xi }^{*}}\left(X\right)\right|X\in A]}^T\hfill \end{array}$$

(59)

for the parametric estimator of $I_Q^{+}$, with ${\psi }_{\theta \xi }\left(x\right)=dlog{P}_{\theta \xi }\left(x\right)/d(\theta ,\xi )$, $({\theta }^{*},{\xi }^{*})$ defined as in (57), and T refering to matrix transposition. Moreover, the significance level of the test (36) of FT, with threshold $I_{\min }$, satisfies

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{log\left[P_{0\xi }\left({\widehat{I}}_n^{+}\ge I_{\min }\right)\right]}{n}=C,\end{array}$$

(60)

with

$$C=p_{\min }{e}^{-B}log\frac{p_{\min }{e}^{-B}}{P_{0\xi }\left(A\right)}+(1-p_{\min }{e}^{-B})log\frac{1-p_{\min }{e}^{-B}}{1-P_{0\xi }\left(A\right)}$$

(61)

for the nonparametric version of the test, with $p_{\min }=P_{0\xi }\left(A\right)exp\left(I_{\min }\right)$. For the parametric versions of the FT-test, and in the special case when θ is a scalar exponential tilting parameter, C is given by (49), with $P_{\min }=P_{{\theta }_{min}\xi }$, and ${\theta }_{\min }$ the solution of $P_{{\theta }_{min}\xi }\left(A\right)=p_{\min }{e}^{-B}$.

Remark 2.

The negative bias term B makes the test of FT in Proposition 6 more conservative than the tests in Propositions 4 and 5. This can be seen, for instance, by comparing the two large deviation rates C in (41) and (61). The rate in (61) is larger, since $p_{\min }$ is multiplied by a term $e^{-B}$. This corresponds to the fact that to falsely reject $H_0$ in Proposition 6 is more difficult.

4.2.2. Two Samples Available

In addition to the first sample (31), suppose a second sample

$$X_{01},\dots ,X_{0n_0}\sim P_{0\xi }$$

(62)

of $n_0$ i.i.d. observations under the null distribution is available. A consistent estimator

$${\widehat{I}}^{+}={\widehat{I}}_{n{n}_0}^{+}=log\frac{\widehat{Q}\left(A\right)}{P_{0\widehat{\xi }}\left(A\right)}$$

(63)

of $I_Q^{+}$ in (52) is then available, with

$$\widehat{\xi }=arg\underset{\xi }{max}\sum _{i=1}^{n_0}log{P}_{0\xi }\left(X_{0i}\right).$$

(64)

The following result provides asymptotic properties of the estimator (63) of actinfo, and the corresponding test (36) of FT with threshold $I_{\min }$ (cf. Section 7 for a proof):

Proposition 7.

Suppose that the null distribution $P_0=P_{0\xi }$ involves an unknown nuisance parameter ξ, and that the active information $I_Q^{+}$ in (52) is estimated by ${\widehat{I}}_{n{n}_0}^{+}$ in (63), making use of two samples (31) and (62), of sizes n and $n_0$, from Q and $P_{0\xi }$, respectively. Further assume that the estimator $\widehat{Q}\left(A\right)$ of $Q\left(A\right)$ is either nonparametric (37) or parametric (55). If $n,n_0\to \infty$ in such a way that

$$\frac{n}{n_0}\to \lambda >0,$$

(65)

then

$$\sqrt{n}({\widehat{I}}_{n{n}_0}^{+}-I_Q^{+})\stackrel{\mathcal{L}}{⟶}N(0,V_1+\lambda V_2),$$

(66)

where

$$V_2=E\left[{\psi }_{\xi }\left(X\right)\right|X\in A]E{\left[{\psi }_{\xi }^T\left(X\right){\psi }_{\xi }\left(X\right)\right]}^{-1}E{\left[{\psi }_{\xi }\left(X\right)\right|X\in A]}^T,$$

(67)

and ${\psi }_{\xi }\left(x\right)=dlog{P}_{0\xi }\left(x\right)/d\xi$. If the nonparametric estimator of $Q\left(A\right)$ is used, then $V_1$ equals V in (39), whereas if the parametric estimator $Q\left(A\right)$ is used, then $V_1$ equals V in (59). The significance level of the test (36) of FT, with threshold $I_{\min }$, satisfies the same type of large deviation result (60) as in Proposition 6, for the nonparametric and parametric versions of the test (in the latter case assuming that θ is a scalar tilting parameter), but in the definitions of the nonparametric and parametric large deviation rates C, the bias term $B=0$.

5. Examples

In this section, we provide five examples. The first cosmology example is a continuation of Section 1.1, with specificity corresponding to a universe that permits life. The second example of student learning was introduced in Section 1.2, with specificity being the test score of a student who prepares for a test. The third example concerns reinforcement learning, with specificity the cumulative reward of a certain trajectory of actions and environments. The last two examples concern evolutionary algorithms for generating molecular machines, with specificity corresponding to the functionality or fitness of these machines. These evolutionary algorithms can be viewed as extensions or variants of the Metropolis–Hastings algorithms of Section 3.2, where proposed moves correspond to mutations, whereas accepted moves correspond to mutations that survive and then possibly spread to a whole population.

Example 1

(Cosmology [26,27]). Suppose that there is a positive constant of nature $X\in \mathrm{\Omega }={\mathbb{R}}^{+}$, a life-permitting interval $A\subset \mathrm{\Omega }$, and a specificity function (6) that equals 1 inside $A=(a,b)$ and zero elsewhere. The maximum entropy distribution under a first moment constraint $\xi =E\left(X\right)$ is exponential with expected value. Consequently,

$$P_{0\xi }\left(A\right)=\frac{1}{\xi }{\int }_a^b{e}^{-x/\xi }dx.$$

The null and alternative hypotheses for the fine-tuning test are given in (50), where under $H_1$, the agent brings about a life-permitting value of X with probability 1 ($P\left(A\right)=1$). Only one universe is observed, with a value $X=X_1$ of the constant. Therefore, there is a sample (31) of size $n=1$, whereas no null sample (62) is available. Since $X_1\in A$ is life-permitting, $\widehat{Q}\left(A\right)=1$. The estimate (53) of actinfo then simplifies to

$${\widehat{I}}^{+}=log\frac{1}{P_{0\max }\left(A\right)}=-log{P}_{0\max }\left(A\right).$$

(68)

Let $x=(a+b)/2$ be the midpoint of the LPI and suppose that half of its relative size $ϵ=(b-a)/\left(2x\right)$ is small. The probability in (68) is then approximated by

$$P_{0\max }\left(A\right)\approx (b-a)\underset{\xi >0}{max}\frac{e^{-x/\xi }}{\xi }\approx 2ϵe^{-1}.$$

From (68), the estimated actinfo

$${\widehat{I}}^{+}\approx 1-log\left(ϵ\right)-log\left(2\right)$$

is a monotone decreasing function of $ϵ$.

Example 2

(Evaluation of student test scores [44]). As a generalization of the example given in Section 1.2, suppose that a number of students perform a test. Let $x=(z,y)=(z_1,\dots ,z_{d-1},y)\in {\mathbb{R}}^d$ summarize the chararcteristics of a student with covariates z that are used to predict the outcome y of the test. The specificity function $f\left(x\right)=x_d=y$ equals the student’s test score, and (5) corresponds to the set of students that pass the test, with a minimally allowed score of $f_0$. The population of students follows a $(d-1)$-dimensional multivariate normal distribution $Z\sim N(,\Sigma )$, where $=(m_1,\dots ,m_{d-1})$ and $\Sigma ={\left({\sigma }_{jk}\right)}_{j,k=1}^{d-1}$ are known. The conditional distribution of the response follows a multiple linear regression model

$$Y|Z=z\sim N\left({\xi }_0+\sum _{j=1}^{d-1}{\xi }_j{z}_j+t({\theta }_0+\sum _{j=1}^{d-1}{\theta }_j{z}_j),{\sigma }^2\right),$$

for a student with a covariate vector z who prepared for the test for a period of length t. The nuisance parameter vector $\xi =({\xi }_0,\dots ,{\xi }_{d-1},{\sigma }^2)$ involves the error variance and the regression parameters for students who did not train for the test, whereas the tuning parameter vector $\theta =({\theta }_0,\dots ,{\theta }_{d-1})$ involves the regression parameters that correspond to the effect of preparing for the test. The unconditional distribution of the response is normal, $Y\sim N(\mu ,V)$, with

$$\begin{array}{ccc}\hfill \mu & =& \mu (\theta ,\xi ,t)=({\xi }_0+t{\theta }_0)+\sum _{j=1}^{d-1}({\xi }_j+t{\theta }_j)m_j,\hfill \\ \hfill V& =& V(\theta ,\xi ,t)={\sigma }^2+\sum _{j,k=1}^{d-1}({\xi }_j+t{\theta }_j)({\xi }_k+t{\theta }_k){\sigma }_{jk}.\hfill \end{array}$$

Therefore, the probability that a randomly chosen student that studied for the test for a period of length t passes is

$$P\left(A\right)=P_{\theta \xi t}\left(A\right)=P(Y\ge f_0)=1-\mathrm{\Phi }\left(\frac{f_0-\mu }{\sqrt{V}}\right),$$

(69)

where $\mathrm{\Phi }$ is the cumulative distribution function of a standard normal distribution. The null distribution $P_0=P_{0\xi }$ corresponds to putting $t=0$ in (69). Thus, the actinfo

$$I^{+}=I^{+}(\theta ,\xi ,t)=log\frac{1-\mathrm{\Phi }\left((f_0-\mu (\theta ,\xi ,t)/\sqrt{V(\theta ,\xi ,t)}\right)}{1-\mathrm{\Phi }\left((f_0-\mu (0,\xi ,0)/\sqrt{V(0,\xi ,0)}\right)}$$

(70)

quantifies how much learning, during a period of length t, increases the probability of passing the test. To compute an estimate ${\widehat{I}}^{+}$ of $I^{+}$ in (70), estimates $\widehat{\xi }$ and $\widehat{\theta }$ of $\xi$ and $\theta$ are needed. This can be achieved by collecting two training samples, as in (63). Another option is to compute the least squares estimates ($\widehat{\xi },\widehat{\theta }$) of the nuisance and the tuning parameters jointly, without bias, from one single dataset ${\left\{(t_i,z_i,y_i)\right\}}_{i=1}^n$, provided that the time periods $t_i$ vary, so that all parameters are identifiable.

Example 3

(Reinforcement learning (RI) [45]). Consider an agent whose purpose is to maximize the reward $f\left(x\right)$ of a trajectory x that they to some extent will be able to control, for a time period of length t. At each time point u, there are m possible environments $\mathcal{S}=\{s_1,\dots ,s_m\}$ and q possible actions $\mathcal{A}=\{a_1,\dots ,a_q\}$ to take. The state space $\mathcal{X}={\mathcal{A}}^t\times {\mathcal{S}}^{t+1}$ consists of all possible trajectories

$$x=(a_0,\dots ,a_{t-1},s_0,\dots ,s_t)$$

of environments and actions, where $s_u$ is the environment and $a_u$ the action taken at time u. A corresponding random trajectory is denoted with capital letters

$$X=(A_0,\dots ,A_{t-1},S_0,\dots ,S_t).$$

If the environment of the system is $S_u=s$ at time u, and action $A_u=a$ is taken, the probability of moving to environment $s^{\prime }$ is $P_a(s,s^{\prime })=P(S_{u+1}=s^{\prime }|S_u=s,A_u=u)$, with an instantaneous reward of $R_a(s,s^{\prime })$. If future rewards are discounted by a factor $\gamma$, the total reward, over a time horizon of length t, is

$$f\left(x\right)=\sum _{u=0}^t{R}_{a_u}(s_u,s_{u+1}){\gamma }^u.$$

Let $f_0$ be a lower bound for a trajectory’s total discounted reward to be acceptable, so that A in (5) is the set of all acceptable trajectories. The agent takes action according to some policy to make the expected total reward of a trajectory as large as possible. To this end, consider stationary policies, where the action $A_u$ taken by the agent at each time point u is only determined by the current environment $s_u$, according to some matrix $\mathrm{\Pi }=\left(\pi \right(s,a);s\in \mathcal{S},a\in \mathcal{A})$ of transition probabilities $\pi (s,a)=P(A_u=u|S_u=s)$. For a completely random policy

$$\pi (s,a)={\xi }_a;a=1,\dots ,q,$$

the action is not influenced by the current environment, and it is completely specified by the vector $\xi =({\xi }_1,\dots ,{\xi }_q)$ of nuisance parameters. Thus, $P_0\left(A\right)=P_{0\xi t}(f\left(X\right)\ge f_0)$ is the probability that an ignorant agent with policy determined by $\xi$, will have an acceptable trajectory. An agent who knows the reward function $R_a$ and the dynamics $P_a$ of the environment will try to take this knowledge into account to formulate a policy that makes the reward as large as possible. A deterministic policy $\theta :\mathcal{S}\to \mathcal{A}$ is a function that takes a unique action for each environment, so that

$$\pi (s,a)=1_{\{a=\theta (s\left)\right\}}.$$

Thus, $P\left(A\right)=P_{\theta t}(f\left(X\right)\ge f_0)$ is the probability that an agent with deterministic policy $\theta$ obtains an acceptable trajectory. The active information

$$I^{+}=I^{+}(\theta ,\xi ,t)=log\frac{P_{\theta }({\sum }_{u=0}^t{R}_{A_u}(S_u,S_{u+1}){\gamma }^u\ge f_0)}{P_{0\xi }({\sum }_{u=0}^t{R}_{A_u}(S_u,S_{u+1}){\gamma }^u\ge f_0)}$$

(71)

quantifies, on a logarithmic scale, how much more likely it is for an agent with policy $\theta$ to obtain an acceptable trajectory, compared to an ignorant agent with policy $\xi$. The values $\xi$ and $\theta$ are varied during the exploration phase of RI, but they are assumed to be known during the exploitation phase of RI. Suppose that we want to compute the actinfo (71) during the exploitation phase. Since $P_0\left(A\right)$ and $P\left(A\right)$ are typically unknown, they have to be estimated by Monte Carlo. To this end, assume we have two samples (31) and (62) of n and $n_0$ trajectories available, from $Q=P_{\theta t}$ and $Q=P_{0\xi t}$, respectively. Then, ${\widehat{I}}^{+}$ in (63) can be used to estimate the actinfo (71).

Example 4

(Molecular machines and Moran models [15,30,41]). Suppose that $\mathrm{\Omega }$ consists of all $2^d$ binary sequences $x=(x_1,\dots ,x_d)$ of length d, with a null distribution $P_0\left(x\right)$ that will be chosen below. The specificity function f is defined as

$$\begin{array}{c}\hfill f\left(x\right)=\left\{\begin{array}{cc}a\left|x\right|,\hfill & x\ne (1,\dots ,1),\hfill \\ 1,\hfill & x=(1,\dots ,1),\hfill \end{array}\right.\end{array}$$

(72)

where $\left|x\right|={\sum }_{i=1}^d{x}_i$ and $a\le 1/d$ is a fixed parameter. We regard x as a molecular machine with d parts, with $x_i=1$ or 0 depending on whether part i functions or not. The specificity $f\left(x\right)$ quantifies how well the machine works, for instance, its ability to regulate activity in vitro or in vivo in a living cell. It is assumed that $f\left(x\right)$ is determined by the number $\left|x\right|$ of functioning parts, with a maximal value $f_{\max }=f(1,\dots ,1)=1$. Using (8), the most stringent definition of high specificity, it follows that $A=\left\{\right(1,\dots ,1\left)\right\}$ only contains one element, a molecular machine for which all parts are in shape. The parameter a is crucial. If $0<a\le 1/d$, it follows that a molecular machine works better the more the parts that are in shape. On the other hand, if $a<0$, then a molecular machine with some parts in shape, but not all, functions worse the more parts are in shape, since all units must work in order for the whole machine to function, and there is a cost $-a$ associated with carrying each part that is in shape, as long as the whole system does not function.

Each state x is interpreted as a population of N subjects, all having the same variant x of the molecular machine. With this interpretation, $X=X_t$ is the outcome of a random evolutionary process where all subjects of the population, at any time point t, have the same state. However, this state may vary over time when all subjects of population simultaneously experience the same change. The question of interest is whether this process can modify the population so that all its members have a functioning molecular machine. A transition of this process from x is caused by a mutation with distribution $q(x,·)$, where $q(x,x)=0$. Suppose a mutation from x to y is possible, i.e., $q(x,y)>0$. A mutation from x to y first occurs in one individual and then it either (momentarily) dies out with probability $1-{\alpha }_{\theta }(x,y)$ or it (momentarily) spreads to the whole population (becomes fixed) with probability

$$\begin{array}{c}\hfill {\alpha }_{\theta }(x,y)=C·{\left(\frac{e^{\theta f\left(y\right)}{P}_0\left(y\right)q(y,x)}{e^{\theta f\left(x\right)}{P}_0\left(x\right)q(x,y)}\right)}^{1/2},\end{array}$$

(73)

where

$$\begin{array}{c}\hfill C={\left(\underset{x,y}{max}\frac{e^{\theta f\left(y\right)}{P}_0\left(y\right)q(y,x)}{e^{\theta f\left(x\right)}{P}_0\left(x\right)q(x,y)}\right)}^{-1/2}\end{array}$$

(74)

is a constant assuring that (73) never exceeds 1, and the maximum is taken over all $x,y$ such that $x\ne y$ and both of $q(x,y)$ and $q(y,x)$ are positive. The Markov chain with transition probabilities (14) and acceptance probability (73) represent the dynamics of the evolutionary process.

As shown in Section 7, the equilibrium distribution of this Markov chain is given by $P_{\theta }$ in (10). In particular, Propositions 2 and 3 remain valid when the Markov chain (14) with acceptance probabilities (73) are used, rather than (15). We will interpret

$$\begin{array}{c}\hfill s\left(x\right)=e^{\theta f\left(x\right)/N}\end{array}$$

(75)

as the selection coefficient or fitness of individuals with a molecular machine of type x, that is, $s\left(x\right)$ is proportional to the fertility rate of individuals of type x.

The MH-type Markov chain with acceptance probability (73) and (74) represents an evolutionary process that closely resembles a Moran model with the selection [46,47,48], which is frequently used for describing evolutionary processes (as can be seen in Section 7). The Moran model is a continuous time Markov chain for a population with overlapping generations where individuals die at the same rate, and are replaced by the offspring of individuals in the population proportionally to their selection coefficients $s\left(x\right)$. New types arise when an offspring of parents of type x mutate with probability $\mu \left(x\right)$. If the mutation rate is small ($\mu \left(x\right)\ll N^{-1}$ for all $x\in \mathrm{\Omega }$), then to a good approximation the whole population will have the same type at any point in time, which is a so-called fixed state assumption.

Even though the Moran model is specified in continuous time, time can be discretized as $t=0,1,2,\dots$ by only recording the population when individuals die. If individuals die at a rate of 1, then the next individual dies at a rate of N, so that time is counted in units of $N^{-1}$ generations. The fixed state assumption is motivated by assuming that newborn offspring with a new mutation either dies out or spreads to the whole population (becoming fixed in the population) right after birth. In this context, q corresponds to the way in which mutations change the type of the individual, whereas ${\alpha }_{\theta }={\alpha }_{\theta N}$ is the probability of fixation. If $q(x,y)$ is the conditional probability that an offspring of a type x parent mutates to y, given that a mutation occurs, then the proposal kernel of the Moran model is

$$\begin{array}{c}\hfill q^{\mathrm{Moran}}(x,y)=\left\{\begin{array}{cc}\mu \left(x\right)q(x,y),\hfill & x\ne y,\hfill \\ 1-\mu \left(x\right),\hfill & x=y.\hfill \end{array}\right.\end{array}$$

(76)

As shown in Section 7, the acceptance (or fixation) probability of the Moran model is

$$\begin{array}{c}\hfill {\alpha }_{\theta N}^{\mathrm{Moran}}(x,y)\approx \frac{1}{N}\left(1+\frac{\theta \left[f\right(y)-f(x\left)\right]}{2}\right)\approx \frac{1}{N}{\left(\frac{e^{\theta f\left(y\right)}}{e^{\theta f\left(x\right)}}\right)}^{1/2}\end{array}$$

(77)

when $\theta \left[f\right(y)-f(x\left)\right]$ is small. From (76) and (77), the Moran model approximates the Metropolis–Hastings kernel with acceptance probabilities (73) and (74) with good accuracy when (i) $\mu \left(x\right)\equiv \mu$; (ii) $P_0$ is uniform; and (iii) the proposal kernel q is symmetric (i.e., $q(x,y)=q(y,x)$), although the time scales of the two processes are different. More specifically, if (i)–(iii) hold, a time-shifted version of the Moran model approximates the MH-type model with acceptance probabilities (73) and (74), so that each time step of the MH-type Markov chain corresponds to $C/\mu$ generations of a Moran model. However, even under assumptions (i)–(iii), the stationary distribution of the Moran model differs slightly from $P_{\theta }$.

The proposal kernel $q(x,y)$ is assumed to be local and satisfying

$$\begin{array}{c}\hfill q(x,y)=\left\{\begin{array}{cc}b/\left[\right|x|+b(d-\left|x\right|\left)\right],\hfill & y=x+e_j,x_j=0,\hfill \\ 1/\left[\right|x|+b(d-\left|x\right|\left)\right],\hfill & y=x+e_j,x_j=1,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

(78)

where $e_j=(0,\dots ,0,1,0,\dots ,0)$ is a row vector of length d with a 1 in position $j\in \{1,\dots ,d\}$ and zeros elsewhere, whereas $x+e_j$ refers to component-wise addition modulo 2, corresponding to a switch of component j of x. A change of component j from 0 to 1 is caused by a beneficial mutation, whereas a change from 1 to 0 corresponds to a deleterious mutation. Consequently, $b>0$ is the ratio between the rates at which beneficial and deleterious mutations occur.

The kernel q in (78) is symmetric only when beneficial and deleterious mutations have the same rate ($b=1$). The more general case of asymmetric q is handled differently by the MH-type algorithm and the Moran model. Whereas the MH-type algorithm elevates the acceptance probability (73) of seldom-proposed states y (those y for which $q(x,y)$ is small for many x), this is not the case for the acceptance probability (77) of the Moran model. To avoid that these states y are reached too often by the MH-type algorithm, the null distribution $P_0$ of no selection has to be chosen so that $P_0\left(y\right)$ is small for rarely proposed states (whereas the Moran model needs no such correction). Therefore $P_0$ in (73) will be chosen as the stationary distribution of a transition kernel (14) for which $\theta =0$ and all candidates are accepted (${\alpha }_0(x,y)=1$). That is, if ${\tilde{\mathrm{\Pi }}}_0$ refers to the transition matrix of such a Markov chain, the initial distribution ${\mathbf{P}}_0$ in (17) is chosen as the solution of

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\mathbf{P}}_0={\mathbf{P}}_0{\tilde{\mathrm{\Pi }}}_0,\hfill \\ \sum _{x\in \mathrm{\Omega }}{P}_0\left(x\right)=1.\hfill \end{array}\right.\end{array}$$

(79)

The null distribution $P_0=P_{0b}$ in (79) involves one single nuisance parameter $\xi =b$. In the special case, when beneficial and deleterious mutations have the same rate ($b=1$), this procedure generates a uniform distribution $P_0\left(x\right)\equiv 2^{-d}$. On the other hand, states x with many functioning parts will be harder to reach by the Markov process ${\tilde{\mathrm{\Pi }}}_0$ when beneficial mutations occur less frequently than deleterious ones ($0<b<1$), resulting in smaller values of $P_0\left(x\right)$. The distribution under the alternative hypothesis, $P=P_{\tilde{\theta }bt}$, involves the nuisance parameter b, the time point t at which the state of the population is recorded, and $\tilde{\theta }=(a,\theta )$, the two parameters that determine how much background information the MH-type evolutionary algorithm makes use of. For simplicity, a and b are here regarded as constants and we only include $\theta$ and t in the notation. This gives rise to an active information

$$I^{+}(\theta ,t)=log\frac{P_{\theta }\left(X_t=(1,\dots ,1)\right)}{P_0\left(X_t=(1,\dots ,1)\right)}.$$

(80)

The MH-type algorithm is studied for $d=5$, and illustrated in Figure 1, Figure 2 and Figure 3. Note that the functional information $I_{f0}$ is a decreasing function of b, since it is more surprising to find a working molecular machine by chance when the rate of beneficial mutations b is small. Moreover, the active information $I^{+}\left(\theta \right)={lim}_{t\to \infty }{I}^{+}(\theta ,t)$ for the equilibrium distribution of the Markov chain as well as the active information $I^{+}(\theta ,t)$ and $I_s^{+}(\theta ,t)$ for a system in non-equilibrium, without and with stopping, are increasing functions of $\theta$, and decreasing functions of a and b. The smaller a or b is, the more external information can be infused to increase the probability of reaching the fine-tuned state of a working molecular machine $(1,\dots ,1)$. When a is small, to leave this state once it is reached becomes more difficult, and consequently $I_s^{+}(\theta ,t)$, is only marginally larger than $I(\theta ,t)$.

Example 5

(Evolutionary programming algorithms). Suppose that $\mathrm{\Omega }={\mathrm{\Omega }}_{\mathrm{ind}}^N$ is a set of genetic variants from some genomic region, $x=(x_1,\dots ,x_N)$, for the members of a population of size N. That is, $x_k\in {\mathrm{\Omega }}_{\mathrm{ind}}$ is the variant of this genomic region for individual k. If, for instance, the region codes for the molecular machine of Example 4, we let $x_k=(x_{k1},\dots ,x_{kd})\in {\{0,1\}}^d={\mathrm{\Omega }}_{\mathrm{ind}}$, with $x_{kj}=1$ or 0 depending on whether component j of this machine works for individual k. Let $g\left(x_k\right)$ be the biological fitness, or the expected number of offspring, of k. In the context of molecular machines, the logarithm of $g\left(x_k\right)$ could be a function of the number of functioning parts of a machine of type $x_k$. The specificity function of a population in state x is the average fitness

$$f\left(x\right)=\frac{1}{N}\sum _{k=1}^{N}g\left(x_k\right)$$

of its individuals. The targeted set A in (5) corresponds to all genetic profiles with an average fitness at least $f_0$. This type of model is frequently used in genetic programming as well as in other types of evolutionary programming algorithms to mimic the evolution of N individuals over time [49,50]. Typically, the output $X=X_t$ of the evolutionary algorithm is the last step of a simulation ${\{X_s=(X_{s1},\dots ,X_{sN})\}}_{s=0}^t$ of the population over t generations. Once the distributions $P_0=P_{0\xi t}$ and $P=P_{\theta \xi t}$ of X are found under the null hypothesis $H_0$ and the alternative hypothesis $H_1$, the actinfo $I^{+}$ can be computed, according to (1). This actinfo quantifies, on a logarithmic scale, how much more likely it is for the average fitness of the population to exceed $f_0$ at time t, for a population with externally infused information ($H_1$) compared to an evolutionary process where no such external information is used ($H_0$). For instance, if a molecular machine needs all its parts in order to function ($g\left(x_k\right)=1\left(\right|x_k|=d)$), then the actinfo at time t equals

$$I^{+}=I^{+}(\theta ,\xi ,t)=log\frac{P_{\theta \xi t}\left(|\{k;1\le k\le N,X_k=(1,\dots ,1)\}|\ge f_{0}N\right)}{P_{0\xi t}\left(|\{k;1\le k\le N,X_k=(1,\dots ,1)\}|\ge f_{0}N\right)},$$

(81)

with $X=(X_1,\dots ,X_N)$. Since the state space $\mathrm{\Omega }$ is very large, it is often complicated to find explicit, analytical expressions for the actinfo $I^{+}$ in (81). Suppose that the nuisance parameters $\xi$ of the null distribution $P_0=P_{0\xi }$ are known. This makes the framework of Section 4.1 applicable, running the evolutionary algorithm n times. That is, n i.i.d. copies ${\left\{X_{is}\right\}}_{s=0}^t$ of the population trajectory are generated up to time t for $i=1,\dots ,n$. Then, $X_i=X_{it}=(X_{it1},\dots ,X_{itN})$, $i=1,\dots ,n$, are used for computing an estimate ${\widehat{I}}_n^{+}$ of the actinfo, and test for fine-tuning, according to Section 4.1.

Recall the fixed state assumption of Example 4, whereby all individuals of the population, at any time point, have the same state. Such an assumption is only realistic when $N\mu \ll 1$, that is, when either the mutation rate $\mu$ and/or the population size N is small. This corresponds to a scenario where $P_0$ and P put all their probability masses along the diagonal

$$\begin{array}{c}\hfill {\mathrm{\Omega }}_{\mathrm{diag}}=\{x\in \mathrm{\Omega };x_1=\dots =x_N\}\end{array}$$

(82)

of $\mathrm{\Omega }$. Since (82) is equivalent to the reduced state space ${\mathrm{\Omega }}_{\mathrm{ind}}$, the fixed state assumption greatly simplifies the analysis. For instance, it often makes it possible to find analytical expressions for the actinfo $I^{+}$, rather than having to estimate it.

6. Discussion

In this article, a general statistical framework is provided for using active information to quantify the amount of pre-specified external knowledge an algorithm makes use of, or equivalently, how tuned the algorithm is. The theory is based on quantifying, for each state x, how specified it is by means of a real-valued function $f\left(x\right)$. An algorithm with external information either directly makes use of knowledge of f, or at least it incorporates knowledge that tends to move the output of the algorithm towards more specified regions. The Metropolis–Hastings Markov chain directly incorporates knowledge of f in terms of the acceptance probability of proposed moves. The learning ability of this algorithm was analyzed by studying its active information, with or without stopping, when the targeted set of highly specified states is reached. When the independent outcomes of an algorithm are available, nonparametric and parametric estimators of the actinfo of the algorithm were also developed, as well as nonparametric and parametric tests of FT.

This work can be extended in different ways. A first extension is to find conditions under which the actinfo $I^{+}(\theta ,t)$ of a stochastic algorithm based on a random start (according to the null distribution of a non-guided algorithm) followed by t iterations of the Metropolis–Hastings Markov chain (without stopping) is a non-decreasing function of t. We conjecture that this is typically the case but have not obtained any general conditions on the distribution q of proposed candidates for this result to hold.

A second extension is to widen the notion of specificity, so that not only the functionality $f\left(x\right)$ but also the rarity $P_0\left(x\right)$ of the outcome x under the null distribution is taken into account. A class of such specificity functions is

$$\begin{array}{c}\hfill g_{\theta }\left(x\right)=\theta f\left(x\right)-log{P}_0\left(x\right),\end{array}$$

(83)

where $\theta >0$ is a parameter that controls the tradeoff between scenarios where either functionality or rarity under the null is the most important determinant of specificity. The case $\theta =0$ in (83) corresponds to the function having no impact, so that $g_0\left(x\right)$ reduces to Shannon’s self information of x. The case $g_1\left(x\right)$ was proposed in [15], whereas $g_{\theta }\left(x\right)$ is solely determined by $f\left(x\right)$ in the limit when $\theta$ becomes large.

A third extension is to generalize the notion of actinfo to include not only the probability of reaching a targeted set of highly specified states A under $H_0$ and $H_1$, but also account for the conditional distribution of the states within A, given that A has been reached. This is related to the way in which functional sequence complexity generalizes the functional information [51,52,53,54]. Let $H\left(Q\right)=-{\sum }_{x}Q\left(x\right)log\left[Q\left(x\right)\right]$ refer to the Shannon entropy of a distribution Q, whereas $H\left(Q_A\right)$ is the Shannon entropy of the corresponding conditional distribution $Q_A\left(x\right)=Q\left(x\right|A)$, given that A has been reached. The functional sequence complexity

$$\begin{array}{cc}\hfill {\mathrm{FSC}}_0& =H\left(P_0\right)-H\left(P_{0A}\right)\hfill \\ \hfill & =E_{P_0}\left\{log\left[P_0(X\mid A)\right]\mid X\in A\right\}-E_{P_0}\{log\left[P_0\left(X\right)\right]\}\hfill \end{array}$$

is the reduction in entropy, under the null hypothesis $H_0$ of the highly specified states in A, compared to the entropy under $H_0$ of all states in $\mathrm{\Omega }$. ${\mathrm{FSC}}_0$ then reduces to the functional information $I_{f0}$ when $P_0$ is uniform over $\mathrm{\Omega }$. In a similar vein, the active uncertainty reduction is introduced:

$$\begin{array}{cc}\hfill {\mathrm{UR}}^{+}& =\sum _{x\in A}{P}_A\left(x\right)logP\left(x\right)-\sum _{x\in A}{P}_{0A}\left(x\right)log{P}_0\left(x\right)\hfill \\ \hfill & =E_P[logP\left(X\right)|X\in A]-E_{P_0}[log{P}_0\left(X\right)|X\in A].\hfill \end{array}$$

Then, ${\mathrm{UR}}^{+}=I^{+}$ when $P_{0A}$ and $P_A$ are uniformly distributed on A. This happens, for instance, when $P_0$ has a uniform distribution on $\mathrm{\Omega }$ and $P=P_{\theta }$ for some $\theta >0$, and if (8) holds. The properties of ${UR}^{+}$ deserve to be analyzed in more detail, for instance, by investigating how it differs from the actinfo $I^{+}$.

A fourth extension would be to apply the concept of actinfo to other genetic models. For instance, Example 4 is the first time that, to our knowledge, actinfo is applied to the Moran model. In the past, however, actinfo was used in population genetics to study fixation times for the Wright–Fisher model of population genetics, a model for which time is discrete and generations do not overlap [55].

7. Proofs

Proof of Proposition 1.

Introduce

$$\begin{array}{cc}\hfill J\left(\theta \right)& =\sum _{x\in A^c}exp\left\{\theta [f\left(x\right)-f\left(x_0\right)]\right\}{P}_0\left(x\right),\hfill \\ \hfill K\left(\theta \right)& =\sum _{x\in A}exp\left\{\theta [f\left(x\right)-f\left(x_0\right)]\right\}{P}_0\left(x\right),\hfill \end{array}$$

(84)

when $\mathrm{\Omega }$ is countable, and replace the sums in (84) by integrals when $\mathrm{\Omega }$ is continuous. Then

$$\begin{array}{cc}\hfill P_{\theta }\left(A\right)& =exp\left[\theta f\left(x_0\right)\right]K\left(\theta \right)/\{exp\left(\theta f\left(x_0\right)\right)[J\left(\theta \right)+K\left(\theta \right)]\}\hfill \\ \hfill & =K\left(\theta \right)/\left[J\right(\theta )+K(\theta \left)\right]\hfill \\ \hfill & =1/\left[J\right(\theta )/K(\theta )+1].\hfill \end{array}$$

(85)

Since $P_0\left(A\right)<1$, it follows that $J\left(\theta \right)$ is a strictly decreasing function of $\theta \ge 0$, whereas $K\left(\theta \right)$ is a non-decreasing function of $\theta$. From this, it follows that $P_{\theta }\left(A\right)$ is a strictly increasing function of $\theta$, and consequently $I^{+}\left(\theta \right)=log[P_{\theta }\left(A\right)/P_0\left(A\right)]$ is a strictly increasing function of $\theta$ as well.

Moreover, $K\left(\theta \right)\ge P_0\left(A\right)>0$ for all $\theta \ge 0$, and $J\left(\theta \right)\to 0$ as $\theta \to \infty$ follows by dominated convergence. In conjunction with (85), this implies that $P_{\theta }\left(A\right)\to 1$ and $I^{+}\left(\theta \right)\to I_{f0}$ as $\theta \to \infty$. □

Proof of Proposition 2.

Equation (20) follows from (17), (19) and the fact that

$$\begin{array}{cc}\hfill P_0\left(A\right)& =\sum _{x\in A}{P}_0\left(x\right)={\mathbf{P}}_0\mathbf{v},\hfill \\ \hfill P_{\theta t}\left(A\right)& =\sum _{x\in A}{P}_{\theta t}\left(x\right)={\mathbf{P}}_{\theta t}\mathbf{v}s.={\mathbf{P}}_0{\mathrm{\Pi }}_{\theta }^t\mathbf{v},\hfill \end{array}$$

since $\mathbf{v}$ is a column vector of length $\left|\mathrm{\Omega }\right|$ with ones in positions $x\in A$ and zeros in positions $x\in A^c$.

Equation (21) is equivalent to proving that

$$\begin{array}{c}\hfill P_{\theta t}\left(A\right)\to P_{\theta }\left(A\right)\mathrm{as}t\to \infty .\end{array}$$

However, this follows from the fact that $P_{\theta }$ is the equilibrium distribution of the Markov chain with transition kernel (18). That is, letting $t\to \infty$ in (19), we find that

$$\begin{array}{c}\hfill {\mathbf{P}}_{\theta t}={\mathbf{P}}_0{\mathrm{\Pi }}_{\theta }^t\to {\mathbf{P}}_{\theta },\end{array}$$

and therefore

$$P_{\theta t}\left(A\right)={\mathbf{P}}_{\theta t}\mathbf{v}s.\to {\mathbf{P}}_{\theta }\mathbf{v}s.=P_{\theta }\left(A\right),\mathrm{as}t\to \infty .$$

□

Proof of Proposition 3.

Equation (28) follows from the definitions of $I^{+}(\theta ,t)$ and $I_s^{+}(\theta ,t)$ in (20) and (27), and the fact that

$$P_{\theta t}\left(A\right)=P(X_t\in A)\le P(X_{t\wedge T}\in A)=P_{\theta ts}\left(A\right),$$

where the inequality is a consequence of the definition of T in (22). Since

$$P_{\theta ts}\left(A\right)=P(T\le t)\le P(T\le t+1)=P_{\theta ,t+1,s}\left(A\right),$$

we proved that $I_s^{+}(\theta ,t)$ is non-decreasing in t. Equation (29) follows from the definition of $I_s^{+}(\theta ,t)$ and the fact that

$$\underset{t\to \infty }{lim}{P}_{\theta ts}\left(A\right)=P(T<\infty )=1.$$

(86)

The last equality of (86) is a consequence of the fact that the Markov chain with transition kernel ${\mathrm{\Pi }}_{\theta }$ is irreducible, so that any state $x\in \mathrm{\Omega }$ will be reached with a probability of 1. In particular, the targeted set A will be reached with a probability of 1. In order to verify (30), we first deduce

$$P(T>t)=1-P_0\left(A\right)e^{I_s^{+}(\theta ,t)}$$

from (24), and then we make use of the equality

$$E\left(T\right)=\sum _{t=0}^{\infty }P(T>t).$$

□

Proof of Proposition 4.

Since $n\widehat{Q}\left(A\right)\sim \mathrm{Bin}(n,Q\left(A\right))$ has a binomial distribution, it follows from the central limit theorem that

$$\sqrt{n}(\widehat{Q}\left(A\right)-Q\left(A\right))\stackrel{\mathcal{L}}{⟶}N(0,Q\left(A\right)[1-Q\left(A\right)]),$$

(87)

as $n\to \infty$. Notice that ${\widehat{I}}^{+}=g\left(\widehat{Q}\left(A\right)\right)$, where $g\left(Q\right)=log[Q/P_0\left(A\right)]$ and $g^{\prime }\left(Q\right)=1/Q$. Equation (38) follows from the Delta method (see, e.g., Theorem 8.12 of [33]) and the fact that

$$V=g^{\prime }{\left(Q\left(A\right)\right)}^2·Q\left(A\right)[1-Q\left(A\right)].$$

In order to establish (40), to begin with, it follows from (34) and the definition of $p_{\min }$ that

$$\begin{array}{cc}\hfill P_{H_0}({\widehat{I}}^{+}\ge I_{\min })& =P_{H_0}(\widehat{Q}\left(A\right)\ge p_{\min })\hfill \\ \hfill & =P_{H_0}\left(\frac{1}{n}\sum _{i=1}^n{Y}_i\ge p_{\min }\right),\hfill \end{array}$$

where $Y_i=I(X_i\in A)\sim Be\left(p_0\right)$ are independent Bernoulli variables under $H_0$ with success probability $p_0=P_0\left(A\right)$. It follows from the large deviations theory that (40) holds, with

$$C=\underset{\varphi >0}{sup}[\varphi p_{\min }-\lambda \left(\varphi \right)]$$

(88)

the Legendre–Fenchel transformation, and

$$\lambda \left(\varphi \right)=logE[exp\left(\varphi Y\right)]=log[1+p_0(e^{\varphi }-1)]$$

(89)

the cumulant generating function of Y [56], pp. 529–533. Inserting (89) into (88), it can be seen that the maximum in (88) is given by (41). □

Proof of Proposition 5.

In order to verify (46), we will first show that the estimator (42) of the tilting parameter $\theta$ is asymptotically normal

$$\sqrt{n}({\widehat{\theta }}_n-{\theta }^{*})\stackrel{\mathcal{L}}{⟶}N(0,U)asn\to \infty ,$$

(90)

with asymptotic variance

$$U=\frac{{\mathrm{Var}}_Q\left[f\left(X\right)\right]}{{\mathrm{Var}}_{P_{{\theta }^{*}}}^2\left[f\left(X\right)\right]}.$$

(91)

To this end, let ${}^{\prime }$ refer to the derivatives with respect to the tilting parameter $\theta$. Define the score function

$${\psi }_{\theta }\left(x\right)=\frac{dlog{P}_{\theta }\left(x\right)}{d\theta }=\frac{P_{\theta }^{\prime }\left(x\right)}{P_{\theta }\left(x\right)}$$

and its derivative

$${\psi }_{\theta }^{\prime }\left(x\right)=\frac{d{\psi }_{\theta }\left(x\right)}{d\theta }.$$

It is a standard result from the asymptotic theory of maximum likelihood estimation and M-estimation (see, e.g., Chapter 6 of [33]) that (90) holds with asymptotic variance

$$U=\frac{{\mathrm{Var}}_Q\left[{\psi }_{{\theta }^{*}}\left(X\right)\right]}{E_Q^2\left[{\psi }_{{\theta }^{*}}^{\prime }\left(X\right)\right]}.$$

(92)

To simplify (92), notice that the score function can be written as

$${\psi }_{\theta }\left(x\right)=f\left(x\right)-\frac{M^{\prime }\left(\theta \right)}{M\left(\theta \right)}=f\left(x\right)-E_{P_{\theta }}\left[f\left(X\right)\right]$$

(93)

for the exponential family of tilted distributions (10) and (11). From this, it follows that

$${\psi }_{\theta }^{\prime }\left(x\right)=\frac{M^{\prime \prime }\left(\theta \right)}{M\left(\theta \right)}-{\left(\frac{M^{\prime }\left(\theta \right)}{M\left(\theta \right)}\right)}^2={\mathrm{Var}}_{P_{\theta }}\left[f\left(X\right)\right]$$

is a constant, not depending on x. Inserting the last two displayed equations into (92), the formula in (91) for the asymptotic variance of $\widehat{\theta }$ is obtained. As a next step, we notice that

$${\widehat{I}}^{+}=g\left(\widehat{\theta }\right),$$

(94)

where

$$g\left(\theta \right)=log\frac{P_{\theta }\left(A\right)}{P_0\left(A\right)}=logh\left(\theta \right)-log{P}_0\left(A\right),$$

(95)

and

$$h\left(\theta \right)=P_{\theta }\left(A\right)=\frac{{\sum }_{x\in A}{e}^{\theta f\left(x\right)}{P}_0\left(x\right)dx}{M\left(\theta \right)}$$

(96)

follows from the definition of $P_{\theta }\left(x\right)$ in (10).

Differentiating (96) with respect to $\theta$, we find that

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill h^{\prime }\left(\theta \right)& =\sum _{x\in A}f\left(x\right)e^{\theta f\left(x\right)}{P}_0\left(x\right)dx/M\left(\theta \right)\hfill \\ \hfill & -M^{\prime }\left(\theta \right)\sum _{x\in A}{e}^{\theta f\left(x\right)}{P}_0\left(x\right)dx/M^2\left(\theta \right).\hfill \end{array}\end{array}$$

(97)

Furthermore, it follows from the RHS of (97) that

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill h^{\prime }\left(\theta \right)& =E_{P_{\theta }}\left[f\left(X\right)I(f\left(X\right)\ge f_0)\right]-P_{\theta }\left(A\right)E_{P_{\theta }}\left[f\left(X\right)\right]\hfill \\ \hfill & ={\mathrm{Cov}}_{P_{\theta }}[f\left(X\right),I(f\left(X\right)\ge f_0)].\hfill \end{array}\end{array}$$

(98)

Then, we combine (95) and (97), and obtain

$$g^{\prime }\left(\theta \right)=\frac{h^{\prime }\left(\theta \right)}{h\left(\theta \right)}=\frac{{\mathrm{Cov}}_{P_{\theta }}[f\left(X\right),I(f\left(X\right)\ge f_0)]}{P_{\theta }\left(A\right)}.$$

(99)

Finally, we use the Delta method to conclude that ${\widehat{I}}^{+}$ is an asymptotic normal estimator (38) of $I^{+}\left({\theta }^{*}\right)$, with asymptotic variance $V=g^{\prime }{\left({\theta }^{*}\right)}^{2}U$, which, in view of (91) and (99), agrees with (47).

In order to prove the large deviation result (48) for the parametric test of FT, let ${\theta }_{\min }$ be the value of the tilting parameter that satisfies $P_{{\theta }_{\min }}\left(A\right)=p_{\min }=P_0\left(A\right)exp\left(I_{\min }\right)$. Then, notice that

$$\begin{array}{cc}\hfill P_{H_0}({\widehat{I}}^{+}\ge I_{\min })& =P_{H_0}(\widehat{Q}\left(A\right)\ge p_{\min })\hfill \\ \hfill & =P_{H_0}(\widehat{\theta }\ge {\theta }_{\min })\hfill \\ \hfill & =P_{H_0}(\sum _{i=1}^n{\psi }_{{\theta }_{\min }}\left(X_i\right)/n\ge 0)\hfill \\ \hfill & =P_{H_0}\left(\sum _{i=1}^{n}f\left(X_i\right)/n\ge E_{p_{\min }}\left[f\left(X\right)\right]\right),\hfill \end{array}$$

where, in the third step, we utilized that $\widehat{\theta }\ge {\theta }_{\min }$ is equivalent to the derivative of the log likelihood of data being non-negative at ${\theta }_{\min }$, and in the fourth step, we made use of (93) and introduced $p_{\min }=P_{{\theta }_{\min }}$. However, this last line is a large deviations probability. It then follows from a large deviations theory that (48) holds, with C the Legendre–Fenchel transformation in (49). □

Proof of Proposition 6.

Since the bias corrected empirical actinfo

$${\widehat{I}}_n^{+}-B=log\frac{\widehat{Q}\left(A\right)}{P_{0\xi }\left(A\right)}$$

(100)

behaves like (34), with $P_0=P_{0\xi }$, the asymptotic normality result for the nonparametric version of the estimator of $I_Q^{+}$ follows from Proposition 4.

For the parametric version of the estimator of $I_Q^{+}$, we will (briefly) generalize the asymptotic normality proof of Proposition 5. It follows from (53) and (55) that

$${\widehat{I}}_n^{+}=g(\widehat{\theta },\widehat{\xi }),$$

where

$$g(\theta ,\xi )=log\frac{P_{\theta \xi }\left(A\right)}{P_{0\max }\left(A\right)}.$$

(101)

Making use of the delta method, it follows that the asymptotic variance of the parametric version of ${\widehat{I}}_n^{+}$ equals

$$V=g^{\prime }({\theta }^{*},{\xi }^{*})\mathrm{AsVar}(\widehat{\theta },\widehat{\xi })g^{\prime }{({\theta }^{*},{\xi }^{*})}^T,$$

(102)

with the asymptotic variance of $(\widehat{\theta },\widehat{\xi })$ defined through

$$\sqrt{n}\left((\widehat{\theta },\widehat{\xi })-({\theta }^{*},{\xi }^{*})\right)\stackrel{\mathcal{L}}{⟶}N(0,\mathrm{AsVar}(\widehat{\theta },\widehat{\xi }))$$

as $n\to \infty$. Since $(\widehat{\theta },\widehat{\xi })$ in (56) is an M-estimator, it follows that its asymptotic variance equals

$$\mathrm{AsVar}(\widehat{\theta },\widehat{\xi })=E{\left[{\psi }_{{\theta }^{*}{\xi }^{*}}^{\prime }\left(X\right)\right]}^{-1}E\left[{\psi }_{{\theta }^{*}{\xi }^{*}}^T\left(X\right){\psi }_{{\theta }^{*}{\xi }^{*}}\left(X\right)\right]E{\left[{\left({\psi }_{{\theta }^{*}{\xi }^{*}}^{\prime }\right)}^T\left(X\right)\right]}^{-1}.$$

(103)

The gradient of (101) is

$$g^{\prime }(\theta ,\xi )=\frac{P_{\theta \xi }^{\prime }\left(A\right)}{P_{\theta \xi }\left(A\right)}=E\left[{\psi }_{\theta \xi }\left(X\right)\right|X\in A],$$

(104)

where ${\psi }_{\theta \xi }=P_{\theta \xi }^{\prime }\left(x\right)/P_{\theta \xi }\left(x\right)$ is the likelihood score function for the combined parameter vector $(\theta ,\xi )$. Putting things together, the asympotic variance formula (59) for the parametric version of ${\widehat{I}}_n^{+}$ follows from (102)–(104).

The significance level of the FT test can be written as

$$P_{0\xi }({\widehat{I}}_n^{+}\ge I_{\min })=P_{0\xi }({\widehat{I}}_n^{+}-B\ge I_{\min }-B).$$

Since $p_{\min }=P_{0\xi }\left(A\right)exp\left(I_{\min }\right)$, we have that

$$I_{\min }-B=log\frac{p_{\min }{e}^{-B}}{P_{0\xi }\left(A\right)}.$$

(105)

From this and (100), it follows that the nonparametric test of FT behaves as the corresponding nonparametric test of Proposition 4, with the null probability $P_0\left(A\right)$ replaced by $P_{0\xi }\left(A\right)$, and $p_{\min }$ replaced by $p_{\min }{e}^{-B}$. Therefore, the large deviation result (61) follows from (41). In a similar way, the large deviation result for the parametric version of the FT-test (in the special case when $\theta$ is a scalar exponential tilting parameter) follows from (100), (105) and Proposition 5. □

Proof of Proposition 7.

Because of (52) and (63), we have that

$$\sqrt{n}({\widehat{I}}_{n{n}_0}^{+}-I_Q^{+})=\sqrt{n}log\frac{\widehat{Q}\left(A\right)}{Q\left(A\right)}-\sqrt{\frac{n}{n_0}}\sqrt{n_0}log\frac{P_{0\widehat{\xi }}\left(A\right)}{P_{0\xi }\left(A\right)},$$

(106)

where

$$\sqrt{n}log\frac{\widehat{Q}\left(A\right)}{Q\left(A\right)}\stackrel{\mathcal{L}}{⟶}N(0,V_1)\mathrm{as}n\to \infty$$

(107)

and

$$\sqrt{n_0}log\frac{P_{0\widehat{\xi }}\left(A\right)}{P_{0\xi }\left(A\right)}\stackrel{\mathcal{L}}{⟶}N(0,V_2)\mathrm{as}{n}_0\to \infty$$

(108)

respectively. It follows from the proofs of Propositions 4 and 5 that the asymptotic variance for $V_1$ in (107) is the same as V in (39) and (59), for the nonparametric and parametric versions of $\widehat{Q}\left(A\right)$, respectively. The asymptotic variance $V_2$ in (108) is given by (67). This is proven using the delta method (similarly as for Proposition 6), making use of the fact that $\widehat{\xi }$ is the maximum likelihood estimator of $\xi$ with asymptotic variance that is the inverse $E{\left[{\psi }_{\xi }^T\left(X\right){\psi }_{\xi }\left(X\right)\right]}^{-1}$ of the Fisher information matrix. The asymptotic normality result (66) then follows from (106)-(108), the fact that $n/n_0\to \lambda$, and the independence of the two samples.

The large deviations results are proven in a similar way as in Proposition 6, replacing $P_{0\max }\left(A\right)$ by $P_{0\widehat{\xi }}\left(A\right)$. Using statistical consistency $\widehat{\xi }\stackrel{p}{⟶}\xi$ as $n_0\to \infty$, it follows that the large deviation rates C of Proposition 7, for the nonparametric and parametric versions of the FT tests, are the same as in Proposition 6, with bias term $B=0$. □

Details from Example 4. In order to prove that the Metropolis–Hastings-type Markov chain (14) with acceptance probabilities (73) has an equilibrium distribution of $P_{\theta }$, we first notice that for any pair of states $x\ne y$, the flow of probability mass

$$\begin{array}{cc}\hfill & P_{\theta }\left(x\right){\pi }_{\theta }(x,y)\hfill \\ \hfill & =P_{\theta }\left(x\right)q(x,y){\alpha }_{\theta }(x,y)\hfill \\ \hfill & =\frac{P_0\left(x\right)e^{\theta f\left(x\right)}}{M\left(\theta \right)}q(x,y)·C{\left[\frac{e^{\theta f\left(y\right)}{P}_0\left(y\right)q(y,x)}{e^{\theta f\left(x\right)}{P}_0\left(x\right)q(x,y)}\right]}^{1/2}\hfill \\ \hfill & =C\frac{{\left(e^{\theta f\left(x\right)}{P}_0\left(x\right)q(x,y)e^{\theta f\left(y\right)}{P}_0\left(y\right)q(y,x)\right)}^{1/2}}{M\left(\theta \right)}\hfill \end{array}$$

(109)

from x to y is symmetric with respect to x and y. Therefore, the flow $P_{\theta }\left(y\right){\pi }_{\theta }(y,x)$ of probability mass in the opposite direction, from y to x, is the same as in (109). A Markov chain with this property is called reversible [57], pp. 11–12. However, it is well known that $P_{\theta }$ is a stationary distribution if the Markov chain is reversible with reversible measure $P_{\theta }$[58], p. 238. If, additionally, the proposal distribution q is such that it is possible to move between any pair of states in a finite number of steps, it follows that the Markov chain is irreducible and hence that $P_{\theta }$ is its unique stationary distribution, which is also the equilibrium distribution of the Markov chain [58], p. 232.

We will then motivate formula (77) for the acceptance probability of a Moran model. Assume that the population evolves over time as a Moran model, and that all individuals have type x. If one individual mutates from x to y, because of (75), the relative fitness between the $N-1$ individuals of type x and the newly mutated individual of type y is

$$s=\frac{e^{\theta f\left(y\right)/N}}{e^{\theta f\left(x\right)/N}}=e^{\theta \left[f\right(y)-f(x\left)\right]/N}.$$

(110)

From the theory of Moran models (e.g., [41,59]), it is well known that the fixation probability for the newly mutated individual is

$${\beta }_N\left(s\right)=\left\{\begin{array}{cc}(1-s^{-1})/(1-s^{-N}),\hfill & s\ne 1,\hfill \\ 1/N,\hfill & s=1.\hfill \end{array}\right.$$

(111)

Inserting (110) into (111), we find (when $s\ne 1$, or equivalently when $\mathrm{\Delta }=\theta \left[f\right(y)-f(x\left)\right]\ne 0$) that

$${\beta }_N\left(s\right)=\frac{1-e^{-\mathrm{\Delta }/N}}{1-e^{-\mathrm{\Delta }}}\approx \frac{1}{N}·\frac{\mathrm{\Delta }}{1-e^{-\mathrm{\Delta }}}\approx \frac{1}{N}·(1+\frac{\mathrm{\Delta }}{2}),$$

which is equivalent to (77).