Weighted Chernoff Information and Optimal Loss Exponent in Context-Sensitive Hypothesis Testing

Abstract

We study binary hypothesis testing for i.i.d. observations under a multiplicative context weight. For the optimal weighted total loss, defined as the sum of weighted type-I and type-II losses, we prove the logarithmic asymptotic $L_n^{\ast }=exp\{-n{D}_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})+o\left(n\right)\},n\to \infty$, where $D_C^{\mathrm{w}}$ is the weighted Chernoff information. The single-letter form of the exponent relies on a structural assumption that the weight factorises across observations, $\phi \left(x_1^n\right)={\displaystyle \prod _{i=1}^n}\phi \left(x_i\right)$; this restriction is essential for the single-letter representation and should be distinguished from the weaker qualitative description “multiplicative context weight”. The proof embeds the weighted geometric mixtures $\phi p^{\alpha }{q}^{1-\alpha }$ into a likelihood-ratio exponential family and identifies the rate through its log-normaliser. We also derive concentration bounds for the tilted weighted log-likelihood, obtain closed forms for Gaussian, Poisson, and exponential models, and extend the exponent characterisation to finitely many hypotheses.

1. Introduction

Let $\mathcal{X}$ be a Polish space with its Borel $\sigma$-algebra and let $X_1^n=(X_1,\dots ,X_n)$ be i.i.d. $\mathcal{X}$-valued observations. We consider the simple hypotheses

$$H_0:X_1^n\sim {\mathbb{P}}^{\otimes n}\mathrm{versus}{H}_1:X_1^n\sim {\mathbb{Q}}^{\otimes n},$$

where $\mathbb{P}$ and $\mathbb{Q}$ are probability measures on $\mathcal{X}$ dominated by a reference measure $\mu$. Without loss of generality, one may take $\mu ={\displaystyle \frac{1}{2}}(\mathbb{P}+\mathbb{Q})$ and write $p={\displaystyle \frac{\mathrm{d}\mathbb{P}}{\mathrm{d}\mu }}$ and $q={\displaystyle \frac{\mathrm{d}\mathbb{Q}}{\mathrm{d}\mu }}$. In the unweighted setting, the optimal sum of type-I and type-II error probabilities is characterized by $\mathrm{TV}({\mathbb{P}}^{\otimes n},{\mathbb{Q}}^{\otimes n})$ and can be written as

$$\underset{{\mathcal{X}}^n}{\int }min\{p\left(x_1^n\right),q\left(x_1^n\right)\}\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right),p\left(x_1^n\right)={\displaystyle \prod _{i=1}^n}p\left(x_i\right),q\left(x_1^n\right)={\displaystyle \prod _{i=1}^n}q\left(x_i\right).$$

(1)

In the standard (unweighted) Bayesian setting, the decay rate of the optimal total error probability is governed by the Chernoff information [1,2]:

$$\begin{array}{cc}\hfill {\rho }_{\alpha }(p,q)& :=\underset{\mathcal{X}}{\int }p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right),\alpha \in [0,1],\hfill \\ \hfill \rho (p,q)& :=\underset{\alpha \in [0,1]}{inf}{\rho }_{\alpha }(p,q),\hfill \\ \hfill D_C(\mathbb{P},\mathbb{Q})& :=-ln\rho (p,q)=\underset{\alpha \in [0,1]}{max}\left[-ln{\rho }_{\alpha }(p,q)\right].\hfill \end{array}$$

(2)

Here ${\rho }_{\alpha }$ is usually called the $\alpha$-skewed Bhattacharyya affinity coefficient, and $\rho (p,q)=\underset{\alpha \in [0,1]}{inf}{\rho }_{\alpha }(p,q)$ is the affinity coefficient. In view of Hölder’s inequality, ${\rho }_{\alpha }(p,q)\in [0, 1]$.

Chernoff also introduced an asymptotic efficiency notion for comparing two experimental designs $e={\displaystyle \frac{ln{\rho }_1}{ln{\rho }_2}}$ such that n observations on one test are equivalent (i.e., they give asymptotically the same total loss as $n\to \infty$) to $en$ observations on another test; see [1].

The paper studies a context-sensitive (weighted) analogue of this criterion and the logarithmic asymptotics of the optimal total loss as $n\to \infty$, in the framework of [3,4]. In the weighted setting, a nonnegative weight function $\phi \left(x_1^n\right)$ reweights the loss of a wrong decision according to the realised sample. Thus, $\phi$ acts as a context factor that changes the relevance of different observations for the statistical task.

Weights of this form arise naturally whenever observations are not equally informative for the inference task. Two canonical mechanisms produce such $\phi$. In importance-type reweighting, samples drawn under a proposal density g are used to perform inference with respect to a target h, and the Radon–Nikodym factor $\phi \left(x\right)=h\left(x\right)/g\left(x\right)$ enters the loss as a strictly positive (non-indicator) tilt; this is the mechanism underlying the context-sensitive framework of [3,4].

In applications, the informational value of an observation often depends on the underlying channel state. A canonical example, directly relevant to multiple hypothesis testing of transmission regimes, is a mobile communication channel modulated by a multi-zone coverage process (e.g., strong/weak/outage) along the receiver trajectory: samples acquired in outage carry little information about the regime and are weighted accordingly. Such reliability-weighted aggregation in multi-state channels was studied within multi-valued frameworks in [5,6].

Under the standard assumption that the modulating state at time i is determined by $X_i$ alone, the resulting weight is a strictly positive bounded function $\phi \left(x\right)$ and extends multiplicatively to $X_1^n$. The weighted Chernoff information $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$ then quantifies the effective discrimination rate under channel-dependent reliability and reduces to the classical rate $D_C(\mathbb{P},\mathbb{Q})$ in the limit $\phi \equiv 1$. Further parametric instances (Gaussian, Poisson, exponential) are worked out in Section 4.

Throughout we assume that the weight is compatible with the i.i.d. structure and factorises across observations; by abuse of notation, $\phi$ denotes both the one-step weight and its product extension.

Assumption 1 

(Factorised weight). The weight function $\phi \left(x_1^n\right)$ satisfies

$$\phi \left(x_1^n\right)={\displaystyle \prod _{i=1}^n}\phi \left(x_i\right),\phi \ge 0.$$

(3)

Assumption 1 is the key single-letter hypothesis. It yields the weighted affinities

$${\rho }_{\alpha }^{\mathrm{w}}(p,q)={\int }_{\mathcal{X}}\phi \left(x\right)p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right),$$

hence an additive logarithmic rate. For one observation and equal priors, the weighted Bayes risk equals

$${\displaystyle \frac{1}{2}}{\int }_{\mathcal{X}}\phi \left(x\right)min\{p\left(x\right),q\left(x\right)\}\mathrm{d}\mu \left(x\right).$$

Since $\min \{a,b\}\le a^{\alpha }{b}^{1-\alpha }$ for every $\alpha \in [0, 1]$,

$${\int }_{\mathcal{X}}\phi \left(x\right)min\{p\left(x\right),q\left(x\right)\}\mathrm{d}\mu \left(x\right)\le {\rho }_{\alpha }^{\mathrm{w}}(p,q),$$

and therefore

$${\int }_{\mathcal{X}}\phi \left(x\right)min\{p\left(x\right),q\left(x\right)\}\mathrm{d}\mu \left(x\right)\le exp\{-D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})\},$$

where $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q}):={max}_{\alpha \in [0,1]}\left[-ln{\rho }_{\alpha }^{\mathrm{w}}(p,q)\right]$ (see Definition 2). Under Assumption 1, the same bound factorises over n observations and yields the exponential scale $exp\{-n{D}_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})\}$. Theorem 1 shows that this scale is exact on the logarithmic level.

1.1. Main Result and Contributions

Let $L_n^{\ast }$ denote the optimal total context-sensitive loss (sum of weighted type-I and type-II losses, minimised over decision rules) for n i.i.d. observations under Assumption 1. Our main theorem (Theorem 1) proves the single-letter logarithmic asymptotic

$$L_n^{\ast }=exp\{-n{D}_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})+o\left(n\right)\},n\to \infty ,$$

(4)

where the rate is the weighted Chernoff information

$$D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=\underset{\alpha \in [0, 1]}{max}\left[-ln{\int }_{\mathcal{X}}\phi \left(x\right)p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right)\right].$$

(5)

For $\phi \equiv 1$, (5) reduces to the classical Chernoff information.

We also extend the exponent characterisation to a finite family of simple hypotheses: the optimal M-ary rate is the minimum pairwise weighted Chernoff information (cf. [7] in the unweighted case). A central technical device is an exponential-family representation of the weighted geometric mixtures $\alpha \mapsto \phi p^{\alpha }{q}^{1-\alpha }$. This embeds the mixtures into a likelihood-ratio exponential family and identifies the exponent through the corresponding log-normaliser. We further derive concentration bounds for tilted weighted log-likelihood ratios and closed-form expressions for $D_C^{\mathrm{w}}$ in several parametric models; see Section 4.

1.2. Contributions

Items (N1)–(N4) below indicate new results; items (A1)–(A3) summarise definitions, geometric context, and tools adopted from the existing literature.

(N1)

(New.) Theorem 1 establishes the logarithmic asymptotic (4) for the optimal weighted total loss under the factorised weight of Assumption 1, with rate given by the weighted Chernoff information (5).

(N2)

(New.) The exponential-family representation of the weighted geometric mixtures $\alpha \mapsto \phi p^{\alpha }{q}^{1-\alpha }$ (Section 3.2) and the resulting uniqueness of the optimal skewing parameter ${\alpha }^{\ast }$.

(N3)

(New.) Concentration bounds for the tilted weighted log-likelihood and the finite-n tail bound of Theorem 2 (Section 3.4).

(N4)

(New.) Closed-form expressions for $D_C^{\mathrm{w}}$ in the Gaussian, Poisson, and exponential models (Section 4), and the M-ary extension showing that the optimal rate equals the minimum pairwise weighted Chernoff information.

(A1)

(Adapted definitions.) The definitions of the weighted Bhattacharyya affinities and the weighted Chernoff information generalise the classical unweighted quantities of [1,2] and follow the context-sensitive framework of [3,4]; their asymptotic and information-geometric consequences developed below are new.

(A2)

(Geometric context.) The information-geometric identities of Section 3.3 are derived in the spirit of the Chentsov–Amari–Nielsen framework [8,9,10,11] but are stated and proved for the tilted log-normaliser $\widehat{F}\left(\theta \right)=ln\int \phi \left(x\right)e^{{\theta }^{\mathrm{T}}t\left(x\right)+k\left(x\right)}\mathrm{d}\mu \left(x\right)$; the unweighted limit $\phi \equiv 1$ recovers the classical statements of [11,12].

(A3)

(Standard tool.) The concentration argument uses the Azuma–Hoeffding/McDiarmid inequality [13,14]; the novelty lies in its application to the tilted weighted log-likelihood.

1.3. Related Work

The exponential theory of testing errors goes back to Chernoff [1] and Hoeffding [2]. The context-sensitive framework and the weighted information quantities used here were developed in [3,4]. The information-geometric viewpoint on Chernoff information originates with Chentsov [9]; the dually flat structure of exponential and mixture families and the associated $\alpha$-divergences are developed in [8,10], and the Chernoff point is characterised as the intersection of an exponential geodesic with the Kullback–Leibler bisector in [11]. For $\phi \equiv 1$, the likelihood-ratio exponential family description is given in [12]; the present paper extends this picture to the tilted integrand $\phi p^{\alpha }{q}^{1-\alpha }$. The minimum-pairwise principle for multiple testing is due to [7]. Weighting mechanisms for covariate-dependent relevance have also been studied outside the asymptotic error-exponent framework, e.g., adaptive-kernel conditional-independence testing [15].

1.4. Structure of the Paper

Section 2 introduces the weighted Bhattacharyya affinities and the weighted Chernoff information. Section 3 proves the main asymptotic result (4) and develops the exponential-family and information-geometric identities. Section 3.4 studies the tilted weighted log-likelihood and derives finite-n concentration bounds. Section 4 examines Gaussian, Poisson, and exponential models and includes the M-ary extension. Auxiliary computations are collected in the appendices.

2. Problem Set-Up and Weighted Divergences

2.1. Context-Sensitive Losses and Weighted Total Variation

We keep the binary i.i.d. model from Section 1 and work under Assumption 1. In particular,

$$p\left(x_1^n\right)={\displaystyle \prod _{i=1}^n}p\left(x_i\right),q\left(x_1^n\right)={\displaystyle \prod _{i=1}^n}q\left(x_i\right),\phi \left(x_1^n\right)={\displaystyle \prod _{i=1}^n}\phi \left(x_i\right).$$

Define the $\phi$-tilted (reweighted) densities

$$p^{\ast }\left(x\right):={\displaystyle \frac{\phi \left(x\right)p\left(x\right)}{E_{\phi }\left(p\right)}},q^{\ast }\left(x\right):={\displaystyle \frac{\phi \left(x\right)q\left(x\right)}{E_{\phi }\left(q\right)}},E_{\phi }\left(p\right):=\int \phi \left(x\right)p\left(x\right)d\mu \left(x\right),$$

(and similarly for $E_{\phi }\left(q\right)$). Throughout this section, we assume that $E_{\phi }\left(p\right),E_{\phi }\left(q\right)\in (0,\infty )$. (Equivalently, ${\rho }_0^{\mathrm{w}}(p,q),{\rho }_1^{\mathrm{w}}(p,q)\in (0,\infty ).$) Then $p^{\ast },q^{\ast }$ are probability densities and, under $\phi \left(x_1^n\right)={\displaystyle \prod _{i=1}^n}\phi \left(x_i\right)$, we have $\phi \left(x_1^n\right)p\left(x_1^n\right)=E_{\phi }{\left(p\right)}^n{\left(p^{\ast }\right)}^{\otimes n}\left(x_1^n\right)$ (and similarly for q).

Under Assumption 1, we have

$${\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)p\left(x_1^n\right)d{\mu }^{\otimes n}={\left(E_{\phi }\left(p\right)\right)}^n,{\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)q\left(x_1^n\right)d{\mu }^{\otimes n}={\left(E_{\phi }\left(q\right)\right)}^n.$$

Let $\mathcal{D}$ denote the class of (possibly randomised) decision rules $D:{\mathcal{X}}^n\to [0,1]$, where $D\left(x_1^n\right)$ is the probability of deciding in favour of $H_1$ after observing $x_1^n$. (Deterministic rules correspond to $D\in \{0,1\}$.)

For $D\in \mathcal{D}$, define the context-sensitive type-I and type-II losses by

$$\begin{array}{cc}\hfill {\alpha }_{\phi }\left(D\right)& :={\mathbb{E}}_{{\mathbb{P}}^{\otimes n}}\left[\phi \left(X_1^n\right)D\left(X_1^n\right)\right]={\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)D\left(x_1^n\right)p\left(x_1^n\right)\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right),\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\beta }_{\phi }\left(D\right)& :={\mathbb{E}}_{{\mathbb{Q}}^{\otimes n}}\left[\phi \left(X_1^n\right)(1-D\left(X_1^n\right))\right]={\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)(1-D\left(x_1^n\right))q\left(x_1^n\right)\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right),\hfill \end{array}$$

(7)

and the corresponding total loss

$$L_n\left(D\right):={\alpha }_{\phi }\left(D\right)+{\beta }_{\phi }\left(D\right),L_n^{\ast }:=\underset{D\in \mathcal{D}}{inf}{L}_n\left(D\right).$$

Proposition 1 

(Pointwise form of the optimal total loss). For each $n\ge 1$,

$$L_n^{\ast }={\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)min\{p\left(x_1^n\right),q\left(x_1^n\right)\}\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right).$$

(8)

Moreover, an optimal (deterministic) decision rule is given by the likelihood-ratio test

$$D_n^{\ast }\left(x_1^n\right)=\mathbf{1}\{q\left(x_1^n\right)\ge p\left(x_1^n\right)\}$$

(with any measurable tie-breaking on $\{p=q\}$).

Proof. 

Fix $x_1^n$. The integrand in $L_n\left(D\right)$ equals

$$\phi \left(x_1^n\right)\left(p\left(x_1^n\right)D\left(x_1^n\right)+q\left(x_1^n\right)(1-D\left(x_1^n\right))\right)=\phi \left(x_1^n\right)\left(q\left(x_1^n\right)+D\left(x_1^n\right)(p\left(x_1^n\right)-q\left(x_1^n\right))\right).$$

Minimising pointwise over $D\left(x_1^n\right)\in [0, 1]$ yields $D_n^{\ast }\left(x_1^n\right)=1$ when $p\left(x_1^n\right)\le q\left(x_1^n\right)$ and $D_n^{\ast }\left(x_1^n\right)=0$ when $p\left(x_1^n\right)>q\left(x_1^n\right)$, giving (8). □

We also use the weighted total variation distance

$${\mathrm{TV}}_{\phi }({\mathbb{P}}^{\otimes n},{\mathbb{Q}}^{\otimes n}):={\displaystyle \frac{1}{2}}{\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)|p\left(x_1^n\right)-q\left(x_1^n\right)|\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right).$$

(9)

Remark 1. 

For $\phi \equiv 1$, this reduces to the usual total variation distance. If φ vanishes on a non-negligible set, ${\mathrm{TV}}_{\phi }$ is, in general, a pseudo-distance; this is sufficient for our purposes since it characterises the weighted losses.

Using $min\{a,b\}={\displaystyle \frac{1}{2}}(a+b-|a-b|)$ in (8) and the definition of ${\mathrm{TV}}_{\phi }$ yields

$$L_n^{\ast }={\displaystyle \frac{1}{2}}\left({\left(E_{\phi }\left(p\right)\right)}^n+{\left(E_{\phi }\left(q\right)\right)}^n\right)-{\mathrm{TV}}_{\phi }({\mathbb{P}}^{\otimes n},{\mathbb{Q}}^{\otimes n}).$$

(10)

2.2. Weighted Affinities and Chernoff Information

We introduce the weighted Bhattacharyya affinities and the weighted Chernoff information. Assume that ${\rho }_{\alpha }^{\mathrm{w}}(p,q)\in (0,\infty )$ for all $\alpha \in [0, 1]$.

Definition 1 

(Weighted Bhattacharyya coefficient and distance). For $\alpha \in [0, 1]$ define the weighted α-skewed Bhattacharyya affinity coefficient

$${\rho }_{\alpha }^{\mathrm{w}}(p,q):={\int }_{\mathcal{X}}\phi \left(x\right)p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right),$$

(11)

and the corresponding weighted Bhattacharyya distance

$$D_{B,\alpha }^{\mathrm{w}}(p,q):=-ln{\rho }_{\alpha }^{\mathrm{w}}(p,q).$$

(12)

Definition 2 

(Weighted Chernoff information). The weighted Chernoff information divergence between $\mathbb{P}$ and $\mathbb{Q}$ is

$$D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=\underset{\alpha \in [0,1]}{max}\left[-ln{\int }_{\mathcal{X}}\phi \left(x\right)p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right)\right]=\underset{\alpha \in [0,1]}{max}{D}_{B,\alpha }^{\mathrm{w}}(p,q).$$

(13)

A maximiser ${\alpha }^{\ast }={\alpha }^{\ast }(p,q)$ in (13) is called the optimal Chernoff parameter.

Remark 2. 

The weighted Chernoff information is symmetric: $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=D_C^{\mathrm{w}}(\mathbb{Q},\mathbb{P})$, since ${\rho }_{\alpha }^{\mathrm{w}}(p,q)={\rho }_{1-\alpha }^{\mathrm{w}}(q,p)$. In general, however, $D_C^{\mathrm{w}}$ does not satisfy the triangle inequality and is therefore a divergence rather than a metric.

Remark 3. 

Under Assumption 1, for every $\alpha \in [0, 1]$,

$${\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)p{\left(x_1^n\right)}^{\alpha }q{\left(x_1^n\right)}^{1-\alpha }\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right)={\left({\rho }_{\alpha }^{\mathrm{w}}(p,q)\right)}^n.$$

Consequently, the weighted Bhattacharyya distances are additive in n and the corresponding Chernoff exponent is of single-letter form.

3. Asymptotics and Information-Geometric Identities

Before stating the main theorem, we separate the two optimisations that appear throughout this section and should not be conflated. The total loss $L_n\left(D\right)={\alpha }_{\phi }\left(D\right)+{\beta }_{\phi }\left(D\right)$ is a non-negative functional of the decision rule and is minimised over $D\in \mathcal{D}$, giving the optimal loss $L_n^{\ast }={inf}_{D\in \mathcal{D}}{L}_n\left(D\right)$. The map $\alpha \mapsto -ln{\rho }_{\alpha }^{\mathrm{w}}(p,q)$ is a non-negative concave functional of the skewing parameter and is maximised over $\alpha \in [0,1]$, giving the weighted Chernoff information $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})={sup}_{\alpha \in [0,1]}[-ln{\rho }_{\alpha }^{\mathrm{w}}(p,q)]$. Theorem 1 below connects the two via the single-letter asymptotic $L_n^{\ast }=exp\{-n{D}_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})+o\left(n\right)\}$.

3.1. Asymptotics of the Optimal Sum of Losses

Recall from Section 2 that

$$L_n^{\ast }:=\underset{D\in \mathcal{D}}{inf}\left[{\alpha }_{\phi }\left(D\right)+{\beta }_{\phi }\left(D\right)\right],$$

and that Proposition 1 yields (8). The next theorem identifies its exact logarithmic asymptotic rate with the weighted Chernoff information from Definition 2.

Theorem 1 

(Optimal sum of context-sensitive losses). Consider the binary hypotheses $H_0:X_1^n\sim {\mathbb{P}}^{\otimes n}$ versus $H_1:X_1^n\sim {\mathbb{Q}}^{\otimes n}$, under Assumption 1. Assume also

$$\underset{\alpha \in [0,1]}{sup}\int \phi \left(x\right)|ln{\displaystyle \frac{p\left(x\right)}{q\left(x\right)}}|p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right)<\infty .$$

(14)

Let $\mathcal{D}$ be the class of (possibly randomised) decision rules $D:{\mathcal{X}}^n\to [0, 1]$, and let ${\alpha }_{\phi }\left(D\right)$, ${\beta }_{\phi }\left(D\right)$ be defined by (6) and (7). Assume that $p,q>0$μ-a.e. and that ${\rho }_{\alpha }^{\mathrm{w}}(p,q)\in (0,\infty )$ for all $\alpha \in [0, 1]$.

Then, as $n\to \infty$,

$$L_n^{\ast }=exp\{-n{D}_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})+o\left(n\right)\}.$$

(15)

Equivalently,

$$\underset{n\to \infty }{lim}-{\displaystyle \frac{1}{n}}ln{L}_n^{\ast }=D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q}).$$

Proof. 

By Proposition 1,

$$L_n^{\ast }={\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)min\{p\left(x_1^n\right),q\left(x_1^n\right)\}\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right).$$

For any $\alpha \in [0,1]$, $min\{a,b\}\le a^{\alpha }{b}^{1-\alpha }$, hence, by factorisation,

$$L_n^{\ast }\le {\left({\rho }_{\alpha }^{\mathrm{w}}(p,q)\right)}^n.$$

Taking the infimum over $\alpha \in [0,1]$ gives

$$\underset{n\to \infty }{lim\; inf}-{\displaystyle \frac{1}{n}}ln{L}_n^{\ast }\ge D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q}).$$

Now fix ${\alpha }^{\ast }\in arg{min}_{\alpha \in [0,1]}{\rho }_{\alpha }^{\mathrm{w}}(p,q)$ and define

$$r_{{\alpha }^{\ast }}\left(x\right):={\displaystyle \frac{\phi \left(x\right)p{\left(x\right)}^{{\alpha }^{\ast }}q{\left(x\right)}^{1-{\alpha }^{\ast }}}{{\rho }_{{\alpha }^{\ast }}^{\mathrm{w}}(p,q)}},S_n:={\displaystyle \sum _{i=1}^n}ln{\displaystyle \frac{p\left(X_i\right)}{q\left(X_i\right)}}.$$

A direct change of measure yields

$$L_n^{\ast }={\left({\rho }_{{\alpha }^{\ast }}^{\mathrm{w}}(p,q)\right)}^n{\mathbb{E}}_{r_{{\alpha }^{\ast }}^{\otimes n}}\left[e^{-{\alpha }^{\ast }{S}_n}\mathbf{1}\{S_n>0\}+e^{(1-{\alpha }^{\ast })S_n}\mathbf{1}\{S_n\le 0\}\right].$$

The bracket is bounded above by 1.

Let $F\left(\alpha \right):=ln{\rho }_{\alpha }^{\mathrm{w}}(p,q)$, it is easy to check that $F\left(\alpha \right)$ is a convex function. Under the regularity assumption of (14),

$$F^{\prime }\left(\alpha \right)={\mathbb{E}}_{r_{\alpha }}\left[ln{\displaystyle \frac{p\left(X\right)}{q\left(X\right)}}\right].$$

If ${\alpha }^{\ast }\in (0, 1)$, then $F^{\prime }\left({\alpha }^{\ast }\right)=0$. Hence, in view of LLN, $S_n/n\to 0$ in $r_{{\alpha }^{\ast }}$-probability. Therefore, for every $\epsilon >0$,

$${\mathbb{E}}_{r_{{\alpha }^{\ast }}^{\otimes n}}\left[e^{(1-{\alpha }^{\ast })S_n}\mathbf{1}\{S_n\le 0\}+e^{-{\alpha }^{\ast }{S}_n}\mathbf{1}\{S_n>0\}\right]\ge e^{-\epsilon n}{r}_{{\alpha }^{\ast }}^{\otimes n}\left(|S_n|\le \epsilon n\right)=exp\left\{o\left(n\right)\right\}.$$

Combined with the upper bound by 1, this implies

$$L_n^{\ast }=exp\{-n{D}_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})+o\left(n\right)\}.$$

In the boundary case ${\alpha }^{\ast }=0$, we have the mean value $m>0$, and $\mathbf{1}(S_n>0)e^{-{\alpha }^{\ast }{S}_n}{|}_{{\alpha }^{\ast }=0}\to 1$ a.s. as $n\to \infty$ by the strong LLN. Similarly, in the case ${\alpha }^{\ast }=1$, we have the mean value $m\le 0$, and $\mathbf{1}(S_n\le 0)e^{(1-{\alpha }^{\ast })S_n}{|}_{{\alpha }^{\ast }=1}\to 1$ a.s. as $n\to \infty$. This completes the proof. □

Corollary 1 

(Asymptotics of the weighted total variation). Under the assumptions of Theorem 1, the weighted total variation satisfies

$${\mathrm{TV}}_{\phi }({\mathbb{P}}^{\otimes n},{\mathbb{Q}}^{\otimes n})={\displaystyle \frac{1}{2}}\left({\left(E_{\phi }\left(p\right)\right)}^n+{\left(E_{\phi }\left(q\right)\right)}^n\right)-exp\{-n{D}_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})+o\left(n\right)\},n\to \infty ,$$

(16)

where $E_{\phi }\left(r\right)={\int }_{\mathcal{X}}\phi \left(x\right)r\left(x\right)\mathrm{d}\mu \left(x\right)$.

Proof. 

Combine the identity (10) with (15). □

Remark 4. 

The weighted α-skewed Bhattacharyya distance (12) appears in many papers, see e.g., [12]. Definition 2 shows that the weighted Chernoff information divergence is the maximally skewed weighted Bhattacharyya distance.

3.2. Exponential-Family Representation and Uniqueness of ${\alpha }^{\ast }$

In order to develop an effective computational procedure and to connect the weighted Chernoff information to information geometry, we embed the weighted geometric mixtures of p and q into a one-parameter likelihood-ratio exponential family. For $\alpha \in [0, 1]$ define

$$Z_{pq}\left(\alpha \right):={\int }_{\mathcal{X}}\phi \left(x\right)p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right)={\rho }_{\alpha }^{\mathrm{w}}(p,q),$$

(17)

and the corresponding normalised density

$${\mathcal{E}}_{pq}=\left\{{\left(pq\right)}_{\alpha }\left(x\right):={\displaystyle \frac{\phi \left(x\right)p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }}{Z_{pq}\left(\alpha \right)}}:\alpha \in [0, 1]\right\}.$$

(18)

By assumption $Z_{pq}\left(\alpha \right)\in (0,\infty )$, so ${\left(pq\right)}_{\alpha }$ is well-defined as a probability density w.r.t. $\mu$.

Set $t\left(x\right):=ln{\displaystyle \frac{p\left(x\right)}{q\left(x\right)}}$ and $k_{pq}\left(x\right):=ln\phi \left(x\right)+lnq\left(x\right)$. Then, ${\left(pq\right)}_{\alpha }$ admits the exponential-family form

$$\begin{array}{cc}\hfill {\left(pq\right)}_{\alpha }\left(x\right)& =exp\left\{\alpha t\left(x\right)-F_{pq}\left(\alpha \right)+k_{pq}\left(x\right)\right\},\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill F_{pq}\left(\alpha \right):& =ln{Z}_{pq}\left(\alpha \right)=-D_{B,\alpha }^{\mathrm{w}}(p,q).\hfill \end{array}$$

(20)

In particular, $t\left(X\right)$ is a sufficient statistic for the family ${\mathcal{E}}_{pq}$.

The log-normaliser $F_{pq}$ is convex on $[0, 1]$; if $ln{\displaystyle \frac{p}{q}}$ is not $\mu$-a.e. constant on $\{\phi >0\}$, then $F_{pq}$ is strictly convex and the maximiser ${\alpha }^{\ast }$ in (13) is unique. By Hölder’s inequality,

$$Z_{pq}\left(\alpha \right)={\int }_{\mathcal{X}}{\left(\phi p\right)}^{\alpha }{\left(\phi q\right)}^{1-\alpha }\mathrm{d}\mu \le {\left(E_{\phi }\left(p\right)\right)}^{\alpha }{\left(E_{\phi }\left(q\right)\right)}^{1-\alpha }.$$

Finally, note that $Z_{pq}\left(1\right)=E_{\phi }\left(p\right)$ and $Z_{pq}\left(0\right)=E_{\phi }\left(q\right)$; hence,

$${\left(pq\right)}_1\left(x\right)={\displaystyle \frac{\phi \left(x\right)p\left(x\right)}{E_{\phi }\left(p\right)}},{\left(pq\right)}_0\left(x\right)={\displaystyle \frac{\phi \left(x\right)q\left(x\right)}{E_{\phi }\left(q\right)}},$$

so ${\mathcal{E}}_{pq}$ is an exponential arc between the tilted versions of $\mathbb{P}$ and $\mathbb{Q}$. By this definition, the following identities hold:

$$\begin{array}{c}{D}_C^{\mathrm{w}}(p,q)=D_{B,{\alpha }^{\ast }(p,q)}^{\mathrm{w}}(p,q)=D_{B,{\alpha }^{\ast }(q,p)}^{\mathrm{w}}(q,p)=D_C^{\mathrm{w}}(q,p).\hfill \end{array}$$

(21)

3.3. Weighted Bregman Divergence and Information-Geometric Identities

3.3.1. Weighted KL Divergence and Weighted Bregman Divergence

This subsection collects information-geometric identities useful for analysing ${\rho }_{\alpha }^{\mathrm{w}}$ and for computing the optimal Chernoff parameter ${\alpha }^{\ast }$. We follow [3] for weighted Bregman divergences.

Let $\mathcal{E}=\{p_{\theta }:\theta \in \mathrm{\Theta }\subset {\mathbb{R}}^d\}$ be a regular exponential family of densities (with respect to $\mu$),

$$p_{\theta }\left(x\right)=exp\{{\theta }^{\mathrm{T}}t\left(x\right)-F\left(\theta \right)+k\left(x\right)\},x\in \mathcal{X}.$$

(22)

For a density r, set $E_{\phi }\left(r\right):={\int }_{\mathcal{X}}\phi \left(x\right)r\left(x\right)\mathrm{d}\mu \left(x\right)$ and write $E_{\phi }\left(\theta \right):=E_{\phi }\left(p_{\theta }\right)$.

Assume $E_{\phi }\left(\theta \right)\in (0,\infty )$ for $\theta \in \mathrm{\Theta }$ and define the tilted log-normaliser

$$\widehat{F}\left(\theta \right):=ln{\int }_{\mathcal{X}}\phi \left(x\right)e^{{\theta }^{\mathrm{T}}t\left(x\right)+k\left(x\right)}\mathrm{d}\mu \left(x\right)=F\left(\theta \right)+ln{E}_{\phi }\left(\theta \right).$$

(23)

Equivalently, the tilted density is

$$p_{\theta }^{\ast }\left(x\right)={\displaystyle \frac{\phi \left(x\right)p_{\theta }\left(x\right)}{E_{\phi }\left(\theta \right)}}.$$

(24)

Definition 3 

(Weighted Kullback–Leibler divergence). For densities $p,q$ on $\mathcal{X}$ define

$$D_{\mathrm{KL}}^{\mathrm{w}}(p\parallel q):={\int }_{\mathcal{X}}\phi \left(x\right)p\left(x\right)ln{\displaystyle \frac{p\left(x\right)}{q\left(x\right)}}\mathrm{d}\mu \left(x\right),$$

(25)

whenever the integral is well defined in $(-\infty ,\infty ]$.

Definition 4 

(Weighted Bregman divergence). The weighted Bregman divergence associated with $(F,\widehat{F})$ is

$$\begin{array}{cc}\hfill B_{\phi ,F}^{\mathrm{w}}({\theta }_1,{\theta }_2):& =e^{\widehat{F}\left({\theta }_2\right)-F\left({\theta }_2\right)}\left[F\left({\theta }_1\right)-F\left({\theta }_2\right)-{({\theta }_1-{\theta }_2)}^{\mathrm{T}}\nabla \widehat{F}\left({\theta }_2\right)\right]\hfill \\ \hfill & =E_{\phi }\left({\theta }_2\right)\left[F\left({\theta }_1\right)-F\left({\theta }_2\right)-{({\theta }_1-{\theta }_2)}^{\mathrm{T}}\nabla \widehat{F}\left({\theta }_2\right)\right].\hfill \end{array}$$

(26)

Proposition 2 

(Weighted KL as weighted Bregman divergence). For a regular exponential family $\mathcal{E}=\{{\mathbb{P}}_{\theta },\theta \in \mathrm{\Theta }\}$, assume that the integral in (25) is well-defined in $(-\infty ,\infty ]$. Then for any ${\theta }_1,{\theta }_2\in \mathrm{\Theta }$,

$$D_{\mathrm{KL}}^{\mathrm{w}}(p_{{\theta }_1}\parallel p_{{\theta }_2})=B_{\phi ,F}^{\mathrm{w}}({\theta }_2,{\theta }_1).$$

(27)

Proof. 

This identity is stated in [3] (Proposition 4.1); we give a short derivation for completeness. By (22),

$$ln{\displaystyle \frac{p_{{\theta }_1}\left(x\right)}{p_{{\theta }_2}\left(x\right)}}={({\theta }_1-{\theta }_2)}^{\mathrm{T}}t\left(x\right)-(F\left({\theta }_1\right)-F\left({\theta }_2\right)).$$

Substituting this into (25) yields

$$D_{\mathrm{KL}}^{\mathrm{w}}(p_{{\theta }_1}\parallel p_{{\theta }_2})={({\theta }_1-{\theta }_2)}^{\mathrm{T}}{\int }_{\mathcal{X}}\phi \left(x\right)t\left(x\right)p_{{\theta }_1}\left(x\right)\mathrm{d}\mu \left(x\right)-(F\left({\theta }_1\right)-F\left({\theta }_2\right))E_{\phi }\left({\theta }_1\right).$$

Using (23) and differentiation under the integral sign (regularity of $\mathcal{E}$),

$$\nabla \widehat{F}\left({\theta }_1\right)={\displaystyle \frac{{\int }_{\mathcal{X}}\phi \left(x\right)t\left(x\right)e^{{\theta }_1^{\mathrm{T}}t\left(x\right)+k\left(x\right)}\mathrm{d}\mu \left(x\right)}{{\int }_{\mathcal{X}}\phi \left(x\right)e^{{\theta }_1^{\mathrm{T}}t\left(x\right)+k\left(x\right)}\mathrm{d}\mu \left(x\right)}}={\displaystyle \frac{{\int }_{\mathcal{X}}\phi \left(x\right)t\left(x\right)p_{{\theta }_1}\left(x\right)\mathrm{d}\mu \left(x\right)}{E_{\phi }\left({\theta }_1\right)}}.$$

Hence, $\int \phi t{p}_{{\theta }_1}\mathrm{d}\mu =E_{\phi }\left({\theta }_1\right)\nabla \widehat{F}\left({\theta }_1\right)$, and therefore

$$D_{\mathrm{KL}}^{\mathrm{w}}(p_{{\theta }_1}\parallel p_{{\theta }_2})=E_{\phi }\left({\theta }_1\right)\left[F\left({\theta }_2\right)-F\left({\theta }_1\right)-{({\theta }_2-{\theta }_1)}^{\mathrm{T}}\nabla \widehat{F}\left({\theta }_1\right)\right]=B_{\phi ,F}^{\mathrm{w}}({\theta }_2,{\theta }_1),$$

which proves (27). □

Proposition 3 

(Primal–dual identities for (weighted) Bregman divergences). Let F be a log-normaliser of a regular exponential family and let $F^{\ast }$ denote its Legendre transform. Write ${\theta }^{\ast }=\nabla F\left(\theta \right)$ and $\theta =\nabla F^{\ast }\left({\theta }^{\ast }\right)$.

(a) Weighted one-parameter identity. Assume $d=1$ (one-parameter case) and let ${\theta }_i^{\ast }:=F^{\prime }\left({\theta }_i\right)$. Then, the following weighted analogue of the classical Bregman duality holds:

$$B_{\phi ,F}^{\mathrm{w}}({\theta }_1,{\theta }_2)=B_{\phi ,F^{\ast }}^{\mathrm{w}}({\theta }_2^{\ast },{\theta }_1^{\ast })-({\theta }_1-{\theta }_2)ln{E}_{\phi }\left({\theta }_2\right)+({\theta }_2^{\ast }-{\theta }_1^{\ast })ln{E}_{\phi }\left({\theta }_1\right),$$

(28)

where $B_{\phi ,F}^{\mathrm{w}}$ is as in Definition 4 and $B_{\phi ,F^{\ast }}^{\mathrm{w}}$ is defined analogously (with F replaced by $F^{\ast }$ and with the convention $E_{\phi }\left({\theta }^{\ast }\right):=E_{\phi }\left(\theta \right)$ under $\theta =\nabla F^{\ast }\left({\theta }^{\ast }\right)$).

(b) Classical identity. For any $d\ge 1$, the (unweighted) Bregman divergence admits the standard Legendre representation

$$B_F({\theta }_0,{\theta }_1)=F\left({\theta }_0\right)+F^{\ast }\left({\theta }_1^{\ast }\right)-{\theta }_0^{\mathrm{T}}{\theta }_1^{\ast },{\theta }_1^{\ast }=\nabla F\left({\theta }_1\right),$$

(29)

where $B_F({\theta }_0,{\theta }_1)=F\left({\theta }_0\right)-F\left({\theta }_1\right)-{({\theta }_0-{\theta }_1)}^{\mathrm{T}}\nabla F\left({\theta }_1\right)$ is the usual (unweighted) Bregman divergence.

Proof. 

Part (a) is a weighted extension of the classical duality $B_F({\theta }_1,{\theta }_2)=B_{F^{\ast }}({\theta }_2^{\ast },{\theta }_1^{\ast })$ and follows by combining the weighted representation (26) with Legendre relations; see also [3]. Part (b) is standard. □

3.3.2. Weighted Chernoff/Bhattacharyya Quantities Inside an Exponential Family

Let $p_{{\theta }_1},p_{{\theta }_2}\in \mathcal{E}$ and ${\theta }_{\alpha }:=\alpha {\theta }_1+(1-\alpha ){\theta }_2$. A direct calculation yields

$$\begin{array}{cc}\hfill {\rho }_{\alpha }^{\mathrm{w}}(p_{{\theta }_1},p_{{\theta }_2})& =ln{\int }_{\mathcal{X}}\phi \left(x\right)p_{{\theta }_1}{\left(x\right)}^{\alpha }{p}_{{\theta }_2}{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right)\hfill \\ \hfill & =\widehat{F}\left({\theta }_{\alpha }\right)-\alpha F\left({\theta }_1\right)-(1-\alpha )F\left({\theta }_2\right).\hfill \end{array}$$

(30)

Consequently,

$$\begin{array}{cc}\hfill D_{B,\alpha }^{\mathrm{w}}(p_{{\theta }_1},p_{{\theta }_2})& =\alpha F\left({\theta }_1\right)+(1-\alpha )F\left({\theta }_2\right)-\widehat{F}\left({\theta }_{\alpha }\right)\hfill \\ \hfill & =U_{F,\alpha }({\theta }_1,{\theta }_2)-ln{E}_{\phi }\left({\theta }_{\alpha }\right),\hfill \end{array}$$

(31)

where $U_{F,\alpha }({\theta }_1,{\theta }_2):=\alpha F\left({\theta }_1\right)+(1-\alpha )F\left({\theta }_2\right)-F\left({\theta }_{\alpha }\right)$ is the (unweighted) Jensen/Burbea–Rao divergence induced by F. In particular, when $\phi \equiv 1$ we have $\widehat{F}\equiv F$ and $D_{B,\alpha }^{\mathrm{w}}(p_{{\theta }_1},p_{{\theta }_2})=U_{F,\alpha }({\theta }_1,{\theta }_2)$.

Remark 5 

(Geometric mixtures and tilting by $\phi$). In particular, when $\phi \equiv 1$, we have $\widehat{F}\equiv F$ and the normalised geometric mixture $p_{{\theta }_1}^{\alpha }{p}_{{\theta }_2}^{1-\alpha }$ belongs to the same exponential family, namely, $\propto p_{{\theta }_{\alpha }}$ with ${\theta }_{\alpha }=\alpha {\theta }_1+(1-\alpha ){\theta }_2$.

Proposition 4 

(Optimal Chernoff parameter in an exponential family). Assume that $\widehat{F}$ is strictly convex on the segment $[{\theta }_1,{\theta }_2]$ and that the maximiser ${\alpha }^{\ast }\in (0,1)$ exists. Then, ${\alpha }^{\ast }$ is unique and satisfies

$${({\theta }_1-{\theta }_2)}^{\mathrm{T}}\nabla \widehat{F}\left({\theta }_{{\alpha }^{\ast }}\right)=F\left({\theta }_1\right)-F\left({\theta }_2\right),{\theta }_{{\alpha }^{\ast }}={\alpha }^{\ast }{\theta }_1+(1-{\alpha }^{\ast }){\theta }_2,$$

(32)

with $\nabla \widehat{F}\left(\theta \right)={\mathbb{E}}_{p_{\theta }^{\ast }}\left[t\left(X\right)\right]$.

Proof. 

Differentiate (31) with respect to $\alpha$ and use ${\displaystyle \frac{\mathrm{d}}{\mathrm{d}\alpha }}{\theta }_{\alpha }={\theta }_1-{\theta }_2$. Strict convexity of $\widehat{F}$ on $[{\theta }_1,{\theta }_2]$ implies strict concavity of $\alpha \mapsto D_{B,\alpha }^{\mathrm{w}}(p_{{\theta }_1},p_{{\theta }_2})$, hence uniqueness. □

Proposition 5 

(Chernoff information as a Jensen-type divergence and a Bregman bisector). Let $p_{{\theta }_1},p_{{\theta }_2}\in \mathcal{E}$ and assume that the maximiser ${\alpha }^{\ast }\in (0,1)$ in Definition 2 exists and is unique. Set ${\theta }_{\alpha }=\alpha {\theta }_1+(1-\alpha ){\theta }_2$. Then

$$D_C^{\mathrm{w}}(p_{{\theta }_1},p_{{\theta }_2})=D_{B,{\alpha }^{\ast }}^{\mathrm{w}}(p_{{\theta }_1},p_{{\theta }_2})={\alpha }^{\ast }F\left({\theta }_1\right)+(1-{\alpha }^{\ast })F\left({\theta }_2\right)-\widehat{F}\left({\theta }_{{\alpha }^{\ast }}\right).$$

(33)

Moreover, ${\theta }_{{\alpha }^{\ast }}$ is characterised by the weighted Bregman bisector condition

$$B_{\phi ,F}^{\mathrm{w}}({\theta }_1,{\theta }_{{\alpha }^{\ast }})=B_{\phi ,F}^{\mathrm{w}}({\theta }_2,{\theta }_{{\alpha }^{\ast }}),$$

(34)

and the common value recovers the Chernoff information as

$$\begin{array}{cc}\hfill D_C^{\mathrm{w}}(p_{{\theta }_1},p_{{\theta }_2})& ={\displaystyle \frac{1}{E_{\phi }\left({\theta }_{{\alpha }^{\ast }}\right)}}{B}_{\phi ,F}^{\mathrm{w}}({\theta }_1,{\theta }_{{\alpha }^{\ast }})-ln{E}_{\phi }\left({\theta }_{{\alpha }^{\ast }}\right)\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill & ={\displaystyle \frac{1}{E_{\phi }\left({\theta }_{{\alpha }^{\ast }}\right)}}{B}_{\phi ,F}^{\mathrm{w}}({\theta }_2,{\theta }_{{\alpha }^{\ast }})-ln{E}_{\phi }\left({\theta }_{{\alpha }^{\ast }}\right).\hfill \end{array}$$

(36)

In the special case $\phi \equiv 1$, we have $\widehat{F}\equiv F$ and $E_{\phi }\left(\theta \right)\equiv 1$, so that (33) reduces to the classical Jensen divergence induced by F and (35) becomes $D_C(p_{{\theta }_1},p_{{\theta }_2})=B_F({\theta }_1,{\theta }_{{\alpha }^{\ast }})=B_F({\theta }_2,{\theta }_{{\alpha }^{\ast }})$.

Proof. 

Equation (33) is (31) at $\alpha ={\alpha }^{\ast }$. For (34), expand $B_{\phi ,F}^{\mathrm{w}}({\theta }_i,{\theta }_{{\alpha }^{\ast }})=E_{\phi }\left({\theta }_{{\alpha }^{\ast }}\right)\left[F\left({\theta }_i\right)-F\left({\theta }_{{\alpha }^{\ast }}\right)-{({\theta }_i-{\theta }_{{\alpha }^{\ast }})}^{\mathrm{T}}\nabla \widehat{F}\left({\theta }_{{\alpha }^{\ast }}\right)\right]$ and use (32) to see that the difference vanishes. Finally, substituting ${\theta }_1-{\theta }_{{\alpha }^{\ast }}=(1-{\alpha }^{\ast })({\theta }_1-{\theta }_2)$ into $B_{\phi ,F}^{\mathrm{w}}({\theta }_1,{\theta }_{{\alpha }^{\ast }})/E_{\phi }\left({\theta }_{{\alpha }^{\ast }}\right)$ and using (32) gives

$${\displaystyle \frac{1}{E_{\phi }\left({\theta }_{{\alpha }^{\ast }}\right)}}{B}_{\phi ,F}^{\mathrm{w}}({\theta }_1,{\theta }_{{\alpha }^{\ast }})={\alpha }^{\ast }F\left({\theta }_1\right)+(1-{\alpha }^{\ast })F\left({\theta }_2\right)-F\left({\theta }_{{\alpha }^{\ast }}\right).$$

Since $\widehat{F}\left({\theta }_{{\alpha }^{\ast }}\right)=F\left({\theta }_{{\alpha }^{\ast }}\right)+ln{E}_{\phi }\left({\theta }_{{\alpha }^{\ast }}\right)$, this yields (35). □

3.3.3. Derivative and Weighted KL

Recall $F_{pq}\left(\alpha \right)=ln{Z}_{pq}\left(\alpha \right)=ln{\rho }_{\alpha }^{\mathrm{w}}(p,q)$. In view of (14 ), the differentiation under the integral sign is justified, and we have

$$F_{pq}^{\prime }\left(\alpha \right)={\mathbb{E}}_{{\left(pq\right)}_{\alpha }}\left[ln{\displaystyle \frac{p\left(X\right)}{q\left(X\right)}}\right],$$

(37)

where ${\left(pq\right)}_{\alpha }$ is the Chernoff-tilted density from (18). In particular,

$$F_{pq}^{\prime }\left(1\right)={\displaystyle \frac{1}{E_{\phi }\left(p\right)}}{\int }_{\mathcal{X}}\phi \left(x\right)p\left(x\right)ln{\displaystyle \frac{p\left(x\right)}{q\left(x\right)}}\mathrm{d}\mu \left(x\right)={\displaystyle \frac{1}{E_{\phi }\left(p\right)}}{D}_{\mathrm{KL}}^{\mathrm{w}}(p\parallel q).$$

(38)

Analogously, $F_{pq}^{\prime }\left(0\right)=-{\displaystyle \frac{1}{E_{\phi }\left(q\right)}}{D}_{\mathrm{KL}}^{\mathrm{w}}(q\parallel p)$.

3.3.4. Chernoff–KL

Lemma 1. 

Let ${\alpha }^{\ast }$ be a maximiser in Definition 2 and assume that ${\alpha }^{\ast }\in (0,1)$ (so that $F_{pq}^{\prime }\left({\alpha }^{\ast }\right)=0$ below). Set $F_{pq}\left(\alpha \right):=ln{\rho }_{\alpha }^{\mathrm{w}}(p,q),r_{\alpha }:={\left(pq\right)}_{\alpha }.$ Then

$$D_C^{\mathrm{w}}(p,q)=D_{\mathrm{KL}}\left(r_{{\alpha }^{\ast }}\parallel r_1\right)-ln{E}_{\phi }\left(p\right)=D_{\mathrm{KL}}\left(r_{{\alpha }^{\ast }}\parallel r_0\right)-ln{E}_{\phi }\left(q\right),$$

(39)

where $r_1\left(x\right)=\phi \left(x\right)p\left(x\right)/E_{\phi }\left(p\right)$ and $r_0\left(x\right)=\phi \left(x\right)q\left(x\right)/E_{\phi }\left(q\right)$.

Here, $D_{\mathrm{KL}}$ denotes the standard (unweighted) Kullback–Leibler divergence.

Proof. 

A direct computation yields, for $\alpha \in [0, 1]$,

$$\begin{array}{cc}\hfill D_{\mathrm{KL}}(r_{\alpha }\parallel r_1)& =-(1-\alpha )F_{pq}^{\prime }\left(\alpha \right)-F_{pq}\left(\alpha \right)+F_{pq}\left(1\right),\hfill \\ \hfill D_{\mathrm{KL}}(r_{\alpha }\parallel r_0)& =\alpha F_{pq}^{\prime }\left(\alpha \right)-F_{pq}\left(\alpha \right)+F_{pq}\left(0\right).\hfill \end{array}$$

At $\alpha ={\alpha }^{\ast }$, we have $F_{pq}^{\prime }\left({\alpha }^{\ast }\right)=0$. Since $F_{pq}\left(1\right)=ln{E}_{\phi }\left(p\right)$ and $F_{pq}\left(0\right)=ln{E}_{\phi }\left(q\right)$, the claim follows from $D_C^{\mathrm{w}}(p,q)=-F_{pq}\left({\alpha }^{\ast }\right)$. □

Corollary 2 

(Chernoff information as a Bregman divergence on the Chernoff arc). Let $F_{pq}\left(\alpha \right):=ln{\rho }_{\alpha }^{\mathrm{w}}(p,q)$ and define the one-dimensional Bregman divergence

$$B_{F_{pq}}(a,b):=F_{pq}\left(a\right)-F_{pq}\left(b\right)-(a-b)F_{pq}^{\prime }\left(b\right).$$

Assume that the maximiser ${\alpha }^{\ast }\in (0,1)$ in Definition 2 is interior, so that $F_{pq}^{\prime }\left({\alpha }^{\ast }\right)=0$. Then

$$D_C^{\mathrm{w}}(p,q)=B_{F_{pq}}(1,{\alpha }^{\ast })-ln{E}_{\phi }\left(p\right)=B_{F_{pq}}(0,{\alpha }^{\ast })-ln{E}_{\phi }\left(q\right).$$

(40)

Equivalently,

$$\begin{array}{cc}\hfill B_{F_{pq}}(1,{\alpha }^{\ast })& =D_{\mathrm{KL}}(r_{{\alpha }^{\ast }}\parallel r_1)=D_C^{\mathrm{w}}(p,q)+ln{E}_{\phi }\left(p\right),\hfill \\ \hfill B_{F_{pq}}(0,{\alpha }^{\ast })& =D_{\mathrm{KL}}(r_{{\alpha }^{\ast }}\parallel r_0)=D_C^{\mathrm{w}}(p,q)+ln{E}_{\phi }\left(q\right),\hfill \end{array}$$

with $r_{\alpha }={\left(pq\right)}_{\alpha }$, $r_1=\phi p/E_{\phi }\left(p\right)$ and $r_0=\phi q/E_{\phi }\left(q\right)$.

Proof. 

In the Chernoff exponential family $\left\{r_{\alpha }\right\}$ with log-normalizer $F_{pq}$, the KL–Bregman identity gives $D_{\mathrm{KL}}(r_{{\alpha }^{\ast }}\parallel r_1)=B_{F_{pq}}(1,{\alpha }^{\ast })$ and $D_{\mathrm{KL}}(r_{{\alpha }^{\ast }}\parallel r_0)=B_{F_{pq}}(0,{\alpha }^{\ast })$. Since $F_{pq}^{\prime }\left({\alpha }^{\ast }\right)=0$ and $F_{pq}\left(1\right)=ln{E}_{\phi }\left(p\right)$, $F_{pq}\left(0\right)=ln{E}_{\phi }\left(q\right)$, we obtain

$$B_{F_{pq}}(1,{\alpha }^{\ast })=-F_{pq}\left({\alpha }^{\ast }\right)+F_{pq}\left(1\right)=D_C^{\mathrm{w}}(p,q)+ln{E}_{\phi }\left(p\right),$$

and similarly for 0. Substituting this into Lemma 1 yields (40). □

Remark 6 

(One-parameter case). Assume $d=1$, ${\theta }_1\ne {\theta }_2$, and that the maximiser ${\alpha }^{\ast }\in (0,1)$ in Definition 2 is interior. Assume moreover that ${\widehat{F}}^{\prime }$ is strictly increasing on Θ and set $\widehat{G}={\left({\widehat{F}}^{\prime }\right)}^{-1}$. Then (32) yields

$${\alpha }^{\ast }={\displaystyle \frac{1}{{\theta }_1-{\theta }_2}}\left(\widehat{G}\left({\displaystyle \frac{F\left({\theta }_1\right)-F\left({\theta }_2\right)}{{\theta }_1-{\theta }_2}}\right)-{\theta }_2\right).$$

(41)

When $\phi \equiv 1$, we have $\widehat{F}\equiv F$ and (41) reduces to the classical formula.

To illustrate the general identities above, we provide explicit expressions for $D_{B,\alpha }^{\mathrm{w}}$ and $D_C^{\mathrm{w}}$ in several parametric settings (Gaussian, Poisson, and exponential); see Section 4.

Remark 7. 

The Bregman representation (40) identifies $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$ with a Bregman divergence on the tilted log-normaliser $\widehat{F}$; geometrically, the optimiser ${\alpha }^{\ast }$ marks the intersection of the exponential geodesic of the tilted family ${\left\{\phi p^{\alpha }{q}^{1-\alpha }\right\}}_{\alpha \in [0,1]}$ with the weighted Kullback–Leibler bisector, generalising the unweighted characterisation of [11,12]. In the exponential-family setting, the computation of $D_C^{\mathrm{w}}$ therefore reduces to $\widehat{F}$ and its gradient: once $\widehat{F}$ is available in closed form, ${\alpha }^{\ast }$ is determined by (41) in the one-parameter case and by a monotone equation in the natural-parameter space in general, without evaluating ${\rho }_{\alpha }^{\mathrm{w}}$ for each α. The examples of Section 4 exemplify this reduction.

3.4. Tilted Weighted Likelihood and Concentration Bounds

Although the optimal rule in the context-sensitive problem is still the usual likelihood-ratio test $q/p$ (cf. Section 3), it is convenient to work with the tilted ratio $q^{\ast }/p^{\ast }$. The factor $\phi$ cancels pointwise in $q^{\ast }/p^{\ast }$ and enters only through the normalisation constants $E_{\phi }\left(p\right)$ and $E_{\phi }\left(q\right)$. We record two consequences: a large-deviation representation for $L^{\ast }/n$ via the cumulant generating functions ${\psi }_{\mathbb{P}},{\psi }_{\mathbb{Q}}$ and their Legendre transforms and a finite-n concentration bound based on a martingale argument.

For the tilted distributions, the log-likelihood takes the form

$$\begin{array}{c}{L}^{\ast }\left(X_1^n\right)=L^{\ast }(X_1,\dots ,X_n)={\displaystyle \sum _{i=1}^n}ln{\displaystyle \frac{q^{\ast }\left(X_i\right)}{p^{\ast }\left(X_i\right)}}\hfill \\ ={\displaystyle \sum _{i=1}^n}ln{\displaystyle \frac{q\left(X_i\right)}{p\left(X_i\right)}}-nln{E}_{\phi }\left(q\right)+nln{E}_{\phi }\left(p\right).\hfill \end{array}$$

(42)

Here $E_{\phi }\left(p\right)=\int \phi \left(x\right)p\left(x\right)\mathrm{d}\mu \left(x\right)$.

In particular, since

$$ln{\displaystyle \frac{q^{\ast }\left(x\right)}{p^{\ast }\left(x\right)}}=ln{\displaystyle \frac{q\left(x\right)}{p\left(x\right)}}+ln{E}_{\phi }\left(p\right)-ln{E}_{\phi }\left(q\right),$$

we may equivalently rewrite likelihood-ratio threshold rules in terms of $L^{\ast }$. For example,

$${\displaystyle \sum _{i=1}^n}ln{\displaystyle \frac{q\left(X_i\right)}{p\left(X_i\right)}}\ge 0⟺L^{\ast }\left(X_1^n\right)\ge n\left(ln{E}_{\phi }\left(p\right)-ln{E}_{\phi }\left(q\right)\right).$$

Thus, $L^{\ast }$ is the usual log-likelihood ratio, shifted by a constant determined by the context weight $\phi$.

The log of the moment generating function and its Legendre transform take the form

$$\begin{array}{cc}\hfill {\psi }_{\mathbb{P}}\left(\alpha \right)& =ln{\mathbb{E}}_{\mathbb{P}}\left[e^{\alpha ln{\displaystyle \frac{q^{\ast }\left(X\right)}{p^{\ast }\left(X\right)}}}\right]\hfill \\ \hfill & =ln{\int }_{\mathcal{X}}q{\left(x\right)}^{\alpha }p{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right)-\alpha ln{E}_{\phi }\left(q\right)+\alpha ln{E}_{\phi }\left(p\right),\hfill \\ \hfill I_{\mathbb{P}}\left(r\right)& =\underset{\alpha }{sup}\left[\alpha r-{\psi }_{\mathbb{P}}\left(\alpha \right)\right],\hfill \end{array}$$

(43)

where $\alpha$ ranges over the set $\{\alpha \in \mathbb{R}:{\psi }_{\mathbb{P}}\left(\alpha \right)<\infty \}$ (and similarly for ${\psi }_{\mathbb{Q}}$).

Similarly,

$$\begin{array}{cc}\hfill {\psi }_{\mathbb{Q}}\left(\alpha \right):& =ln{\mathbb{E}}_{\mathbb{Q}}\left[e^{\alpha ln{\displaystyle \frac{q^{\ast }\left(X\right)}{p^{\ast }\left(X\right)}}}\right]=ln{\mathbb{E}}_{\mathbb{P}}\left[e^{ln{\displaystyle \frac{q\left(X\right)}{p\left(X\right)}}+\alpha ln{\displaystyle \frac{q^{\ast }\left(X\right)}{p^{\ast }\left(X\right)}}}\right]\hfill \\ \hfill & =ln{\int }_{\mathcal{X}}q{\left(x\right)}^{\alpha +1}p{\left(x\right)}^{-\alpha }\mathrm{d}\mu \left(x\right)-\alpha ln{E}_{\phi }\left(q\right)+\alpha ln{E}_{\phi }\left(p\right).\hfill \end{array}$$

(44)

This implies the relation of Legendre transforms

$$I_{\mathbb{Q}}\left(r\right)=I_{\mathbb{P}}\left(r\right)-r+ln{E}_{\phi }\left(p\right)-ln{E}_{\phi }\left(q\right).$$

(45)

In particular, $I_{\mathbb{P}}\left(0\right)$ may be treated as a natural weighted version of the Chernoff divergence between q and p:

$$\begin{array}{cc}\hfill I_{\mathbb{P}}\left(0\right)& =\underset{\alpha }{sup}\left[-ln{\int }_{\mathcal{X}}q{\left(x\right)}^{\alpha }p{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right)+\alpha ln{E}_{\phi }\left(q\right)-\alpha ln{E}_{\phi }\left(p\right)\right]\hfill \\ \hfill & =:{\widehat{D}}_C^{\mathrm{w}}(q,p).\hfill \end{array}$$

(46)

Interpretation. The value $I_{\mathbb{P}}\left(0\right)$ is the Chernoff–Cramér exponent controlling the tail event $\{L^{\ast }/n\ge 0\}$ under $\mathbb{P}$, i.e., (under standard regularity assumptions), $\mathbb{P}(L^{\ast }\left(X_1^n\right)\ge 0)\asymp e^{-n{I}_{\mathbb{P}}\left(0\right)}$. In the unweighted case $\phi \equiv 1$, we have $E_{\phi }\left(p\right)=E_{\phi }\left(q\right)=1$ and $I_{\mathbb{P}}\left(0\right)$ reduces to the classical Chernoff information between p and q. We also stress that ${\widehat{D}}_C^{\mathrm{w}}(q,p)=I_{\mathbb{P}}\left(0\right)$ is a tilted-likelihood exponent and is distinct from the weighted Chernoff information $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$ from Definition 2, which governs the optimal sum of context-sensitive losses.

Non-Asymptotic Concentration via a Doob Martingale

The rate functions $I_{\mathbb{P}}$ and $I_{\mathbb{Q}}$ capture the exponential scale of deviations of $L^{\ast }/n$ as $n\to \infty$. To obtain explicit finite-n bounds, we now apply a standard martingale method to $L^{\ast }$ under $\mathbb{Q}$.

Consider the filtration ${\mathcal{F}}_k=\sigma (X_1,\dots ,X_k)$ and define the random variables $\{U_k,k=0,\dots ,n\}$ by

$$\begin{array}{cc}\hfill U_k& ={\mathbb{E}}_{\mathbb{Q}}\left[L^{\ast }(X_1,\dots ,X_n)|{\mathcal{F}}_k\right]\hfill \\ \hfill & ={\displaystyle \sum _{j=1}^k}ln{\displaystyle \frac{q\left(X_j\right)}{p\left(X_j\right)}}+(n-k)D_{\mathrm{KL}}(\mathbb{Q}\parallel \mathbb{P})-nln{E}_{\phi }\left(q\right)+nln{E}_{\phi }\left(p\right),\hfill \end{array}$$

(47)

where $D_{\mathrm{KL}}(\mathbb{Q}\parallel \mathbb{P})=\int q\left(x\right)ln{\displaystyle \frac{q\left(x\right)}{p\left(x\right)}}\mathrm{d}\mu \left(x\right)$ stands for the (unweighted) Kullback–Leibler divergence of $\mathbb{Q}$ and $\mathbb{P}$. Then

$$U_k-U_{k-1}=ln{\displaystyle \frac{q\left(X_k\right)}{p\left(X_k\right)}}-D_{\mathrm{KL}}(\mathbb{Q}\parallel \mathbb{P}),k=1,\dots ,n.$$

(48)

Observe that $\{U_k,{\mathcal{F}}_k\}$ is a martingale w.r.t. $\mathbb{Q}$.

Assume now that $d<\infty$ and ${\sigma }^2<\infty$, where $d=\underset{x\in \mathcal{X}}{sup}\left|ln{\displaystyle \frac{q\left(x\right)}{p\left(x\right)}}-D_{\mathrm{KL}}(\mathbb{Q}\parallel \mathbb{P})\right|$ and

$$\begin{array}{cc}\hfill {\sigma }^2& ={\mathbb{E}}_{\mathbb{Q}}\left[{(U_k-U_{k-1})}^2|{\mathcal{F}}_{k-1}\right]\hfill \\ \hfill & ={\int }_{\mathcal{X}}q\left(x\right){\left(ln{\displaystyle \frac{q\left(x\right)}{p\left(x\right)}}-D_{\mathrm{KL}}(\mathbb{Q}\parallel \mathbb{P})\right)}^2\mathrm{d}\mu \left(x\right).\hfill \end{array}$$

(49)

In view of a refined Azuma–Hoeffding/McDiarmid inequality [13,14],

$$\begin{array}{c}\hfill {\mathbb{P}}_{\mathbb{Q}}\left(L^{\ast }(X_1,\dots ,X_n)>(D_{\mathrm{KL}}(\mathbb{Q}\parallel \mathbb{P})+\beta )n-(ln{E}_{\phi }\left(q\right)-ln{E}_{\phi }\left(p\right))n\right)\\ \hfill \le exp\left\{-nD\left({\displaystyle \frac{\delta +{\gamma }^{\ast }}{1+{\gamma }^{\ast }}}\parallel {\displaystyle \frac{{\gamma }^{\ast }}{1+{\gamma }^{\ast }}}\right)\right\},\end{array}$$

(50)

where $\delta ={\displaystyle \frac{{\sigma }^2}{d^2}}$, ${\gamma }^{\ast }={\displaystyle \frac{{\beta }^{\ast }}{d}}$, ${\beta }^{\ast }=\beta -ln{E}_{\phi }\left(q\right)+ln{E}_{\phi }\left(p\right)$ and $D(p\parallel q)=pln{\displaystyle \frac{p}{q}}+(1-p)ln{\displaystyle \frac{1-p}{1-q}}$ stands for the Kullback–Leibler divergence between the two Bernoulli distributions $(p,1-p)$ and $(q,1-q)$.

We use the following modified version of Azuma–Hoeffding inequality; see [14].

Lemma 2. 

Let $\{U_k,{\mathcal{F}}_k\}$ be a discrete-time real-valued martingale. Assume that, for some constants $d,\sigma >0$, the following two requirements are satisfied a.s. for every $k\in \{1,\dots ,n\}:$

$$\begin{array}{c}|U_k-U_{k-1}|\le d,\hfill \\ \mathrm{Var}[U_k-U_{k-1}|{\mathcal{F}}_{k-1}]\le {\sigma }^2.\hfill \end{array}$$

(51)

Then, for every $\beta \ge 0$,

$$\mathbb{P}\left(|U_n-U_0|\ge \beta n\right)\le 2exp\left\{-nD\left({\displaystyle \frac{\delta +\gamma }{1+\gamma }}\parallel {\displaystyle \frac{\gamma }{1+\gamma }}\right)\right\},$$

(52)

where $\delta ={\displaystyle \frac{{\sigma }^2}{d^2}}$ and $\gamma ={\displaystyle \frac{\beta }{d}}$. (Note that $\delta \in [0,1]$ automatically whenever $|U_k-U_{k-1}|\le d$ a.s.)

In particular, the one-sided bound $\mathbb{P}(U_n-U_0\ge \beta n)\le exp\{-nD(·)\}$ holds.

Theorem 2. 

Set

$${\beta }^{\ast }=\beta -ln{E}_{\phi }\left(p\right)+ln{E}_{\phi }\left(q\right),{\gamma }^{\ast }={\displaystyle \frac{{\beta }^{\ast }}{d}}.$$

Under conditions $d<\infty$, ${\sigma }^2<\infty$, and ${\beta }^{\ast }\ge 0$,

$${\mathbb{P}}_{\mathbb{Q}}\left(L^{\ast }(X_1,\dots ,X_n)\ge \beta n\right)\le exp\left\{-nD\left({\displaystyle \frac{\delta +{\gamma }^{\ast }}{1+{\gamma }^{\ast }}}\parallel {\displaystyle \frac{{\gamma }^{\ast }}{1+{\gamma }^{\ast }}}\right)\right\}.$$

(53)

Lemma 2 is quoted from [14]. Theorem 2 is its direct application to the tilted log-likelihood ratio (42); the only dependence on the context weight $\phi$ is through $E_{\phi }\left(p\right)$ and $E_{\phi }\left(q\right)$.

4. Examples and Applications

The identities of Section 3 reduce the computation of $D_{B,\alpha }^{\mathrm{w}}$ and $D_C^{\mathrm{w}}$ to the single-letter weighted affinity

$${\rho }_{\alpha }^{\mathrm{w}}(p,q)=\int \phi \left(x\right)p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right),$$

followed by optimisation over $\alpha \in [0,1]$. We work this out for Gaussian, Poisson, and exponential families, highlighting how the context weight $\phi$ modifies the classical formulas. When $\phi \equiv 1$, the expressions reduce to the standard unweighted Bhattacharyya and Chernoff quantities; more involved non-exponential-family computations (such as the Cauchy location–scale family) are deferred to the Appendix A.

4.1. A Numerical Illustration

This subsection illustrates the behaviour of ${\alpha }^{\ast }$ and $D_C^{\mathrm{w}}$ under a non-trivial factorised weight and serves as a direct numerical verification of the Bregman identities of Section 3.3: for the model below the affinity ${\rho }_{\alpha }^{\mathrm{w}}$ is available in closed form, and we check that closed-form evaluation and direct numerical integration agree to machine precision.

Consider the asymmetric Gaussian hypotheses

$$H_0:\mathbb{P}=\mathcal{N}({\mu }_0,{\sigma }_0^2),H_1:\mathbb{Q}=\mathcal{N}({\mu }_1,{\sigma }_1^2),({\mu }_0,{\sigma }_0^2)=(0,1),({\mu }_1,{\sigma }_1^2)=(3,2),$$

with a non-indicator factorised weight

$$\phi \left(x\right)=exp\left(-\beta {(x-x_0)}^2\right),x_0=0,\beta \ge 0.$$

(54)

At $\beta =0$, one has $\phi \equiv 1$ and the unweighted Chernoff information is recovered. For $\beta >0$, the weight concentrates near $x_0={\mu }_0$; in particular, (54) is not an indicator-type weight, so the weighted problem does not reduce to the unweighted Chernoff information on a restricted domain.

The asymmetry ${\sigma }_0^2\ne {\sigma }_1^2$ and the centring of $\phi$ at the $H_0$ mean are essential for the illustration. In a fully symmetric configuration (${\sigma }_0={\sigma }_1$, ${\mu }_1=-{\mu }_0$, $x_0=0$), the problem is invariant under $\alpha \leftrightarrow 1-\alpha$, so the optimum is pinned to ${\alpha }^{\ast }=1/2$ for every $\beta$ and the effect of the weight on the Chernoff compromise is invisible. Asymmetric hypotheses are precisely where the weighted formalism is operationally distinct from the classical one, and it is this distinction that the numerics below is designed to expose.

Writing $A\left(\alpha \right)=\alpha /{\sigma }_0^2+(1-\alpha )/{\sigma }_1^2+2\beta$, $B\left(\alpha \right)=\alpha {\mu }_0/{\sigma }_0^2+(1-\alpha ){\mu }_1/{\sigma }_1^2+2\beta x_0$, and $C\left(\alpha \right)=\alpha {\mu }_0^2/{\sigma }_0^2+(1-\alpha ){\mu }_1^2/{\sigma }_1^2+2\beta x_0^2$, a direct Gaussian integration yields

$$ln{\rho }_{\alpha }^{\mathrm{w}}(p,q)=\frac{1}{2}ln\frac{2\pi }{A\left(\alpha \right)}-\frac{\alpha }{2}ln\left(2\pi {\sigma }_0^2\right)-\frac{1-\alpha }{2}ln\left(2\pi {\sigma }_1^2\right)-\frac{1}{2}\left(C\left(\alpha \right)-\frac{B{\left(\alpha \right)}^2}{A\left(\alpha \right)}\right),$$

(55)

and maximising (55) over $\alpha \in [0, 1]$ gives ${\alpha }^{\ast }\left(\beta \right)$ and $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})\left(\beta \right)$. Table 1 reports their values at three selected $\beta$. Direct numerical integration of ${\rho }_{\alpha }^{\mathrm{w}}$ from its definition agrees with (55) to machine precision on all tabulated entries, which confirms the Bregman identities of Section 3.3 numerically for this example.

Table 1. Optimal skewing parameter and weighted Chernoff information for the Gaussian example (54), obtained by maximising (55).

$\mathit{\beta }$	${\mathit{\alpha }}^{\ast }\left(\mathit{\beta }\right)$	${\mathit{D}}_{\mathit{C}}^{\mathbf{w}}(\mathbb{P},\mathbb{Q})$
0 (unweighted)	$0.4153$	$0.8018$
$1/16$	$0.3355$	$0.9827$
$1/4$	$0.0963$	$1.4935$

The monotone growth of $\beta \mapsto D_C^{\mathrm{w}}$ and the leftward shift of ${\alpha }^{\ast }\left(\beta \right)$ illustrate a qualitative conclusion of Section 3: localising $\phi$ near ${\mu }_0$ preferentially retains observations that are more probable under $H_0$ and thereby increases the effective discrimination rate, while simultaneously moving the optimal tilting towards the $H_0$ side. The classical unweighted limit is recovered at $\beta =0$.

In the language of hypothesis testing, ${\alpha }^{\ast }$ is the parameter that balances the exponential rates of the type-I and type-II losses at the Bayes optimum: the type-I exponent equals ${\alpha }^{\ast }{D}_C^{\mathrm{w}}$ and the type-II exponent equals $(1-{\alpha }^{\ast })D_C^{\mathrm{w}}$ (cf. Section 3). A leftward shift of ${\alpha }^{\ast }$ therefore corresponds to reallocating the available exponential budget towards faster decay of the type-II loss at the expense of the type-I loss, which is the optimal response to a weight that concentrates mass in the region where $H_0$ is more plausible.

Data and Code Availability

A Jupyter/Colab notebook reproducing Table 1 and Figure 1, Figure 2 and Figure 3, together with the direct-integration verification of (55), is archived on Zenodo [16] and mirrored on GitHub.

Figure 1. The map $\alpha \mapsto -ln{\rho }_{\alpha }^{\mathrm{w}}(p,q)$ for the Gaussian hypotheses $\mathcal{N}(0,1)$, $\mathcal{N}(3,2)$ with weight (54), for $\beta \in \{0,1/16,1/4\}$. The optimum ${\alpha }^{\ast }$ is marked by a bullet on each curve and shifts to the left as $\beta$ increases.

Figure 2. Optimal skewing parameter ${\alpha }^{\ast }\left(\beta \right)$ for the Gaussian example. The dashed line marks the unweighted value ${\alpha }^{\ast }\left(0\right)$.

Figure 3. Weighted Chernoff information $\beta \mapsto D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$. The dashed line marks the classical value $D_C$ recovered at $\beta =0$.

4.2. Gaussian Models

Throughout this subsection, the reference measure is the Lebesgue measure on ${\mathbb{R}}^d$. We compute the weighted Bhattacharyya coefficient

$${\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})={\int }_{{\mathbb{R}}^d}\phi \left(x\right)p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}x,\alpha \in [0,1],$$

together with the weighted Bhattacharyya distance $D_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q}):=-ln{\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$ and the weighted Chernoff information $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})={max}_{\alpha \in [0, 1]}{D}_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$ (Definition 2). Note that, unlike the unweighted case, ${\rho }_{\alpha }^{\mathrm{w}}$ is not restricted to $[0, 1]$ and $D_{B,\alpha }^{\mathrm{w}}$ (hence also $D_C^{\mathrm{w}}$) may take negative values.

Example 1 

(Gaussian weighted Bhattacharyya coefficient with exponential weight). Let $\mathbb{P}=\mathcal{N}({\mu }_1,{\mathrm{\Sigma }}_1)$ and $\mathbb{Q}=\mathcal{N}({\mu }_2,{\mathrm{\Sigma }}_2)$ on ${\mathbb{R}}^d$, where ${\mathrm{\Sigma }}_1\succ 0$ and ${\mathrm{\Sigma }}_2\succ 0$, and let $\phi \left(x\right)=e^{{\gamma }^{\mathrm{T}}x}$ for some $\gamma \in {\mathbb{R}}^d$. Denote by $p,q$ the corresponding densities. For $\alpha \in [0, 1]$ define

$$\begin{array}{c}{\mathrm{\Sigma }}_{\alpha }:={\left(\alpha {\mathrm{\Sigma }}_1^{-1}+(1-\alpha ){\mathrm{\Sigma }}_2^{-1}\right)}^{-1},\hfill \\ {\tilde{\mu }}_{\alpha }:={\mathrm{\Sigma }}_{\alpha }\left(\alpha {\mathrm{\Sigma }}_1^{-1}{\mu }_1+(1-\alpha ){\mathrm{\Sigma }}_2^{-1}{\mu }_2+\gamma \right).\hfill \end{array}$$

Then

$$\begin{array}{cc}\hfill & {\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})={\int }_{{\mathbb{R}}^d}{e}^{{\gamma }^{\mathrm{T}}x}p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}x\hfill \\ \hfill & ={\displaystyle \frac{|{\mathrm{\Sigma }}_{\alpha }{|}^{1/2}}{|{\mathrm{\Sigma }}_1{|}^{\alpha /2}{|{\mathrm{\Sigma }}_2|}^{(1-\alpha )/2}}}exp\left\{-{\displaystyle \frac{1}{2}}\left(\alpha {\mu }_1^{\mathrm{T}}{\mathrm{\Sigma }}_1^{-1}{\mu }_1+(1-\alpha ){\mu }_2^{\mathrm{T}}{\mathrm{\Sigma }}_2^{-1}{\mu }_2-{\tilde{\mu }}_{\alpha }^{\mathrm{T}}{\mathrm{\Sigma }}_{\alpha }^{-1}{\tilde{\mu }}_{\alpha }\right)\right\}.\hfill \end{array}$$

(56)

Consequently,

$$\begin{array}{cc}\hfill & D_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}\left(\alpha {\mu }_1^{\mathrm{T}}{\mathrm{\Sigma }}_1^{-1}{\mu }_1+(1-\alpha ){\mu }_2^{\mathrm{T}}{\mathrm{\Sigma }}_2^{-1}{\mu }_2-{\tilde{\mu }}_{\alpha }^{\mathrm{T}}{\mathrm{\Sigma }}_{\alpha }^{-1}{\tilde{\mu }}_{\alpha }+ln{\displaystyle \frac{|{\mathrm{\Sigma }}_1{|}^{\alpha }{|{\mathrm{\Sigma }}_2|}^{1-\alpha }}{|{\mathrm{\Sigma }}_{\alpha }|}}\right).\hfill \end{array}$$

(57)

In particular, setting $\gamma =0$ (i.e., $\phi \equiv 1$) reduces (57) to the classical (unweighted) Gaussian Bhattacharyya distance; see, e.g., [12].

Corollary 3 

(Common covariance). In Example 1, assume ${\mathrm{\Sigma }}_1={\mathrm{\Sigma }}_2=\mathrm{\Sigma }\succ 0$ and keep the exponential weight $\phi \left(x\right)=e^{{\gamma }^{\mathrm{T}}x}$. Set $\delta :={\mu }_1-{\mu }_2$, ${\parallel v\parallel }_{{\mathrm{\Sigma }}^{-1}}^2:=v^{\mathrm{T}}{\mathrm{\Sigma }}^{-1}v$, and ${\mu }_{\alpha }:=\alpha {\mu }_1+(1-\alpha ){\mu }_2$. Then, for any $\alpha \in [0, 1]$,

$$\begin{array}{cc}\hfill {\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})& ={\int }_{{\mathbb{R}}^d}{e}^{{\gamma }^{\mathrm{T}}x}p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}x\hfill \\ \hfill & =exp\left\{-{\displaystyle \frac{\alpha (1-\alpha )}{2}}{\parallel \delta \parallel }_{{\mathrm{\Sigma }}^{-1}}^2+{\gamma }^{\mathrm{T}}{\mu }_{\alpha }+{\displaystyle \frac{1}{2}}{\gamma }^{\mathrm{T}}\mathrm{\Sigma }\gamma \right\},\hfill \end{array}$$

(58)

and therefore

$$D_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=-ln{\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})={\displaystyle \frac{\alpha (1-\alpha )}{2}}{\parallel \delta \parallel }_{{\mathrm{\Sigma }}^{-1}}^2-{\gamma }^{\mathrm{T}}{\mu }_{\alpha }-{\displaystyle \frac{1}{2}}{\gamma }^{\mathrm{T}}\mathrm{\Sigma }\gamma .$$

(59)

If ${\mu }_1\ne {\mu }_2$ and the unconstrained maximiser

$$\tilde{\alpha }={\displaystyle \frac{1}{2}}-{\displaystyle \frac{{\gamma }^{\mathrm{T}}\delta }{{\parallel \delta \parallel }_{{\mathrm{\Sigma }}^{-1}}^2}}$$

belongs to $(0, 1)$, then ${\alpha }^{\ast }=\tilde{\alpha }$; otherwise the maximum over $\alpha \in [0, 1]$ is attained at the nearest boundary point ${\alpha }^{\ast }\in \{0,1\}$. In all cases,

$$D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=\underset{\alpha \in [0,1]}{max}{D}_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=D_{B,{\alpha }^{\ast }}^{\mathrm{w}}(\mathbb{P},\mathbb{Q}).$$

In particular, for $\gamma =0$ (i.e., $\phi \equiv 1$) we recover ${\alpha }^{\ast }=1/2$ and $D_C(\mathbb{P},\mathbb{Q})={\parallel \delta \parallel }_{{\mathrm{\Sigma }}^{-1}}^2/8$.

Proof. 

The expression for ${\rho }_{\alpha }^{\mathrm{w}}$ follows by simplifying Example 1 under ${\mathrm{\Sigma }}_1={\mathrm{\Sigma }}_2=\mathrm{\Sigma }$, which makes the determinant prefactor equal to 1 and yields a Gaussian MGF term $exp({\gamma }^{\mathrm{T}}{\mu }_{\alpha }+\frac{1}{2}{\gamma }^{\mathrm{T}}\mathrm{\Sigma }\gamma )$. The maximiser follows by differentiating (59) in $\alpha$. □

Choosing an exponential weight $\phi \left(x\right)=e^{{\gamma }^{\mathrm{T}}x}$ corresponds to exponential tilting: for a Gaussian $X\sim \mathcal{N}(\mu ,\mathrm{\Sigma })$, this tilting keeps the covariance and shifts the mean to $\mu +\mathrm{\Sigma }\gamma$ (with normalisation factor $exp({\gamma }^{\mathrm{T}}\mu +\frac{1}{2}{\gamma }^{\mathrm{T}}\mathrm{\Sigma }\gamma )$), which is why the weighted affinities remain available in closed form. In particular, the optimal Chernoff parameter is no longer forced to be ${\alpha }^{\ast }=1/2$ and, as (59) shows, sufficiently strong tilting can push the maximiser to the boundary ${\alpha }^{\ast }\in \{0,1\}$.

4.3. Poisson Models

Example 2 

(Poisson model with exponential weight). Let $\mathcal{X}={\mathbb{N}}_0=\{0,1,2,\dots \}$ and let μ be the counting measure on $\mathcal{X}$. Fix two hypotheses $\mathbb{P}=\mathrm{Poi}\left({\lambda }_1\right)$ and $\mathbb{Q}=\mathrm{Poi}\left({\lambda }_2\right)$ with ${\lambda }_1,{\lambda }_2>0$, and write $p=p_{{\lambda }_1}$ and $q=q_{{\lambda }_2}$. Throughout this subsection we work under the standing assumption of Section 2, namely, that the observations $X_1,\dots ,X_n$ are i.i.d. (distributed as $\mathbb{P}$ under $H_0$ and as $\mathbb{Q}$ under $H_1$), and that the weight φ factorises across observations (Assumption 1). We still consider the exponential weight ${\phi }_{\gamma }\left(k\right)=e^{\gamma k},\gamma \in \mathbb{R}.$ (For $\gamma =0$ we recover the unweighted case $\phi \equiv 1$.) Equivalently, setting $\epsilon :=e^{\gamma }>0$, the weight takes the form $\phi \left(k\right)={\epsilon }^k$; this reparameterisation is convenient in applications where $\epsilon \in (0, 1)$ models a per-event discount factor, while $\gamma \in \mathbb{R}$ is the natural parameter for the exponential-family calculations below. The two parameterisations are equivalent.

For $\alpha \in [0, 1]$, set

$${\lambda }_{\alpha }:={\lambda }_1^{\alpha }{\lambda }_2^{1-\alpha }.$$

(a) Weighted Bhattacharyya coefficient and Chernoff arc.   A direct summation gives

$$\begin{array}{cc}\hfill {\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})& ={\displaystyle \sum _{k=0}^{\infty }}{\phi }_{\gamma }\left(k\right)p{\left(k\right)}^{\alpha }q{\left(k\right)}^{1-\alpha }=exp\left\{-\alpha {\lambda }_1-(1-\alpha ){\lambda }_2+e^{\gamma }{\lambda }_{\alpha }\right\}.\hfill \end{array}$$

(60)

Hence, by Definition 1,

$$D_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=-ln{\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=\alpha {\lambda }_1+(1-\alpha ){\lambda }_2-e^{\gamma }{\lambda }_{\alpha }.$$

(61)

Moreover, the normalised weighted geometric mixture (Chernoff-tilted density) from (18) takes the form

$${\left(pq\right)}_{\alpha }\left(k\right)={\displaystyle \frac{{\phi }_{\gamma }\left(k\right)p{\left(k\right)}^{\alpha }q{\left(k\right)}^{1-\alpha }}{{\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})}}=exp\{-e^{\gamma }{\lambda }_{\alpha }\}{\displaystyle \frac{{\left(e^{\gamma }{\lambda }_{\alpha }\right)}^k}{k!}}=\mathrm{Poi}\left(e^{\gamma }{\lambda }_{\alpha }\right).$$

(b) Optimal Chernoff parameter.   If ${\lambda }_1\ne {\lambda }_2$, then $\alpha \mapsto D_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$ is strictly concave on $[0, 1]$ since

$${\displaystyle \frac{{\mathrm{d}}^2}{\mathrm{d}{\alpha }^2}}{D}_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=-e^{\gamma }{\lambda }_{\alpha }{\left(ln{\displaystyle \frac{{\lambda }_1}{{\lambda }_2}}\right)}^2<0;$$

hence, the maximiser ${\alpha }^{\ast }$ in Definition 2 is unique. Differentiating (61) yields the critical point condition

$$0={\displaystyle \frac{\mathrm{d}}{\mathrm{d}\alpha }}{D}_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})={\lambda }_1-{\lambda }_2-e^{\gamma }{\lambda }_{\alpha }ln\left({\displaystyle \frac{{\lambda }_1}{{\lambda }_2}}\right).$$

Equivalently, the (unconstrained) critical point $\alpha =\tilde{\alpha }$ satisfies

$${\lambda }_{\tilde{\alpha }}=e^{-\gamma }L({\lambda }_1,{\lambda }_2),L({\lambda }_1,{\lambda }_2):={\displaystyle \frac{{\lambda }_1-{\lambda }_2}{ln{\lambda }_1-ln{\lambda }_2}}.$$

In contrast to the unweighted case ($\gamma =0$), the context tilt γ may push the optimal Chernoff parameter to the boundary ${\alpha }^{\ast }\in \{0,1\}$.

Thus, the unconstrained maximiser is

$$\tilde{\alpha }={\displaystyle \frac{lnL({\lambda }_1,{\lambda }_2)-\gamma -ln{\lambda }_2}{ln{\lambda }_1-ln{\lambda }_2}},$$

and the maximiser on $[0, 1]$ is ${\alpha }^{\ast }={\mathrm{\Pi }}_{[0,1]}\left(\tilde{\alpha }\right)$, where

$${\mathrm{\Pi }}_{[0,1]}\left(a\right):=min\{1,max\{0,a\}\}.$$

Finally,

$$D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=\underset{\alpha \in [0,1]}{max}{D}_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=D_{B,{\alpha }^{\ast }}^{\mathrm{w}}(\mathbb{P},\mathbb{Q}).$$

If ${\lambda }_1={\lambda }_2$, then ${\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$ does not depend on α and $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=D_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$ for any $\alpha \in [0, 1]$.

Derivation of (60). For $k\in {\mathbb{N}}_0$,

$$p{\left(k\right)}^{\alpha }q{\left(k\right)}^{1-\alpha }=exp\{-\alpha {\lambda }_1-(1-\alpha ){\lambda }_2\}{\displaystyle \frac{{\lambda }_{\alpha }^k}{k!}},$$

so multiplying by $e^{\gamma k}$ and summing over k gives

$$\begin{array}{cc}\hfill {\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})& =exp\{-\alpha {\lambda }_1-(1-\alpha ){\lambda }_2\}\sum _{k\ge 0}{\left(e^{\gamma }{\lambda }_{\alpha }\right)}^k/k!\hfill \\ \hfill & =exp\{-\alpha {\lambda }_1-(1-\alpha ){\lambda }_2+e^{\gamma }{\lambda }_{\alpha }\}.\hfill \end{array}$$

4.4. Exponential Models

Example 3 

(Exponential model with exponential weight). Let $\mathcal{X}={\mathbb{R}}_{+}=[0,\infty )$ with Lebesgue measure. Fix two hypotheses $\mathbb{P}=\mathrm{Exp}\left({\lambda }_1\right)$ and $\mathbb{Q}=\mathrm{Exp}\left({\lambda }_2\right)$ with rates ${\lambda }_1,{\lambda }_2>0$, and write

$$p\left(x\right)={\lambda }_1{e}^{-{\lambda }_{1}x}\mathbf{1}\{x\ge 0\},q\left(x\right)={\lambda }_2{e}^{-{\lambda }_{2}x}\mathbf{1}\{x\ge 0\}.$$

Consider the exponential weight ${\phi }_{\gamma }\left(x\right)=e^{\gamma x}$ with

$$\gamma <min\{{\lambda }_1,{\lambda }_2\},$$

so that ${\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})\in (0,\infty )$ for all $\alpha \in [0, 1]$. For $\alpha \in [0, 1]$ set

$${\lambda }_{\alpha }:=\alpha {\lambda }_1+(1-\alpha ){\lambda }_2.$$

(a) Weighted Bhattacharyya coefficient and Chernoff arc.   A direct computation gives

$${\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})={\int }_0^{\infty }{e}^{\gamma x}p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }\mathrm{d}x={\displaystyle \frac{{\lambda }_1^{\alpha }{\lambda }_2^{1-\alpha }}{{\lambda }_{\alpha }-\gamma }}.$$

Hence

$$D_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=-ln{\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=ln({\lambda }_{\alpha }-\gamma )-\alpha ln{\lambda }_1-(1-\alpha )ln{\lambda }_2.$$

Moreover, the Chernoff-tilted density ${\left(pq\right)}_{\alpha }$ from (18) is again exponential:

$${\left(pq\right)}_{\alpha }\left(x\right)={\displaystyle \frac{e^{\gamma x}p{\left(x\right)}^{\alpha }q{\left(x\right)}^{1-\alpha }}{{\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})}}=({\lambda }_{\alpha }-\gamma )e^{-({\lambda }_{\alpha }-\gamma )x}\mathbf{1}\{x\ge 0\}=\mathrm{Exp}({\lambda }_{\alpha }-\gamma ).$$

(b) Optimal Chernoff parameter.   If ${\lambda }_1\ne {\lambda }_2$, then $\alpha \mapsto D_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})$ is strictly concave on $[0, 1]$; hence, the maximiser ${\alpha }^{\ast }$ in Definition 2 is unique. Differentiating yields the critical point condition

$${\displaystyle \frac{{\lambda }_1-{\lambda }_2}{{\lambda }_{\alpha }-\gamma }}=ln\left({\displaystyle \frac{{\lambda }_1}{{\lambda }_2}}\right).$$

Equivalently, the (unconstrained) critical point $\alpha =\tilde{\alpha }$ satisfies

$${\lambda }_{\tilde{\alpha }}-\gamma =L({\lambda }_1,{\lambda }_2),L({\lambda }_1,{\lambda }_2):={\displaystyle \frac{{\lambda }_1-{\lambda }_2}{ln{\lambda }_1-ln{\lambda }_2}},$$

so that

$$\tilde{\alpha }={\displaystyle \frac{\gamma +L({\lambda }_1,{\lambda }_2)-{\lambda }_2}{{\lambda }_1-{\lambda }_2}}.$$

The maximiser on $[0, 1]$ is ${\alpha }^{\ast }={\mathrm{\Pi }}_{[0,1]}\left(\tilde{\alpha }\right)$ (projection onto $[0, 1]$), and

$$D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=\underset{\alpha \in [0,1]}{max}{D}_{B,\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=D_{B,{\alpha }^{\ast }}^{\mathrm{w}}(\mathbb{P},\mathbb{Q}).$$

If ${\lambda }_1={\lambda }_2=\lambda$, then ${\rho }_{\alpha }^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=\lambda /(\lambda -\gamma )$ does not depend on α, so any $\alpha \in [0, 1]$ is optimal and $D_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})=ln(\lambda -\gamma )-ln\lambda$. Setting $\gamma =0$ (i.e., $\phi \equiv 1$) recovers the classical unweighted expressions.

Additional Example (Baseline, Non-Exponential Family)

Appendix A contains a closed-form illustration for the Cauchy location–scale family. Since the Cauchy family is not an exponential family, this example complements the main text by showing that, even in the unweighted baseline case $\phi \equiv 1$, the Bhattacharyya coefficient (in particular ${\rho }_{1/2}$) and the Chernoff information may involve special functions (complete elliptic integrals). For nontrivial weights $\phi$, the symmetry ${\rho }_{\alpha }={\rho }_{1-\alpha }$ (hence ${\alpha }^{\ast }=1/2$) typically fails and a comparable closed form is not available, so we keep the baseline Cauchy computation in the appendix.

4.5. Extension to M-ary Hypothesis Testing

We now record the finite-M analogue of Theorem 1. The key observation is that the optimal M-ary pointwise loss is squeezed between pairwise minima (Lemma 3), and each pairwise term has logarithmic rate given by the corresponding weighted Chernoff information. Hence the overall M-ary rate is determined by the closest pair in terms of $D_C^{\mathrm{w}}$.

Fix an integer $M\ge 2$ and let ${\mathbb{P}}_1,\dots ,{\mathbb{P}}_M$ be probability measures on $\mathcal{X}$ dominated by $\mu$, with strictly positive densities $p_1,\dots ,p_M$. Let $X_1^n=(X_1,\dots ,X_n)$ be i.i.d. under each hypothesis $H_i:X_1^n\sim {\mathbb{P}}_i^{\otimes n}$. Assume that the weight function factorises as in Assumption 1.

Assume moreover that for every $1\le i<j\le M$ and every $\alpha \in [0, 1]$,

$${\rho }_{\alpha }^{\mathrm{w}}(p_i,p_j)={\int }_{\mathcal{X}}\phi \left(x\right)p_i{\left(x\right)}^{\alpha }{p}_j{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right)\in (0,\infty ),$$

so that all pairwise weighted Chernoff information values are well-defined and inequality

$$\underset{i\ne j}{max}\underset{\alpha \in [0, 1]}{sup}\int \phi \left(x\right)|ln{\displaystyle \frac{p_i\left(x\right)}{p_j\left(x\right)}}|p_i{\left(x\right)}^{\alpha }{p}_j{\left(x\right)}^{1-\alpha }\mathrm{d}\mu \left(x\right)<\infty$$

(62)

holds true.

A (deterministic) M-ary decision rule is a measurable map ${\delta }_n:{\mathcal{X}}^n\to \{1,\dots ,M\}$. Define the context-sensitive loss under $H_i$ by

$$L_{i,n}\left({\delta }_n\right):={\mathbb{E}}_{{\mathbb{P}}_i^{\otimes n}}\left[\phi \left(X_1^n\right)\mathbf{1}\{{\delta }_n\left(X_1^n\right)\ne i\}\right],$$

and the total loss

$$L_{n,M}\left({\delta }_n\right):={\displaystyle \sum _{i=1}^M}{L}_{i,n}\left({\delta }_n\right),L_{n,M}^{\ast }:=\underset{{\delta }_n}{inf}{L}_{n,M}\left({\delta }_n\right).$$

Proposition 6 

(Pointwise form of the optimal M-ary total loss). For each $n\ge 1$,

$$L_{n,M}^{\ast }={\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)\left({\displaystyle \sum _{i=1}^M}{p}_i\left(x_1^n\right)-\underset{1\le j\le M}{max}{p}_j\left(x_1^n\right)\right)\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right),$$

(63)

where $p_i\left(x_1^n\right)={\prod }_{k=1}^n{p}_i\left(x_k\right)$. Moreover, an optimal rule is given by the maximum-likelihood classifier

$${\delta }_n^{\ast }\left(x_1^n\right)\in arg\underset{1\le j\le M}{max}{p}_j\left(x_1^n\right)$$

(with any measurable tie-breaking).

Proof. 

Fix ${\delta }_n$. Using ${\sum }_{i=1}^M\mathbf{1}\{{\delta }_n\ne i\}{p}_i={\sum }_{i=1}^M{p}_i-p_{{\delta }_n}$ pointwise, we obtain

$$L_{n,M}\left({\delta }_n\right)={\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)\left({\displaystyle \sum _{i=1}^M}{p}_i\left(x_1^n\right)-p_{{\delta }_n\left(x_1^n\right)}\left(x_1^n\right)\right)\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right).$$

Minimisation over ${\delta }_n$ is therefore pointwise in $x_1^n$ and is achieved by selecting an index maximising $p_j\left(x_1^n\right)$, yielding (63). □

Lemma 3 

(Pairwise minima). For any non-negative numbers $a_1,\dots ,a_M$,

$$\underset{1\le i<j\le M}{max}min(a_i,a_j)\le {\displaystyle \sum _{k=1}^M}{a}_k-\underset{1\le k\le M}{max}{a}_k\le \sum _{1\le i<j\le M}min(a_i,a_j).$$

(64)

Consequently, defining for $i<j$

$$I_n^{i,j}:={\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)min\{p_i\left(x_1^n\right),p_j\left(x_1^n\right)\}\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right),$$

we have the sandwich inequality

$$\underset{i<j}{max}{I}_n^{i,j}\le L_{n,M}^{\ast }\le \sum _{i<j}{I}_n^{i,j}.$$

(65)

Proof. 

Let $a_{\left(1\right)}\ge a_{\left(2\right)}\ge \dots \ge a_{\left(M\right)}$ be the decreasing rearrangement of $(a_1,\dots ,a_M)$. Then ${\sum }_k{a}_k-{max}_k{a}_k={\sum }_{r=2}^M{a}_{\left(r\right)}\ge a_{\left(2\right)}$. Moreover, $\underset{i<j}{max}min(a_i,a_j)=a_{\left(2\right)}$, proving the left inequality in (64).

For the right inequality, let $k^{\ast }\in arg{max}_k{a}_k$. Then

$$\sum _{1\le i<j\le M}min(a_i,a_j)\ge \sum _{k\ne k^{\ast }}min(a_k,a_{k^{\ast }})=\sum _{k\ne k^{\ast }}{a}_k={\displaystyle \sum _{k=1}^M}{a}_k-\underset{k}{max}{a}_k.$$

Applying (64) pointwise to $a_i=p_i\left(x_1^n\right)$, multiplying by $\phi \left(x_1^n\right)$ and integrating yields (65). □

Theorem 3 

(M-ary exponent equals the minimum pairwise weighted Chernoff information). For $1\le i<j\le M$, let $D_C^{\mathrm{w}}({\mathbb{P}}_i,{\mathbb{P}}_j)$ be the weighted Chernoff information as in Definition 2, and (62) holds true. Set

$$C_M^{\mathrm{w}}:=\underset{1\le i<j\le M}{min}{D}_C^{\mathrm{w}}({\mathbb{P}}_i,{\mathbb{P}}_j).$$

Then the optimal M-ary total loss satisfies

$$L_{n,M}^{\ast }=exp\{-n{C}_M^{\mathrm{w}}+o\left(n\right)\},n\to \infty ,$$

(66)

or equivalently,

$$\underset{n\to \infty }{lim}-{\displaystyle \frac{1}{n}}ln{L}_{n,M}^{\ast }=C_M^{\mathrm{w}}.$$

Proof. 

Fix $1\le i<j\le M$ and consider the binary testing problem between ${\mathbb{P}}_i$ and ${\mathbb{P}}_j$ with the same factorised weight $\phi$. The optimal binary total loss equals

$$I_n^{i,j}={\int }_{{\mathcal{X}}^n}\phi \left(x_1^n\right)min\{p_i\left(x_1^n\right),p_j\left(x_1^n\right)\}\mathrm{d}{\mu }^{\otimes n}\left(x_1^n\right),$$

and by Theorem 1 applied to the pair $({\mathbb{P}}_i,{\mathbb{P}}_j)$,

$$I_n^{i,j}=exp\{-n{D}_C^{\mathrm{w}}({\mathbb{P}}_i,{\mathbb{P}}_j)+o_{i,j}\left(n\right)\}.$$

Since the number of pairs is finite, letting $r_n:={max}_{i<j}|o_{i,j}\left(n\right)|$ yields $r_n=o\left(n\right)$ and

$$I_n^{i,j}=exp\{-n{D}_C^{\mathrm{w}}({\mathbb{P}}_i,{\mathbb{P}}_j)+O\left(r_n\right)\}\mathrm{uniformly}\mathrm{over}i<j.$$

Now use the sandwich inequality (65). Let $(i^{\ast },j^{\ast })$ attain the minimum $C_M^{\mathrm{w}}=D_C^{\mathrm{w}}({\mathbb{P}}_{i^{\ast }},{\mathbb{P}}_{j^{\ast }})$. From the lower bound,

$$L_{n,M}^{\ast }\ge I_n^{i^{\ast },j^{\ast }}=exp\{-n{C}_M^{\mathrm{w}}+O\left(r_n\right)\}.$$

From the upper bound,

$$L_{n,M}^{\ast }\le \sum _{i<j}{I}_n^{i,j}\le \left({\displaystyle \genfrac{}{}{0pt}{}{M}{2}}\right)exp\{-n{C}_M^{\mathrm{w}}+O\left(r_n\right)\}.$$

Taking $-{\displaystyle \frac{1}{n}}ln(·)$ and letting $n\to \infty$ yields (66). □

Remark 8 

(Nonzero priors do not change the exponent). Let $w_1,\dots ,w_M>0$, ${\sum }_i{w}_i=1$, and consider the Bayesian weighted total loss $L_{n,M}^{\left(w\right)}\left({\delta }_n\right)={\displaystyle \sum _{i=1}^M}{w}_i{L}_{i,n}\left({\delta }_n\right)$ with optimum $L_{n,M}^{\left(w\right)\ast }:={inf}_{{\delta }_n}{L}_{n,M}^{\left(w\right)}\left({\delta }_n\right)$. Then, the exponent remains $C_M^{\mathrm{w}}$. Indeed, writing $w_{min}:={min}_i{w}_i$ and $w_{max}:={max}_i{w}_i$, for any ${\delta }_n$,

$$w_{min}{L}_{n,M}\left({\delta }_n\right)\le L_{n,M}^{\left(w\right)}\left({\delta }_n\right)\le w_{max}{L}_{n,M}\left({\delta }_n\right),$$

and taking infimum over ${\delta }_n$ gives $w_{min}{L}_{n,M}^{\ast }\le L_{n,M}^{\left(w\right)\ast }\le w_{max}{L}_{n,M}^{\ast }$. Hence, $-{\displaystyle \frac{1}{n}}ln{L}_{n,M}^{\left(w\right)\ast }$ and $-{\displaystyle \frac{1}{n}}ln{L}_{n,M}^{\ast }$ have the same limit $C_M^{\mathrm{w}}$.

5. Conclusions

We studied context-sensitive simple hypothesis testing under a multiplicative weight and proved that the optimal total loss admits the single-letter logarithmic asymptotic

$$L_n^{\ast }=exp\{-n{D}_C^{\mathrm{w}}(\mathbb{P},\mathbb{Q})+o\left(n\right)\}.$$

The rate is the weighted Chernoff information. The main structural ingredient is an exponential-family embedding of the weighted geometric mixtures $\phi p^{\alpha }{q}^{1-\alpha }$, which yields the characterisation of the optimal Chernoff parameter through the log-normaliser and leads to weighted information-geometric identities. We also derived finite-n concentration bounds for the tilted weighted log-likelihood, obtained explicit formulas in Gaussian, Poisson, and exponential models, and extended the logarithmic asymptotic to finitely many hypotheses through the minimum pairwise weighted Chernoff information.

Open Problems

The single-letter representation (4) rests on Assumption 1 (factorised weights), on the integrability of the tilted log-normaliser $\widehat{F}$, and on i.i.d. sampling of simple hypotheses. Relaxing these assumptions defines several natural open directions.

(a)

Non-factorised weights. Replacing Assumption 1 by a weight $\phi \left(x_1^n\right)=\psi \left(\frac{1}{n}{\sum }_{i=1}^{n}h\left(x_i\right)\right)$ or a pairwise-interaction weight; the single-letter rate is then expected to be replaced by a variational formula over the space of probability measures, in the spirit of Sanov/Gibbs conditioning.

(b)

Integrability. Weights with heavy tails in the sufficient statistic $t\left(x\right)$ may violate the finiteness of $\widehat{F}$ near ${\alpha }^{\ast }{\theta }_1+(1-{\alpha }^{\ast }){\theta }_2$; the boundary cases ${\alpha }^{\ast }\in \{0,1\}$ (cf. Proposition 5 and [3]) call for a systematic treatment via truncation or a change in base measure.

(c)

Dependent observations. A weighted counterpart of the Gärtner–Ellis theorem for stationary ergodic sequences, along the lines of the weighted extensions in [4].

(d)

Composite hypotheses. A weighted analogue of the generalised likelihood-ratio test and its exponent, with sup/inf characterisations in terms of $D_C^{\mathrm{w}}$ over the composite parameter sets.

(e)

Information geometry of weighted manifolds. Extending the Chentsov–Amari framework [8,9,10,11,12] to weighted statistical manifolds; the Fisher metric and the dually flat structure depend on symmetries that $\phi$ breaks in a controlled way.

Appendix A.1. KL Divergence (Closed Form)

Proposition A1 

(KL divergence between two Cauchy laws). Let $\mathbb{P}=\mathrm{Cauchy}(l_1,s_1)$ and $\mathbb{Q}=\mathrm{Cauchy}(l_2,s_2)$ with $s_1,s_2>0$. Then

$$D_{\mathrm{KL}}(\mathbb{P}||\mathbb{Q})={\int }_{\mathbb{R}}{p}_{l_1,s_1}\left(x\right)ln{\displaystyle \frac{p_{l_1,s_1}\left(x\right)}{p_{l_2,s_2}\left(x\right)}}\mathrm{d}x=ln{\displaystyle \frac{{(s_1+s_2)}^2+{(l_1-l_2)}^2}{4s_1{s}_2}}.$$

(A1)

Appendix A.2. Chernoff Parameter and Bhattacharyya Coefficient

Recall the (unweighted) $\alpha$-Bhattacharyya coefficient and distance

$${\rho }_{\alpha }(\mathbb{P},\mathbb{Q})={\int }_{\mathbb{R}}{p}_{l_1,s_1}{\left(x\right)}^{\alpha }{p}_{l_2,s_2}{\left(x\right)}^{1-\alpha }\mathrm{d}x,D_{B,\alpha }(\mathbb{P},\mathbb{Q})=-ln{\rho }_{\alpha }(\mathbb{P},\mathbb{Q}),\alpha \in [0, 1].$$

Proposition A2 

(Chernoff parameter for Cauchy). For the Cauchy location–scale family (with $\phi \equiv 1$), one has the symmetry

$${\rho }_{\alpha }(\mathbb{P},\mathbb{Q})={\rho }_{1-\alpha }(\mathbb{P},\mathbb{Q}),\alpha \in [0,1];$$

(A2)

equivalently, $D_{B,\alpha }(\mathbb{P},\mathbb{Q})=D_{B,1-\alpha }(\mathbb{P},\mathbb{Q})$. Consequently, if $\mathbb{P}\ne \mathbb{Q}$, then the Chernoff maximiser is unique and equals

$${\alpha }^{\ast }={\displaystyle \frac{1}{2}},\mathit{and}\mathit{hence}{D}_C(\mathbb{P},\mathbb{Q})=\underset{\alpha \in [0,1]}{max}{D}_{B,\alpha }(\mathbb{P},\mathbb{Q})=D_{B,1/2}(\mathbb{P},\mathbb{Q}).$$

If $\mathbb{P}=\mathbb{Q}$, then $D_{B,\alpha }(\mathbb{P},\mathbb{Q})\equiv 0$ and every $\alpha \in [0,1]$ is a maximiser.

Proof. 

Set $F\left(\alpha \right):=ln{\rho }_{\alpha }(\mathbb{P},\mathbb{Q})$. By Hölder’s inequality, F is convex on $[0, 1]$, and it is strictly convex unless $p_{l_1,s_1}/p_{l_2,s_2}$ is $\mu$-a.e. constant (equivalently, unless $\mathbb{P}=\mathbb{Q}$). Hence $D_{B,\alpha }(\mathbb{P},\mathbb{Q})=-F\left(\alpha \right)$ is concave (strictly concave when $\mathbb{P}\ne \mathbb{Q}$). For Cauchy laws, the symmetry (A2) holds; see, e.g., [17] for an invariance-based proof. Therefore, $D_{B,\alpha }$ is symmetric about $1/2$, and concavity implies that it attains its maximum at $\alpha =1/2$. Strict concavity yields uniqueness when $\mathbb{P}\ne \mathbb{Q}$. □

Appendix A.3. Closed Form for ρ_1/2 and D_C

Lemma A1 

(A standard elliptic-integral identity). Let $a,b>0$ and $d\in \mathbb{R}$. Define the complete elliptic integral of the first kind

$$K\left(m\right):={\int }_0^{\pi /2}{\displaystyle \frac{\mathrm{d}u}{\sqrt{1-m{sin}^{2}u}}},m\in [0,1).$$

Then

$${\int }_{-\infty }^{\infty }{\displaystyle \frac{\mathrm{d}x}{\sqrt{(x^2+a^2)\left({(x-d)}^2+b^2\right)}}}={\displaystyle \frac{4}{\sqrt{{(a+b)}^2+d^2}}}K\left({\displaystyle \frac{{(a-b)}^2+d^2}{{(a+b)}^2+d^2}}\right).$$

(A3)

Proposition A3 

(Bhattacharyya coefficient for Cauchy, closed form). Let $\mathbb{P}=\mathrm{Cauchy}(l_1,s_1)$ and $\mathbb{Q}=\mathrm{Cauchy}(l_2,s_2)$ with $s_1,s_2>0$, and set $\delta :=l_1-l_2$. Then

$${\rho }_{1/2}(\mathbb{P},\mathbb{Q})={\int }_{\mathbb{R}}\sqrt{p_{l_1,s_1}\left(x\right)p_{l_2,s_2}\left(x\right)}\mathrm{d}x={\displaystyle \frac{4\sqrt{s_1{s}_2}}{\pi \sqrt{{(s_1+s_2)}^2+{\delta }^2}}}K\left({\displaystyle \frac{{(s_1-s_2)}^2+{\delta }^2}{{(s_1+s_2)}^2+{\delta }^2}}\right).$$

(A4)

Consequently,

$$D_C(\mathbb{P},\mathbb{Q})=D_{B,1/2}(\mathbb{P},\mathbb{Q})=-ln{\rho }_{1/2}(\mathbb{P},\mathbb{Q}),$$

(A5)

with ${\rho }_{1/2}(\mathbb{P},\mathbb{Q})$ given by (A4).

Proof. 

By definition,

$$\sqrt{p_{l_1,s_1}\left(x\right)p_{l_2,s_2}\left(x\right)}={\displaystyle \frac{\sqrt{s_1{s}_2}}{\pi \sqrt{\left({(x-l_1)}^2+s_1^2\right)\left({(x-l_2)}^2+s_2^2\right)}}}.$$

Shift $x\mapsto x+l_2$ to obtain

$${\rho }_{1/2}(\mathbb{P},\mathbb{Q})={\displaystyle \frac{\sqrt{s_1{s}_2}}{\pi }}{\int }_{-\infty }^{\infty }{\displaystyle \frac{\mathrm{d}x}{\sqrt{(x^2+s_2^2)\left({(x-\delta )}^2+s_1^2\right)}}}.$$

Apply Lemma A1 with $a=s_2$, $b=s_1$, $d=\delta$ to get (A4). Finally, Proposition A2 yields $D_C(\mathbb{P},\mathbb{Q})=D_{B,1/2}(\mathbb{P},\mathbb{Q})$, hence (A5). □

Remark A1 

(Useful special cases). If $l_1=l_2$ (common location), then $\delta =0$ and (A4) reduces to

$${\rho }_{1/2}(\mathbb{P},\mathbb{Q})={\displaystyle \frac{4\sqrt{s_1{s}_2}}{\pi (s_1+s_2)}}K\left({\left({\displaystyle \frac{s_1-s_2}{s_1+s_2}}\right)}^2\right).$$

If $s_1=s_2=s$ (common scale), then

$${\rho }_{1/2}(\mathbb{P},\mathbb{Q})={\displaystyle \frac{4s}{\pi \sqrt{4s^2+{\delta }^2}}}K\left({\displaystyle \frac{{\delta }^2}{4s^2+{\delta }^2}}\right).$$

Remark A2 

(Weighted case). For a general weight φ, one has

$${\rho }_{1/2}^{\mathrm{w}}(\mathbb{P},\mathbb{Q})={\int }_{\mathbb{R}}\phi \left(x\right)\sqrt{p_{l_1,s_1}\left(x\right)p_{l_2,s_2}\left(x\right)}\mathrm{d}x.$$

The symmetry (A2) (and hence ${\alpha }^{\ast }=1/2$) typically fails unless φ is compatible with the invariance argument used in [17]. We therefore use the Cauchy model mainly as a closed-form baseline example at $\phi \equiv 1$.