Next Article in Journal
The Entropy of Words—Learnability and Expressivity across More than 1000 Languages
Previous Article in Journal
An Information-Spectrum Approach to the Capacity Region of the Interference Channel

Entropy 2017, 19(6), 276; https://doi.org/10.3390/e19060276

Article
Noise Enhancement for Weighted Sum of Type I and II Error Probabilities with Constraints
College of Communication Engineering, Chongqing University, Chongqing 400044, China
*
Author to whom correspondence should be addressed.
Received: 12 April 2017 / Accepted: 12 June 2017 / Published: 14 June 2017

Abstract

:
In this paper, the noise-enhanced detection problem is investigated for the binary hypothesis-testing. The optimal additive noise is determined according to a criterion proposed by DeGroot and Schervish (2011), which aims to minimize the weighted sum of type I and II error probabilities under constraints on type I and II error probabilities. Based on a generic composite hypothesis-testing formulation, the optimal additive noise is obtained. The sufficient conditions are also deduced to verify whether the usage of the additive noise can or cannot improve the detectability of a given detector. In addition, some additional results are obtained according to the specificity of the binary hypothesis-testing, and an algorithm is developed for finding the corresponding optimal noise. Finally, numerical examples are given to verify the theoretical results and proofs of the main theorems are presented in the Appendix.
Keywords:
noise enhancement; hypothesis testing; weighted sum; error probability

1. Introduction

In the binary hypothesis testing problem, there are usually a null hypothesis H 0 and an alternative hypothesis H 1 , and the objective of testing is to be determine truthfulness of them based on the observation data and a decision rule. Due to the presence of noise, the decision result obviously cannot be absolutely correct. Generally, two erroneous decisions may occur in the signal detection: type I error that rejects a true null hypothesis and type II error that accepts a false null hypothesis [1].
In the classical statistical theory, the Neyman–Pearson criterion is usually applied to obtain a decision rule that minimizes the type II error probability β with a constraint on the type I error probability α . However, the minimum β may not always correspond to the optimal decision result. For instance, in the example 1 of [2], a binary hypothesis testing is designed to determine the mean of a normal data: the mean equals to −1 under H 0 and equals to 1 under H 1 . Under the constraint that the type I error probability is fixed to 0.05, the type II error probability is decreased from 0.0091 to 0.00000026 when the data size increases from 20 to 100, whereas the rejection region of the null hypothesis is changed from (0.1, +∞) to (−0.51, +∞). In such case, more information brings a worse decision result even a smaller type II error probability is achieved with the fixed type I error probability. Similarly, the decision rule that minimizes the type I error probability for a fixed type II error probability may not perform well. Therefore, it could not be appropriate to simply minimize one of the two error probabilities in practice. The ideal case is to develop a decision criterion that minimizes the two types of error probabilities simultaneously, but it is almost impossible in practical applications.
In order to obtain a better decision result to balance the type I and II error probabilities, DeGroot and Schervish [1] proposed a criterion to minimize a weighted sum of type I and II error probabilities, i.e., min ϕ [ c 1 α ( ϕ ) + c 2 β ( ϕ ) ] , where ϕ represents the decision rule, c 1 and c 2 are the weight coefficients corresponding to α and β , respectively, and c 1 , c 2 > 0 . Furthermore, DeGroot also provided the optimal decision procedure to minimize the weighted sum. The decision rule is given as follows. If c 1 f ( x | H 0 ) < c 2 f ( x | H 1 ) , the null hypothesis H 0 is rejected, where f ( x | H 0 ) and f ( x | H 1 ) are the respective probability density functions (pdfs) of the observation x under H 0 and H 1 . If c 1 f ( x | H 0 ) > c 2 f ( x | H 1 ) , the alternative hypothesis H 1 is rejected. In addition, if c 1 f ( x | H 0 ) = c 2 f ( x | H 1 ) , the hypothesis H 0 can be either rejected or not. The optimal detector in this case is closely related to the distribution of the observation. This implies that once the distribution changes, the detector should be adjusted accordingly. But in the cases where the detector is fixed, this weighted sum rule cannot be directly applied. In such a case, finding an alternative method to minimize the weighted sum of type I and II error probabilities instead of changing the detector is important. Fortunately, the stochastic resonance (SR) theory provides a means to solve this problem.
The SR, first discovered by Benzi et al. [3] in 1981, is a phenomenon where noise plays a positive role in enhancing signal and system through a nonlinear system under certain conditions. The phenomenon of SR in the signal detection is also called noise-enhanced detection. Recent studies indicate that the system output performance can be improved significantly by adding noise to the system input or increasing the background noise level [4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22]. The improvements achieved via noise can be measured in the forms of increased signal-to-noise ratio (SNR) [7,8,9,10], mutual information (MI) [11,12] or detection probability [13,14,15,16], or in the form of decreased Bayes risk [17,18]. For example, the SNR gain of a parallel uncoupled array of bistable oscillators, operating in a mixture of sinusoidal signal and Gaussian white noise, is maximized via extra array noise [8]. In addition, due to the added array noise, the performance of a finite array closely approaches to an infinite array. In [11], the throughput MI of threshold neurons is increased by increasing the intensity of faint input noise. The optimal additive noise to maximize the detection probability with a constraint on false-alarm probability is studied in [13], and the sufficient conditions for improvability and non-improvability are deduced. In [17], the effects of additive independent noise on the performance of suboptimal detectors are investigated according to the restricted Bayes criterion, where the minimum noise modified Bayes risk is explored with certain constraints on the conditional risks. Inspired by this concept, it is reasonable to conjecture that a proper noise can decrease the weighted sum of type I and II error probabilities for a fixed detector.
In the absence of constraints, it is obvious that the additive noise that minimizes the weighted sum is a constant vector, whereas the corresponding type I or II error probability may exceed a certain value to cause a bad decision result. To avoid this problem, two constraints are enforced on type I and II error probabilities, respectively, to keep a balance. The aim of this work is to find the optimal additive noise that minimizes the weighted sum of type I and II error probabilities with the constraints on type I and II error probabilities for a fixed detector. Furthermore, the work can also be extended to some applications, such as the energy detection in sensor networks [23,24] and the independent Bernoulli trials [25]. The main contributions of this paper are summarized as follows:
  • Formulation of the optimization problem for minimizing the noise modified weighted sum of type I and II error probabilities under the constraints on the two error probabilities is presented.
  • Derivations of the optimal noise that minimizes the weighted sum and sufficient conditions for improvability and nonimprovability for a general composite hypothesis testing problem are provided.
  • Analysis of the characteristics of the optimal additive noise that minimizes the weighted sum for a simple hypothesis testing problem is studied and the corresponding algorithm to solve the optimization problem is developed.
  • Numerical results are presented to verify the theoretical results and to demonstrate the superior performance of the proposed detector.
The remainder of this paper is organized as follows: in Section 2, a noise modified composite hypothesis testing problem is formulated first for minimizing the weighted sum of type I and II error probabilities under different constraints. Then the sufficient conditions for improvability and nonimprovability are given and the optimal additive noise is derived. In Section 3, additional theoretical results are analyzed for a simple hypothesis testing problem. Finally, simulation results are shown in Section 4 and conclusions are made in Section 5.
Notation: Lower-case bold letters denote vectors, with υ i denoting the i-th element of υ ; θ denotes the value of parameter Θ ; f υ ( υ | θ ) denotes the pdf of υ for a given parameter value Θ = θ ; Ω i denotes the set of all possible parameter values of Θ under H i ; δ ( ) denotes the Dirac function; , and denote intersection, union and null set, respectively; , ( ) T , , E { } , min , max and arg denote convolution, transpose, integral, expectation, minimum, maximum and argument operators, respectively; inf { } and sup { } denote the infimum and supremum operators, respectively; means summation; and H denote the respective gradient and Hessian operators.

2. Noise Enhanced Composite Hypothesis Testing

2.1. Problem Formulation

Consider the following binary composite hypothesis testing problem:
{ H 0 : f x ( x | θ ) , θ Ω 0 H 1 : f x ( x | θ ) , θ Ω 1
where x N is the observation vector, H 0 and H 1 are the null and the alternative hypothesizes, respectively, θ denotes the value of parameter Θ , f x ( x | θ ) represents the pdf of x for a given parameter value Θ = θ . The parameter Θ has multiple possible values under each hypothesis and denote the pdf of any parameter value Θ = θ under H 0 and H 1 by ϖ 0 ( θ ) and ϖ 1 ( θ ) . In addition, Ω 0 and Ω 1 denote the respective sets of all possible values of Θ under H 0 and H 1 . It is true that Ω 0 Ω 1 = and the union of them forms the parameter space Ω , i.e., Ω = Ω 0 Ω 1 .
Without loss of generality, a decision rule (detector) is considered as:
ϕ ( x ) = { 1 , x Γ 1 0 , x Γ 0
where Γ 0 and Γ 1 form the observation space Γ . Actually, the detector chooses H 1 if x Γ 1 , otherwise chooses H 0 if x Γ 0 .
In order to investigate the performance of the detector achieved via an additive noise, a noise modified observation y is obtained by adding an independent additive noise n to the original observation x , i.e., y = x + n . For a given parameter value Θ = θ , the pdf of y is calculated by the convolution of the pdfs of x and n , given by:
f y ( y | θ ) = f x ( x | θ ) p n ( n ) = N p n ( n ) f x ( y n | θ ) d n
where p n ( n ) denotes the pdf of n . For a fixed detector, the noise modified type I and II error probabilities of the detector for given parameter values now is expressed as:
α y ( ϕ ; θ ) = Γ ϕ ( y ) f y ( y | θ ) d y = Γ 1 f y ( y | θ ) d y , θ Ω 0
β y ( ϕ ; θ ) = Γ ( 1 ϕ ( y ) ) f y ( y | θ ) d y = Γ 0 f y ( y | θ ) d y , θ Ω 1
Correspondingly, the average noise modified type I and II error probabilities are calculated by:
α y ( ϕ ) = Ω 0 α y ( ϕ ; θ ) ϖ 0 ( θ ) d θ
β y ( ϕ ) = Ω 1 β y ( ϕ ; θ ) ϖ 1 ( θ ) d θ
From (6) and (7), the weighted sum of the two types of average error probabilities is obtained as:
E r y = c 1 α y ( ϕ ) + c 2 β y ( ϕ )   = c 1 Ω 0 α y ( ϕ ; θ ) ϖ 0 ( θ ) d θ + c 2 Ω 1 β y ( ϕ ; θ ) ϖ 1 ( θ ) d θ
where c 1 and c 2 are the weights assigned for the type I and II error probabilities, which can be predefined according to the actual situations. For example, if the prior probabilities are known, the value of c 1 and c 2 equal the prior probabilities corresponding to H 0 and H 1 , respectively. Besides, the values of c 1 and c 2 can also be determined based on the expected decision results.
In this work, the aim is to find the optimal independent additive noise, which minimizes the weighted sum of the average error probabilities under the constraints on the maximum type I and II error probabilities for different parameter values. The optimization problem can be formulated as below:
p n o p t ( n ) = arg min p n ( n ) E r y
subject to
{ max θ Ω 0 α y ( ϕ ; θ ) α o max θ Ω 1 β y ( ϕ ; θ ) β o
where α o and β o are the upper limits for the type I and II error probabilities, respectively.
In order to explicitly express the optimization problem described in (9) and (10), substituting (3) into (4) produces:
α y ( ϕ ; θ ) = Γ 1 N p n ( n ) f x ( y n | θ ) d n d y   = N p n ( n ) Γ 1 f x ( y n | θ ) d y d n   = N p n ( n ) A θ ( n ) d n   = E { A θ ( n ) } , θ Ω 0
where
A θ ( n ) = Γ 1 f x ( y n | θ ) d y = Γ ϕ ( y ) f x ( y n | θ ) d y , θ Ω 0
It should be noted that A θ ( n ) can be viewed as the type I error probability obtained by adding a constant vector n to x for θ Ω 0 . Therefore, α x ( ϕ ; θ ) = A θ ( 0 ) = Γ ϕ ( x ) f x ( x | θ ) d x denotes the type I error probability for the original observation x .
Similarly, β y ( ϕ ; θ ) in (5) can be expressed as:
β y ( ϕ ; θ ) = E { B θ ( n ) } ,   θ Ω 1
where
B θ ( n ) = Γ 0 f x ( y n | θ ) d y = Γ ( 1 ϕ ( y ) ) f x ( y n | θ ) d y ,   θ Ω 1
The B θ ( n ) can be treated as the type II error probability obtained by adding a constant vector n to x for θ Ω 1 and β x ( ϕ ; θ ) = B θ ( 0 ) = Γ ( 1 ϕ ( x ) ) f x ( x | θ ) d x is the original type II error probability without adding noise for θ Ω 1 .
With (11) and (13), (8) becomes:
E r y = N p n ( n ) [ c 1 Ω 0 A θ ( n ) ϖ 0 ( θ ) d θ + c 2 Ω 1 B θ ( n ) ϖ 1 ( θ ) d θ ] d n = E { E r ( n ) }
where
E r ( n ) = c 1 Ω 0 A θ ( n ) ϖ 0 ( θ ) d θ + c 2 Ω 1 B θ ( n ) ϖ 1 ( θ ) d θ
Accordingly, E r ( n ) is the weighted sum of two types of average error probabilities achieved by adding a constant vector n to the original observation x . Naturally, E r x = E r ( 0 ) denotes the weighted sum of type I and II average error probabilities for the original observation x .
Combined (11), (13) and (15), the optimization problem in (9) and (10) now is:
p n o p t ( n ) = arg min p n ( n ) E { E r ( n ) }
subject to
{ max θ Ω 0 E { A θ ( n ) } α o max θ Ω 1 E { B θ ( n ) } β o

2.2. Sufficient Conditions for Improvability and Non-improvability

In practice, the solution of the optimization problem in (17) and (18) requires a research over all possible noises and this procedure is complicated. Therefore, it is worthwhile to determine whether the detector can or cannot be improved by adding additive noise in advance. From (17) and (18), a detector is considered to be improvable if there exists one noise n that satisfies E { E r ( n ) } < E r x = E r ( 0 ) , max θ Ω 0 E { A θ ( n ) } α o and max θ Ω 0 E { B θ ( n ) } β o simultaneously; otherwise, the detector is considered to be non-improvable.
The sufficient conditions for non-improvability can be obtained according to the characteristics of A θ ( n ) , B θ ( n ) and E r ( n ) , which are provided in Theorem 1.
Theorem 1.
If there exists θ * Ω 0 ( θ * Ω 1 ) such that A θ * ( n ) α o ( B θ * ( n ) β o ) implies E r ( n ) E r ( 0 ) for any n P n , where P n represents the convex set of all possible additive noises, and if A θ * ( n ) ( B θ * ( n ) ) and E r ( n ) are convex functions over P n , then the detector is non-improvable.
The proof is provided in Appendix A.
Under the conditions in Theorem 1, the detector cannot be improved and it is unnecessary to solve the optimization problem in (17) and (18). In other words, if the conditions in Theorem 1 are satisfied, the three inequities E r y E r x , max θ Ω 0 E { A θ ( n ) } α o and max θ Ω 1 E { B θ ( n ) } β o cannot be achieved simultaneously by adding any additive noise. In addition, even if the conditions in Theorem 1 are not satisfied, the detector can also be non-improvable. This implies the sufficient conditions for improvability need to be addressed.
The sufficient conditions for improvability are discussed now. Suppose that A θ ( x ) ( θ Ω 0 ), B θ ( x ) ( θ Ω 1 ) and E r ( x ) are second-order continuously differentiable around x = 0 . In order to facilitate the subsequent analysis, six auxiliary functions are predefined as follows based on the first and second partial derivatives of A θ ( x ) , B θ ( x ) and E r ( x ) with respect to the elements of x . The first three auxiliary functions a θ ( 1 ) ( x , g ) , b θ ( 1 ) ( x , g ) and e r ( 1 ) ( x , g ) are defined as the weight sums of the first partial derivatives of A θ ( x ) , B θ ( x ) and E r ( x ) , respectively, based on the coefficient vector g . Specifically:
a θ ( 1 ) ( x , g ) i = 1 N g i A θ ( x ) x i = g T A θ ( x ) , θ Ω 0
b θ ( 1 ) ( x , g ) i = 1 N g i B θ ( x ) x i = g T B θ ( x ) , θ Ω 1
e r ( 1 ) ( x , g ) i = 1 N g i E r ( x ) x i = g T E r ( x )
where g is a N -dimensional column vector, g T is the transposition of g , x i and g i are the i-th elements of x and g , respectively. In addition, denotes the gradient operator, thereby A θ ( x ) ( B θ ( x ) , E r ( x ) ) is a N -dimensional column vector with i-th element A θ ( x ) / x i ( B θ ( x ) / x i , E r ( x ) / x i ), i = 1 , , N . The last three auxiliary functions a θ ( 2 ) ( x , g ) , b θ ( 2 ) ( x , g ) and e r ( 2 ) ( x , g ) are defined as the weight sums of the second partial derivatives of A θ ( x ) , B θ ( x ) and E r ( x ) based on the N × N coefficient matrix g g T , i.e.,
a θ ( 2 ) ( x , g ) j = 1 N i = 1 N g j g i 2 A θ ( x ) ( x j x i ) = g T H ( A θ ( x ) ) g , θ Ω 0
b θ ( 2 ) ( x , g ) j = 1 N i = 1 N g j g i 2 B θ ( x ) ( x j x i ) = g T H ( B θ ( x ) ) g , θ Ω 1
e r ( 2 ) ( x , g ) j = 1 N i = 1 N g j g i 2 E r ( x ) ( x j x i ) = g T H ( E r ( x ) ) g
where H denote the Hessian operator, H ( A θ ( x ) ) ( H ( B θ ( x ) ) , H ( E r ( x ) ) ) is a N × N matrix with its ( j , i ) -th element denoted by 2 A θ ( x ) / ( x j x i ) ( 2 B θ ( x ) / ( x j x i ) , 2 E r ( x ) / ( x j x i ) ), where i , j = 1 , , N .
Based on the definitions in (19)–(24), Theorem 2 presents the sufficient conditions for improvability.
Theorem 2.
Suppose that Λ 0 and Λ 1 are the sets of all possible values of θ that maximize A θ ( 0 ) and B θ ( 0 ) , respectively, α o = max θ Ω 0 A θ ( 0 ) and β o = max θ Ω 1 B θ ( 0 ) . The detector is improvable, if there exists a N -dimensional column vector g that satisfies one of the following conditions for all θ 0 Λ 0 and θ 1 Λ 1 :
(1) 
e r ( 1 ) ( x , g ) | x = 0 < 0 , a θ 0 ( 1 ) ( x , g ) | x = 0 < 0 , b θ 1 ( 1 ) ( x , g ) | x = 0 < 0 ;
(2) 
e r ( 1 ) ( x , g ) | x = 0 > 0 , a θ 0 ( 1 ) ( x , g ) | x = 0 > 0 , b θ 1 ( 1 ) ( x , g ) | x = 0 > 0 ;
(3) 
e r ( 2 ) ( x , g ) | x = 0 < 0 , a θ 0 ( 2 ) ( x , g ) | x = 0 < 0 , b θ 1 ( 2 ) ( x , g ) | x = 0 < 0 .
The proof is presented in Appendix B.
Theorem 2 indicates that under the condition (1), (2) or (3), there always exist noises that decrease the weighted sum of average error probabilities under the constraints on the type I and II error probabilities. In addition, alternative sufficient conditions for improvability can be obtained by defining the following two functions, and they are:
I ( t ) = inf { E r ( n ) | max θ Ω 0 A θ ( n ) = t , n N }
S ( t ) = sup { max θ Ω 1 B θ ( n ) | max θ Ω 0 A θ ( n ) = t , n N }
where I ( t ) and S ( t ) are the minimum weighted sum of two types of average error probabilities and the maximum type II error probability for a given maximum type I error probability obtained via adding a constant vector, respectively. If there is a t 0 α o such that I ( t 0 ) E r ( 0 ) and S ( 0 ) β o , the detector is improvable. More specifically, there exists a constant vector n 0 that satisfies max θ Ω 0 A θ ( n 0 ) = t 0 α o , E r ( n 0 ) E r ( 0 ) and max θ Ω 0 B θ ( n 0 ) β o simultaneously. However, in most cases, the solution of the optimization problem in (17) and (18) is not a constant vector. A more practical sufficient condition for improvability is shown in Theorem 3.
Theorem 3.
Let α ˜ = max θ Ω 0 α x ( ϕ ; θ ) and β ˜ = max θ Ω 1 β x ( ϕ ; θ ) be the respective maximum type I and II error probabilities without adding any noise, and suppose that α ˜ α o , β ˜ β o and S ( α ˜ ) = β ˜ . If I ( t ) and S ( t ) are second-order continuously differentiable around t = α ˜ , and I ( α ˜ ) < 0 and S ( α ˜ ) < 0 hold at the same time, then the detector is improvable.
The proof is given in Appendix C.
Additionally, the following functions J ( ε ) and G ( ε ) are defined:
J ( ε ) = inf { E r ( n ) | max θ Ω 1 B θ ( n ) = ε , n N }
G ( ε ) = sup { max θ Ω 0 A θ ( n ) | max θ Ω 1 B θ ( n ) = ε , n N }
A similar conclusion to the Theorem 3 can be made as well, provided in Corollary 1.
Corollary 1.
The detector is improvable, if J ( β ˜ ) 0 and G ( β ˜ ) 0 hold, where J ( ε ) and G ( ε ) are second-order continuously differentiable around ε = β ˜ , and G ( β ˜ ) = α ˜ .
The proof is similar to that of Theorem 3 and it is omitted here.

2.3. Optimal Additive Noise

In general, it is difficult to solve the optimization problem in (17) and (18) directly, because the solution is obtained based on the search over all possible additive noises. Hence, in order to reduce the computational complexity, one can utilize Parzen window density estimation to obtain an approximate solution. Actually, the pdf of the optimal additive noise can be approximated by:
p n ( n ) = l = 1 L η l ϑ l ( n )
where η l 0 and l = 1 L η l = 1 , while ϑ l ( ) represents the window function that satisfies ϑ l ( x ) 0 for any x and ϑ l ( x ) d x = 0 for l = 1 , , L . The window function can be a cosine window, rectangular window, or Gauss window function. With (29), the optimization problem is simplified to obtain the parameter values corresponding to each window function. In such cases, global optimization algorithms can be applied such as Particle swarm optimization (PSO), Ant colony algorithm (ACA), and Genetic algorithm (GA) [26,27,28].
If the numbers of parameter values in Ω 0 and Ω 1 are finite, the optimal additive noise for (17) and (18) is a randomization of no more than M + K constant vectors. In this case, Ω 0 and Ω 1 can be expressed by Ω 0 = { θ 01 , θ 02 , , θ 0 M } and Ω 1 = { θ 11 , θ 12 , , θ 1 K } , where M and K are finite positive integers. The Theorem 4 states this claim.
Theorem 4.
Suppose that each component in the optimal additive noise is finite, namely n i [ a i , b i ] for i = 1 , , N , where a i and b i are two finite values. If A θ 0 i ( ) and B θ 1 i ( ) are continuous functions, the pdf of the optimal additive noise for the optimization problem in (17) and (18) can be expressed as:
p n ( n ) = l = 1 M + K η l δ ( n n l )
where η l 0 and l = 1 M + K η l = 1 .
The proof is similar to that of Theorem 4 in [17] and Theorem 3 in [13], and omitted here. In some special cases, the optimal additive can be solved directly based on the characteristics of I ( t ) ( H ( ε ) ). For example, let E r min = min t I ( t ) = I ( t m ) ( E r min = min ε H ( ε ) = I ( ε m ) ) and max θ Ω 0 A θ ( n m ) = t m ( max θ Ω 1 B θ ( n m ) = ε m ). If t m α o ( ε m β o ) and max θ Ω 1 B θ ( n m ) β o ( max θ Ω 0 A θ ( n m ) α o ), the optimal additive noise is a constant vector with pdf of p n ( n ) = δ ( n n m ) . In addition, equality of max θ Ω 0 E { A θ ( n ) } = α o ( max θ Ω 1 E { B θ ( n ) } = β o ) holds if t m > α o ( ε m > β o ).

3. Noise Enhanced Simple Hypothesis Testing

In this section, the noise enhanced binary simple hypothesis testing problem is considered, which is a special case of the optimization problem in (9) and (10). Therefore, the conclusions obtained in Section 2 are also applicable in this section. Furthermore, due to the specificity of simple binary hypothesis testing problem, some additional results are also obtained.

3.1. Problem Formulation

When Ω i = { θ i } , i = 0 , 1 , the composite binary hypothesis testing problem described in (1) is simplified to a simple binary hypothesis testing problem. In this case, the probability of θ i under H i equals to 1, i.e., ϖ i ( θ ) = 1 for i = 0 , 1 . Therefore, the corresponding noise modified type I and II error probabilities is rewritten as:
α y ( ϕ ) = α y ( ϕ ; θ 0 ) = N p n ( n ) Γ 1 f 0 ( y n ) d y d n = E { A 0 ( n ) }
β y ( ϕ ) = β y ( ϕ ; θ 1 ) = N p n ( n ) Γ 0 f 1 ( y n ) d y d n = E { B 1 ( n ) }
where f 0 ( ) and f 1 ( ) represent the pdfs of x under H 0 and H 1 , respectively, and A 0 ( n ) and B 1 ( n ) are:
A 0 ( n ) = Γ 1 f 0 ( y n ) d y
B 1 ( n ) = Γ 0 f 1 ( y n ) d y
Correspondingly, the weighted sum of noise modified type I and II error probabilities is calculated by:
E r y = c 1 α y ( ϕ ) + c 2 β y ( ϕ ) = c 1 E { A 0 ( n ) } + c 2 E { B 1 ( n ) }   = N p n ( n ) ( c 1 A 0 ( n ) + c 2 B 1 ( n ) ) d n   = E { E r ( n ) }
where
E r ( n ) = c 1 A 0 ( n ) + c 2 B 1 ( n )
As a result, the optimization problem in (9) and (10) becomes:
p n o p t ( n ) = arg min p n ( n ) E { E r ( n ) }
subject to
{ E { A 0 ( n ) } α o E { B 1 ( n ) } β o
Based on the definitions in (33) and (34), A 0 ( n ) and B 1 ( n ) are viewed as the noise modified type I and II error probabilities obtained by adding a constant vector noise. Furthermore, A 0 ( 0 ) and B 1 ( 0 ) are the original type I and II error probabilities, respectively.

3.2. Algorithm for the Optimal Additive Noise

According to the Theorem 4 in Section 2.3, the optimal additive noise for the optimization problem in (37) and (38) is a randomization of most two constant vectors with the pdf p n o p t ( n ) = η δ ( n n 1 ) + ( 1 η ) δ ( n n 2 ) . In order to find the values of η , n 1 and n 2 , we first divide each constant vector n into four disjoint sets according to the relationships of A 0 ( n ) and α o , B 1 ( n ) and β o . To be specific, the four disjoint sets are Q 1 = { n | A 0 ( n ) α o , B 1 ( n ) β o } , Q 2 = { n | A 0 ( n ) α o , B 1 ( n ) > β o } , Q 3 = { n | A 0 ( n ) > α o , B 1 ( n ) β o } , and Q 4 = { n | A 0 ( n ) > α o , B 1 ( n ) > β o } . Then, we calculate the minimum E r ( n ) , the corresponding set of all possible values of n is denoted by Q e = { n | n = arg min n E r ( n ) } . It should be noted that n Q e is the optimal additive noise that minimizes the weighted sum without constraints.
It is obvious that Q 1 , Q 2 and Q 3 do not exist if all the elements of Q e belong to Q 4 . In other words, if Q e Q 4 , there is no additive noise that satisfies E { E r ( n ) } < E r ( 0 ) under the constraints of E { A 0 ( n ) } α o and E { B 1 ( n ) } β o . Therefore, if the detector is improvable, the elements of Q e must come from Q 1 , Q 2 and/or Q 3 . Theorem 5 is now provided to find the values of η , n 1 and n 2 .
Theorem 5.
Let η 1 = α o A 0 ( n 2 ) A 0 ( n 1 ) A 0 ( n 2 ) and η 2 = β o B 1 ( n 2 ) B 1 ( n 1 ) B 1 ( n 2 ) .
(1) 
If Q e Q 1 , then η = 1 and n 1 Q e Q 1 such that E r o p t y = E r ( n 1 ) = min n E r ( n ) .
(2) 
If Q e Q 2 and Q e Q 3 are true, then we have n 1 Q e Q 2 , n 2 Q e Q 3 , η 1 η η 2 , and E r o p t y = min n E r ( n ) .
(3) 
If Q e Q 2 , then E r o p t y is obtained when η = η 2 , and the corresponding E { A 0 ( n ) } achieves the minimum and E { B 1 ( n ) } = β o .
(4) 
If Q e Q 3 , then E r o p t y is achieved when η = η 1 , and the corresponding E { A 0 ( n ) } = α o and E { B 1 ( n ) } reaches the minimum.
The corresponding proofs are provided in Appendix D.
From (3) and (4) in Theorem 5, under the constraints on E { A 0 ( n ) } α o and E { B 1 ( n ) } β o , the solution of the optimization problem in (37) and (38) is identical with the additive noise that minimizes E { A 0 ( n ) } ( E { B 1 ( n ) } ) when Q e Q 2 ( Q e Q 3 ). In such cases, the optimal solution can be obtained easily by referring the algorithm provided in [14].

4. Numerical Results

In this section, a binary hypothesis testing problem is studied to verify the theoretical analysis, and it is:
{ H 0 : x = v H 1 : x = Θ + v
where x is an observation, Θ is a constant or random variable, and v is the background noise with pdf p v ( ) . From (39), the pdf of x under H 0 is f 0 ( x ) = p v ( x ) , and the pdf of x under H 1 for a given parameter value Θ = θ is denoted by f θ ( x ) = p v ( ) p θ ( ) , where p θ ( ) represents the pdf of Θ = θ . A noise modified observation y is obtained via adding an additive independent noise n to the observation x , i.e., y = x + n . If the additive noise n is a constant vector, the pdf of y under H 0 is calculated as f 0 ( y ) = f 0 ( x n ) , and the pdf of y under H 1 for Θ = θ is f θ ( y ) = f θ ( x n ) . In addition, a linear- quadratic detector is utilized here, given by:
T ( y ) = d 0 y 2 + d 1 y + d 2 > < H 0 H 1 γ
where d 0 , d 1 and d 2 are detector parameters, and γ denotes the detection threshold. In the numerical examples, α o = α x and β o = β x , where α x and β x are the original type I and II error probabilities, respectively.

4.1. Rayleigh Distribution Background Noise

Suppose that Θ = s is a constant, the problem shown in (39) represents a simple binary hypothesis testing problem. Here, we set d 0 = d 2 = 0 and d 1 = 1 , then the detector becomes
T ( y ) = y > < H 0 H 1 γ
It is assumed that the background noise v obeys the mixture of Rayleigh distributions with zero-means such that p v ( v ) = i = 1 M m i φ i ( v μ i ) , where m i 0 for i = 1 , , M , i = 1 M m i = 1 , and
φ i ( x ) = { x σ i 2 exp ( x 2 2 σ i 2 ) , x 0 0 , x < 0
In the simulations, the variances of all the Rayleigh components are assumed to be the same, i.e., σ i = σ for i = 1 , , M . In addition, the parameters are specified as M = 4 , u 1 = 0.2 , u 2 = 0.4 , u 3 = 2 σ π 2 0.2 , u 4 = 2 σ π 2 0.4 and m i = 0.25 for i = 1 , , 4 . From (33) and (34), the noise modified type I error probability A 0 ( n ) and type II error probability B 1 ( n ) obtained by adding a constant vector n is calculated as:
A 0 ( n ) = γ + f 0 ( y ) d y = i = 1 4 m i Φ ( γ μ i n )
B 1 ( n ) = γ f 1 ( y ) d y = 1 i = 1 4 m i Φ ( γ μ i n s )
where Φ ( x ) = x x σ 2 exp ( x 2 2 σ 2 ) d t , when x > 0 ; Φ ( x ) = 1 , when x 0 . Accordingly, α x = A 0 ( 0 ) = i = 1 4 m i Φ ( γ μ i ) and β x = B 1 ( 0 ) = 1 i = 1 4 m i Φ ( γ μ i s ) . Let c 1 = β x / ( α x + β x ) and c 2 = α x / ( α x + β x ) , the noise modified weighted sum of the two types of error probabilities obtained via adding a constant vector is E r ( n ) = c 1 i = 1 4 m i Φ ( γ μ i n ) + c 2 i = 1 4 m i Φ ( γ μ i n s ) .
From Section 3.2, the pdf of the optimal additive noise that minimizes weighted sum of type I and II error probabilities is denoted by p n o p t ( n ) = η δ ( n n 1 ) + ( 1 η ) δ ( n n 2 ) , under the two constraints that α y α x and β y β x . Moreover, the optimal additive noise for the case without any constraints is a constant vector.
Figure 1 plots the minimum noise modified weighted sums of type I and II error probabilities obtained under no constraint and two constraints that α y α x and β y β x , and the original weighted sum without adding any noise for different values of σ when s = 3 and γ = s / 2 . When σ σ 1 , there is no noise that decreases the weighted sum. With the increase of σ , noise exhibits a positive effect on the detection performance. To be specific, when σ 1 < σ < σ 2 , the weighted sum can be decreased by adding a constant vector for the no constraint case. When σ > σ 2 , the weighted sum can be decreased adding the noise under two constraints. The noise modified weighted sum obtained without any constraints is less than or equal to that obtained under the two constraints, and the difference between them first decreases to zero for σ 3 < σ < σ 4 and then gradually increases when σ > σ 4 . In addition, once σ exceeds a certain value, no noise exists that can decrease the weighted sum for any cases.
Figure 2 shows the type I and II error probabilities corresponding to the weighted sum in Figure 1. From both Figure 1 and Figure 2, it is observed that one of the noise modified Type I and II error probabilities performs worse than the original one for the no constraint case. Therefore, though the noise modified weighted sum obtained with no constraint is less than that obtained under the two constraints, the corresponding noise is actually not suitable to add to the observation. Furthermore, when the minimum value of the noise modified weighted sum is obtained under the two constraints, the corresponding type II probability equals to the original one and the type I probability achieves the minimum for σ 2 < σ < σ 3 . Conversely, when σ > σ 4 , the corresponding type I probability equals to the original one and the type II probability achieves the minimum. The results are consistent with part (3) and part (4) in Theorem 5. Especially, for σ 3 < σ < σ 4 , the minimum values of the noise modified weighted sum obtained under no constraint is equal to that obtained under two constraints, and the corresponding type I and II error probabilities are the same, which also agrees with part (2) in Theorem 5. In order to further illustrate the results in Figure 1 and Figure 2, Table 1 provides the optimal additive noises added for the two different cases.
Figure 3 depicts the minimum noise modified weighted sums of the type I and II error probabilities versus s for the cases of no constraint and two constraints, and the original weighted sum, when σ = 1 and γ = s / 2 . The corresponding type I and II error probabilities are depicted in Figure 4a,b, respectively. It is seen in Figure 3, the improvement of the weighted sum obtained by adding noise first increases and then decreases with the increase of s , and finally they all converge to the same value. The differences for the cases with and without constraints are very small in most cases. In the small interval of s , i.e., s ( s 1 , s 2 ) , the difference even decreases to zero. On the other hand, the noise modified type I error probability obtained under no constraint is significantly greater than the original one for s < s 1 , while the corresponding type II error probability is less than that obtained under the two constraints. The situation, however, is reversed for s 2 < s < s 3 . When s > s 3 , there is no noise that decreases the weighted sum under the two constraints, while the weighted sum is still decreased by adding a constant vector for no constraint case. When s > s 4 , the weighted sum cannot be decreased by adding any noise for all the cases. Furthermore, Table 2 shows the optimal additive noises that minimize the weighted sum under the cases of no and two constraints.
Figure 5 shows the minimum noise modified weighted sums of type I and II error probabilities versus γ for the cases of no constraint and two constraints, and the original weighted sum, when σ = 1 and s = 3 . The corresponding type I and II error probabilities are depicted in Figure 6a,b, respectively. As illustrated in Figure 5, when γ is close to zero, the original weighted sum E r x approaches to zero. In such case, no additive noise exists to decrease the weighted sum. For the case of two constraints, the improvement of the weighted sum first increases for γ 1 < γ < γ 2 and then decreases for γ 2 < γ < γ 3 , and no improvement can be obtained when γ > γ 2 . On the other hand, the minimum noise modified weighted sum obtained under no constraint is smaller than that obtained under the two constraints for γ 1 < γ < γ 3 , and the difference between them first increases and then decreases for both γ 1 < γ < γ 2 and γ 2 < γ < γ 3 . When γ > γ 3 , there still exists a constant vector that decreases the weighted sum, but it may be not a suitable noise in the practical application according to the type II probability depicted in Figure 6b. Furthermore, in order to study the results illustrated in Figure 5 and Figure 6, Table 3 shows the optimal additive noises that minimize the weighted sum for the cases of no and two constraints.

4.2. Gaussian Mixture Background Noise

Suppose that Θ is a random variable with following pdf:
ϖ 1 ( θ ) = ρ δ ( θ s ) + ( 1 ρ ) δ ( θ + s )
Therefore, we have Ω 0 = { 0 } and Ω 1 = { s , s } . In the simulations, we set d 0 = 1 , d 1 = 0 , d 2 = s 2 / 4 and γ = 0 , the detector is expressed as:
T ( y ) = y 2 s 2 4 > < H 0 H 1 0
Moreover, we assume that v is a zero-mean symmetric Gaussian mixture noise with pdf of p v ( v ) = i = 1 M m i ψ i ( v μ i ) , where m i 0 , i = 1 M m i = 1 and:
ψ i ( v ) = 1 2 π σ i 2 exp ( v 2 2 σ i 2 )
Let M = 4 and the mean values of the symmetric Gaussian components are set as [0.05 0.52 −0.52 −0.05] with corresponding weights [0.35 0.15 0.15 0.35]. In addition, the variances of Gaussian components are the same, i.e., σ i = σ for i = 0 , , 4 . According to (12) and (14), the noise modified type I error probability obtained by adding a constant vector n to x is calculated by:
A 0 ( n ) = i = 1 4 m i ( Ψ ( s / 2 + μ i + n σ ) + Ψ ( s / 2 μ i n σ ) )
and the corresponding type II error probabilities for Θ = s and s are respectively calculated as:
B s ( n ) = 1 i = 1 4 m i ( Ψ ( 3 s / 2 + μ i + n σ ) + Ψ ( s / 2 μ i n σ ) )
B s ( n ) = 1 i = 1 4 m i ( Ψ ( s / 2 + μ i + n σ ) + Ψ ( 3 s / 2 μ i n σ ) )
where Ψ ( x ) = x 1 2 π exp ( t 2 2 σ 2 ) d t . Accordingly:
B 1 ( n ) = ρ B s ( n ) + ( 1 ρ ) B s ( n )
Therefore, the original type I and type II error probabilities for Θ = s and s are α x ( ϕ ; 0 ) = A 0 ( 0 ) , β x ( ϕ ; s ) = B s ( 0 ) and β x ( ϕ ; s ) = B s ( 0 ) , respectively.
Due to the symmetry property of v , one obtains B s ( 0 ) = B s ( 0 ) . In this case, the original average type II error probability is β x = B 1 ( 0 ) = ρ β x ( ϕ ; s ) + ( 1 ρ ) β x ( ϕ ; s ) = B s ( 0 ) = B s ( 0 ) . The noise modified weighted sum of type I and average type II error probabilities corresponding to the constant vector is expressed by E r ( n ) = c 1 A 0 ( n ) + c 2 B 1 ( n ) . The values of c 1 and c 2 are still specified as β x / ( α x + β x ) and α x / ( α x + β x ) , respectively. From Theorem 4 in Section 2.3, the optimal additive noise that minimizes the weighted sum is a randomization with a pdf of p n o p t ( n ) = η 1 δ ( n n 1 ) + η 2 δ ( n n 2 ) + η 3 δ ( n n 3 ) , where η i 0 for i = 1 , , 3 , and i = 1 3 η i = 1 .
Figure 7 shows the detection performance of the original detector and the noise enhanced detector that minimizes the weighted sum of type I and average type II error probabilities under the constraints that α y ( ϕ ; 0 ) α o and max θ Ω 1 β y ( ϕ ; θ ) β o , for different values of σ where s = 1 and ρ = 0.6 . The minimum achievable noise modified weighted sum is plotted in Figure 7a, and the corresponding type I error probability and type II error probabilities for Θ = s and s are depicted in Figure 7b–d, respectively.
From Figure 7, the original weighted sums, type I error probabilities, and type II error probabilities for Θ = s and s increase as σ decreases towards zero. In Figure 7a, when σ is close to zero, the weighted sum can be decreased significantly. With the increase of σ , the improvement obtained by adding noise is reduced gradually to zero. In other words, the phenomenon of noise-enhanced detection performance cannot occur when σ exceeds a certain value. In Figure 7b, the noise modified type I error probability stays at 0.1500 for σ < 0.07 and then increases gradually to equal to the original type I error probability. Moreover, the noise modified type II error probabilities for Θ = s corresponding to the minimum weighted sum increases from zero to that of original detector, shown in Figure 7c, while the type II error probabilities for Θ = s of the noise enhanced detector is equal to that of the original detector all the time. In fact, the type II error probability for Θ = s also reaches the minimums under the constraints that α y ( ϕ ; 0 ) α o and max θ Ω 1 β y ( ϕ ; θ ) β o in this example. In addition, Table 4 offers the optimal additive noises that minimize the weighted sum for different values of σ to explain the results in Figure 7. It should be noted that the optimal noise is not unique.y.
Figure 8a demonstrates the weighted sums of type I and average type II error probabilities of the original detector and the noise enhanced detector versus s , where σ = 0.08 and ρ = 0.6 . The corresponding type I error probability and type II error probabilities for Θ = s and s are depicted in Figure 8b–d, respectively. From Figure 8a, the weighted sum cannot be decreased under the constraints on different error probabilities for s < s 1 and s < s 2 . Conversely, there exists additive noise under the constraints that reduces the weighted sum for s 1 < s < s 2 , and the corresponding improvement first increases and then decreases with the increase of s . Comparing Figure 8b with Figure 8a, it is noted that the change of the noise modified type I error probability is similar to that of the noise modified weighted sum. In Figure 8c, the noise modified type II error probability for Θ = s first decreases to the minimum and then increases as s increases, while the type II error probability for Θ = s of the noise modified detector is always equal to that of the original detector, shown in Figure 8d. In addition, in order to further illustrate the results in Figure 8, Table 5 shows the optimal noises that minimize the weighted sum under the case of two constraints.

5. Conclusions

In this paper, a noise-enhanced detection problem has been investigated for a general composite hypothesis testing. Under the constraints of type I and II error probabilities, the minimization of the weighted sum of average type I and II error probabilities has been explored by adding an additive independent noise. The sufficient conditions for improvability of the weighted sum are provided, and a simple algorithm to search the optimal noise is developed. Then some additional theoretical results are made based on the specificity of the binary simple hypothesis testing problem. The studies on different noise distributions confirm the theoretical analysis that the optimal additive noise indeed minimizes the weighted sum under certain conditions. To be noted that, theoretical results can also be extended to a broad class of noise enhanced optimization problems under two inequality constraints such as the minimization of Bayes risk under the different constraints of condition risks for a binary hypothesis testing problem.

Acknowledgments

This research is partly supported by the Basic and Advanced Research Project in Chongqing (Grant No. cstc2016jcyjA0134, No. cstc2016jcyjA0043), the National Natural Science Foundation of China (Grant No. 61501072, No. 41404027, No. 61571069, No. 61675036, No. 61471073) and the Project No. 106112017CDJQJ168817 supported by the Fundamental Research Funds for the Central Universities.

Author Contributions

Shujun Liu raised the idea of the framework to solve noise enhanced detection problems. Ting Yang and Shujun Liu contributed to the drafting of the manuscript, interpretation of the results, some experimental design and checked the manuscript. Kui Zhang contributed to develop the algorithm of finding the corresponding optimal noise and Ting Yang contributed to the proofs of the theories developed in this paper. All authors have read and approved the final manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Proof of Theorem 1

Proof. 
Due to the convexity of A θ * ( n ) and according to the Jensen’s inequality, the type I error probability in (4) is calculated as:
α y ( ϕ ; θ * ) = E { A θ * ( n ) } A θ * ( E { n } )
The contradiction method is utilized to prove this theorem. Suppose that the detector can be improved by adding noise. The improvability means that α y ( ϕ ; θ * ) α o for any θ * Ω 0 , and then A θ * ( E { n } ) α o from (A1). Since E { n } P n , A θ * ( E { n } ) α o implies E r ( E { n } ) E r ( 0 ) based on the assumption in Theorem 1, (15) can be recalculated as:
E r y = E { E r ( n ) } E r ( E { n } ) E r ( 0 ) = E r x
where the first inequality holds according to the convexity of E r ( n ) . From (A1) and (A2), the inequality E r y < E r x cannot be achieved by adding any noise under the conditions presented in Theorem 1. Therefore, the detector is nonimprovable, which contradicts the assumption. Similarly, the alternative conditions for nonimprovability stated in the parentheses can also be proved. □

Appendix B. Proof of Theorem 2

Proof. 
According to the definitions in (9) and (10), improvability for a detector means that there exists at least one pdf p n ( n ) to satisfy three conditions, i.e., E r y = N p n ( n ) E r ( n ) d n < E r ( 0 ) , N p n ( n ) A θ ( n ) d n α o for any θ Ω 0 and N p n ( n ) B θ ( n ) d n β o for any θ Ω 1 . Suppose that the noise pdf p n ( n ) consists of L infinitesimal noise components, i.e., p n ( n ) = l = 1 L λ l δ ( n ε l ) . The three conditions can be rewritten as follows:
E r y = l = 1 L λ l E r ( ε l ) < E r ( 0 )
l = 1 L λ l A θ ( ε l ) α o , θ Ω 0
l = 1 L λ l B θ ( ε l ) β o , θ Ω 1
Since ε l , l = 1 , , L , is infinitesimal, E r ( ε l ) , A θ ( ε l ) and B θ ( ε l ) can be expressed approximately with Taylor series expansion as E r ( 0 ) + ε l T E r + 0.5 ε l T H ε l , A θ ( 0 ) + ε l T A θ + 0.5 ε l T H θ A ε l and B θ ( 0 ) + ε l T B θ + 0.5 ε l T H θ B ε l , where E r ( A θ , B θ ) and H ( H θ A , H θ B ) are the gradient and the Hessian matrix of E r ( x ) ( A θ ( x ) , B θ ( x ) ) around x = 0 , respectively. Therefore, (A3)–(A5) are rewritten as:
l = 1 L λ l ε l T E r + 0.5 l = 1 L λ l ε l T H ε l < 0
l = 1 L λ l ε l T A θ + 0.5 l = 1 L λ l ε l T H θ A ε l α o A θ ( 0 ) , θ Ω 0
l = 1 L λ l ε l T B θ + 0.5 l = 1 L λ l ε l T H θ B ε l β o B θ ( 0 ) , θ Ω 1
Let ε l be expressed by ε l = τ l g , where g is a N -dimensional real vector and τ l is an infinitesimal real value, l = 1 , , L . Accordingly, one obtains:
l = 1 L λ l τ l g T E r + 0.5 l = 1 L λ l τ l 2 g T H g < 0
l = 1 L λ l τ l g T A θ + 0.5 l = 1 L λ l τ l 2 g T H θ A g α o A θ ( 0 ) , θ Ω 0
l = 1 L λ l τ l g T B θ + 0.5 l = 1 L λ l τ l 2 g T H θ B g β o B θ ( 0 ) , θ Ω 1
Based on the definitions given in (19)–(24), (A9)–(A11) are simplified as:
( k e r ( 1 ) ( x , g ) + e r ( 2 ) ( x , g ) ) | x = 0 < 0
( k a θ ( 1 ) ( x , g ) + a θ ( 2 ) ( x , g ) ) | x = 0 < 2 ( α o A θ ( 0 ) ) l = 1 L λ l τ l 2 , θ Ω 0
( k b θ ( 1 ) ( x , g ) + b θ ( 2 ) ( x , g ) ) | x = 0 < 2 ( β o B θ ( 0 ) ) l = 1 L λ l τ l 2 , θ Ω 1
where k = 2 l = 1 L λ l τ l / l = 1 L λ l τ l 2 . As α o = A θ ( 0 ) for θ Λ 0 and α o > A θ ( 0 ) for θ Ω 0 Λ 0 , the right-hand side of (A13) approaches to plus infinity for θ Ω 0 Λ 0 . Similarly, when β o = B θ ( 0 ) for θ Λ 1 and β o > B θ ( 0 ) for θ Ω 1 Λ 1 , the right-hand side of (A14) also goes to plus infinity for θ Ω 1 Λ 1 . Therefore, we only need to consider the cases of θ Λ 0 and θ Λ 1 . In doing so, (A12)–(A14) are now:
( k e r ( 1 ) ( x , g ) + e r ( 2 ) ( x , g ) ) | x = 0 < 0
( k a θ ( 1 ) ( x , g ) + a θ ( 2 ) ( x , g ) ) | x = 0 < 0 , θ Λ 0
( k b θ ( 1 ) ( x , g ) + b θ ( 2 ) ( x , g ) ) | x = 0 < 0 , θ Λ 1
It is obvious that k can be set as any real value by choosing appropriate λ l and τ l . As a result, (A15)–(A17) can be satisfied by selecting a suitable value of k under each condition in Theorem 2. That is:
(1)
Inequalities (A15)–(A17) can be satisfied by setting k as a sufficiently large positive number, if e r ( 1 ) ( x , g ) | x = 0 < 0 , a θ 0 ( 1 ) ( x , g ) | x = 0 < 0 , b θ 1 ( 1 ) ( x , g ) | x = 0 < 0 hold.
(2)
Inequalities (A15)–(A17) can be satisfied by setting k as a sufficiently large negative number, if e r ( 1 ) ( x , g ) | x = 0 > 0 , e a θ 0 ( 1 ) ( x , g ) | x = 0 > 0 , b θ 1 ( 1 ) ( x , g ) | x = 0 > 0 hold.
(3)
Inequalities (A15)–(A17) can be satisfied by setting k as zero, if e r ( 2 ) ( x , g ) | x = 0 < 0 , a θ 0 ( 2 ) ( x , g ) | x = 0 < 0 , b θ 1 ( 2 ) ( x , g ) | x = 0 < 0 hold. □

Appendix C. Proof of Theorem 3

Proof. 
Since I ( t ) and S ( t ) are second-order continuously differentiable around t = α ˜ , there exists a ξ > 0 such that I ( ) < 0 and S ( ) < 0 for Δ = ( α ˜ ξ , α ˜ + ξ ) . If one adds a noise with pdf p n ^ ( n ) = 0.5 δ ( n n 1 ) + 0.5 δ ( n n 2 ) , where max θ Ω 0 A θ ( n 1 ) = α ˜ + ξ and max θ Ω 0 A θ ( n 2 ) = α ˜ ξ , to the original observation x , the maximum values of corresponding noise modified type I and II error probabilities are:
max θ Ω 0 E { A θ ( n ^ ) } E { max θ Ω 0 A θ ( n ^ ) } 0.5 ( α ˜ + ξ ) + 0.5 ( α ˜ ξ ) = α ˜ α o
max θ Ω 1 E { B θ ( n ^ ) } E { max θ Ω 1 B θ ( n ^ ) } 0.5 S ( α ˜ + ξ ) + 0.5 S ( α ˜ ξ ) S ( α ˜ ) = β ˜ β o
In addition:
E { E r ( n ^ ) } = E { I ( t ) } = 0.5 I ( α ˜ + ξ ) + 0.5 I ( α ˜ ξ ) < I ( α ˜ )
One obtains E { E r ( n ^ ) } < E r ( 0 ) because I ( α ˜ ) E r ( 0 ) according to the definition of I ( t ) . As a result, the detector is improvable. □

Appendix D. Proof of Theorem 5

Proof. 
Part (1): If Q e Q 1 , any n 1 Q e Q 1 satisfies the constraints of A 0 ( n 1 ) α o and B 1 ( n 1 ) β o based on the definition of Q 1 and E r o p t y = E r ( n 1 ) = min n E r ( n ) < E r ( 0 ) according to the definition of Q e .
Part (2): If Q e Q 2 and Q e Q 3 simultaneously, there exists n 1 Q e Q 2 and n 2 Q e Q 3 such that E r ( n 1 ) = E r ( n 2 ) = min n E r ( n ) based on the definition of Q e . In order to meet the constraints that E { A 0 ( n ) } α o and E { B 1 ( n ) } β o , the noise components η , n 1 and n 2 should satisfy the following two inequalities:
η A 0 ( n 1 ) + ( 1 η ) A 0 ( n 2 ) α o
η B 1 ( n 1 ) + ( 1 η ) B 1 ( n 2 ) β o
Consequently, η η 1 = α o A 0 ( n 2 ) A 0 ( n 1 ) A 0 ( n 2 ) and η η 2 = β o B 1 ( n 2 ) B 1 ( n 1 ) B 1 ( n 2 ) according to the definitions of Q 2 and Q 3 . If η 1 η η 2 , the noise with pdf p n o p t ( n ) = η δ ( n n 1 ) + ( 1 η ) δ ( n n 2 ) can minimize E { E r ( n ) } and satisfy the two inequalities, and E r o p t y = η E r ( n 1 ) + ( 1 η ) E r ( n 2 ) = min n E r ( n ) .
Part (3): If Q e Q 2 , the optimal additive noise is not a constant vector, i.e., η 1 . Therefore, one of n 1 and n 2 belongs to Q 2 and the second one comes from Q 1 or Q 3 . In addition, η , n 1 and n 2 should also satisfy the two constraints in (A21) and (A22).
First, suppose that n 1 Q 2 and n 2 Q 1 , then (A21) holds based on the definitions of Q 1 and Q 2 . We should only consider the constraint in (A22), which implies η η 2 . It is true that A 0 ( n 2 ) α o and B 1 ( n 2 ) β o according to the definition of Q 1 . If E r ( n 1 ) > E r ( n 2 ) , we have E r ( n 2 ) < E r o p t y = η E r ( n 1 ) + ( 1 η ) E r ( n 2 ) , which contradicts with the definition of p n o p t ( n ) . Hence, E r ( n 1 ) < E r ( n 2 ) and the minimum of E { E r ( n ) } is obtained when η = η 2 .
Next, suppose that n 1 Q 2 and n 2 Q 3 . The two inequalities in (A21) and (A22) require that η 1 η η 2 . If E r ( n 1 ) > E r ( n 2 ) , the minimum of E { E r ( n ) } is obtained when η = η 1 . In such case, there exists a noise with pdf p n ^ ( n ) = ς p n o p t ( n ) + ( 1 ς ) δ ( n n e ) that satisfies E { A 0 ( n ^ ) } α o and E { B 1 ( n ^ ) } β o simultaneously, where n e Q e and 0 ς 1 . Therefore, E { E r ( n ^ ) } = ς E r o p t y + ( 1 ς ) E r ( n e ) < E r o p t y since E r ( n e ) = min n E r ( n ) < E r o p t y , which contradicts with the definition of p n o p t ( n ) . As a result, E r ( n 1 ) < E r ( n 2 ) and the minimum of E { E r ( n ) } is obtained when η = η 2 .
When η = η 2 , one obtains E { B 1 ( n ) } = η B 1 ( n 1 ) + ( 1 η ) B 1 ( n 2 ) = β o . In other words, the minimum of E { E r ( n ) } is obtained when E { A 0 ( n ) } achieves the minimum and E { A 0 ( n ) } . Accordingly, one obtains E r o p t y = c 1 α o p t y + c 2 β o .
Part (4): The proof of Part (4) is similar to that of Part (3) and it is omitted here. □

References

  1. DeGroot, M.H.; Sxhervish, M.J. Probability and Statistics, 4nd ed.; Addison-Wesley: Boston, MA, USA, 2011. [Google Scholar]
  2. Pericchi, L.; Pereira, C. Adaptative significance levels using optimal decision rules: Balancing by weighting the error probabilities. Braz. J. Probab. Stat. 2016, 30, 70–90. [Google Scholar] [CrossRef]
  3. Benzi, R.; Sutera, A.; Vulpiani, A. The mechanism of stochastic resonance. J. Phys. A Math. 1981, 14, 453–457. [Google Scholar] [CrossRef]
  4. Patel, A.; Kosko, B. Noise benefits in quantizer-array correlation detection and watermark decoding. IEEE Trans. Signal Process. 2011, 59, 488–505. [Google Scholar] [CrossRef]
  5. Han, D.; Li, P.; An, S.; Shi, P. Multi-frequency weak signal detection based on wavelet transform and parameter compensation band-pass multi-stable stochastic resonance. Mech. Syst. Signal Process. 2016, 70–71, 995–1010. [Google Scholar] [CrossRef]
  6. Addesso, P.; Pierro, V.; Filatrella, G. Interplay between detection strategies and stochastic resonance properties. Commun. Nonlinear Sci. Numer. Simul. 2016, 30, 15–31. [Google Scholar] [CrossRef]
  7. Gingl, Z.; Makra, P.; Vajtai, R. High signal-to-noise ratio gain by stochastic resonance in a double well. Fluct. Noise Lett. 2001, 1, L181–L188. [Google Scholar] [CrossRef]
  8. Makra, P.; Gingl, Z. Signal-to-noise ratio gain in non-dynamical and dynamical bistable stochastic resonators. Fluct. Noise Lett. 2002, 2, L147–L155. [Google Scholar] [CrossRef]
  9. Makra, P.; Gingl, Z.; Fulei, T. Signal-to-noise ratio gain in stochastic resonators driven by coloured noises. Phys. Lett. A 2003, 317, 228–232. [Google Scholar] [CrossRef]
  10. Duan, F.; Chapeau-Blondeau, F.; Abbott, D. Noise-enhanced SNR gain in parallel array of bistable oscillators. Electron. Lett. 2006, 42, 1008–1009. [Google Scholar] [CrossRef]
  11. Mitaim, S.; Kosko, B. Adaptive stochastic resonance in noisy neurons based on mutual information. IEEE Trans. Neural Netw. 2004, 15, 1526–1540. [Google Scholar] [CrossRef] [PubMed]
  12. Patel, A.; Kosko, B. Mutual-Information Noise Benefits in Brownian Models of Continuous and Spiking Neurons. In Proceedings of the 2006 International Joint Conference on Neural Network, Vancouver, BC, Canada, 16–21 July 2006; pp. 1368–1375. [Google Scholar]
  13. Chen, H.; Varshney, P.K.; Kay, S.M.; Michels, J.H. Theory of the stochastic resonance effect in signal detection: Part I – fixed detectors. IEEE Trans. Signal Process. 2007, 55, 3172–3184. [Google Scholar] [CrossRef]
  14. Patel, A.; Kosko, B. Optimal noise benefits in Neyman–Pearson and inequality constrained signal detection. IEEE Trans. Signal Process. 2009, 57, 1655–1669. [Google Scholar]
  15. Bayram, S.; Gezici, S. Stochastic resonance in binary composite hypothesis-testing problems in the Neyman–Pearson framework. Digit. Signal Process. 2012, 22, 391–406. [Google Scholar] [CrossRef]
  16. Bayrama, S.; Gultekinb, S.; Gezici, S. Noise enhanced hypothesis-testing according to restricted Neyman–Pearson criterion. Digit. Signal Process. 2014, 25, 17–27. [Google Scholar] [CrossRef]
  17. Bayram, S.; Gezici, S.; Poor, H.V. Noise enhanced hypothesis-testing in the restricted Bayesian framework. IEEE Trans. Signal Process. 2010, 58, 3972–3989. [Google Scholar] [CrossRef]
  18. Bayram, S.; Gezici, S. Noise enhanced M-ary composite hypothesis-testing in the presence of partial prior information. IEEE Trans. Signal Process. 2011, 59, 1292–1297. [Google Scholar] [CrossRef]
  19. Chen, H.; Varshney, L.R.; Varshney, P.K. Noise-enhanced information systems. Proc. IEEE 2014, 102, 1607–1621. [Google Scholar] [CrossRef]
  20. Weber, J.F.; Waldman, S.D. Stochastic Resonance is a Method to Improve the Biosynthetic Response of Chondrocytes to Mechanical Stimulation. J. Orthop. Res. 2015, 34, 231–239. [Google Scholar] [CrossRef] [PubMed]
  21. Duan, F.; Chapeau-Blondeau, F.; Abbott, D. Non-Gaussian noise benefits for coherent detection of narrow band weak signal. Phys. Lett. A 2014, 378, 1820–1824. [Google Scholar] [CrossRef]
  22. Lu, Z.; Chen, L.; Brennan, M.J.; Yang, T.; Ding, H.; Liu, Z. Stochastic resonance in a nonlinear mechanical vibration isolation system. J. Sound Vib. 2016, 370, 221–229. [Google Scholar] [CrossRef]
  23. Rossi, P.S.; Ciuonzo, D.; Ekman, T.; Dong, H. Energy Detection for MIMO Decision Fusion in Underwater Sensor Networks. IEEE Sen. J. 2015, 15, 1630–1640. [Google Scholar] [CrossRef]
  24. Rossi, P.S.; Ciuonzo, D.; Kansanen, K.; Ekman, T. Performance Analysis of Energy Detection for MIMO Decision Fusion in Wireless Sensor Networks Over Arbitrary Fading Channels. IEEE Trans. Wirel. Commun. 2016, 15, 7794–7806. [Google Scholar] [CrossRef]
  25. Ciuonzo, D.; de Maio, A.; Rossi, P.S. A Systematic Framework for Composite Hypothesis Testing of Independent Bernoulli Trials. IEEE Signal Proc. Lett. 2015, 22, 1249–1253. [Google Scholar] [CrossRef]
  26. Parsopoulos, K.E.; Vrahatis, M.N. Particle Swarm Optimization Method for Constrained Optimization Problems; IOS Press: Amsterdam, The Netherlands, 2002; pp. 214–220. [Google Scholar]
  27. Hu, X.; Eberhart, R. Solving constrained nonlinear optimization problems with particle swarm optimization. In Proceedings of the sixth world multiconference on systemics, cybernetics and informatics, Orlando, FL, USA, 14–18 July 2002. [Google Scholar]
  28. Price, K.V.; Storn, R.M.; Lampinen, J.A. Differential Evolution: A Practical Approach to Global Optimization; Springer: New York, NY, USA, 2005. [Google Scholar]
Figure 1. The minimum noise modified weighted sums of the type I and II error probabilities obtained under no constraint and two constraints, and the original weighted sum for different σ when s = 3 and γ = s/2.
Figure 1. The minimum noise modified weighted sums of the type I and II error probabilities obtained under no constraint and two constraints, and the original weighted sum for different σ when s = 3 and γ = s/2.
Entropy 19 00276 g001
Figure 2. The type I (a) and II (b) error probabilities corresponding to the weighted sum in Figure 1.
Figure 2. The type I (a) and II (b) error probabilities corresponding to the weighted sum in Figure 1.
Entropy 19 00276 g002
Figure 3. The minimum noise modified weighted sums of the type I and II error probabilities obtained under no constraint and two constraints, and the original weighted sum for different s when σ = 1 and γ = s/2.
Figure 3. The minimum noise modified weighted sums of the type I and II error probabilities obtained under no constraint and two constraints, and the original weighted sum for different s when σ = 1 and γ = s/2.
Entropy 19 00276 g003
Figure 4. The type I and II error probabilities corresponding to the weighted sum in Figure 3 are shown in (a) and (b), respectively.
Figure 4. The type I and II error probabilities corresponding to the weighted sum in Figure 3 are shown in (a) and (b), respectively.
Entropy 19 00276 g004
Figure 5. The minimum noise modified weighted sums of the type I and II error probabilities obtained under no constraint and two constraints, and the original weighted sum for different γ when σ = 1 and s = 3.
Figure 5. The minimum noise modified weighted sums of the type I and II error probabilities obtained under no constraint and two constraints, and the original weighted sum for different γ when σ = 1 and s = 3.
Entropy 19 00276 g005
Figure 6. The type I and II error probabilities corresponding to the weighted sum in Figure 5 are shown in (a) and (b), respectively.
Figure 6. The type I and II error probabilities corresponding to the weighted sum in Figure 5 are shown in (a) and (b), respectively.
Entropy 19 00276 g006
Figure 7. The weighted sums, type I error probabilities, and type II error probabilities for Θ = s and −s of the original detector and the noise enhanced detector for different σ where s = 1 and ρ = 0.6 shown in (a), (b), (c) and (d), respectively
Figure 7. The weighted sums, type I error probabilities, and type II error probabilities for Θ = s and −s of the original detector and the noise enhanced detector for different σ where s = 1 and ρ = 0.6 shown in (a), (b), (c) and (d), respectively
Entropy 19 00276 g007
Figure 8. The weighted sums, type I error probabilities, and type II error probabilities for Θ = s and −s of the original detector and the noise enhanced detector for different s where σ = 0.08 and ρ = 0.6 shown in (a), (b), (c) and (d), respectively.
Figure 8. The weighted sums, type I error probabilities, and type II error probabilities for Θ = s and −s of the original detector and the noise enhanced detector for different s where σ = 0.08 and ρ = 0.6 shown in (a), (b), (c) and (d), respectively.
Entropy 19 00276 g008
Table 1. The optimal additive noises that minimize the weighted sum under two constraints and no constraint for various σ where s = 3 and γ = s/2.
Table 1. The optimal additive noises that minimize the weighted sum under two constraints and no constraint for various σ where s = 3 and γ = s/2.
σ Two ConstraintsNo Constraints
n 1 n 2 η n o
0.950---−1.7089
1.250−1.90821.79630.6950−1.9218
2.125−2.51363.18960.7862−2.5136/3.1896
3.000−3.37714.69420.37704.7449
Table 2. The optimal additive noises that minimize the weighted sum under two constraints and no constraint for various s where σ = 1 and γ = s/2.
Table 2. The optimal additive noises that minimize the weighted sum under two constraints and no constraint for various s where σ = 1 and γ = s/2.
s Two ConstraintsNo Constraints
n 1 n 2 η n o
1.25−1.36821.73270.29181.7474
1.75−1.44081.65630.7265−1.4408/1.6563
2.5−1.60521.46900.6983−1.6201
3.25---−0.5866
Table 3. The optimal additive noises that minimize the weighted sum under two constraints and no constraint for various γ where σ = 1 and s = 3.
Table 3. The optimal additive noises that minimize the weighted sum under two constraints and no constraint for various γ where σ = 1 and s = 3.
γ Two ConstraintsNo Constraints
n 1 n 2 η n o
0.050----
1.100−2.12130.93410.28780.9691
1.425−1.79471.25850.5355−1.7957
2.250−0.96932.08360.8867−1.1763
3.375---−0.5775
Table 4. The optimal additive noises that minimize the weighted sum under two constraints for various σ where s = 1 and ρ = 0.6.
Table 4. The optimal additive noises that minimize the weighted sum under two constraints for various σ where s = 1 and ρ = 0.6.
σ n 1 n 2 n 3 η 1 η 2 η 3
0.00010.2286--1.0000--
0.020.2286−0.2255-0.84130.1587-
0.050.2287−0.22080.24210.53100.34460.1244
0.080.2180−0.2185−0.21680.59430.24490.1608
Table 5. The optimal additive noises that minimize the weighted sum under two constraints for various s where σ = 0.08 and ρ = 0.6.
Table 5. The optimal additive noises that minimize the weighted sum under two constraints for various s where σ = 0.08 and ρ = 0.6.
s n 1 n 2 n 3 η 1 η 2 η 3
0.650.1613−0.1613-0.62670.3733-
0.750.2026−0.2026-0.79490.2051-
0.850.2148−0.2149-0.21500.82620.13000.0438
0.950.2195−0.2196-0.21900.70060.19160.1078

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Back to TopTop