Search Results (18)

The current replication crisis relating to the non-replicability and the untrustworthiness of published empirical evidence is often viewed through the lens of the Positive Predictive Value (PPV) in the context of the Medical Diagnostic Screening (MDS) model. The PPV is misconstrued as a measure that evaluates ‘the probability of rejecting $H_0$ when false’, after being metamorphosed by replacing its false positive/negative probabilities with the type I/II error probabilities. This perspective gave rise to a widely accepted diagnosis that the untrustworthiness of published empirical evidence stems primarily from abuses of frequentist testing, including p-hacking, data-dredging, and cherry-picking. It is argued that the metamorphosed PPV misrepresents frequentist testing and misdiagnoses the replication crisis, promoting ill-chosen reforms. The primary source of untrustworthiness is statistical misspecification: invalid probabilistic assumptions imposed on one’s data. This is symptomatic of the much broader problem of the uninformed and recipe-like implementation of frequentist statistics without proper understanding of (a) the invoked probabilistic assumptions and their validity for the data used, (b) the reasoned implementation and interpretation of the inference procedures and their error probabilities, and (c) warranted evidential interpretations of inference results. A case is made that Fisher’s model-based statistics offers a more pertinent and incisive diagnosis of the replication crisis, and provides a well-grounded framework for addressing the issues (a)–(c), which would unriddle the non-replicability/untrustworthiness problems. Full article

The isotropic Cauchy distribution is a member of the central $\alpha$-stable family that plays a role in the set of heavy-tailed distributions similar to that of the Gaussian density among finite second-moment laws. Given a sequence of n observations, we are interested in characterizing the performance of Likelihood Ratio Tests, where two hypotheses are plausible for the observed quantities: either isotropic Cauchy or isotropic Gaussian. Under various setups, we show that the probability of error of such detectors is not always exponentially decaying with n, with the leading term in the exponent shown to be logarithmic instead, and we determine the constants in that leading term. Perhaps surprisingly, the optimal Bayesian probabilities of error are found to exhibit different asymptotic behaviors. Full article

Statistical hypothesis testing, as formalized by 20th century statisticians and taught in college statistics courses, has been a cornerstone of 100 years of scientific progress. Nevertheless, the methodology is increasingly questioned in many scientific disciplines. We demonstrate in this paper how many of the worrisome aspects of statistical hypothesis testing can be ameliorated with concepts and methods from evidential analysis. The model family we treat is the familiar normal linear model with fixed effects, embracing multiple regression and analysis of variance, a warhorse of everyday science in labs and field stations. Questions about study design, the applicability of the null hypothesis, the effect size, error probabilities, evidence strength, and model misspecification become more naturally housed in an evidential setting. We provide a completely worked example featuring a two-way analysis of variance. Full article

Although large data sets are generally viewed as advantageous for their ability to provide more precise and reliable evidence, it is often overlooked that these benefits are contingent upon certain conditions being met. The primary condition is the approximate validity (statistical adequacy) of the probabilistic assumptions comprising the statistical model ${\mathcal{M}}_{\mathit{\theta }}\left(\mathbf{x}\right)$ applied to the data. In the case of a statistically adequate ${\mathcal{M}}_{\mathit{\theta }}\left(\mathbf{x}\right)$ and a given significance level $\alpha$, as n increases, the power of a test increases, and the p-value decreases due to the inherent trade-off between type I and type II error probabilities in frequentist testing. This trade-off raises concerns about the reliability of declaring ‘statistical significance’ based on conventional significance levels when n is exceptionally large. To address this issue, the author proposes that a principled approach, in the form of post-data severity (SEV) evaluation, be employed. The SEV evaluation represents a post-data error probability that converts unduly data-specific ‘accept/reject $H_0$ results’ into evidence either supporting or contradicting inferential claims regarding the parameters of interest. This approach offers a more nuanced and robust perspective in navigating the challenges posed by the large n problem. Full article

This paper considers the appearance of indications of useful acoustic signals in the signal/noise mixture. Various information characteristics (information entropy, Jensen–Shannon divergence, spectral information divergence and statistical complexity) are investigated in the context of solving this problem. Both time and frequency domains are studied for the calculation of information entropy. The effectiveness of statistical complexity is shown in comparison with other information metrics for different signal-to-noise ratios. Two different approaches for statistical complexity calculations are also compared. In addition, analytical formulas for complexity and disequilibrium are obtained using entropy variation in the case of signal spectral distribution. The connection between the statistical complexity criterion and the Neyman–Pearson approach for hypothesis testing is discussed. The effectiveness of the proposed approach is shown for different types of acoustic signals and noise models, including colored noises, and different signal-to-noise ratios, especially when the estimation of additional noise characteristics is impossible. Full article

Ocean Acoustic Waveguide Remote Sensing (OAWRS) enables fish population density distributions to be instantaneously quantified and continuously monitored over wide areas. Returns from seafloor geology can also be received as background or clutter by OAWRS when insufficient fish populations are present in any region. Given the large spatial regions that fish inhabit and roam over, it is important to develop automatic methods for determining whether fish are present at any pixel in an OAWRS image so that their population distributions, migrations and behaviour can be efficiently analyzed and monitored in large data sets. Here, a statistically optimal automated approach for distinguishing fish from seafloor geology in OAWRS imagery is demonstrated with Neyman–Pearson hypothesis testing which provides the highest true-positive classification rate for a given false-positive rate. Multispectral OAWRS images of large herring shoals during spawning migration to Georges Bank are analyzed. Automated Neyman-Pearson hypothesis testing is shown to accurately distinguish fish from seafloor geology through their differing spectral responses at any space and time pixel in OAWRS imagery. These spectral differences are most dramatic in the vicinity of swimbladder resonances of the fish probed by OAWRS. When such significantly different spectral dependencies exist between fish and geologic scattering, the approach presented provides an instantaneous, reliable and statistically optimal means of automatically distinguishing fish from seafloor geology at any spatial pixel in wide-area OAWRS images. Employing Kullback–Leibler divergence or the relative entropy in bits from Information Theory is shown to also enable automatic discrimination of fish from seafloor by their distinct statistical scattering properties across sensing frequency, but without the statistical optimal properties of the Neyman–Pearson approach. Full article

This paper establishes a hybrid distributed phased array multiple-input multiple-output (PA-MIMO) radar system model to improve the target detection performance by combining coherent processing gain and spatial diversity gain. First, the radar system signal model and array space configuration model for the PA-MIMO radar are established. Then, a novel likelihood ratio test (LRT) detector is derived based on the Neyman–Pearson (NP) criterion in a fixed noise background. It can jointly optimize the coherent processing gain and spatial diversity gain of the system by implementing subarray level and array element level optimal configuration at both receiver and transmitter ends in a uniform blocking manner. On this basis, three typical optimization problems are discussed from three aspects, i.e., the detection probability, the effective radar range, and the radar system equipment volume. The approximate closed-form solutions of them are constructed and solved by the proposed quantum particle swarm optimization-based stochastic rounding (SR-QPSO) algorithm. Through the simulations, it is verified that the proposed optimal configuration of the hybrid distributed PA-MIMO radar system offers substantial improvements compared to the other typical radar systems, detection probability of 0.98, and an effective range of 1166.3 km, which significantly improves the detection performance. Full article

Radar network configuration and power allocation are of great importance in military applications, where the entire surveillance area needs to be searched under resource budget constraints. To pursue the joint antenna placement and power allocation (JAPPA) optimization, this paper develops a JAPPA strategy to improve target detection performance in a widely distributed multiple-input and multiple-output (MIMO) radar network. First, the three variables of the problem are incorporated into the Neyman–Pearson (NP) detector by using the antenna placement optimization and the Lagrange power allocation method. Further, an improved iterative greedy dropping heuristic method based on a two-stage local search is proposed to solve the NP-hard issues of high-dimensional non-linear integer programming. Then, the sum of the weighted logarithmic likelihood ratio test (LRT) function is constructed as optimization criteria for the JAPPA approach. Numerical simulations and the theoretical analysis confirm the superiority of the proposed algorithm in terms of achieving effective overall detection performance. Full article

Channel-based physical-layer authentication, which is capable of detecting spoofing attacks in dual-hop wireless networks with low cost and low complexity, attracted a great deal of attention from researchers. In this paper, we explore the likelihood ratio test (LRT) with cascade channel frequency response, which is optimal according to the Neyman–Pearson theorem. Since it is difficult to derive the theoretical threshold and the probability of detection for LRT, majority voting (MV) algorithm is employed as a trade-off between performance and practicality. We make decisions according to the temporal variations of channel frequency response in independent subcarriers separately, the results of which are used to achieve a hypothesis testing. Then, we analyze the theoretical false alarm rate (FAR) and miss detection rate (MDR) by quantifying the upper bound of their sum. Moreover, we develop the optimal power allocation strategy between the transmitter and the relay by minimizing the derived upper bound with the optimal decision threshold according to the relay-to-receiver channel gain. The proposed power allocation strategy takes advantage of the difference of noise power between the relay and the receiver to jointly adjust the transmit power, so as to improve the authentication performance on condition of fixed total power. Simulation results demonstrate that the proposed power allocation strategy outperforms the equal power allocation in terms of FAR and MDR. Full article

A generalized likelihood ratio test (GLRT) with the constant false alarm rate (CFAR) property was recently developed for adaptive detection of moving targets in focusing synthetic aperture radar (SAR) images. However, in the multichannel SAR-ground moving-target indication (SAR-GMTI) system, image defocus is inevitable, which will remarkably degrade the performance of the GLRT detector, especially for the lower radar cross-section (RCS) and slower radial velocity moving targets. To address this issue, based on the generalized steering vector (GSV), an extended GLRT detector is proposed and its performance is evaluated by the optimum likelihood ratio test (LRT) in the Neyman-Pearson (NP) criterion. The joint data vector formulated by the current cell and its adjacent cells is used to obtain the GSV, and then the extended GLRT is derived, which coherently integrates signal and accomplishes moving-target detection and parameter estimation. Theoretical analysis and simulated SAR data demonstrate the effectiveness and robustness of the proposed detector in the defocusing SAR images. Full article

This paper investigates the detection of the transmitted power violation (TPV) in the satellite-terrestrial integrated network, where the terrestrial base station may break the spectrum policies so that severe damages are made to the satellite systems. Due to the lack of prior information on specific abnormal behaviors, this problem is complex and challenging. To tackle it, we first turn to the geolocation spectrum database based detecting framework, where not only the tasks of each segment but also the spectrum policies are specified. Then, the ternary hypothesis test and the generalized Neyman–Pearson (GMNP) test criterion are applied to maximize the detection probability under the false-alarm constraint. What is more, the Abnormal after Normal (AaN) detector is developed to simplify the analysis. Finally, simulations are conducted to demonstrate that the proposed detector can realize the detection of TPV in most cases at the expense of less than 10% detection probability. Full article

In social and health sciences, many statistical procedures and estimation techniques rely on the underlying distributional assumption of normality of the data. Non-normality may lead to incorrect statistical inferences. This study evaluates the performance of selected normality tests within the stringency framework for skewed alternative space. The stringency concept allows us to rank the tests uniquely. The Bonett and Seier test (T_w) turns out to represent the best statistics for slightly skewed alternatives and the Anderson–Darling (AD); Chen–Shapiro (CS); Shapiro–Wilk (W); and Bispo, Marques, and Pestana (BCMR) statistics are the best choices for moderately skewed alternative distributions. The maximum loss of Jarque–Bera (JB) and its robust form (RJB), in terms of deviations from the power envelope, is greater than 50%, even for large sample sizes, which makes them less attractive in testing the hypothesis of normality against the moderately skewed alternatives. On balance, all selected normality tests except T_w and Daniele Coin’s COIN-test performed exceptionally well against the highly skewed alternative space. Full article

This paper focuses on test procedures under corrupted data. We assume that the observations $Z_i$ are mismeasured, due to the presence of measurement errors. Thus, instead of $Z_i$ for $i=1,\dots ,n$, we observe $X_i=Z_i+\sqrt{\delta }{V}_i$, with an unknown parameter $\delta$ and an unobservable random variable $V_i$. It is assumed that the random variables $Z_i$ are i.i.d., as are the $X_i$ and the $V_i$. The test procedure aims at deciding between two simple hyptheses pertaining to the density of the variable $Z_i$, namely $f_0$ and $g_0$. In this setting, the density of the $V_i$ is supposed to be known. The procedure which we propose aggregates likelihood ratios for a collection of values of $\delta$. A new definition of least-favorable hypotheses for the aggregate family of tests is presented, and a relation with the Kullback-Leibler divergence between the sets ${\left(f_{\delta }\right)}_{\delta }$ and ${\left(g_{\delta }\right)}_{\delta }$ is presented. Finite-sample lower bounds for the power of these tests are presented, both through analytical inequalities and through simulation under the least-favorable hypotheses. Since no optimality holds for the aggregation of likelihood ratio tests, a similar procedure is proposed, replacing the individual likelihood ratio by some divergence based test statistics. It is shown and discussed that the resulting aggregated test may perform better than the aggregate likelihood ratio procedure. Full article

We study a binary hypothesis testing problem in which a defender must decide whether a test sequence has been drawn from a given memoryless source $P_0$ , while an attacker strives to impede the correct detection. With respect to previous works, the adversarial setup addressed in this paper considers an attacker who is active under both hypotheses, namely, a fully active attacker, as opposed to a partially active attacker who is active under one hypothesis only. In the fully active setup, the attacker distorts sequences drawn both from $P_0$ and from an alternative memoryless source $P_1$ , up to a certain distortion level, which is possibly different under the two hypotheses, to maximize the confusion in distinguishing between the two sources, i.e., to induce both false positive and false negative errors at the detector, also referred to as the defender. We model the defender–attacker interaction as a game and study two versions of this game, the Neyman–Pearson game and the Bayesian game. Our main result is in the characterization of an attack strategy that is asymptotically both dominant (i.e., optimal no matter what the defender’s strategy is) and universal, i.e., independent of $P_0$ and $P_1$ . From the analysis of the equilibrium payoff, we also derive the best achievable performance of the defender, by relaxing the requirement on the exponential decay rate of the false positive error probability in the Neyman–Pearson setup and the tradeoff between the error exponents in the Bayesian setup. Such analysis permits characterizing the conditions for the distinguishability of the two sources given the distortion levels. Full article

The main objective of this paper is to find the relation between the adaptive significance level presented here and the sample size. We statisticians know of the inconsistency, or paradox, in the current classical tests of significance that are based on p-value statistics that are compared to the canonical significance levels (10%, 5%, and 1%): “Raise the sample to reject the null hypothesis” is the recommendation of some ill-advised scientists! This paper will show that it is possible to eliminate this problem of significance tests. We present here the beginning of a larger research project. The intention is to extend its use to more complex applications such as survival analysis, reliability tests, and other areas. The main tools used here are the Bayes factor and the extended Neyman–Pearson Lemma. Full article