The Extreme Value Support Measure Machine for Group Anomaly Detection

Abstract

Group anomaly detection is a subfield of pattern recognition that aims at detecting anomalous groups rather than individual anomalous points. However, existing approaches mainly target the unusual aggregate of points in high-density regions. In this way, unusual group behavior with a number of points located in low-density regions is not fully detected. In this paper, we propose a systematic approach based on extreme value theory (EVT), a field of statistics adept at modeling the tails of a distribution where data are sparse, and one-class support measure machines (OCSMMs) to quantify anomalous group behavior comprehensively. First, by applying EVT to a point process model, we construct an analytical model describing the likelihood of an aggregate within a group with respect to low-density regions, aimed at capturing anomalous group behavior in such regions. This model is then combined with a calibrated OCSMM, which provides probabilistic outputs to characterize anomalous group behavior in high-density regions, enabling improved assessment of overall anomalous group behavior. Extensive experiments on simulated and real-world data demonstrate that our method outperforms existing group anomaly detectors across diverse scenarios, showing its effectiveness in quantifying and interpreting various types of anomalous group behavior.

1. Introduction

Anomaly detection is a critical task in data analysis that differs substantially from traditional classification. While classification typically involves the modeling of two or more classes with labeled data and comparable sample sizes, anomaly detection focuses on distinguishing between “normal” and “anomalous” samples, often characterized by an imbalanced class distribution, where anomalous samples are rare. A key assumption in anomaly detection is that the training data comprise only normal samples (throughout this paper, the term “normal” consistently signifies what is usual and regular. When discussing distributions, we employ the term “Gaussian distribution” instead of “normal distribution”) [1]. Let $\mathit{X}=(X_1,\dots ,X_d)$ be a multivariate random variable representing d feature components with distribution function $F_{\mathit{X}}$. Traditional approaches to anomaly detection focus on a point model ($M\left(\mathit{x}\right)$) in the feature space of $\mathit{X}$ that captures distribution characteristics of normal point samples (which typically reside in high-density regions) to identify anomalies deviating from this model (so-called pointwise anomaly detection).

Group anomaly detection extends pointwise anomaly detection to scenarios where a sample is not an individual point ($\mathit{x}$) but a group of points ($\tilde{\mathit{x}}=\{{\mathit{x}}_1,\dots ,{\mathit{x}}_n\}$) representing a sample of the group variable ($\tilde{\mathit{X}}=\{{\mathit{X}}_1,\dots ,{\mathit{X}}_n\}$, where ${\mathit{X}}_1,\dots ,{\mathit{X}}_n$ share a common distribution function ($F_{\mathit{X}}$)). Group anomalies can arise in diverse contexts, such as irregular sequences of measured heartbeats [2]. When applied to the detection of wind turbine failures, group anomalies are observed as shifts of signal sequences [3]. Group anomalies also appear in the usage patterns of urban bicycle rental systems [4]. Although pointwise anomaly detectors could theoretically be applied in group scenarios if a representative feature set for the group variable were extractable [5], these methods face the following significant challenges in practical applications: (i) Constructing a representative feature set that adequately encapsulates diverse group-level characteristics is often difficult, especially when the number of training groups is limited and the variable ($\mathit{X}$) comprising the group is high-dimensional. (ii) Achieving interpretability of the group-level feature set in terms of the spatial configuration of points in the original feature space of $\mathit{X}$ remains nontrivial. As a result, specialized methods for group anomaly detection have been developed, which bypass group feature extraction and allow for direct analysis within the original feature space of $\mathit{X}$.

This paper presents a statistical approach for group anomaly detection, aiming to construct a framework to analytically quantify various types of anomalous group behavior. Given normal groups in the training set, the points across all normal groups collectively define a common, potentially complex distribution with a density of $y=p\left(\mathit{x}\right)$. Thresholding the density ($p\left(\mathit{x}\right)$) partitions the feature space of $\mathit{X}$ into high-density and low-density regions, thereby enabling all possible anomalous group behavior to be systematically classified into two types: (i) point-based anomalous group behavior, where a group exhibits unusual collective behavior among its points labeled as belonging to low-density regions, and (ii) distribution-based anomalous group behavior, referring to a group whose distribution pattern of points labeled as belonging to high-density regions differs from that in any normal group in the training set. Such categorization is in line with existing work but is statistically underpinned through the density function expressed as $y=p\left(\mathit{x}\right)$ [6,7,8,9,10]. Figure 1 provides a visual illustration of the two types of group anomalies. The blue points in the two-dimensional input space represent points from all normal groups and collectively characterize the underlying point distribution, where densely clustered areas correspond to high-density regions and sparse areas indicate low-density regions. Figure 1a shows groups with point-based anomalous group behavior, where each group with red borders in the test set contains an unusual aggregate of points located in low-density regions. Figure 1b illustrates groups with distribution-based anomalous group behavior. Although the purple points are situated in high-density regions, the groups with purple borders exhibit distribution patterns that substantially differ from any of the patterns observed among the normal training groups. Groups exhibiting both types of behavior are considered anomalous on both fronts.

Existing models for group anomaly detection face two critical limitations, which motivate the development of our novel statistical approach. First, while a wide range of popular models excel at detecting distribution-based anomalous group behavior, they exhibit a significant weakness in identifying point-based anomalous group behavior. For example, models like the One-Class Support Measure Machine (OCSMM) [7], Support Measure Data Description (SMDD) [11], and the neural network model proposed by Chalapathy et al. [8] primarily assess how the distribution pattern of points (labeled as belonging to high-density regions) in a test group deviates from that of normal training groups while largely neglecting the crucial aggregation behavior of points in low-density regions within this test group. In domains like health monitoring, both types of group anomalies are crucial, and the failure to effectively detect point-based anomalous group behavior can lead to spurious alarms. Although models like the Mixture of Gaussian Mixture Model (MGMM) [6] and the Flexible Genre Model (FGM) [12] attempt to address both types of group anomalies, they often exhibit higher sensitivity towards distribution-based anomalous group behavior. This imbalance in sensitivity is further illustrated in Section 2 and Section 5.

The second limitation lies in modeling point-based anomalous group behavior, where the strategy of most models that combine (pointwise) anomaly scores is flawed. The main reason is that low-density region points need to be considered as a whole to capture their aggregate behavior; otherwise, the multiple hypothesis-testing problem will manifest. The point-based group anomaly detection problem can be stated as follows:

H₀.

$\tilde{\mathit{x}}$ is a group of points drawn from the distribution of $\mathit{X}$;

H₁.

$\tilde{\mathit{x}}$ is an anomalous group with respect to the distribution of $\mathit{X}$.

Clearly, it is a multiple hypothesis-testing problem where more than one hypothesis is being tested—more specifically, one hypothesis for each individual point being anomalous or not. For a group size of $n=1$, this simplifies to classic (pointwise) anomaly detection, where the likelihood ($p\left({\mathit{x}}_1\right)$) can be directly used as an anomaly score [13]. However, for $n>1$, the risk of wrongly classifying a group ($\tilde{\mathit{x}}$) as anomalous considerably increases. Indeed, while the type-I error for ${\mathit{x}}_i\in \tilde{\mathit{x}}$ can be controlled at a significance level of $\alpha$, the likelihood of making at least one type-I error among all n hypotheses corresponds to $P\left(\text{type-Ierror}\right)=1-{(1-\alpha )}^n>\alpha$. As a result, the decisions made by these models become unreliable.

This work aims to develop an effective method to address the aforementioned limitations and accurately evaluate the anomalousness of a group. We integrate a likelihood model based on Extreme Value Theory (EVT) with a calibrated OCSMM within a unified framework. Each component is designed to specifically target a distinct form of anomalous group behavior, with the likelihood model being employed to characterize the low-density regions and avoid the multiple hypothesis-testing problem, whereas the calibrated OCSMM is intended to capture deviations in the distributional structure of high-density regions across groups. Leveraging the complementary information captured by each component, these two components are subsequently combined through a constructed uninorm to generate an overall anomaly score. Specifically, the contributions of this paper are described as follows:

• We offer a novel framework where EVT is applied to a Point Process Model (PPM) in order to fully capture the spatial configuration that is hidden in the data points situated in low-density regions.
• An analytical result is proven to approximate the distribution of the PPM. Based on this result, a point-based anomaly score is defined to evaluate the anomalous extent of a group with respect to low-density regions, effectively addressing the multiple hypothesis-testing problem.
• We extend the existing OCSMM framework to yield probabilistic outputs, thereby enabling the construction of a distribution-based anomaly score that exclusively assesses the anomalous extent of a group with respect to high-density regions. OCSMM is chosen for this purpose because it effectively captures the distributional characteristics of normal groups in high-density regions, without redundantly incorporating information from low-density regions, thereby avoiding overlap with the previously constructed point-based anomaly score.
• A uninorm is constructed to effectively combine the above two scores, each focusing on the anomalous extent of one of the group’s two complementary aspects. The resulting overall anomaly score provides a comprehensive measure of the group’s anomalousness.

The remainder of this paper is structured as follows. In Section 2, existing approaches for group anomaly detection are reviewed and analyzed. Section 3 provides the necessary knowledge of EVT and PPM, which is crucial for constructing our model. Subsequently, Section 4 introduces our novel method for group anomaly detection. In Section 5, the method is evaluated on both synthetic and real-world datasets. Section 6 concludes this paper.

2. Related Work

As our study is concerned with group anomaly detection, we focus our literature review on this area rather than (pointwise) anomaly detection methods. Following classic pointwise anomaly detection, early developments in group anomaly detection methods combined the individual predicted scores generated by pointwise anomaly detectors into a single score for a statistical decision [14]. For instance, in the image analysis research of Hazel [15], a pointwise detector was used to identify anomalous pixels that were subsequently gathered into anomalous groups. Das et al. [16] classified groups according to a high fraction of individual anomalous points within a group. Belhadi et al. [17] combined different data mining techniques to identify groups of sequence outliers. Mohod and Janeja [18] adapted density-based clustering methods to discover spatially anomalous groups (windows) of arbitrary shape. Das et al. [19] also introduced a group anomaly detection framework based on scan statistics to detect clusters with increased counts of spatial data. More recently, Pehlivanian and Neill [20] extended scan statistics to detect contiguous group anomalies in ordered data. However, scan statistics are based on the number of instances within a region but discard their relative location in space. Moreover, these methods are ineffective in addressing the multiple hypothesis-testing problem.

With the advancement of statistical techniques, there is a growing interest in detecting unusual group behavior, especially in high-density regions. Some representative models have been proposed. Muandet and Schölkopf [7] introduced OCSMM, which extends the One-Class Support Vector Machine (OCSVM) [21] to groups. By adapting the kernel in OCSVM for the mapping of points into feature space to the kernel mean embedding mapping of groups into feature space, OCSMM extracts distribution information from normal groups and separates them from the origin. SMDD [11] also utilizes the kernel mean embedding technique, but it is based on the idea of constructing a minimum enclosing ball [22]. Ting et al. [23] proposed the isolation distribution kernel to quantify distribution similarities within groups. Fisch et al. [24] introduced a statistical approach for detecting both collective and point anomalies with a linear computational cost. Other approaches, such as those proposed by Yu et al. [25], Song et al. [9], and Chalapathy et al. [8], are tailored for specific application areas. However, these models consistently fail to handle scenarios where both point-based and distribution-based anomalous group behavior occur within the same group. Some efforts have been made to detect both types of anomalous group behavior, such as MGMM [6] and FGM [12], which are strongly related to the topic model of Latent Dirichlet Allocation (LDA) [26]. Topic models are often used in natural language processing to classify a collection of documents, each of which is typically concerned with multiple topics and where each topic is represented by a cluster of multiple words. MGMM considers each group as a mixture of Gaussian-distributed topics, while FGM defines flexible structures called “genres” to model topic distributions. Scores are then defined to measure point-based and distribution-based anomalous group behavior and combined using a weight function. However, these models tend to focus primarily on the distribution of points in high-density regions, as the topic generation process is less affected by points in low-density regions, particularly when these points are sparse. As a result, they exhibit significant bias in detecting point-based anomalous group behavior.

In recent years, to develop a suitable model for point-based anomalous group behavior, researchers have explored considering low-density region points as an integral component. Luca et al. [2,27,28] investigated the use of EVT to extract features characterizing the aggregate of points in low-density regions (also termed extreme risk regions) of $p\left(\mathit{x}\right)$. This involves summarizing aggregation information from extreme points in terms of the mean, maximum excess of $p\left(\mathit{x}\right)$, or the number of extremes with respect to some threshold. While these models address the multivariate hypothesis-testing problem by employing such features to characterize aggregated extreme behavior, questions remain about how to identify a comprehensive feature set that effectively captures this behavior.

3. Background Knowledge

In this section, the necessary concepts of EVT and the related PPM are introduced, which play an essential role in the remainder of the paper. For more details about the proofs of the presented theorems, we refer to [29]. The key notations and abbreviations used throughout this paper are summarized in

Table 1. Summary of notations and abbreviations.

Symbol/Abbreviation	Definition
$\mathit{X}$,   $\mathit{x}$	A d-dimensional random variable and a sample thereof
$p\left(\mathit{x}\right)$	Estimated probability density at point $\mathit{x}$
Y,   y	A 1-dimensional random variable defined as $Y=p\left(\mathit{X}\right)$ and a sample thereof
Z,   z	A 1-dimensional random variable defined as $Z=-logY=-logp\left(\mathit{X}\right)$ and a sample thereof
$\tilde{\mathit{x}}=\{{\mathit{x}}_1,\dots ,{\mathit{x}}_n\}$	A group sample consisting of n point samples
$\tilde{\mathit{y}}=\{y_1,\dots ,y_n\}$	A transformed group sample obtained from $\tilde{\mathit{x}}$, where $y_i=p\left({\mathit{x}}_i\right)$
$\tilde{\mathit{z}}=\{z_1,\dots ,z_n\}$	A transformed group sample obtained from $\tilde{\mathit{x}}$, where $z_i=-logp\left({\mathit{x}}_i\right)$
$u_n$	Threshold used to determine exceedances for Z
$c_n$	Location parameter for the right tail of Z
$d_n$	Scale parameter for the right tail of Z
u	Normalized $u_n$, where $u={\displaystyle \frac{u_n-c_n}{d_n}}$
${\tilde{\mathit{z}}}^{\mathrm{exc}}$	Set of exceedances within group $\tilde{\mathit{z}}$
$f_n\left({\tilde{\mathit{z}}}^{\mathrm{exc}}\right)$	Likelihood of the exceedance pattern in group $\tilde{\mathit{z}}$
$V_n^{\mathrm{exc}}$	Likelihood variable, where $V_n^{\mathrm{exc}}=f_n\left({\tilde{\mathit{z}}}^{\mathrm{exc}}\right)$
$G\left(v^{\mathrm{exc}}\right)$	Analytical cumulative distribution function of the likelihood, where $G\left(v^{\mathrm{exc}}\right)=\underset{n\to +\infty }{lim}P\left(V_n^{\mathrm{exc}}\le v^{\mathrm{exc}}\right)$
$\mathbb{B}=\left\{{\tilde{\mathit{x}}}_{l+1},\dots ,{\tilde{\mathit{x}}}_{l+h}\right\}$	A set of h simulated anomalous groups used for calibration
$\gamma$	Kernel bandwidth in OCSMM
$U_{3\Pi }(a,b)$	$3\Pi$ operator
$U_{\mathit{a}no}(a,b)$	A constructed uninorm used in the final combination
e	Threshold in uninorm $U_{\mathit{a}no}(a,b)$
$\mathbb{A}=\left\{{\tilde{\mathit{x}}}_1,\dots ,{\tilde{\mathit{x}}}_l\right\}$	A training set of groups consisting of l normal group samples
$\mathbb{T}=\{{\tilde{\mathit{x}}}_1^{\left(t\right)},\dots ,{\tilde{\mathit{x}}}_{l^{\prime }}^{\left(t\right)}\}$	A test set consisting of $l^{\prime }$ group samples
$\alpha \left(\tilde{\mathit{x}}\right)$	Point-based group anomaly score for group sample $\tilde{\mathit{x}}$
$\beta \left(\tilde{\mathit{x}}\right)$	Distribution-based group anomaly score for group sample $\tilde{\mathit{x}}$
$\varphi \left(\tilde{\mathit{x}}\right)$	EVSMM score for group sample $\tilde{\mathit{x}}$
EVT	Extreme Value Theory
OCSVM	One-Class Support Vector Machine
OCSMM	One-Class Support Measure Machine
PPM	Point Process Model
SMDD	Support Measure Data Description
MGMM	Mixture of Gaussian Mixture Model

.

3.1. Extreme Value Theory

EVT is a statistical discipline with the objective of modeling the stochastic behavior of a univariate process at unusually large (or small) levels. It has already been proven useful in many applications, such as reliability analysis, structural health monitoring, meteorology, and financial risk assessment [30,31,32].

The central result in EVT is the Fisher–Tippett theorem concerning the limiting distribution of

$$M_n=max\{Z_1,Z_2,\dots ,Z_n\},\mathrm{as}n\to +\infty ,$$

where $Z_1,Z_2,\dots ,Z_n$ are i.i.d. univariate random variables with a common distribution function ($F_Z$).

Theorem 1 (Fisher-Tippet-Gnedenko).

If there exist sequences of normalizing constants $c_n$ and $d_n$ such that the following convergence in distribution holds:

$$P\left({\displaystyle \frac{M_n-c_n}{d_n}}\le z\right)\to G_{\xi }\left(z\right),asn\to +\infty ,$$

(1)

then the non-degenerate limiting distribution ($G_{\xi }\left(z\right)$) is a member of the so-called Generalized Extreme Value (GEV) family of distributions:

$$G_{\xi }\left(z\right)=\left\{\begin{array}{cc}exp\left\{-{\left[1+\xi z\right]}^{-{\displaystyle \frac{1}{\xi }}}\right\}\hfill & ,if\xi \ne 0\hfill \\ exp\{-exp(-z\left)\right\}\hfill & ,if\xi =0.\hfill \end{array}\right.$$

(2)

If $\xi \ne 0$, then the domain of the distribution is restricted to the set expressed as $\{z\mid 1+\xi z>0\}$.

If the shape parameter ($\xi$) is negative, zero, or positive, then the corresponding distributions ($G_{\xi }$) belongs to the Weibull, Gumbel, or Fréchet family, respectively. Thus, the shape parameter ($\xi$) thus determines the behavior in the tails of the distribution ($F_Z$).

The GEV family provides a model for block maxima, i.e., maxima of a sequence $(\{Z_1,Z_2,\dots ,Z_n\}$). However, for relatively large n values, few block maxima are available from the data, which can lead to unreliable estimates of $\xi$ [30]. To overcome this problem, the peaks-over-threshold method can be used, which avoids the need for blocking. This method infers a model from all observations that exceed some high threshold.

Theorem 2 (Pickands-Balkema-De Haan).

For a given $u\in \mathbb{R}$ sufficiently close to 0, the sequence of thresholds expressed as $u_n=c_n+u{d}_n$ is considered, where $c_n$ and $d_n$ denote the normalizing constants as defined in (1). If (1) holds for some member ($G_{\xi }\left(z\right)$) of the GEV family, then the distribution of the exceedances ($Z-u_n$), conditional on $Z>u_n$, satisfies the following limiting property:

$$\underset{n\to +\infty }{lim}P\left({\displaystyle \frac{Z-u_n}{a\left(u_n\right)}}<z\mid Z>u_n\right)=H_{\xi }\left(z\right),$$

(3)

where $a\left(u_n\right)$ is the scale parameter for exceedances relative to the threshold ($u_n$). The distribution expressed as

$$H_{\xi }\left(z\right)=\left\{\begin{array}{cc}1-{\left(1+{\displaystyle \frac{\xi }{1+\xi u}}z\right)}^{-{\displaystyle \frac{1}{\xi }}}\hfill & ,if\xi \ne 0\hfill \\ 1-e^{-z}\hfill & ,if\xi =0\hfill \end{array}\right.$$

(4)

denotes the family of Generalized Pareto Distributions (GPDs) with $z\ge 0$ in the case of $\xi \ge 0$ and $0\le z\le -{\displaystyle \frac{1}{\xi }}$ in the case of $\xi <0$.

These two classical models represent the cornerstone of Extreme Value Theory. When the prior distribution of Z is unknown, the three parameters, i.e., the shape parameter ($\xi$) and the normalizing constants ($c_n$ and $d_n$), need to be estimated for practical applications. For a range of commonly encountered distributions, such as the gamma, exponential, Gaussian, and beta distributions, Embrechts et al. [29] presented direct sample estimation formulas for the $\xi$, $c_n$, and $d_n$ parameters. In cases where populations have the extreme value distribution of the Gumbel case ($\xi =0$), the scale parameter ($a\left(u_n\right)=d_n$) and the normalizing constants ($c_n$ and $d_n$) can be estimated as follows [29]:

$$c_n=inf\left\{z\mid P(Z\le z)\ge 1-{\displaystyle \frac{1}{n}}\right\},d_n=E(Z-c_n\mid Z>c_n),$$

(5)

where the latter is the mean excess function.

3.2. Point Process Model

A (spatial) PPM in Euclidean space can be viewed as a random variable ($\tilde{\mathit{X}}$) defined over all possible point patterns in some subspace ($\mathcal{D}$) of ${\mathbb{R}}^d$ ($d\in {\mathbb{N}}_0$), where a realized point pattern is given by [33] $\tilde{\mathit{x}}=\{{\mathit{x}}_1,{\mathit{x}}_2,\dots \},{\mathit{x}}_i\in {\mathbb{R}}^d.$ A PPM integrates stochastic information about the number of points and their individual locations in the feature space. It can be shown that the stochastic properties of a PPM are fully characterized by so-called counting measures with respect to the corresponding subsets [34], which are obtained by mapping a pattern ($\tilde{\mathit{x}}$) to the number of points that fall in a subset (A) of ${\mathbb{R}}^d$:

$$N_A\left(\tilde{\mathit{x}}\right)=\sum _{i\ge 1}{\chi }_A\left({\mathit{x}}_i\right),$$

(6)

where ${\chi }_A\left({\mathit{x}}_i\right)=1$ for ${\mathit{x}}_i\in A$ and ${\chi }_A\left({\mathit{x}}_i\right)=0$ for ${\mathit{x}}_i\notin A$. An important concept associated with a counting measure is its expectation ($\mathrm{\Lambda }\left(A\right)=\mathrm{E}\left(N_A\left(\tilde{\mathit{X}}\right)\right)$ ). Clearly, a realization ($\tilde{\mathit{x}}$) can be fully reconstructed by considering the counting measures corresponding to all possible subsets (A) of ${\mathbb{R}}^d$.

The PPM can be used to elegantly characterize the extremes in a way that unifies the models presented in Theorems 1 and 2 [30].

Theorem 3 (Point Process Characterization).

When the limiting distribution (3) of the block maxima of the univariate random variable (Z) holds, the PPM of patterns consisting of exceedances of the block $\{Z_1,\dots ,Z_n\}$ with respect to the threshold ($u_n$), i.e.,

$${\tilde{Z}}_n^{exc}=\{Z_1,Z_2,\dots ,Z_n\}\cap [u_n,+\infty ),$$

converges to a Poisson point process as $n\to +\infty$. The corresponding counting measure ($N_A\left({\tilde{Z}}_n^{exc}\right)$) converges in distribution to a Poisson distribution:

$$\begin{array}{c}\underset{n\to +\infty }{lim}P(N_A\left({\tilde{Z}}_n^{exc}\right)=k)={\displaystyle \frac{{\lambda }^k}{k!}}{e}^{-\lambda },\end{array}$$

(7)

where $\lambda =\mathrm{E}\left(N_A\left({\tilde{Z}}_n^{exc}\right)\right)$ is the expectation of the counting measure.

For the Gumbel case, where $\xi =0$, it can be shown that $\lambda =\underset{n\to +\infty }{lim}nP(Z>u_n)=e^{-u}$.

4. A Statistical Framework for Modeling Group Behavior

This section introduces the Extreme Value Support Measure Machine (EVSMM), a novel statistical method for comprehensively quantifying how anomalous a group is compared to normal training groups. The foundation of EVSMM involves characterizing the group-level normal behavior of normal training groups within the d-dimensional point feature space (${\mathbb{R}}^d$). We quantitatively analyze group behavior by modeling the statistical characteristics of two complementary parts within the group: the low-density region and the high-density region. The EVSMM operates through a two-step process. First, two scores are constructed to quantify how unusual the parts of a group situated in the low-density and high-density regions are. Second, an effective way to combine these two scores is proposed by using a constructed uninorm aggregation function, whose roots trace back to the fusion of certainty factors in highly successful early-day expert systems. The following sections detail the rationale and necessity behind the construction steps of EVSMM.

4.1. Two Key Scores

We start with the construction of two scores to quantify the anomaly extent of a group’s low-density and high-density regions.

4.1.1. PPM of Exceedances

In this section, we propose a PPM framework that models the spatial configuration of exceedances with respect to a decision boundary in its full generality. We then derive an asymptotic distribution, valid as $n\to +\infty$, which is used as an approximation to model the aggregate of exceedances.

Let $\mathit{X}$ be a d-dimensional random variable, with no distributional assumptions imposed on $\mathit{X}$. Consider a training set of normal groups ($\mathbb{A}=\left\{{\tilde{\mathit{x}}}_1,\dots ,{\tilde{\mathit{x}}}_l\right\}$), where each group (${\tilde{\mathit{x}}}_{\ell }=\{{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_{n_{\ell }}\}\subset {\mathbb{R}}^d$) has a varying size ($n_{\ell }$), with $\mathrm{E}\left(n_{\ell }\right)=n$. The objective is to comprehensively quantify the unusual behavior of a group with respect to low-density regions, which entails characterizing the aggregate behavior of exceedances, as determined by a certain number of exceedances, and their various locations within low-density regions. The aggregate of exceedances within group $\tilde{\mathit{x}}$ is given by

$$\{\mathit{x}\in \tilde{\mathit{x}}\mid p\left(\mathit{x}\right)\le e^{-u_n}\}$$

for large values of $u_n$. The low-density regions are defined by the decision boundary ($p\left(\mathit{x}\right)\le e^{-u_n}$, where $u_n$ is the threshold in the peak over threshold model introduced in Theorem 2).

As previously described, this study begins by analyzing the distribution of points across all training groups, which describes the probability space of d-dimensional variable $\mathit{X}$. As our framework is not restricted to a particular density estimator, any suitable method could be adopted to estimate the probability density function ($p\left(\mathit{x}\right)$). To illustrate this flexibility, We mention several representative alternatives, such as parametric models like Gaussian Mixture Models (GMMs), non-parametric approaches like Projection Pursuit Density Estimation (PPDE), and random projection-based techniques for high-dimensional data. In our experiments, we adopt Kernel Density Estimation (KDE), a widely used non-parametric method that does not require prior assumptions about the data distribution. We explicitly clarify that KDE is used solely as an example and is not necessarily the optimal choice for all scenarios. Next, the transformation of $Y=p\left(\mathit{X}\right)$ is applied, mapping the multivariate random variable ($\mathit{X}$) to the univariate random variable (Y). Consequently, within each group ($\tilde{\mathit{x}}$), sample ${\mathit{x}}_i$ is transformed into sample $y_i$ within $\tilde{\mathit{y}}$ as follows:

$$\tilde{\mathit{y}}=\{y_1,y_2,\dots ,y_n\}=\left\{p\left({\mathit{x}}_1\right),p\left({\mathit{x}}_2\right),\dots ,p\left({\mathit{x}}_n\right)\right\}.$$

Therefore, the exceedances of $\tilde{\mathit{x}}$, whether they are extremely large (the right tail) or small (the left tail) values in the case of a univariate X or occupy diverse locations within low-density regions in the case of a multivariate $\mathit{X}$, correspond solely to the left tail of Y. As Y is the probability density, it is bounded below by zero, which guarantees the existence of an extreme value distribution in its left tail. Moreover, Clifton et al. [35] proved that the distribution of minima ($y^{\mathrm{ext}}:=min\{y_1,y_2,\dots ,y_n\}$) can be approximated by a Weibull distribution. A Weibull distribution, with a lower bound at zero, describes the behavior of small values of densities near zero [30]. To correct the skewness in the distribution of Y near zero, a logarithmic transformation ($Z=-log\left(Y\right)$) is applied:

$$\tilde{\mathit{z}}=\{z_1,z_2,\dots ,z_n\}=\left\{-log\left(y_1\right),-log\left(y_2\right),\dots ,-log\left(y_n\right)\right\}$$

mapping the short tail of the Weibull distribution near zero to the right tail of a Gumbel distribution for maxima:

$$P\left(max\{Z_1,\dots ,Z_n\}\le z\right)\approx G\left(z\right),\mathrm{as}n\to +\infty ,$$

where $G\left(z\right)$ denotes a cumulative distribution within the Gumbel family. In this way, the multivariate, complex, low-density regions ($\{\mathit{x}\mid p\left(\mathit{x}\right)\le e^{-u_n}\}$) are transformed into regions $\{z\mid z\ge u_n\}$ on the real line. The significance of transforming $\mathit{X}$ to Z not only lies in mapping low-density regions from a high-dimensional space to the right tail in a one-dimensional space but also in making it possible to exploit the well-established theoretical results for the Gumbel case ($\xi =0$) in EVT. Within a group ($\tilde{\mathit{z}}=\{z_1,z_2,\dots ,z_n\}$) of n observations of Z, we study those points that exceed the threshold ($u_n$) and consider them as an aggregate:

$${\tilde{\mathit{z}}}^{\mathrm{exc}}=\left\{z_1^{\mathrm{exc}},\dots ,z_{K_n}^{\mathrm{exc}}\right\},$$

where $K_n$ denotes the number of exceedances in $\tilde{\mathit{z}}$ w.r.t. $u_n$. In what follows, we derive an analytical result to determine the asymptotic distribution of such aggregate of exceedances as $n\to +\infty$. Note that, although i.i.d. random variables are considered, the results can be applied to time series data by considering the residuals after fitting a time series model.

To tackle the above problem in its full generality, ${\tilde{\mathit{z}}}^{\mathrm{exc}}$ is viewed as a realized non-empty point pattern of a PPM. First, the probabilistic model of individual exceedances is derived using the limiting property (3) in Theorem 2 for $\xi =0$ (i.e., the Gumbel case). As a result, for large n, the likelihood of an exceedance $z_i^{\mathrm{exc}}$ can be approximated by:

$$p_n\left(z_i^{\mathrm{exc}}\right)\approx {\displaystyle \frac{1}{d_n}}{e}^{-{\displaystyle \frac{z_i^{\mathrm{exc}}-u_n}{d_n}}},$$

(8)

where $u_n=c_n+u{d}_n$ and $c_n$ and $d_n$ are defined in (5). Second, the aggregating behavior of these exceedances is analyzed. The aggregate of exceedances with respect to low-density regions is fully characterized by the counting measure introduced in Section 3.2. From Section 3.1, it follows that

$$P(K_n=k)\approx {\displaystyle \frac{{\lambda }^k}{k!}}{e}^{-\lambda },$$

where $\lambda =e^{-u}$ and $u\ge 0$. These approximations motivate the definition of a probability measure, referred to as the PPM of exceedances, with the following likelihood:

$$f_n\left({\tilde{\mathit{z}}}^{\mathrm{exc}}\right):=k!{\mu }_k^n\prod _{i=1}^k{\displaystyle \frac{1}{d_n}}{e}^{-{\displaystyle \frac{z_i^{\mathrm{exc}}-u_n}{d_n}}},$$

(9)

where k denotes the observed size of ${\tilde{\mathit{z}}}^{\mathrm{exc}}$ and ${\mu }_k^n=P(K_n=k)$. In the theorem below, we prove that the distribution of the likelihoods ($f_n\left({\tilde{\mathit{z}}}^{\mathrm{exc}}\right)$) is asymptotically given by a random variable ($V^{\mathrm{exc}}$) of mixed type. Random variables of mixed type are neither discrete nor continuous but are a mixture of both [36]. The discrete component results from the likelihoods (${\mu }_0^n>0$) that describe cases when there are no exceedances in $\tilde{\mathit{z}}$. This corresponds to a discontinuity in the CDF that can be described using the Heaviside step function:

$$H\left(v\right)=\left\{\begin{array}{cc}1\hfill & ,\mathrm{if}v\ge 0\hfill \\ 0\hfill & ,\mathrm{if}v<0.\hfill \end{array}\right.$$

(10)

Theorem 4.

Consider a random variable (Z) that satisfies the following limiting property:

$$\underset{n\to +\infty }{lim}P\left({\displaystyle \frac{Z-u_n}{d_n}}<z\mid Z>u_n\right)=1-e^{-z},$$

(11)

where $u_n=c_n+u{d}_n$, with $u\ge 0$, is determined by a sequence of thresholds in which

$$\begin{array}{cc}\hfill & c_n=inf\left\{z\mid P(Z\le z)\ge 1-{\displaystyle \frac{1}{n}}\right\},\hfill \\ \hfill & d_n=E(Z-c_n\mid Z>c_n).\hfill \end{array}$$

(12)

The random variables ($V_n^{\mathrm{exc}}$) of likelihoods ($f_n\left({\tilde{\mathit{z}}}^{\mathrm{exc}}\right)$), as defined in (9), converge in distribution to a random variable ($V^{\mathrm{exc}}$) with the CDF:

$$\begin{array}{cc}\hfill G\left(v^{\mathrm{exc}}\right)={\mu }_0& H(v^{\mathrm{exc}}-{\mu }_0)+\sum _{k\ge 1}{\mu }_k\left[1-{\mathcal{E}}_{k,1}\left(-log{\displaystyle \frac{v^{\mathrm{exc}}{\left(d_n\right)}^k}{k!{\mu }_k}}\right)\right],\hfill \end{array}$$

(13)

where ${\mathcal{E}}_{k,1}$ denotes the CDF of an Erlang distribution with shape parameter k and scale 1 and ${\mu }_k={\displaystyle \frac{{\lambda }^k}{k!}}{e}^{-\lambda }$. In particular, it holds that

$$\underset{n\to +\infty }{lim}P\left(V_n^{\mathrm{exc}}\le v^{\mathrm{exc}}\right)=G\left(v^{\mathrm{exc}}\right).$$

(14)

Proof. 

First, we determine the distribution of $W_n^{\mathrm{exc}}=-log\left(V_n^{\mathrm{exc}}\right)$ given the number of exceedances ($K_n=k$). We have

$$\begin{array}{cc}\hfill W^{\mathrm{exc}}& =-log\left(f_n\left({\tilde{z}}^{\mathrm{exc}}\right)\right)\hfill \\ \hfill & =-log(k!{\mu }_k^n)+klog\left(d_n\right)+\sum _{i=1}^k{\displaystyle \frac{z_i^{\mathrm{exc}}-u_n}{d_n}}.\hfill \end{array}$$

According to (11), the rescaled exceedances (${\displaystyle \frac{Z_i^{\mathrm{exc}}-u_n}{d_n}}$) converge in distribution to an exponential random variable with a scale of $\sigma =1$ as $n\to +\infty$. Therefore, according to the continuous mapping theorem (stating that convergence is preserved by a continuous transformation [29]), the sum of k such independent exceedances converges to the distribution of a sum of k exponential random variables with a scale of $\sigma =1$. Thus, the limiting distribution of $W_n^{*}:=W_n^{\mathrm{exc}}+log(k!{\mu }_k^n)-klog\left(d_n\right)$, conditioned on $K_n=k>0$, is given by an Erlang distribution with shape parameter k and rate parameter 1.

The cumulative distribution function of $W_n^{\mathrm{exc}}$ can be found using the law of total probability:

$$\begin{array}{cc}\hfill \overline{G}\left(w^{\mathrm{exc}}\right)=& \underset{n\to +\infty }{lim}P\left(W_n^{\mathrm{exc}}<w^{\mathrm{exc}}\right)\hfill \\ \hfill =& \underset{n\to +\infty }{lim}\sum _{k\ge 0}{\mu }_k^{n}P\left(W_n^{\mathrm{exc}}<w^{\mathrm{exc}}\mid k\right)\hfill \\ \hfill =& \sum _{k\ge 0}{\mu }_k\underset{n\to +\infty }{lim}P\left(W_n^{*}<w^{\mathrm{exc}}+log(k!{\mu }_k^n)-klog\left(d_n\right)\mid k\right)\hfill \\ \hfill =& {\mu }_{0}H\left(w^{\mathrm{exc}}+log\left({\mu }_0\right)\right)+\sum _{k\ge 1}{\mu }_k{\mathcal{E}}_{k,1}\left(w^{\mathrm{exc}}+log(k!{\mu }_k)-klog\left(d_n\right)\right).\hfill \end{array}$$

(15)

The latter part is the limiting CDF of $W_n^{\mathrm{exc}}$, which we denote as $\overline{G}\left(w^{\mathrm{exc}}\right)$. Transforming back to the original distribution by means of $V^{\mathrm{exc}}=e^{-W^{\mathrm{exc}}}$, one obtains the desired result for $G\left(v^{\mathrm{exc}}\right)$:

$$\begin{array}{cc}\hfill & G\left(v^{\mathrm{exc}}\right)=\underset{n\to +\infty }{lim}P\left(V_n^{\mathrm{exc}}\le v^{\mathrm{exc}}\right)\hfill \\ \hfill & =1-\overline{G}\left(-log\left(v^{\mathrm{exc}}\right)\right)\hfill \\ \hfill & =\sum _{k\ge 0}{\mu }_k-[{\mu }_{0}H\left(-log\left(v^{\mathrm{exc}}\right)+log\left({\mu }_0\right)\right)+\sum _{k\ge 1}{\mu }_k{\mathcal{E}}_{k,1}\left(-log\left(v^{\mathrm{exc}}\right)+log(k!{\mu }_k)-klog\left(d_n\right)\right)]\hfill \\ \hfill & ={\mu }_0\left(1-H\left(-log\left(v^{\mathrm{exc}}\right)+log\left({\mu }_0\right)\right)\right)+\sum _{k\ge 1}{\mu }_k\left(1-{\mathcal{E}}_{k,1}\left(-log\left(v^{\mathrm{exc}}\right)+log(k!{\mu }_k)-klog\left(d_n\right)\right)\right)\hfill \\ \hfill & ={\mu }_{0}H\left(v^{\mathrm{exc}}-{\mu }_0\right)+\sum _{k\ge 1}{\mu }_k\left[1-{\mathcal{E}}_{k,1}\left(-log{\displaystyle \frac{v^{\mathrm{exc}}{\left(d_n\right)}^k}{k!{\mu }_k}}\right)\right].\hfill \end{array}$$

   □

As a brief summary, we propose the likelihood measure ($v^{\mathrm{exc}}$) to evaluate group samples. The likelihood measure of a group is constructed based on the pattern of its exceedances—taking into account both the individual locations of low-density regions and their aggregate behavior. The derived analytical form ($G\left(v^{\mathrm{exc}}\right)$) describes the distribution of this likelihood measure and plays a central role in quantifying the group’s probabilistic anomaly extent. Notably, similar to the Central Limit Theorem, a key advantage of EVT—the foundation of our framework—is that, while the asymptotic theory requires large n values, in practice, the method often performs well even for moderate values of n. To validate the analytical expression in (13), we conducted a simulation experiment. For this purpose, we generated 2000 groups from a one-dimensional standard Gaussian distribution ($N(0,1)$), with each group size sampled from a Poisson distribution with a mean of 50. In practice, such group size selection entails a trade-off between bias and variance, as a small group size leads to an estimation bias, whereas a large group size results in an increased estimation variance. Next, the likelihood ($v^{\mathrm{exc}}=f_n\left({\tilde{\mathit{z}}}^{\mathrm{exc}}\right)$) is calculated for each group with respect to an aggregate of exceedances, and the CDF of the likelihood is examined. The empirical CDF of $v^{\mathrm{exc}}$ is then compared with the asymptotic CDF given by (13). As shown in Figure 2, the empirical CDF approximates the analytical CDF quite well.

Based on the above analytical result, the first score is constructed using the expression of CDF $G\left(v^{\mathrm{exc}}\right)$ in (13), which assesses the anomaly extent of a group ($\tilde{\mathit{x}}=\left\{{\mathit{x}}_1,\dots ,{\mathit{x}}_n\right\}$) with respect to low-density regions. Addressing the multiple-hypothesis problem is critical, because as the number of points forming a group increases, the exceedances are expected to become more extreme, and their number is expected to grow. The proposed likelihood concept encompasses both the probability of each individual exceedance location and the probability of the size of the aggregate of exceedances within a group. Through the likelihood concept, all exceedances within a group are treated as a unified whole. Accordingly, the first score, termed the point-based group anomaly score, is defined as follows:

$$\alpha \left(\tilde{\mathit{x}}\right)=1-G\left(v^{\mathrm{exc}}\right),$$

(16)

where $v^{\mathrm{exc}}$ denotes the likelihood, as defined in (9), of the aggregate of exceedances ($z_i=-logp\left({\mathit{x}}_i\right)$) with respect to the threshold ($u_n$). A high anomaly score ($\alpha \left(\tilde{\mathit{x}}\right)$) indicates that there is a small probability of observing a group with a lower likelihood of the aggregate of exceedances than $v^{\mathrm{exc}}$ in any other group. We summarize the practical implementation of the model in Algorithm 1.

4.1.2. A Calibrated OCSMM Model

In this section, we construct the distribution-based group anomaly score by calibrating the OCSMM to quantify the anomaly extent of the group with respect to high-density regions. Basically, the OCSMM extends OCSVM by classifying groups ($\tilde{\mathit{x}}$) instead of individual points ($\mathit{x}$) [37]. The core concept behind OCSMM involves applying the kernel trick to map the mean embeddings of training groups into a feature space, where they are separated from the origin by a hyperplane. For more details about OCSMM, we refer to Appendix A.

Algorithm 1 PPM of exceedances: point-based group anomaly score.

Input: A set of normal groups $\mathbb{A}=\left\{{\tilde{\mathit{x}}}_1,\dots ,{\tilde{\mathit{x}}}_l\right\}$; a test group set $\mathbb{T}=\{{\tilde{\mathit{x}}}_1^{\left(t\right)},\dots ,{\tilde{\mathit{x}}}_{l^{\prime }}^{\left(t\right)}\}$; the expected group size n; Output: The point-based anomaly score $\alpha \left(\mathbb{T}\right)$ of the test group set $\mathbb{T}$. 1: Select an appropriate density estimator to estimate $p\left(\mathit{x}\right)$ based on all point samples $\mathit{x}$ from the normal groups in $\mathbb{A}$. Dimensionality reduction may optionally be applied prior to estimation, depending on the chosen estimator. 2: Apply the transformations $Y=p\left(\mathit{X}\right)$ and $Z=-log\left(Y\right)$ to all point samples $\mathit{x}$ (collected from the normal groups in $\mathbb{A}$), yielding samples z of the variable Z. 3: Use the mean residual life plot based on the sample z to select a threshold $u_n$ for the variable Z. 4: Estimate the location parameter $c_n$ and scale parameter $d_n$ based on samples z using Equation (5), and compute the normalized threshold $u={\displaystyle \frac{u_n-c_n}{d_n}}$. 5: for each ${\tilde{\mathit{x}}}_j^{\left(t\right)}$ in the test group set $\mathbb{T}$ do 6:    Transform each point ${\mathit{x}}_i\in {\tilde{\mathit{x}}}_j^{\left(t\right)}$ to $z_i=-logp\left({\mathit{x}}_i\right)$, and obtain ${\tilde{\mathit{z}}}_j^{\left(t\right)}$. 7:    Extract the exceedances ${\tilde{\mathit{z}}}^{\mathrm{exc}}=\{z_i\in {\tilde{\mathit{z}}}_j^{\left(t\right)}:z_i>u_n\}$ and compute the likelihood $v^{\mathrm{exc}}=f_n\left({\tilde{\mathit{z}}}^{\mathrm{exc}}\right)$ using Equation (9). 8:    According to Theorem 4, compute the point-based anomaly score as $\alpha \left({\tilde{\mathit{x}}}_j^{\left(t\right)}\right)=1-G\left(v^{\mathrm{exc}}\right)$. 9: end for 10: Obtain the point-based group anomaly scores $\alpha \left(\mathbb{T}\right)$ for the test set as $\alpha \left(\mathbb{T}\right)=\{\alpha \left({\tilde{\mathit{x}}}_1^{\left(t\right)}\right),\dots ,\alpha \left({\tilde{\mathit{x}}}_{l^{\prime }}^{\left(t\right)}\right)\}$.

There are several key reasons for selecting OCSMM. First, it is specifically designed to capture the distribution characteristics within each normal group and demonstrates robustness to outliers in low-density regions, providing a stable and reliable representation of distribution information for the training group set with respect to high-density regions. This capability complements the model proposed in the previous section, which focuses on low-density regions. Second, OCSMM leverages the kernel trick, enabling efficient modeling of the decision boundaries for complex group structures. By using the mean kernel embeddings, OCSMM computes inner products in high-dimensional spaces without requiring explicit group feature transformations. Third, OCSMM is firmly grounded in statistical learning theory, kernel mean embedding theory, and optimization theory. Its formulation as a convex optimization problem, with clearly defined constraints, ensures that a unique global optimum can be reliably determined.

Although OCSMM provides a flexible and mathematically robust framework, its output is inherently binary, producing classifications without any probability estimates. While the decision function generates continuous scores, classification is determined solely by the sign of the output. These scores indicate relative distances from the decision boundary, but they do not provide a direct probabilistic interpretation.

As far as we are aware, there has been no prior research specifically focused on the probabilistic calibration of OCSMM. A critical challenge in this process is generating anomalous group samples, a necessary step for calibration due to the inherent scarcity of anomalous samples in both classical anomaly detection and group anomaly detection tasks. The following section details our calibration procedure, including the choice of the calibration method and the steps for generating anomalous groups.

We argue that, among various calibration methods, sigmoid fitting is particularly suitable for calibrating OCSMM. Unlike binning methods and isotonic regression, which often introduce discontinuities and risk overfitting, sigmoid fitting is specifically designed for binary classification problems and performs robustly on imbalanced data. In addition, sigmoid fitting offers flexibility and computational efficiency, which makes it advantageous for handling large datasets. Furthermore, sigmoid fitting assumes a monotonic relationship between decision scores and class probabilities, an assumption that aligns well with the behavior of OCSMM’s decision scores. Based on these considerations, sigmoid fitting was chosen as the calibration method.

The decision function of the OCSMM model trained on the training group set is denoted as $\gamma \left(\tilde{\mathit{x}}\right)$. Let $\beta \left(\tilde{\mathit{x}}\right)$ represent the calibrated score of a group ($\tilde{\mathit{x}}$) based on the output of $\gamma \left(\tilde{\mathit{x}}\right)$. Then, sigmoid fitting is performed as follows:

$$P\left(\beta \left(\tilde{\mathit{x}}\right)=1\mid \tilde{\mathit{x}}\right)\approx J_{g,q}\left(\gamma \left(\tilde{\mathit{x}}\right)\right)\equiv {\displaystyle \frac{1}{1+exp\left(g\gamma \left(\tilde{\mathit{x}}\right)+q\right)}},$$

(17)

where g and q are parameters estimated by solving the following regularized, unconstrained optimization problem:

$$\begin{array}{cc}{\displaystyle \underset{g,q}{max}}\hfill & {\displaystyle \sum _{j=1}^{l^{*}}{\zeta }_{j}log\left(J_j\right)+\left(1-{\zeta }_j\right)log\left(1-J_j\right),}\hfill \end{array}$$

(18)

where $J_j=J_{g,q}\left(\gamma \left({\tilde{\mathit{x}}}_j\right)\right)$ and

$${\zeta }_j=\left\{\begin{array}{cc}{\displaystyle \frac{N_{+}+1}{N_{+}+2}}\hfill & ,\mathrm{if}{\beta }_j=1\hfill \\ {\displaystyle \frac{1}{N_{-}+2}}\hfill & ,\mathrm{if}{\beta }_j=-1\hfill \end{array}\right.,$$

for $j=1,\dots ,l^{*}$. The numbers of samples labeled as 1 (${\beta }_j=1$) and $-1$ (${\beta }_j=-1$) are denoted by $N_{+}$ and $N_{-}$, respectively. For calibration related to a pointwise classifier (e.g., SVM or OCSVM), Platt used the Levenberg–Marquardt (LM) algorithm to solve (18) [38]. Later, this was improved by Lin et al. [39], who demonstrated that (18) is a convex optimization problem and presented a more robust algorithm with proven theoretical convergence to solve it.

To calibrate the group classifier OCSMM, it is essential to generate anomalous groups rather than individual anomalous points. Inspired by previous work on anomalous points [40,41,42], we adapted these ideas for groups. To generate anomalous points, the central assumption is that, spatially, all such points lie outside high-density regions. Building upon this assumption, previous studies have focused on constructing anomalous points for an optimal calibration, where a threshold on the density ($y=p\left(\mathit{x}\right)$) is determined to classify a point as anomalous or not. However, anomalous groups cannot be generated by the same approach as anomalous points, as they exhibit more complex structural behavior. Unlike anomalous points, anomalous groups may not only deviate spatially from high-density regions but can also significantly overlap with normal groups. These simulated groups are specifically constructed to introduce mild deviations from normal patterns near the decision boundary, serving as informative references for the calibration of the OCSMM score. Based on the above analysis, we propose the following algorithm for the generation of anomalous groups.

Algorithm 2 starts by generating the group centers. A key difference between Algorithm 2 and the pointwise calibration methods discussed earlier is that the threshold, a hyperparameter used to define the boundary of high-density regions, is removed. This modification allows group centers to be located in both high-density and low-density regions. The next step focuses on ensuring that anomalous groups exhibit as much diversity in their distribution characteristics as possible. To achieve this, the covariance matrix is generated randomly under constraints derived from the covariance matrices of the normal groups in the training set. Then, a new anomalous group for the calibration is generated by sampling from a multivariate Gaussian distribution, using the group center as the location and the generated covariance matrix as the scale.

Algorithm 2 Generation of a set of anomalous groups ($\mathbb{B}$) for the calibration.

Input: A set of normal groups $\mathbb{A}=\left\{{\tilde{\mathit{x}}}_1,\dots ,{\tilde{\mathit{x}}}_l\right\}$; the number of anomalous groups h; the expected size ${\lambda }^{\left(l\right)}$ of each anomalous group; Output: A set of anomalous groups $\mathbb{B}$ of size h. 1: Compute the centers ${\mathit{c}}_1,{\mathit{c}}_2,\dots ,{\mathit{c}}_l$ and covariance matrices ${\mathit{v}}_1,{\mathit{v}}_2,\dots ,{\mathit{v}}_l$ of the normal groups in the training set. Compute the center ${\mathit{c}}^{*}$ of all the centers ${\mathit{c}}_1,{\mathit{c}}_2,\dots ,{\mathit{c}}_l$ and the average Euclidean distance r from the centers ${\mathit{c}}_1,{\mathit{c}}_2,\dots ,{\mathit{c}}_l$ to ${\mathit{c}}^{*}$. 2: Generate a set of h points $B=\left\{{\mathit{b}}_{l+1},\dots ,{\mathit{b}}_j,\dots ,{\mathit{b}}_{l+h}\right\}$ uniformly within a hypersphere with center ${\mathit{c}}^{*}$ and radius $2r$. 3: for each ${\mathit{b}}_j$ in B do 4:    Identify the nearest normal group in $\mathbb{A}$, where the distance ${\Delta }_{{\mathit{b}}_j}$ is computed as the minimal Euclidean distance between ${\mathit{b}}_j$ and the centers in $\mathbb{A}$. A symmetric covariance matrix ${\mathit{v}}_{{\mathit{b}}_j}$ is generated based on the covariance matrix of the nearest group. Each element is randomly sampled from a uniform distribution between 0 and the corresponding element of the nearest group’s covariance matrix. The resulting matrix is symmetrized by adding it to its transpose and dividing by two. 5:    Use ${\mathit{b}}_j$ as the center and the covariance matrix ${\mathit{v}}_{{\mathit{b}}_j}$ to generate a group ${\tilde{\mathit{x}}}_{{\mathit{b}}_j}$ of points randomly from a multivariate Gaussian distribution, with the number of points drawn from a Poisson distribution with mean ${\lambda }^{\left(l\right)}$. 6: end for 7: Obtain a set of anomalous groups $\mathbb{B}=\left\{{\tilde{\mathit{x}}}_{l+1},\dots ,{\tilde{\mathit{x}}}_{{\mathit{b}}_j},\dots ,{\tilde{\mathit{x}}}_{l+h}\right\}$.

Algorithm 2 avoids generating extreme groups that would be too easily detected and eliminates the need for the intricate threshold determination step required in methods of generating anomalous points. This design provides a pragmatic approach to addressing the scarcity of labeled group anomalies and aims to improve probabilistic interpretability rather than explicitly simulating specific unknown anomalous behavior. After obtaining the anomalous groups, the sigmoid calibration technique is applied. We then define a distribution-based anomaly score as the calibrated OCSMM score ($\beta \left(\tilde{\mathit{x}}\right)$) to quantify how anomalous the high-density region of a group is. The overall steps for obtaining the distribution-based anomaly score ($\beta \left(\tilde{\mathit{x}}\right)$) using calibrated OCSMM are summarized in Algorithm 3.

Algorithm 3 The calibrated OCSMM: distribution-based group anomaly score.

Input: A set of normal groups $\mathbb{A}=\left\{{\tilde{\mathit{x}}}_1,\dots ,{\tilde{\mathit{x}}}_l\right\}$; a test group set $\mathbb{T}=\{{\tilde{\mathit{x}}}_1^{\left(t\right)},\dots ,{\tilde{\mathit{x}}}_{l^{\prime }}^{\left(t\right)}\}$; the number of anomalous groups h; the expected size n of each group; Output: The distribution-based anomaly score $\beta \left(\mathbb{T}\right)$ of the test group set $\mathbb{T}$. 1: Train an OCSMM model based on the normal group set $\mathbb{A}=\left\{{\tilde{\mathit{x}}}_1,\dots ,{\tilde{\mathit{x}}}_l\right\}$. 2: Using Algorithm 2, generate h anomalous groups, $\mathbb{B}=\left\{{\tilde{\mathit{x}}}_{l+1},\dots ,{\tilde{\mathit{x}}}_{{\mathit{b}}_{\mathbf{j}}},\dots ,{\tilde{\mathit{x}}}_{l+h}\right\}$, with each group size drawn from a Poisson distribution with expectation parameter n. 3: Compute the OCSMM scores for the normal groups, $\gamma \left(\mathbb{A}\right)=\left\{\gamma \left({\tilde{\mathit{x}}}_1\right),\dots ,\gamma \left({\tilde{\mathit{x}}}_l\right)\right\}$, and the anomalous groups, $\gamma \left(\mathbb{B}\right)=\{\gamma \left({\tilde{\mathit{x}}}_{l+1}\right),\dots ,\gamma \left({\tilde{\mathit{x}}}_{{\mathit{b}}_j}\right),\dots ,\gamma \left({\tilde{\mathit{x}}}_{l+h}\right)\}$, using the decision function $\gamma (·)$ of the trained OCSMM model. 4: Fit a sigmoid model to $\gamma \left(\mathbb{A}\right)$ and $\gamma \left(\mathbb{B}\right)$ by estimating the parameters g and q. 5: Apply the trained OCSMM model to compute the OCSMM scores for the test set, $\gamma \left(\mathbb{T}\right)$, and then use the trained sigmoid model to obtain the calibrated scores as $\beta \left(\mathbb{T}\right)=\{\beta \left({\tilde{\mathit{x}}}_1^{\left(t\right)}\right),\dots ,\beta \left({\tilde{\mathit{x}}}_{l^{\prime }}^{\left(t\right)}\right)\}$.

4.2. Extreme Value Support Measure Machine

In previous sections, we introduced two group anomaly scores to evaluate two complementary parts of the group: a point-based anomaly score for a group’s low-density region, denoted as $\alpha \left(\tilde{\mathit{x}}\right)$, and a distribution-based anomaly score for a group’s high-density region, denoted as $\beta \left(\tilde{\mathit{x}}\right)$. Specifically, $\alpha \left(\tilde{\mathit{x}}\right)$ assesses the anomalousness of the aggregate entity of exceedances within the low-density regions. In contrast, $\beta \left(\tilde{\mathit{x}}\right)$ evaluates whether the distributional characteristics of points, considered as an aggregate within the high-density regions, deviate significantly from those observed in the training groups. The next step is to develop an integrated framework that effectively leverages both probabilistic scores to capture the overall level of anomalousness.

For this purpose, we take inspiration from the way certainty factors are combined, encountered in the early days of expert systems such as PROSPECTOR [43] and MYCIN [44] capable of reasoning under uncertainty. Functions such as Van Melle’s combining function [45], typically of type ${[-1,1]}^2\to [-1,1]$, are well thought-out functions designed for that purpose. Only in the 1990s [45], after linearly rescaling to functions of type ${[0,1]}^2\to [0,1]$ was it realized that they were examples of a broader class of aggregation functions on the unit interval called uninorms. Uninorms operate on the unit interval of $[0,1]$, providing a systematic way to combine probabilistic scores. In our case, the two probabilistic scores, each within the range of $[0,1]$, align particularly well with uninorms, which are binary operations capable of meeting these engineering demands.

We now argue that uninorms with a neutral element (e), which acts as a hyperparameter, are ideally suited for our purpose. In engineering applications with multiple measures assessing different aspects of a system, it is essential that when more than one measure exceeds a given threshold (triggering alarms for specific aspects), the overall measure should reflect a more severe status of the system. For instance, in health monitoring, when a patient exhibits multiple anomalous health indicators, we tend to consider the overall health condition to be worse. Constructing a single measure that exceeds any anomalous health indicator when assessed on the same scale provides more informative insights into the patient’s condition. In such cases, traditional weighting methods, such as weighted averaging and t-conorms (e.g., maximum), either dilute dominant signals or fail to model mutual reinforcement. Uninorms offer a principled mechanism to capture both synergistic and antagonistic effects, which is critical in group anomaly scenarios involving dual sources of evidence.

A uninorm is a symmetric, associative, increasing binary operation on $[0,1]$ with a neutral element ($e\in [0,1]$). The case of $e=1$ corresponds to the well-known t-norms (including the minimum operation), and the case $e=0$ corresponds to the well-known t-conorms (including the maximum operation). Due to its associativity, any uninorm (U) is either conjunctive (i.e., $U(0,1)=U(1,0)=0$) or disjunctive (i.e., $U(0,1)=U(1,0)=0$). The structure of a uninorm with a neutral element (e) is expressed as follows [46]: on ${[0,e]}^2$, it behaves as a rescaled t-norm, taking values below the minimum; on ${[e,1]}^2$, it behaves as a rescaled t-conorm, taking values above the maximum; an on the remaining parts of the unit square (i.e., $[0,e]\times [e,1]\cup [e,1]\times [0,e]$), it behaves as a compensatory (also called averaging) operator, taking values between the minimum and the maximum. Given a t-norm and a t-conorm, one can always build a uninorm by considering the minimum or maximum (one of the two extremes) as a compensatory operator.

In our case, suppose that exactly one of the anomaly scores takes a value higher than e; then, one would consider this an indication of the group being anomalous, irrespective of the other score and, thus, consider it as the overall anomaly score. Hence, to construct a uninorm, we consider the maximum as compensatory operator a viable choice. However, if both anomaly scores exceed e, then they could be seen as reinforcing one another and act synergistically, yielding an overall score exceeding the maximum of the individual scores. Similarly, if both anomaly scores take a value lower than e, then they should act antagonistically, yielding an overall score lower than their minimum. The symmetry of the uninorm expresses that both anomaly scores play the same role.

The most famous uninorm exhibiting the desired synergistic and antagonistic behavior is the $3\Pi$ operator $U_{3\Pi }:{[0,1]}^2\to [0,1]$ with neutral element ${\displaystyle \frac{1}{2}}$:

$$U_{3\Pi }(a,b)={\displaystyle \frac{ab}{ab+(1-a)(1-b)}}$$

with either $U_{3\Pi }(0,1)=U_{3\Pi }(1,0)=0$ or $U_{3\Pi }(0,1)=U_{3\Pi }(1,0)=1$. This operation is nothing else but a rescaled version of the function used for combining certainty factors in the PROSPECTOR rule-based system. Without going into detail, we note that all members of the important class of representable uninorms are isomorphic to this iconic $3\Pi$ operator, illustrating its central role.

Recalling our argumentation above that, on $[0,e]\times [e,1]\cup [e,1]\times [0,e]$, the selected uninorm should preferably act as the maximum, we can modify the $3\Pi$ operator (keeping the underlying t-norm and t-conorm), resulting in the uninorm ($U_{\mathit{a}no}$) used in our final aggregation method. It is given by

$$U_{\mathit{a}no}(a,b)=\left\{\begin{array}{cc}2e{U}_{3\Pi }({\displaystyle \frac{a}{2e}},{\displaystyle \frac{b}{2e}})\hfill & ,\mathrm{if}(a,b)\in {[0,e]}^2\hfill \\ e+(1-e)\left[2U_{3\Pi }({\displaystyle \frac{1}{2}}+{\displaystyle \frac{a-e}{2-2e}},{\displaystyle \frac{1}{2}}+{\displaystyle \frac{b-e}{2-2e}})-1\right]\hfill & ,\mathrm{if}(a,b)\in {(e,1]}^2\hfill \\ max(a,b)\hfill & ,\mathrm{elsewhere}.\hfill \end{array}\right.$$

(19)

The detailed derivation of the uninorm ($U_{\mathit{ano}}$) based on $3\Pi$ operator $U_{3\Pi }$ is provided in Appendix A. As a result, the combination of the two anomaly scores is completed. We present the full procedure of the proposed model EVSMM in Algorithm 4.

Algorithm 4 Extreme value support measure machine (EVSMM).

Input: A set of normal groups $\mathbb{A}=\left\{{\tilde{\mathit{x}}}_1,\dots ,{\tilde{\mathit{x}}}_l\right\}$; A test group set $\mathbb{T}=\{{\tilde{\mathit{x}}}_1^{\left(t\right)},\dots ,{\tilde{\mathit{x}}}_{l^{\prime }}^{\left(t\right)}\}$; Output: The anomaly scores $\varphi \left({\tilde{\mathit{x}}}^{\left(t\right)}\right)$ of the test group set $\mathbb{T}$. 1: Apply KDE on the normal groups in $\mathbb{A}$ to have an estimation of the density of each point. Then transform the normal training groups $\tilde{\mathit{x}}$ to $\mathit{z}$ as shown in Section 4.1. Compute $c_n$, $d_n$ through (5), respectively. 2: Train an OCSMM model based on the training set $\mathbb{A}$. Then apply Algorithm 2 and the objective function (18) to get a sigmoid model $\beta \left(\tilde{\mathit{x}}\right)$ for the calibration of the OCSMM score. 3: for each ${\tilde{\mathit{x}}}_j^{\left(t\right)}$ in $\mathbb{T}$ do 4:    (Probabilistic point-based score) The likelihood of a group w.r.t. an aggregate of exceedances in low-density regions is obtained from (9). Then anomaly score $\alpha \left({\tilde{\mathit{x}}}_j^{\left(t\right)}\right)$ is determined from (16) and (13) based on the analytical asymptotic CDF of the likelihood. 5:    (Probabilistic distribution-based score) Compute the OCSMM score $f\left({\tilde{\mathit{x}}}_j^{\left(t\right)}\right)$. Then calibrate the OCSMM score through the trained sigmoid model (17) to obtain the probabilistic distribution-based score $\beta \left({\tilde{\mathit{x}}}_j^{\left(t\right)}\right)$. 6:    (Final score) Compute the final anomaly score of the group ${\tilde{\mathit{x}}}_j^{\left(t\right)}$ as $\varphi \left({\tilde{\mathit{x}}}_j^{\left(t\right)}\right)=U_{\mathit{a}no}\left(\alpha \left({\tilde{\mathit{x}}}_j^{\left(t\right)}\right),\beta \left({\tilde{\mathit{x}}}_j^{\left(t\right)}\right)\right)$ by applying (A4). 7: end for

5. Experiments

In this section, we present extensive experiments to demonstrate the effectiveness of the proposed EVSMM in detecting unusual group behavior. The performance of the EVSMM is compared with five other group anomaly detectors: the discriminant SMDD model (https://github.com/jorjasso/SMDD-group-anomaly-detection, accessed on 28 May 2025) [11], the OCSMM (https://github.com/kdgutier/ocsmm, accessed on 28 May 2025) [7] and its calibrated output based on a sigmoid transformation (Cali-OCSMM), the generative MGMM model (https://www.cs.cmu.edu/%7Elxiong/gad/gad.html, accessed on 28 May 2025) [6], and an extreme value model (EVM) based on Extreme Value Theory [2].

5.1. Experiments on Synthetic Data

We performed an experiment using synthetic data with a set-up comparable to the one used in [6,7,8,9,11]. Following these references, the normal groups are simulated from two Gaussian mixture distributions in the plane that each consists of four components. The mixing proportions for the Gaussian mixtures are expressed as $(0.22,0.64,0.03,0.11)$ and $(0.22,0.03,0.64,0.11)$. To generate a normal group, probabilities $(0.48,0.52)$ are assigned to determine which Gaussian mixing proportion is applied. The selected Gaussian mixing proportion is then used to generate the group. The components of both mixtures are centered at $(-1,-1)$, $(1,-1)$, $(0,1)$, and $(1,1)$ and share the covariance matrix ($\mathrm{\Sigma }=0.15\times {\mathit{I}}_2$, where ${\mathit{I}}_2$ is the $2\times 2$ identity matrix).

Three types of anomalous groups are considered: (i) point-based anomalous groups, (ii) distribution-based anomalous groups, and (iii) groups that are anomalous as a mixture of unusual low-density and high-density regions. The threshold for identifying low-density regions is determined based on a mean residual plot. To generate groups with varying extents of point-based anomalous behavior, the number of low-density regions is randomly chosen according to a Poisson distribution with a mean of ${\lambda }^{\mathrm{ext}}=3$, truncated at zero to ensure the presence of at least one anomalous point. For a more detailed investigation of the impact of ${\lambda }^{\mathrm{ext}}$, additional experiments were conducted with varying ${\lambda }^{\mathrm{ext}}$ values (as presented later). These experiments reveal that as ${\lambda }^{\mathrm{ext}}$ increases, other group anomaly detectors, such as MGMM, begin to achieve performance comparable to that of our model. This phenomenon occurs because an increase in the number of exceedances gradually transforms point-based anomalous group behavior into distribution-based anomalous group behavior. Groups exhibiting various distribution-based anomalous behaviors are simulated through a Gaussian mixture distribution with the same centers as the normal groups but different mixing proportions. The proportions $(0.6,0.1,0.07,0.23)$, as used in [7], are complemented by including three additional configurations: $(0.80,0.01,0.06,0.13)$, $(0,0,0.06,0.94)$, and $(0.64,0.22,0.03,0.11)$. A mixed anomalous group contains both an aggregate of points in low-density regions and an aggregate of points randomly simulated from unusual mixing proportions in high-density regions.

To evaluate our method with varying extents of distribution-based and point-based anomalous behavior, nine distinct cases were designed to ensure a diverse representation of anomalous group configurations in the test set. The corresponding experiments consider varying fractions of the three types of anomalous groups, defined as $[1:0:0]$, $[0:1:0]$, $[0:0:1]$, $[0.5:0.5:0]$, $[0:0.5:0.5]$, $[0.5:0:0.5]$, $[0.5:0.25:0.25]$, $[0.25:0.5:0.25]$, and $[0.25:0.25:0.5]$ (the first component representing point-based anomalous groups, the second representing distribution-based groups, and the third representing a mixture thereof). In each of the experiments, we injected 60 anomalous groups into the test set according to the corresponding proportions.

We used 1000 normal groups for training. The training involves a 10-fold cross-validation: in each run, the dataset is randomly partitioned into ten parts, of which seven are used for training, two for hyperparameter optimization in a validation step, and one for testing. In the proposed EVSMM approach, two hyperparameters are subject to tuning: $\gamma$ in the Cali-OCSMM model, controlling the kernel width, and e, the threshold in the uninorm that determines the aggregation behavior when combining anomaly scores designed to evaluate complementary aspects of the group. These hyperparameters are optimized using a grid search strategy guided by nested cross-validation, where $\gamma \in \{0.5,0.8,1.1,1.4\}$ and $e\in \{0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9\}$. The group size is simulated using a Poisson distribution with a mean of ${\lambda }^{\left(l\right)}=50$. Model performance is analyzed based on the ROC curve and the AUC metric. The detailed results are presented in Figure 3, Figure 4 and Figure 5 and Appendix A.

To ensure robust performance, $\gamma$ and e were jointly tuned via grid search, using AUC as the selection criterion within a nested 10-fold cross-validation framework. Across all folds and experimental cases, the optimal $\gamma$ value consistently converged to $1.4$. As a result, only the values of e are reported in

Table 2. Values of hyperparameter e across 10-fold cross-validation for synthetic data cases.

	Case	Case 1	Case 2	Case 3	Case 4	Case 5	Case 6	Case 7	Case 8	Case 9
Fold
Fold 1		0.4	0.1	0.1	0.1	0.1	0.1	0.7	0.2	0.4
Fold 2		0.2	0.6	0.8	0.8	0.1	0.1	0.6	0.1	0.1
Fold 3		0.5	0.1	0.2	0.7	0.3	0.1	0.8	0.1	0.3
Fold 4		0.1	0.6	0.1	0.2	0.2	0.7	0.2	0.1	0.1
Fold 5		0.1	0.1	0.1	0.1	0.5	0.7	0.8	0.1	0.5
Fold 6		0.2	0.1	0.4	0.7	0.3	0.8	0.1	0.2	0.1
Fold 7		0.2	0.7	0.1	0.2	0.9	0.1	0.1	0.2	0.1
Fold 8		0.6	0.6	0.8	0.1	0.8	0.6	0.7	0.1	0.7
Fold 9		0.5	0.1	0.9	0.9	0.8	0.2	0.9	0.2	0.1
Fold 10		0.2	0.6	0.9	0.1	0.9	0.1	0.6	0.1	0.2
Optimal		0.2	0.1	0.1	0.1	0.3	0.1	0.7	0.1	0.1

, which summarizes the fold-wise optimal e for each synthetic anomaly case. The final row reports the optimal value of e for each case based on its frequency across folds.

In contrast to the other approaches, from the results shown in Figure 3, it follows that EVSMM achieves consistently competitive performance across all cases. MGMM is also relatively effective compared to other models, achieving AUC scores exceeding $0.75$ in eight out of nine cases. However, it is observed that MGMM is less performant in evaluating point-based anomalous groups that have a limited number of exceedances. Although MGMM is designed to detect group anomalies of different types, its ability to detect point-based anomalous group behavior requires a sufficient number of exceedances, as examined further through experiments with varying ${\lambda }^{\mathrm{ext}}$ values. Based on the results in the cases of $[1:0:0]$ and $[0:1:0]$, it is clear that the EVM is particularly suited to effectively detect point-based group anomalies. Although half of the groups in the case of $[0.5:0:0.5]$ exhibit varying extents of distribution-based anomalous group behavior, EVM stands out for its ability to quantify point-based anomalous group behavior across all groups. However, when compared with the outcomes from the case of $[0:0.5:0.5]$, it becomes evident that EVM is unable to address the distribution-based anomalous group behavior present in half of the groups in the case of $[0.5:0:0.5]$. SMDD, MGMM, and OCSMM are particularly suited for groups with an unusual aggregate of points in high-density regions, as shown in the cases of $[0:1:0]$ and $[0:0.5:0.5]$, where every group exhibits a varying extent of distribution-based anomalous behavior, resulting in higher AUC values for the these models. Also in the cases of $[0.25:0.5:0.25]$ and $[0.25:0.25:0.5]$, with a high proportion of distribution-based group anomalies, three models show their detection capabilities. In Appendix A, we show comprehensive ROC curves for all nine cases, allowing for comparison of the performances of various approaches.

Figure 4 shows the variance of the AUC scores on a logarithmic scale. Overall, EVSMM shows a satisfactory stability of AUC scores. In cases with a relatively low proportion of point-based group anomalies, MGMM has a lower variance than EVSMM. OCSMM and Cali-OCSMM have the same variance in all cases, confirming that the sigmoid calibration is effective. Appendix A shows an alternative graphical representation using box plots to compare the AUC scores among various methods for each case.

Furthermore, we conducted an experiment focused exclusively on point-based group anomalies, specifically by varying the number of points located in low-density regions within groups. In our previous experiments, we assumed that the number of points in low-density regions is governed by a Poisson distribution with an expectation parameter of ${\lambda }^{\mathrm{ext}}=3$. Here, we vary this parameter from 0 to 10 and look at the effect on the performance of MGMM, EVM, and EVSMM, all of which possess the ability to evaluate point-based anomalous group behavior. Figure 5 reveals that as the number of points in low-density regions increases, the performance of MGMM becomes comparable with that of EVM and EVSMM. However, when the number is lower, MGMM fails to quantify point-based anomalous behavior. This can be attributed to the gradual transition of point-based anomalous group behavior into distribution-based anomalous group behavior as the number of exceedances increases. As indicated the previous results, although the overall performance of MGMM is not as strong as that of EVSMM, MGMM exhibits greater sensitivity to this transition compared to other models that excel in detecting distribution-based group anomalies.

5.2. Experiments on the Sloan Digital Sky Survey Data

In this section, we evaluate the EVSMM on the Sloan Digital Sky Survey (SDSS) dataset. This dataset includes information on millions of celestial objects, such as stars, galaxies, and various other astronomical phenomena. It provides detailed measurements of the positions, brightness, and spectra of these objects. Our focus is on studying a subset of galaxies that consists of a large volume of 4000-dimensional spectra of about $7\times {10}^5$ galaxies. We adopt the 1000-dimensional feature vectors derived from the original 4000-dimensional spectra, as provided in the publicly available benchmark dataset used in prior studies [6,7,9,11]. We then apply principal component analysis (PCA) to further reduce the data to four dimensions. The resulting four principal components retain approximately $85\%$ of the total variance. Specifically, PC1, PC2, PC3, and PC4 account for $77.2\%$, $5.8\%$, $1.6\%$, and $0.5\%$ of the total variance, respectively. This reduced representation is subsequently used as input for density estimation. Specifically, KDE is performed on the resulting four-dimensional features, thereby mitigating the impact of high dimensionality on the estimation process.

Xiong et al. [6] were the first to utilize the SDSS dataset for group anomaly detection. They identified 505 spatial groups of galaxies using a constructed neighborhood graph, where each connected component in the graph represents a group. Then, they conducted group anomaly detection on these 505 groups without prior knowledge of any group anomaly labels. The results gained positive recognition from astronomers. Furthermore, in order to ensure a statistically meaningful comparison among various methods, artificial anomaly injections were employed to address the absence of labels, a method that was commonly adopted in subsequent research [6,7,9,11,47].

Building on the injection method used in previous studies, we increased the complexity of the injected anomalous groups to avoid configurations that are trivial to distinguish. Point-based, distribution-based, and mixed anomalous groups are constructed by randomly selecting existing points from the training group set. Specifically, point-based group anomalies are generated by aggregating galaxies located in low-density regions, with aggregate sizes following a Poisson distribution with ${\lambda }^{\mathrm{ext}}=3$, whereas distribution-based group anomalies are constructed by randomly selecting galaxies from high-density regions, with the constraint that the selected galaxies are not all drawn from the same group. We consider six cases, where, in each case, a total of 50 anomalous groups are injected with proportions of point-based, distribution-based, and mixed anomalous groups set to $[0.5:0.5:0]$, $[0:0.5:0.5]$, $[0.5:0:0.5]$, $[0.5:0.25:0.25]$, and $[0.25:0.5:0.25]$, respectively.

During a nested 10-fold cross-validation procedure, the optimal value of $\gamma$ was consistently selected as 1.4 across all folds. The corresponding values of e selected in each fold are reported in

Table 3. Values of hyperparameter e across 10-fold cross-validation for SDSS data cases.

	Case	Case 1	Case 2	Case 3	Case 4	Case 5	Case 6
Fold
Fold 1		0.5	0.1	0.9	0.9	0.5	0.1
Fold 2		0.7	0.1	0.9	0.9	0.9	0.9
Fold 3		0.9	0.2	0.9	0.9	0.9	0.1
Fold 4		0.6	0.1	0.8	0.6	0.9	0.9
Fold 5		0.8	0.4	0.9	0.9	0.1	0.9
Fold 6		0.6	0.7	0.3	0.9	0.9	0.1
Fold 7		0.9	0.1	0.1	0.9	0.9	0.1
Fold 8		0.1	0.1	0.1	0.1	0.1	0.1
Fold 9		0.3	0.1	0.9	0.9	0.4	0.9
Fold 10		0.8	0.1	0.9	0.9	0.7	0.9
Optimal		0.9	0.1	0.9	0.9	0.9	0.1

. This table shows the fold-wise tuning results for each SDSS anomaly case, and the final row presents the most frequently selected value across folds.

As shown in Figure 6, EVSMM demonstrates consistently high AUC scores across all cases, outperforming achieving performance comparable to that of other methods. In Figure 7, which shows the variance of the AUC scores on a logarithmic scale, we observe that EVSMM stands out for both high AUC scores and low variance, showcasing its ability to effectively and reliably handle various types of group anomalies. While MGMM achieves high AUC scores in many cases, its performance is less stable in configurations dominated by point-based group anomalies, as reflected in its higher variance. EVM and Cali-OCSMM perform well at detecting distribution-based group anomalies but exhibit less stability compared to EVSMM. On this real-world dataset, we observe that SMDD exhibits notable variability in its AUC score variances across different data configurations. This may suggest that, compared to other methods, the stability of SMDD is more affected by the dataset properties, resulting in marked discrepancies in variance values.

The performances of OCSMM and Cali-OCSMM are very similar, indicating that the calibration method for the OCSMM is effective for the SDSS data. In Appendix A, we show the ROC curves and box plots of the AUC scores for each case. These results highlight the effectiveness and robustness of EVSMM, particularly in scenarios with mixed anomaly types with varying extents of anomalous behavior. While MGMM shows strong potential, its sensitivity to specific configurations (e.g., $[0.5:0:0.5]$, $[0.5:0.25:0.25]$) warrants further investigation to enhance its stability and performance.

6. Conclusions and Future Work

In this work, we have proposed an efficient method for comprehensively detecting group anomalies, termed EVSMM. The method integrates a PPM based on EVT and a calibrated OCSMM, which evaluate complementary aspects of the target group, with the scores subsequently being integrated using a constructed uninorm. Indeed, through the PPM, we formally construct the likelihood for an aggregate of low-density regions within a group and, more importantly, derive the analytical distribution of the likelihood. The calibrated OCSMM quantifies the aggregate of points within high-density regions, addressing the limitation of its raw output, which provides only classification without probability estimates. Through this calibration, a novel approach is introduced for generating diverse anomalous groups, which offers a reference for future research on the calibration and validation of group models. A uninorm is constructed to combine two anomaly scores with the ability to locally synergize, antagonize, and weigh these scores relative to the threshold e.

Extensive experiments show that existing group anomaly detection approaches, such as OCSMM, MGMM, and SMDD, are mainly focused on the detection of distribution-based group anomalies but lack the ability to comprehensively quantify diverse group anomalies. Alternative approaches that are built upon methods centered around the detection of individual anomalous points tend to inaccurately model point-based anomalous group behavior and face challenges managing the multiple-hypothesis problem. The proposed EVSMM effectively quantifies various anomalous group behaviors and, unlike existing methods, provides a probability-based anomaly score that enhances interpretability and utility in practical applications.

Future work opens several promising and exciting directions. One direction lies in extending our model beyond anomaly detection to risk prediction tasks in climate-related applications. For example, EVSMM can be applied to evaluate heat waves, which are defined as consecutive periods of high temperatures. Each period can be treated as a group, with its severity assessed based on the joint characteristics of duration and intensity. In addition to retrospective evaluation, the model can be used for risk assessment of future extreme events. Another relevant direction is the assessment of persistent extreme rainfall, where clusters of intense precipitation across consecutive time intervals can be modeled as groups using high-resolution precipitation data. The model can further support prediction by estimating the likelihood of future events under varying environmental conditions. These applications demonstrate how group-level modeling can be leveraged not only for anomaly detection but also for probabilistic risk assessment in climate science. Another focus will be a deeper investigation into the performance of our model in group anomaly detection tasks. This involves two aspects. On the one hand, we should keep exploring the model’s ability to detect varying extents of anomalous behavior, extending research beyond evaluating the test-set accuracy. Specifically, our plan is to study the relationship between the entropy of groups and the anomaly scores to gain further insights into its capabilities. On the other hand, we intend to explore more meaningful real-world group anomaly detection tasks to assess the practical utility and robustness of our approach. One particular aspect that can be studied is the robustness of the method against noisy data. Since the likelihood of a group is evaluated based on the entire aggregate of exceedances, we expect that, to some extent, noisy observations—such as random low-density points scattered in otherwise normal groups—do not affect the distribution of the likelihood substantially unless their number exceeds a statistically meaningful level. However, further investigation could provide deeper insight into the precise effect of noisy data on the detection of the different types of group anomalies examined in this work. These directions are expected to provide a clearer understanding of the model’s strengths and uncover novel applications.