Face Spoofing Detection with Stacking Ensembles in Work Time Registration System

Abstract

This paper introduces a passive face-authenticity detection system, designed for integration into an employee work time registration platform. The system is implemented as a stacking ensemble of multiple models. Each model independently assesses whether a camera is capturing a live human face or a spoofed representation, such as a photo or video. The ensemble comprises a convolutional neural network (CNN), a smartphone bezel-detection algorithm to identify faces displayed on electronic devices, a face context analysis module, and additional CNNs for image processing. The outputs of these models are aggregated by a neural network that delivers the final classification decision. We examined various combinations of models within the ensemble and compared the performance of our approach against existing methods through experimental evaluation.

1. Introduction

In the context of this article, an authentic face refers to the face of a live person in front of a camera. An inauthentic or spoofed face refers to a face shown on an electronic device screen, in video footage, or printed on paper.

This paper introduces a method for verifying the authenticity of faces, intended for use within a work time management system.

The specific issue addressed in the paper is the detection of scenarios in which an employee places a photo or video of another employee in front of the camera to falsely record the attendance of someone who is not physically present. Before detailing the proposed method, we will provide a brief overview of the work time management system, explain how our method interacts with other modules of the system, and outline the requirements it must meet.

The described work time management system is currently in use by several companies. Based on the requirements received, our new facial authenticity recognition module is to be integrated into the system as illustrated in Figure 1.

The facial authenticity recognition module will receive photos of employees whose identities (first and last names) have already been confirmed by the employee identification module, along with the coordinates of their faces within those photos. It must then evaluate the likelihood that the face is authentic (i.e., not spoofed) and forward the result to the work time registration module.

The technical specifications of the work time management system are as follows: the entire system consists of a client-side and a server-side part. The client-side application is implemented in Android, iOS, and in a web browser, and most often uses simple, front smartphone cameras or, sometimes, USB cameras, without infrared, to take pictures of people who register their work time (Figure 1). The server side is based on a server with two AMD EPYC 7282 16-core processors, 256 GB RAM, Nvidia RTX 3050 GPU, 1 TB FDD, and 8 TB HDD, using the Windows Operating System. The client-side application takes the photos, compresses them, and sends them to the server side, where further processing takes place and where our module, presented in this paper, operates.

According to the technical specifications, the authors of this paper were not permitted to modify any system modules other than the one they were developing. They were also not allowed to install infrared-equipped cameras or use Apple’s TrueDepth Camera technology [1] for face recognition. This restriction stemmed from the lack of consent from both clients and the original system developers to alter a well-functioning system and incur additional costs.

The client-side application is activated each time an employee starts or ends their work shift by scanning a QR code. This mechanism ensures that the system does not capture unnecessary photos of individuals merely passing by the camera or images of empty rooms. Once activated, the camera captures photos of the employee and uses the BlazeFace algorithm [2] to locate the face. Only images in which a face is successfully detected are compressed into the WEBP format (5–6 kB) for fast transmission. These are then sent to the server, where they are stored in a database for potential future analysis.

On the server side, the employee identification module processes the incoming photos. It verifies the identity of the person in each image by matching them with the employee associated with the scanned QR code, using the ArcFace algorithm [3]. This module performs with near-perfect accuracy.

Due to the constraints of the system, existing modules could not be modified. As a result, alternatives such as the RetinaFace algorithm [4] could not be integrated for face detection. However, given the high accuracy of the current solution, this limitation was not deemed problematic.

Per the provided specifications, our method receives photos along with face coordinates from the employee identification module, which is already implemented on the server. The image quality is generally suboptimal—photos are highly compressed (WEBP, 5–6 kB), often taken under poor lighting conditions, and frequently depict employees in a hurry, not properly positioned in front of the camera. Several example images taken under these conditions are presented in Figure 2. Nonetheless, our method is designed to operate effectively under these constraints, as we had no control over photo quality.

Typically, the problem of facial authenticity detection is studied in the context of users logging into an operating system or application. Many well-established methods exist for that use case and are thoroughly documented in the literature. However, the conditions and requirements in a work time registration system differ substantially, as previously described. In fact, state-of-the-art methods designed for login systems performed poorly in our settings, achieving only around 50% accuracy, and most authentic faces were incorrectly classified as spoofed. This issue also extended to a prominent commercial solution discussed in the experimental section. These systems perform well under ideal conditions: good lighting, a properly aligned face, and a high-quality USB camera. In contrast, the work time registration environment caused significant performance drops, leading us to conclude that image quality is a critical factor.

To explore this hypothesis, we captured several high-quality images using a full-frame camera and displayed them on a 4K-resolution notebook screen, simulating spoofing attempts. These images were often recognized as authentic by existing systems. This confirmed that photo quality can heavily influence the results. Since no available solution performed reliably under our specific conditions, we were compelled to develop a custom method, which is presented in this paper.

One of the key problems in the industry is the potential for fraud in work time registration. To prevent this, it is essential to ensure that the person registering work time is physically present and not being impersonated via photo or video. Therefore, we evaluated various face recognition algorithms and methods for determining whether a face belongs to a live person or is a spoofed image. This task is very different from facial authentication for login purposes and thus comes with its own set of unique requirements.

First, employees often do not actively cooperate with the system, so we cannot expect them to align their faces perfectly with the camera. Moreover, we cannot require active liveness detection (e.g., blinking, smiling).

Second, the system must function with low-cost, widely available cameras—such as front-facing smartphone cameras—where the user interface appears on the same screen.

Third, the system must be capable of serving thousands of employees logging their start times each morning through a centralized server. This requires strong image compression to save bandwidth, and the face-authenticity detection must be computationally efficient to provide feedback within one second, ensuring responsiveness under high load.

Fourth, there is a limited amount of training data, particularly examples of fraudulent attempts. We cannot ask users to intentionally spoof the system just to expand the dataset.

Importantly, the system is not expected to detect all fraudulent attempts, especially if doing so would result in a high number of false positives, which could disrupt company operations. If one fraud attempt is missed, the individual is likely to try again, increasing the likelihood of being detected on a subsequent attempt.

Furthermore, systems designed to detect screens or smartphones displaying a face often failed in our scenario, where only a partial view of the screen might be visible. For example, YOLO struggled to detect partially visible objects effectively. This limitation necessitated the creation of a custom database containing real and spoofed examples and the development of a bespoke facial authenticity detection system.

2. Review of Existing Solutions

Facial recognition systems are constantly gaining popularity. In particular, they are used to confirm the user’s authenticity in many applications, for example, logging into a computer. In particular, solutions based on deep learning have been rapidly gaining popularity in recent years. The most successful methods include DeepFace [7], DeepIDs [8], VGG Face [9], SphereFace [10], and the solution that we used in our system: ArcFace.

However, a common problem in all face recognition systems is the possibility of fraud, where someone else can obtain unauthorized access to the system using a photo or video of the legitimate user’s face.

Several systems aiming for face spoofing prevention were constructed using two cameras, a 3D camera, or an additional infrared camera. However, there are cases when we can use only a simple, cheap 2D camera, in particular the front camera of a smartphone or a webcam embedded into a laptop, as in our case.

However, the lack of 3D is not the only indicator of fraud, and the existence of 3D does not protect against using 3D masks for an attack. There also exist other indicators of an attack, and since we do not have access to the 3D information, we need to use them.

Frequently, we do not know what the signals of fraud are. However, in this case, we can use the photos of the real persons’ authentications and of the attempts of fraud to train a predictive model to recognize the fraudulent situation.

However, the predictive models usually need examples of two classes for the learning process: the real class and the fake class, and during the prediction phase, they predict the face recognition based on the similarity of examples in the training set. For that reason, several solutions were proposed.

In previous years, many traditional handcrafted feature-based methods were proposed for face spoofing detection. These algorithms contained methods such as LBP [11,12], SIFT [13], SURF [14], DoG, and HOG [15], and were designed for extracting spoofing patterns from various color spaces (RGB, HSV, and YCbCr).

Also, many methods to detect liveness cues were developed, which tried to detect eye-blinking [16,17,18], face and head movement, gaze tracking [19], and remote physiological signals such as rPPG [20,21,22,23].

However, it is easy to cheat the liveness cues with a proper video attack, which, on the one hand, frequently makes them less reliable, and on the other hand, users do not like this (in our application, it would be unacceptable; no one would use it).

Although deep learning and CNNs have achieved great success in many computer vision tasks (image classification, semantic segmentation, object detection, etc.), they contain many tuning parameters (weights) and therefore require large and diverse training datasets in order to avoid overfitting.

Thus, in response to that, several hybrid methods were proposed, where features are first extracted from the images, and then the CNNs operate on the extracted features rather than on the original images, which allows reducing the number of parameters and thus avoids overfitting. Below, we briefly list some recently proposed solutions.

Traditional image-based approaches focus on image quality and characteristics and therefore employ hand-made features, such as LBP, SIFT, HOG, and SURF, with shallow classifiers to discriminate live and fake faces [24].

Sthevanie et al. [25] combined the LBP and GLCM methods to extract the features used to detect spoof images. The LBP and GLCM are based on texture features, but GLCM uses statistical functions, so the extraction result represents the overall image texture features.

Chen et al. [26] proposed a two-stream convolutional neural network (TSCNN) that works in two complementary spaces: RGB and multi-scale retinex (MSR) space to take advantage of both texture details in RGB space and illumination invariance of MSR.

Rajeswaran and Kumar [27] developed an approach to convert the image to the L*a*b* color space, which is then passed through HOG (Histogram of Oriented Gradients) for face detection. The detected face is then advanced through the VGG7 CNN architecture for the extraction of features and then classified using a fully connected layer for spoof detection.

Hashemifard et al. [28], instead of completely extracting handcrafted texture features or relying only on deep neural networks, addressed the problem by fusing both wide and deep features in a unified neural architecture. The main idea was to take advantage of the strengths of both methods to derive a well-generalized solution for the problem.

Balamurali [9] performed face detection using MTCNN (Multi-Task Cascaded Convolutional Neural Network) and used the Support Vector Classifier (SVC) to classify real and fake faces.

Ab Wahab et al. [29] presented an Online Attendance System Using Face Recognition to record the daily attendance of students in online classes.

Shu et al. [30] proposed a model that consists of two streams. One stream converts the input RGB images into grayscale ones and conducts multi-scale color inversion to obtain the MSCI images, which are then put into the MobileNet to extract face reflection features. The other stream directly feeds RGB images into the MobileNet to extract face color features. Finally, the features extracted separately from the two branches are fused and then used for face spoofing detection.

Bahia et al. [31] presented face spoofing detection based on the extraction of Heterogeneous Auto-Similarities of Characteristics (HASC) descriptor from the HSV and YCbCr color spaces. The HASC descriptor encodes dependencies between low-level dense features of the color face image using covariance and information-theoretic measures to explore how the features are interrelated in real and fake faces.

Zhang et al. [32] proposed Dual Probabilistic Modeling (DPM), with two dedicated modules, DPM-LQ (Label Quality aware learning) and DPM-DQ (Data Quality aware learning). Both modules were designed based on the assumption that data and labels should form coherent probabilistic distributions. The task of DPM-LQ was to produce feature representations without overfitting to the distribution of noisy semantic labels, and the task of DPM-DQ was to eliminate data noise from ‘False Reject’ and ‘False Accept’ during inference by correcting the prediction confidence of noisy data based on its quality distribution.

Huang et al. [33] since the vision transformer (ViT) has demonstrated significant performance gains in various computer vision tasks by effectively capturing long-range dependencies, recent FAS methods have investigated the potential of leveraging the channel-wise features derived from self-attention (SA) in ViT. While channel-wise features effectively capture discriminative local attention, the complementary information existing between different channels remained undiscovered in prior methods. This paper investigates the unexplored characteristics of complementary channel information within ViT and proposes to incorporate both channel-wise and complementary channel information to learn long-range and discriminative features for FAS. Two modules, Channel Difference Self-Attention (CDSA) and Multi-head Channel Difference Self-Attention (MCDSA), are used to facilitate learning complementary channel characteristics and enhancing both feature discriminability and representational capacity. Using CDSA and MCDSA, the paper presents a custom Channel Difference Transformer (CDformer).

Antil et al. [34] provide a comprehensive review of the state-of-the-art works published over the past decade and discuss the temporal evolution of the FAS/PAD field. The paper reviews different types of attacks against facial authentication systems and covers key features used for FAS models. It also discusses the FAS design approaches followed by the backbone architectures used to design these methods. It also discusses publicly available databases for FAS models, standard protocols, and benchmarking methods. An extensive comparative analysis of experimental results from different PAD methods over the past decade is provided, highlighting limitations and current challenges. It observes a lack of a robust, large-scale general dataset for FAS and underscores the need for new developments in the field.

3. The Proposed Method

This work addresses a specific problem of developing a passive face spoofing detection system that does not require user interaction. The system is tailored for integration with an employee work time registration platform, using photos provided by that system, which were usually taken with a single front smartphone camera without infrared sensors.

The proposed method receives photos from the “Identification of the person in the photo” module, as shown in Figure 1, and detects whether the face visible in the photo is a face of a live person standing in front of a camera or the face was shown to the camera on a photo, video, or in another way.

Our contributions and core innovations comprise:

• Development of a heterogeneous ensemble combining statistical and deep learning models.
• Design and implementation of several new models that are used together with already existing models within the ensemble.
• Designing the CNN architecture presented in Figure 3 and in

  Table 1. Details about the proposed neural network architecture for the image analysis module.

  Layer (Type: Depth-Idx)	Output Shape	Param #
  SpoofDetectionNet	[32, 2]	–
  Conv2d: 1-1	[32, 16, 64, 64]	448
  BatchNorm2d: 1-2	[32, 16, 64, 64]	32
  Conv2d: 1-3	[32, 16, 64, 64]	2320
  BatchNorm2d: 1-4	[32, 16, 64, 64]	32
  MaxPool2d: 1-5	[32, 16, 32, 32]	–
  Dropout: 1-6	[32, 16, 32, 32]	–
  Conv2d: 1-7	[32, 32, 32, 32]	4640
  BatchNorm2d: 1-8	[32, 32, 32, 32]	64
  Conv2d: 1-9	[32, 32, 32, 32]	9248
  BatchNorm2d: 1-10	[32, 32, 32, 32]	64
  MaxPool2d: 1-11	[32, 32, 16, 16]	–
  Dropout: 1-12	[32, 32, 16, 16]	–
  Flatten: 1-13	[32, 8192]	–
  Linear: 1-14	[32, 64]	524,352
  BatchNorm1d: 1-15	[32, 64]	128
  Dropout: 1-16	[32, 64]	–
  Linear: 1-17	[32, 2]	130
  Total params:		541,458
  Trainable params:		541,458
  Non-trainable params:		0

  , which outperforms both domain-specific and state-of-the-art general-purpose deep learning models. It should be noted that a vital component of this architecture is the pre-processing step.
• Creation of a smartphone bezel-detection algorithm, which was consistently included in the highest-performing model combinations (see the detailed tables in the Appendix A).
• Development of a CNN-based smartphone detection model presented in Figure 4 and

  Table 2. Details about the proposed neural network architecture for the smartphone detection module.

  Layer (Type: Depth-Idx)	Output Shape	Param #
  BezelDetectionNet	[32, 2]	–
  Conv2d: 1-1	[32, 32, 128, 128]	896
  BatchNorm2d: 1-2	[32, 32, 128, 128]	64
  MaxPool2d: 1-3	[32, 32, 64, 64]	–
  Conv2d: 1-4	[32, 64, 64, 64]	18,496
  BatchNorm2d: 1-5	[32, 64, 64, 64]	128
  MaxPool2d: 1-6	[32, 64, 32, 32]	–
  Conv2d: 1-7	[32, 128, 32, 32]	73,856
  BatchNorm2d: 1-8	[32, 128, 32, 32]	256
  MaxPool2d: 1-9	[32, 128, 16, 16]	–
  Conv2d: 1-10	[32, 256, 16, 16]	295,168
  BatchNorm2d: 1-11	[32, 256, 16, 16]	512
  MaxPool2d: 1-12	[32, 256, 8, 8]	–
  Linear: 1-13	[32, 1024]	16,778,240
  Linear: 1-14	[32, 512]	524,800
  Linear: 1-15	[32, 2]	1026
  Total params:		17,693,442
  Trainable params:		17,693,442
  Non-trainable params:		0

  .
• Implementation of a novel face context analysis algorithm.
• Identification of optimal combinations of models for the ensembles.
• Integration of the proposed solution into a work time registration system to detect face spoofing attempts.

The proposed method is structured as a two-tiered stacking ensemble, as presented in Figure 5.

The first level of the model consists of the following modules:

• Smartphone bezel-detection algorithm
• Smartphone detection using a CNN
• Face context analysis
• Image analysis using a CNN
• (optionally) Image analysis using the second deep neural network
• (optionally) Image analysis using the third deep neural network

First, each of the modules predicts whether the face visible in the photo is real or spoofed. Then, the predictions of the modules are fed into a neural network that acts as a meta-classifier, producing the final authenticity decision. The individual modules are presented in detail in the following sections.

Explanation why a cascade of models is used for recognition: The experiments showed that the convolutional neural network that we proposed was the best-performing model of the 10 models we tested. However, using a cascade committee of various models in the form of a stacking ensemble yielded further accuracy improvements, justifying our choice of a committee-based approach.

3.1. Smartphone Bezel-Detection Algorithm

The first developed module is a bezel-detection algorithm designed to detect whether a spoof face is displayed on an electronic device such as a smartphone or tablet. It does so by detecting high-contrast edges around the displayed face, and if enough edges of a certain size are detected, then the spoof is detected.

First, pre-processing is performed on each image to normalize it and adapt it to different sizes of input images and positions of faces. The input images undergo resizing to 256 × 256 pixels. Then, the face is detected, and the image is converted to grayscale. An example of this pre-processing is presented in Figure 6.

To detect a bezel, the center point of the detected face is selected, and then the image is analyzed in four directions toward the edges. Iteratively, smaller and smaller fragments of the image are selected, and their average brightness is calculated. If the average is below a set value (in this case, a threshold of 28 for values between 0 and 255), then the fragment is considered a part of the bezel of an electronic device. This is presented in Algorithm 1. If bezels are detected in at least two directions, then the algorithm detects a spoofing attempt. For most cases, this ensures that even if a single bezel is detected due to uniform background colors or elements of the face (like dark hair), the face will not be mistaken for a spoof. This process is depicted in Figure 7.

This algorithm is optimized for computational efficiency to minimize processing time. In addition, all algorithms are executed in parallel, further accelerating the overall analysis.

Algorithm 1: Algorithm used for detecting smartphone bezels.

# Requires a grayscale image with a detected face  def detect_bezel(gray): while current_bezel_size >= min_bezel_size: # Select a fragment of the image bezel = gray[y_start:y_start + current_bezel_size, x_start:x_end] # Check if the average value of the pixels in the fragment is below the threshold average = np.mean(bezel) if average <= gray_threshold: candidates.append((x_start, y_start, x_end, y_start + current_bezel_size, average))    # Move the fragment y_start += 1    if y_start + current_bezel_size >= y_end: current_bezel_size -= 1 y_start = 0

3.2. Smartphone Detection Using CNN

Complementing the bezel detector, a smartphone detection algorithm was developed using a convolutional neural network architecture. This module works closely with other parts of the system and helps to detect if a face is displayed on an electronic device, such as a smartphone or a tablet, through features such as shape, bezel, or on-screen elements.

First, a model based on YOLOv4 was developed, but it did not work well in this case, as it was trained to detect whole objects, while in this use case, most of the time, only parts of the device are visible in the camera view. This necessitated the creation of a separate model for this task. The architecture of this model is presented in Figure 4.

The CNN was trained to detect whether there is a smartphone in the image and whether it appears as part of the image (for example, the presented person is holding it), or is used to actively display a spoofed face. Therefore, while this algorithm can provide useful information about the authenticity of the photo, it is best used as part of the whole system.

3.3. Face Context Analysis

This module enhances spoof detection by analyzing regions surrounding the face for inconsistencies. The algorithm aims to detect irregularities in the input image, such as sharp edges or objects, by comparing the area close to the detected face against more distant areas. The advantage of this approach is that it allows the system to more accurately detect spoofed faces displayed on electronic devices or printed by analyzing not just the face, but also its close surroundings.

The area close to the face (the "near" context) is defined as 40% of the width and height of the face outside of the detected face rectangle in all directions. The area further away from the face (the “far” context) is defined as an additional 40% in all directions outside of the close area.

For both areas, edge detection is performed using the Sobel operator. Then, for the detected edges, average pixel intensity is calculated separately for the area closer to the face and further away from it. Then, histograms are computed with the calculated intensities over 64 bins. This method could also be interpreted as a sharpness analysis of the image, as it is based on the amount and intensity of the detected edges. A sample output of this part of the algorithm is shown in Figure 8.

Then, for the histograms generated this way-separately for the close and far areas, four metrics are calculated and compared to each histogram from the dataset already generated this way. Those four metrics are Correlation, Chi-Square, Intersection, and Bhattacharyya distance. The distance between the histograms is then defined as the sum of all four metrics. This calculation is shown in Equation (5). It is important to normalize the metrics by scaling them down to a range of [0; 1], as they all cover different ranges. For Correlation and Intersection, the calculated value is subtracted from 1, as the value of zero means that the histograms are different. This ensures that the value of 1 represents two very different histograms for all matrices and thus a possible spoofing attempt.

$$\begin{array}{cc}\hfill correlation(H_1,H_2)& =1-norm\left({\displaystyle \frac{{\sum }_I(H_1\left(I\right)-\overline{H_1})(H_2\left(I\right)-\overline{H_2})}{\sqrt{{\sum }_I{(H_1\left(I\right)-\overline{H_1})}^2{\sum }_I{(H_2\left(I\right)-\overline{H_2})}^2}}}\right)\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill chi\_square(H_1,H_2)& =norm\left(\sum _I{\displaystyle \frac{{\left(H_1\left(I\right)-H_2\left(I\right)\right)}^2}{H_1\left(I\right)}}\right)\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill intersection(H_1,H_2)& =1-norm\left(\sum _{I}min(H_1\left(I\right),H_2\left(I\right))\right)\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill bhattacharyya(H_1,H_2)& =norm\left(\sqrt{1-{\displaystyle \frac{1}{\sqrt{\overline{H_1}\overline{H_2}{N}^2}}}\sum _I\sqrt{H_1\left(I\right)·H_2\left(I\right)}}\right)\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill D(H_1,H_2)& =\left(correlation(H_1,H_2)+chi\_square(H_1,H_2)\right)\hfill \\ \hfill & +\left(intersection(H_1,H_2)+bhattacharyya(H_1,H_2)\right)\hfill \end{array}$$

(5)

where $norm\left(x\right)={\displaystyle \frac{1}{1+e^{-x}}}$ and $\overline{H_k}={\displaystyle \frac{1}{N}}{\sum }_J{H}_k\left(J\right)$. $H_1$ and $H_2$ represent the compared histograms.

Based on these distances, a kNN algorithm is performed to select the three closest neighbors (with the lowest distance to the new histogram), and the spoofing probability is calculated depending on how many of those histograms belong to authentic images and spoof images.

This method strengthens the system’s ability to detect inconsistencies by focusing not only on the face but also on the broader context of the surrounding image. By comparing edge intensities in two areas around the face to known distributions, it is possible to detect sudden changes and edges in the image’s foreground that could indicate a spoof attempt.

3.4. Image Analysis Using CNN

A dedicated CNN was developed to analyze full images and assess face authenticity based on subtle cues, such as face quality, edge presence, and visual consistency. The goal of this module is to analyze elements of the image that would otherwise be difficult to process with algorithms, such as the quality of the face, the appearance of objects or edges close to the face, or the cohesion of the image.

The network architecture consists of two sets of convolutional layers followed by batch normalization layers, ReLU activation functions, pooling, and dropout layers (CONV => RELU => CONV => RELU => POOL => DROPOUT). Then, a single fully connected layer is added to the network. Finally, a SoftMax classifier is deployed to output the probabilities of the face being authentic or spoofed. The full architecture is shown in Figure 3. The face score is determined by the highest of those probabilities.

To better account for real use cases, the images were augmented. The enhancements included a random horizontal flip (mirror flip), a random rotation by up to 5 degrees, and a random change in the brightness and saturation of the image (by up to 10%). Such parameters were intentionally small and were selected to represent real-world conditions. The operations also included scaling the images down to a size of 64 × 64 pixels.

The images used for training and testing were additionally preprocessed by centering them around the detected face; this is to ensure that the network can analyze the face and its close surroundings, and is especially useful if the face is only part of a bigger image. Experimental evaluation has shown a great increase in accuracy after this centering step, compared to when the images are scaled down without centering. This is therefore an integral part of this module that makes it perform better.

The output layer of the convolutional neural network contains a SoftMax classifier, which has two outputs. Each output can be interpreted as the probability that the image belongs to the specific class, which in the case of this system is “authentic” or “spoof”.

3.5. Calculating the Result and Final Probability

A lightweight neural network was developed to aggregate outputs of the above-presented models and to compute the final spoofing probability, instead of relying on pre-defined weights for each module individually. The network takes each partial result and its probability (for example, a result could be “spoof” with probability 80%) and produces a final result, classifying the face as either “authentic” or “spoof” with an associated probability.

The proposed architecture consists of two fully connected layers with ReLU activation functions and an output layer using a sigmoid function classifier (Figure 9 and

Table 3. Details about the proposed neural network architecture used as the final classifier.

Layer (Type: Depth-Idx)	Output Shape	Param #
ProbabilityNN	[1, 1]	–
Linear: 1-1	[1, 10]	50
Linear: 1-2	[1, 1]	11
Sigmoid: 1-3	[1, 1]	–
Total params:		61
Trainable params:		61
Non-trainable params:		0

). The network accepts floating-point values, which represent the absolute probability of the face being inauthentic. For example, if a face is determined to be authentic with a probability of 80%, then the input would be 20% as the probability that the face is spoofed. The output of the network is a single value [0; 1] that represents the probability that the face is spoofed. If the probability is greater than 0.5, the face is marked as inauthentic.

Due to the sigmoid transfer function in the last layer of this network, its output can be interpreted as the probability that a given face is real. To do this, the network training time should be limited before the predicted values in the final training phase start to be very close to zero or one, but the training time should also be long enough to achieve high classification accuracy.

4. Experimental Evaluation

4.1. Experimental Setup

The proposed face-authenticity detection system was evaluated using four datasets:

• A small dataset ($N=477$) prepared by us, which includes faces captured in the described scenario. There are 242 authentic face images and 234 spoof face images. This is the most realistic database for the purpose of our system. However, this dataset includes faces of individuals who did not agree to publish their images, and for this reason, we cannot share the data. The three remaining datasets are available for download.
• The open Spoofing Dataset [35] (N = 1054), which includes authentic faces and those displayed on electronic devices (522 authentic images, 532 spoof images).
• The Large Crowd-collected Facial Anti-Spoofing Dataset by D. Timoschenko et al. [5] (N = 16,284, 1942 authentic images, 14,342 spoof images).
• Parts of the Celeb-A Dataset by Zhang et al. [6]. This dataset contains over 200 thousand images, which is too much to process with our available resources. Instead, a total of N = 23,052 images were randomly sampled (13,543 authentic images, 9509 spoof images).

Our system was developed in Python using PyTorch, Scikit-Learn, OpenCV, and other libraries. The source code is available at https://github.com/Stukeley/FaceAuthenticityDetection.

Technical specifications of the development and testing system: We used three computers, and the tests were split among them. Two computers were equipped with AMD Ryzen 9 7950X CPU, 128 GB RAM, NVidia RTX4070 GPU, 512 GB FDD, 8 TB HDD, Windows Operating System, and one with 2 x AMD EPYC 7282, 512 GB RAM, NVidia Tesla T4 GPU, 512 GB FDD, 6 TB HDD, Windows Operating System.

We tested our solutions and compared the results with the results obtained with other methods. The other methods include two implementations of facial spoofing detection, for which we were able to find the source code and train them on our data. Unfortunately, many authors either did not provide the source code, which made their methods practically impossible to use, or provided non-working code and did not respond to our emails when we asked for help to run their code. The remaining methods are the newest and well-known deep learning models, where we used the PyTorch implementation.

• Depth Prediction with CNN [36], a method developed for facial spoofing detection using an open-source implementation by Anand Pandey [37].
• MobileNet [38] implemented for facial spoofing detection in an open-source code by Nguyen Dinh Quy [39].
• The ConvNeXt architecture [40] without pre-trained weights for the “BASE” model, using PyTorch implementation.
• The EfficientNetV2 architecture [41] without pre-trained weights for the “S” model size, using PyTorch implementation.
• The Vision Transformer architecture [42] without pre-trained weights for the b_16 architecture.
• The Swin Transformer architecture [43] without pre-trained weights for the V2_S architecture.

To evaluate each system, five runs of a stratified 5-fold cross-validation were performed, starting each time from a different random data split and a different seed for training the models to account for stochasticity in training and testing performance, for a total of 25 training/testing pairs per dataset for each model. In a given cross-validation run, each model was trained using the same data split.

For the bezel-detection algorithm, no training was performed at this step; instead, the parameters, including the grayscale threshold for detected bezels, the minimum size, and the number of bezels detected to consider an image spoofed, were empirically adjusted based on images from previous evaluations.

The following models: ConvNeXt, EfficientNetV2, Vision Transformer, and Swin Transformer were trained with the same parameters: $LR=0.001$, $BS=32$, $EPOCHS=10$, $DROPOUT\_RATE=0.25$, which were the default parameters in the PyTorch implementations we used. The image size for all models was set to (64 × 64), except for the Vision Transformer and Swin Transformer architectures, which were instead trained and evaluated using images of size (224 × 224), as these architectures were built to work with larger image sizes. Values different from those chosen were also tested, resulting in similar or worse performance. No pre-trained weights were used, as this makes the comparison between the proposed system and other tested architectures more reliable. All architectures were trained and evaluated on the same datasets.

4.2. Experimental Results

First, we present the partial results of individual models of the proposed system as well as of other tested systems. Then, the results obtained with different stacking and voting committees are presented. The following configurations were tested:

• Our four developed modules: bezel detection, smartphone detection CNN, context analysis, and image analysis CNN (baseline).
• Other methods used for face spoofing analysis: Depth prediction, MobileNet, ConvNeXt, EfficientNetV2, Vision Transformer, and Swin Transformer.
• Various combinations of the above modules and methods in the stacking ensemble (between 2 and 7 models).
• Various combinations of the above modules and methods in the voting ensemble (between 2 and 7 models).

The obtained results, including the ensembles with 3, 4, 5, 6, and 7 models with the best accuracy across all datasets, are presented in

Table 4. Prediction accuracy [%] of the evaluated methods and ensembles in five runs of a 5-fold cross-validation (a total of 25 training/test pairs). At the bottom, the best-performing ensembles of 3, 4, 5, 6, and 7 models are presented. The complete results of 240 different stacking and voting ensembles are presented in the Appendix A.

Our Methods	Small dataset	Spoofing Data	Crowd-collected	CelebA
Bezel (B)	70.99 ± 4.22	57.67 ± 3.05	37.63 ± 0.69	47.05 ± 0.50
Smartphone CNN (S)	81.35 ± 3.83	86.78 ± 2.63	90.27 ± 0.59	89.26 ± 0.47
Context (C)	77.98 ± 4.21	91.95 ± 1.86	90.47 ± 1.05	98.22 ± 0.17
Image analysis CNN (N)	85.64 ± 4.59	91.03 ± 1.98	98.20 ± 0.36	94.71 ± 0.24
Other Methods	Small dataset	Spoofing Data	Crowd-collected	CelebA
EfficientNetV2 (E)	83.21 ± 7.91	91.48 ± 3.80	88.53 ± 0.95	91.50 ± 0.44
ConvNeXt (X)	54.71 ± 6.47	56.03 ± 4.60	90.47 ± 1.05	90.37 ± 4.05
MobileNet	71.73 ± 2.33	73.86 ± 0.89	90.83 ± 0.19	87.96 ± 0.34
Depth prediction	53.25 ± 3.33	41.27 ± 5.25	89.37 ± 7.01	85.75 ± 8.27
ViT (V)	52.52 ± 6.04	56.56 ± 10.46	87.92 ± 0.65	75.49 ± 0.55
Swin Transformer (T)	50.32 ± 3.43	59.48 ± 5.54	88.07 ± 0.67	58.75 ± 0.32
Ensembles	Small dataset	Spoofing Data	Crowd-collected	CelebA
C,N,E (stacking)	91.18 ± 3.95	96.35 ± 1.69	98.89 ± 0.22	98.94 ± 0.17
S,C,N,E (stacking)	92.29 ± 3.16	96.77 ± 1.46	98.93 ± 0.22	99.14 ± 0.11
S,C,N,X,E (stacking)	92.15 ± 3.36	96.85 ± 1.51	99.00 ± 0.17	99.19 ± 0.10
B,S,C,N,X,E (stacking)	92.46 ± 3.17	96.82 ± 1.54	98.99 ± 0.19	99.18 ± 0.12
B,S,C,N,X,E,V (stacking)	92.62 ± 3.05	96.88 ± 1.48	98.99 ± 0.17	99.20 ± 0.12

. The ensembles were chosen based on mean rank across the four datasets, and we selected the best 3-model ensemble, the best 4-model ensemble, and so forth. The ranks and classification accuracy of the ensembles are presented in

Table A3. All obtained accuracies [%] for different stacking and voting ensemble combinations over five runs of a 5-fold cross-validation, part 1. The rank was obtained by averaging the position for all four datasets, for each combination. The models used are Bezel (B), Smartphone Detection CNN (S), Context (C), Image Analysis CNN (N), EfficientNetV2 (E), ConvNeXt (X), and Vision Transformer (V).

Combination	Small	Spoofing	Large	CelebA	Rank
B,S,C,N,X,E,V (stacking)	92.62 ± 3.05	96.88 ± 1.48	98.99 ± 0.17	99.20 ± 0.12	2.75
S,C,N,X,E (stacking)	92.15 ± 3.36	96.85 ± 1.51	99.00 ± 0.17	99.19 ± 0.10	4.25
B,S,C,N,X,E (stacking)	92.46 ± 3.17	96.82 ± 1.54	98.99 ± 0.19	99.18 ± 0.12	4.5
B,S,C,N,E (stacking)	92.63 ± 3.04	96.88 ± 1.53	98.95 ± 0.20	99.12 ± 0.13	7.25
S,C,N,X,E,V (stacking)	91.69 ± 3.73	96.93 ± 1.39	98.98 ± 0.18	99.19 ± 0.10	7.5
B,S,C,N,E,V (stacking)	92.80 ± 3.21	96.80 ± 1.57	98.95 ± 0.19	99.10 ± 0.13	9
S,C,N,E,V (stacking)	92.11 ± 4.04	96.89 ± 1.44	98.94 ± 0.21	99.12 ± 0.12	10.75
S,C,N,E (stacking)	92.29 ± 3.16	96.77 ± 1.46	98.93 ± 0.22	99.14 ± 0.11	11.5
B,S,C,N,X (stacking)	91.61 ± 2.67	96.12 ± 1.31	98.99 ± 0.17	99.17 ± 0.11	11.75
B,C,N,X,E,V (stacking)	92.04 ± 3.47	96.30 ± 1.66	98.95 ± 0.14	99.09 ± 0.10	13.25
B,S,C,N,X,V (stacking)	91.43 ± 2.91	96.07 ± 1.33	98.99 ± 0.17	99.17 ± 0.09	13.75
C,N,X,E,V (stacking)	91.28 ± 4.00	96.38 ± 1.69	98.97 ± 0.15	99.05 ± 0.15	16.5
S,C,N,X (stacking)	90.92 ± 2.86	95.96 ± 1.27	99.01 ± 0.18	99.16 ± 0.12	17
B,C,N,X,E (stacking)	91.69 ± 3.83	96.18 ± 1.66	98.95 ± 0.14	99.08 ± 0.13	17
S,C,N,X,V (stacking)	90.73 ± 3.05	95.88 ± 1.28	98.99 ± 0.16	99.16 ± 0.09	20.25
B,C,N,E,V (stacking)	92.16 ± 3.66	96.34 ± 1.66	98.89 ± 0.21	98.94 ± 0.18	20.5
B,S,C,N,V (stacking)	91.48 ± 2.72	95.91 ± 1.39	98.94 ± 0.20	99.09 ± 0.13	21.25
B,S,C,N (stacking)	91.29 ± 2.72	95.86 ± 1.34	98.95 ± 0.21	99.11 ± 0.12	22
B,C,N,E (stacking)	91.85 ± 4.07	96.18 ± 1.65	98.91 ± 0.21	98.93 ± 0.17	23
C,N,X,E (stacking)	90.61 ± 3.80	96.25 ± 1.80	98.95 ± 0.16	99.05 ± 0.11	23.25
S,C,N,V (stacking)	90.73 ± 3.06	95.93 ± 1.38	98.94 ± 0.22	99.07 ± 0.13	25
C,N,E,V (stacking)	91.19 ± 4.28	96.30 ± 1.62	98.89 ± 0.21	98.96 ± 0.18	27
C,N,E (stacking)	91.18 ± 3.95	96.35 ± 1.69	98.89 ± 0.22	98.94 ± 0.17	27
S,C,N (stacking)	90.65 ± 2.99	95.81 ± 1.32	98.94 ± 0.23	99.04 ± 0.10	29
B,C,N,X (stacking)	89.75 ± 3.80	94.43 ± 1.97	98.96 ± 0.16	98.98 ± 0.10	38.25
B,C,N,X,V (stacking)	89.40 ± 4.05	94.33 ± 2.00	98.97 ± 0.15	98.98 ± 0.10	40.25
B,S,C,X,E,V (stacking)	90.02 ± 4.15	95.97 ± 1.74	96.53 ± 0.41	99.03 ± 0.12	41.25
B,S,N,X,E (stacking)	91.74 ± 3.22	95.62 ± 1.88	98.37 ± 0.29	97.15 ± 0.28	41.25
B,S,C,X,E (stacking)	90.11 ± 4.12	95.80 ± 1.78	96.57 ± 0.42	99.01 ± 0.12	43.75
B,S,N,X,E,V (stacking)	91.60 ± 3.46	95.55 ± 2.12	98.37 ± 0.32	97.10 ± 0.26	43.75
S,N,X,E,V (stacking)	91.55 ± 3.90	95.58 ± 2.25	98.37 ± 0.31	97.09 ± 0.27	44
B,S,N,E (stacking)	92.16 ± 3.28	95.74 ± 2.12	98.30 ± 0.36	96.68 ± 0.34	44.75
S,N,X,E (stacking)	91.40 ± 3.70	95.68 ± 1.96	98.35 ± 0.32	97.10 ± 0.28	45.25
B,S,N,E,V (stacking)	92.12 ± 3.24	95.69 ± 2.19	98.31 ± 0.34	96.63 ± 0.35	45.75
C,N,X (stacking)	87.78 ± 4.20	94.43 ± 2.01	98.96 ± 0.16	99.00 ± 0.13	47.75
C,N,X,V (stacking)	87.97 ± 3.85	94.33 ± 2.05	98.96 ± 0.15	98.99 ± 0.10	47.75
B,C,N (stacking)	89.76 ± 3.96	94.38 ± 1.92	98.90 ± 0.22	98.84 ± 0.13	47.75
B,C,N,V (stacking)	89.73 ± 3.87	94.31 ± 1.99	98.91 ± 0.21	98.87 ± 0.12	48.5
S,C,X,E,V (stacking)	88.75 ± 5.02	95.87 ± 1.80	96.46 ± 0.39	99.01 ± 0.13	50.5
S,N,E,V (stacking)	91.74 ± 3.86	95.59 ± 2.18	98.29 ± 0.37	96.64 ± 0.34	50.5
S,C,E,V (stacking)	89.16 ± 4.38	96.04 ± 1.83	96.13 ± 0.35	98.93 ± 0.18	50.75
S,C,X,E (stacking)	88.80 ± 4.70	95.86 ± 1.80	96.49 ± 0.43	98.99 ± 0.11	51
B,S,C,E,V (stacking)	89.60 ± 4.38	95.84 ± 1.78	96.13 ± 0.37	98.93 ± 0.17	52
S,C,N,E (voting)	91.92 ± 3.47	96.27 ± 1.54	94.67 ± 1.30	98.02 ± 0.35	53.25
B,S,C,N,E (voting)	92.04 ± 3.26	94.61 ± 2.19	96.70 ± 0.40	97.37 ± 0.51	54
S,N,E (stacking)	91.21 ± 3.81	95.66 ± 2.08	98.29 ± 0.36	96.66 ± 0.36	54
B,S,C,E (stacking)	89.44 ± 4.38	95.84 ± 1.73	96.08 ± 0.37	98.91 ± 0.16	54.5
S,C,N (voting)	90.48 ± 2.19	94.87 ± 1.71	97.14 ± 0.34	98.69 ± 0.17	56.25
S,C,E (stacking)	88.70 ± 5.00	95.93 ± 1.93	96.11 ± 0.37	98.92 ± 0.15	56.5
C,N (stacking)	88.02 ± 3.58	94.43 ± 1.94	98.89 ± 0.22	98.83 ± 0.13	57
B,S,N,X,V (stacking)	91.10 ± 2.11	93.98 ± 1.43	98.36 ± 0.32	97.01 ± 0.34	57.75
B,N,X,E,V (stacking)	91.24 ± 4.21	94.71 ± 2.19	98.31 ± 0.31	96.42 ± 0.25	58.75
C,N,V (stacking)	87.88 ± 3.68	94.33 ± 2.00	98.88 ± 0.22	98.86 ± 0.14	59
B,N,X,E (stacking)	90.58 ± 4.79	94.73 ± 2.11	98.31 ± 0.34	96.39 ± 0.30	61.75
B,S,N,X (stacking)	90.42 ± 2.91	93.67 ± 1.68	98.36 ± 0.32	96.97 ± 0.36	62.75
C,N,E (voting)	89.00 ± 3.84	95.47 ± 1.66	96.53 ± 0.38	98.10 ± 0.22	63
N,X,E,V (stacking)	89.69 ± 4.79	94.69 ± 2.22	98.31 ± 0.31	96.42 ± 0.32	65
B,S,N,V (stacking)	91.41 ± 2.85	93.85 ± 1.58	98.30 ± 0.37	96.32 ± 0.23	65.25
S,N,X,V (stacking)	88.89 ± 3.36	93.93 ± 1.35	98.38 ± 0.32	96.97 ± 0.39	66.5

.

We have performed hyperparameter optimization for our own neural network models (smartphone detection CNN and image analysis CNN) by varying the learning rate, batch size, epoch count, and dropout rate over the baseline values of $LR=0.001$, $BS=32$, $EPOCHS=10$, $DROPOUT\_RATE=0.25$. The reason for using these baseline values as a starting point was to start from the same hyperparameters as the other models (ConvNeXt, EfficientNetV2, Vision Transformer, and Swin Transformer) used. The results are presented in

Table A1. Obtained accuracies [%] in a 5-fold cross-validation for two proposed convolutional neural networks (Smartphone Detection CNN and Image Analysis CNN) with different hyperparameter combinations (learning rate (LR), batch size (BS), epochs, and dropout rate).

LR	BS	Epochs	Dropout	Smartphone CNN	Image CNN
Small dataset
0.01	32	10	0.25	68.07 ± 5.56	73.95 ± 2.70
0.001	32	10	0.25	80.74 ± 4.28	85.92 ± 2.19
0.001	16	10	0.25	83.57 ± 4.12	87.80 ± 3.80
0.001	64	10	0.25	77.47 ± 2.97	87.55 ± 3.86
0.001	32	15	0.25	80.51 ± 4.64	86.62 ± 3.31
0.001	32	20	0.25	85.20 ± 2.68	89.91 ± 1.71
0.001	32	10	0.15	80.51 ± 3.04	81.00 ± 2.63
0.001	32	10	0.35	82.16 ± 2.03	82.86 ± 3.48
0.0001	32	10	0.25	88.73 ± 3.05	81.22 ± 3.24
Spoofing Dataset
0.01	32	10	0.25	69.86 ± 2.72	84.59 ± 1.80
0.001	32	10	0.25	86.70 ± 1.92	92.25 ± 0.77
0.001	16	10	0.25	88.13 ± 1.64	91.67 ± 1.86
0.001	64	10	0.25	81.34 ± 2.20	92.06 ± 1.50
0.001	32	15	0.25	88.42 ± 1.97	93.01 ± 0.83
0.001	32	20	0.25	86.12 ± 0.68	92.25 ± 1.77
0.001	32	10	0.15	86.22 ± 2.06	90.95 ± 0.63
0.001	32	10	0.35	86.41 ± 1.23	91.58 ± 2.32
0.0001	32	10	0.25	91.10 ± 1.20	90.81 ± 2.19
Large Crowd-collected
0.01	32	10	0.25	87.80 ± 0.34	85.03 ± 0.19
0.001	32	10	0.25	90.63 ± 0.55	98.26 ± 0.09
0.001	16	10	0.25	90.93 ± 0.33	98.16 ± 0.12
0.001	64	10	0.25	89.91 ± 0.18	98.24 ± 0.27
0.001	32	15	0.25	90.16 ± 0.45	98.50 ± 0.14
0.001	32	20	0.25	90.96 ± 0.10	98.64 ± 0.13
0.001	32	10	0.15	90.16 ± 0.52	97.10 ± 0.15
0.001	32	10	0.35	90.16 ± 0.53	97.33 ± 0.29
0.0001	32	10	0.25	91.74 ± 0.20	97.35 ± 0.24
CelebA
0.01	32	10	0.25	79.84 ± 0.62	84.57 ± 0.38
0.001	32	10	0.25	89.47 ± 0.52	93.80 ± 0.39
0.001	16	10	0.25	90.99 ± 0.34	94.39 ± 0.13
0.001	64	10	0.25	86.72 ± 0.37	93.17 ± 0.21
0.001	32	15	0.25	90.10 ± 0.50	95.16 ± 0.30
0.001	32	20	0.25	89.94 ± 0.28	95.87 ± 0.11
0.001	32	10	0.15	88.75 ± 0.49	94.61 ± 0.13
0.001	32	10	0.35	89.45 ± 0.36	94.95 ± 0.31
0.0001	32	10	0.25	88.83 ± 0.27	92.87 ± 0.21

and

Table A2. Obtained accuracies [%] in a 5-fold cross-validation for the stacking ensembles from Table 4 with default and optimal hyperparameters for our models-Smartphone Detection CNN (S) and Image Analysis CNN (N). The other models used in the ensembles: Bezel (B), Context (C), EfficientNetV2 (E), ConvNeXt (X), and Vision Transformer (V).

Combination	Default Parameters	Optimal Parameters
Small dataset
C,N,E	89.20 ± 1.71	91.78 ± 1.29
S,C,N,E	91.31 ± 2.54	91.02 ± 3.60
S,C,N,X,E	90.84 ± 3.28	91.55 ± 3.44
B,S,C,N,X,E	91.78 ± 2.69	91.55 ± 3.61
B,S,C,N,X,E,V	91.78 ± 2.36	91.55 ± 3.61
Spoofing Dataset
C,N,E	95.56 ± 0.56	96.27 ± 1.30
S,C,N,E	96.36 ± 0.83	96.84 ± 0.23
S,C,N,X,E	96.46 ± 0.78	97.22 ± 0.36
B,S,C,N,X,E	96.75 ± 0.98	97.03 ± 0.36
B,S,C,N,X,E,V	96.56 ± 1.26	97.13 ± 0.68
Large Crowd-collected
C,N,E	98.94 ± 0.12	99.02 ± 0.08
S,C,N,E	98.98 ± 0.23	99.13 ± 0.06
S,C,N,X,E	99.05 ± 0.17	99.12 ± 0.06
B,S,C,N,X,E	99.01 ± 0.22	99.14 ± 0.11
B,S,C,N,X,E,V	99.00 ± 0.20	99.16 ± 0.09
CelebA
C,N,E	99.04 ± 0.10	99.10 ± 0.13
S,C,N,E	99.15 ± 0.08	99.18 ± 0.07
S,C,N,X,E	99.21 ± 0.06	99.25 ± 0.10
B,S,C,N,X,E	99.20 ± 0.10	99.23 ± 0.07
B,S,C,N,X,E,V	99.17 ± 0.11	99.26 ± 0.10

in the Appendix A.

Based on the experimental results presented in Table A1, we determined the set of optimal hyperparameters with which the Smartphone Detection CNN and the Image Analysis CNN obtained the highest classification accuracies. The optimal hyperparameters for the smartphone detection CNN: $LR=0.0001$, $BS=32$, $EPOCHS=10$, $DROPOUT\_RATE=0.25$ The optimal hyperparameters for the image analysis CNN: $LR=0.001$, $BS=32$, $EPOCHS=20$, $DROPOUT\_RATE=0.25$

As can be seen from Table A2, when the Smartphone Detection CNN and the Image Analysis CNN are not used as standalone models but as part of a stacking ensemble, the hyperparameter optimization still improves the results, though the differences are much smaller, especially for the two largest datasets.

In the analysis conducted here, the threshold of the final classifier network output above which a face is considered spoofed was set to $0.50$. If the probability returned by the network is greater than or equal to 50%, then the system predicts the face as spoofed. Although other threshold values were tested, the results were not sensitive to the change in the threshold value, and for that reason, a threshold value of 50% could be used, as presented in

Table A7. Obtained metrics [%] in a 5-fold cross-validation for different values of threshold of the stacking ensemble B,S,C,N,X,E,V. The threshold is the probability above which images are considered spoofed. Small dataset.

Threshold	Accuracy	Precision	Recall	F1-Score
0.05	93.60 ± 1.75	88.25 ± 5.24	98.84 ± 1.42	93.13 ± 2.55
0.10	95.13 ± 2.42	90.9 ± 6.86	98.84 ± 1.42	94.53 ± 3.36
0.15	95.74 ± 3.09	92.19 ± 7.78	98.84 ± 1.42	95.19 ± 3.99
0.20	96.05 ± 2.81	92.66 ± 7.51	98.84 ± 1.42	95.46 ± 3.87
0.25	96.35 ± 2.47	93.33 ± 6.43	98.84 ± 1.42	95.86 ± 3.22
0.30	96.65 ± 2.23	93.82 ± 6.18	98.84 ± 1.42	96.14 ± 3.13
0.35	96.96 ± 1.93	94.55 ± 5.01	98.84 ± 1.42	96.56 ± 2.46
0.40	96.05 ± 2.06	94.43 ± 5.19	96.73 ± 3.16	95.46 ± 3.02
0.45	96.04 ± 2.29	95.65 ± 3.86	95.12 ± 4.18	95.33 ± 3.32
0.50	96.04 ± 2.29	95.65 ± 3.86	95.12 ± 4.18	95.33 ± 3.32
0.55	95.74 ± 2.24	95.61 ± 3.87	94.52 ± 3.60	95.02 ± 3.22
0.60	96.04 ± 2.07	96.56 ± 2.82	94.52 ± 3.60	95.48 ± 2.57
0.65	95.43 ± 1.67	96.56 ± 2.82	93.27 ± 3.31	94.83 ± 1.99
0.70	95.13 ± 2.01	96.56 ± 2.82	92.71 ± 4.13	94.51 ± 2.19
0.75	95.13 ± 2.01	96.56 ± 2.82	92.71 ± 4.13	94.51 ± 2.19
0.80	95.43 ± 2.15	97.13 ± 2.55	92.71 ± 4.13	94.80 ± 2.39
0.85	95.74 ± 2.22	98.28 ± 2.25	91.71 ± 5.15	94.82 ± 3.30
0.90	95.13 ± 2.00	99.38 ± 1.25	89.50 ± 5.88	94.06 ± 3.15
0.95	93.92 ± 2.53	99.33 ± 1.33	87.13 ± 5.57	92.73 ± 3.15

,

Table A8. Obtained metrics [%] in a 5-fold cross-validation for different values of threshold of the stacking ensemble B,S,C,N,X,E,V. The threshold is the probability above which images are considered spoofed. Spoofing dataset.

Threshold	Accuracy	Precision	Recall	F1-Score
0.05	97.23 ± 1.77	96.56 ± 3.97	93.40 ± 7.42	94.67 ± 3.38
0.10	97.23 ± 1.77	95.64 ± 4.12	95.49 ± 6.75	95.32 ± 3.20
0.15	97.60 ± 1.32	95.43 ± 3.92	96.46 ± 6.10	95.74 ± 3.01
0.20	97.96 ± 1.12	95.52 ± 3.71	96.99 ± 5.58	96.08 ± 2.86
0.25	97.96 ± 1.12	95.59 ± 3.56	97.34 ± 5.17	96.31 ± 2.74
0.30	98.20 ± 0.94	95.75 ± 3.44	97.55 ± 4.83	96.51 ± 2.63
0.35	98.44 ± 1.05	95.98 ± 3.38	97.66 ± 4.56	96.70 ± 2.56
0.40	98.32 ± 1.04	96.19 ± 3.30	97.68 ± 4.36	96.83 ± 2.49
0.45	98.44 ± 1.12	96.38 ± 3.25	97.70 ± 4.18	96.94 ± 2.44
0.50	98.44 ± 1.12	96.54 ± 3.19	97.71 ± 4.04	97.03 ± 2.39
0.55	98.32 ± 1.04	96.67 ± 3.14	97.70 ± 3.91	97.10 ± 2.34
0.60	98.20 ± 1.01	96.78 ± 3.08	97.67 ± 3.78	97.14 ± 2.29
0.65	98.20 ± 1.27	96.89 ± 3.04	97.62 ± 3.69	97.18 ± 2.26
0.70	98.44 ± 1.12	97.04 ± 3.01	97.57 ± 3.61	97.23 ± 2.24
0.75	98.44 ± 1.12	97.16 ± 2.97	97.53 ± 3.54	97.28 ± 2.21
0.80	98.44 ± 1.12	97.28 ± 2.94	97.49 ± 3.47	97.32 ± 2.18
0.85	98.44 ± 1.12	97.39 ± 2.90	97.44 ± 3.44	97.36 ± 2.16
0.90	98.32 ± 1.34	97.50 ± 2.86	97.38 ± 3.44	97.38 ± 2.16
0.95	97.24 ± 1.12	97.59 ± 2.83	97.19 ± 3.53	97.33 ± 2.15

,

Table A9. Obtained metrics [%] in a 5-fold cross-validation for different values of threshold of the stacking ensemble B,S,C,N,X,E,V. The threshold is the probability above which images are considered spoofed. Large Crowd-collected dataset.

Threshold	Accuracy	Precision	Recall	F1-Score
0.05	97.97 ± 0.34	97.60 ± 2.76	97.32 ± 3.49	97.40 ± 2.13
0.10	98.45 ± 0.29	97.64 ± 2.70	97.43 ± 3.45	97.48 ± 2.11
0.15	98.56 ± 0.28	97.68 ± 2.65	97.54 ± 3.41	97.55 ± 2.09
0.20	98.70 ± 0.21	97.72 ± 2.60	97.64 ± 3.37	97.62 ± 2.08
0.25	98.80 ± 0.16	97.76 ± 2.56	97.72 ± 3.33	97.69 ± 2.06
0.30	98.92 ± 0.17	97.81 ± 2.52	97.80 ± 3.29	97.76 ± 2.05
0.35	98.95 ± 0.13	97.86 ± 2.49	97.88 ± 3.25	97.82 ± 2.03
0.40	98.94 ± 0.14	97.90 ± 2.45	97.94 ± 3.21	97.87 ± 2.02
0.45	98.96 ± 0.16	97.94 ± 2.42	98.00 ± 3.17	97.93 ± 2.00
0.50	98.92 ± 0.19	97.98 ± 2.39	98.06 ± 3.13	97.98 ± 1.99
0.55	98.87 ± 0.21	98.02 ± 2.36	98.10 ± 3.09	98.02 ± 1.97
0.60	98.85 ± 0.24	98.06 ± 2.33	98.15 ± 3.05	98.06 ± 1.95
0.65	98.80 ± 0.20	98.10 ± 2.31	98.18 ± 3.01	98.10 ± 1.94
0.70	98.69 ± 0.22	98.13 ± 2.28	98.21 ± 2.97	98.13 ± 1.92
0.75	98.58 ± 0.25	98.17 ± 2.26	98.24 ± 2.94	98.16 ± 1.90
0.80	98.49 ± 0.20	98.21 ± 2.24	98.25 ± 2.90	98.19 ± 1.88
0.85	98.21 ± 0.33	98.24 ± 2.22	98.26 ± 2.86	98.21 ± 1.86
0.90	97.99 ± 0.32	98.27 ± 2.20	98.26 ± 2.82	98.23 ± 1.84
0.95	97.37 ± 0.34	98.31 ± 2.18	98.23 ± 2.79	98.24 ± 1.81

and

Table A10. Obtained metrics [%] in a 5-fold cross-validation for different values of threshold of the stacking ensemble B,S,C,N,X,E,V. The threshold is the probability above which images are considered spoofed. CelebA dataset.

Threshold	Accuracy	Precision	Recall	F1-Score
0.05	98.54 ± 0.10	98.27 ± 2.16	98.27 ± 2.76	98.24 ± 1.79
0.10	98.78 ± 0.15	98.25 ± 2.14	98.31 ± 2.74	98.25 ± 1.77
0.15	98.96 ± 0.12	98.24 ± 2.12	98.34 ± 2.71	98.26 ± 1.75
0.20	99.03 ± 0.13	98.24 ± 2.09	98.37 ± 2.69	98.27 ± 1.73
0.25	99.08 ± 0.12	98.24 ± 2.07	98.40 ± 2.66	98.29 ± 1.71
0.30	99.12 ± 0.12	98.24 ± 2.05	98.42 ± 2.64	98.30 ± 1.70
0.35	99.13 ± 0.11	98.25 ± 2.03	98.45 ± 2.62	98.32 ± 1.68
0.40	99.18 ± 0.12	98.25 ± 2.00	98.47 ± 2.59	98.33 ± 1.67
0.45	99.22 ± 0.14	98.26 ± 1.99	98.49 ± 2.57	98.35 ± 1.65
0.50	99.24 ± 0.12	98.28 ± 1.97	98.50 ± 2.54	98.36 ± 1.64
0.55	99.25 ± 0.12	98.29 ± 1.95	98.52 ± 2.52	98.38 ± 1.63
0.60	99.27 ± 0.12	98.31 ± 1.93	98.53 ± 2.50	98.39 ± 1.61
0.65	99.24 ± 0.13	98.33 ± 1.92	98.54 ± 2.47	98.40 ± 1.60
0.70	99.20 ± 0.13	98.34 ± 1.91	98.54 ± 2.45	98.42 ± 1.59
0.75	99.10 ± 0.15	98.36 ± 1.89	98.54 ± 2.43	98.43 ± 1.58
0.80	99.07 ± 0.16	98.38 ± 1.88	98.54 ± 2.41	98.43 ± 1.56
0.85	98.90 ± 0.18	98.40 ± 1.87	98.53 ± 2.39	98.44 ± 1.55
0.90	98.71 ± 0.17	98.42 ± 1.86	98.50 ± 2.37	98.44 ± 1.54
0.95	98.26 ± 0.17	98.45 ± 1.85	98.46 ± 2.37	98.43 ± 1.53

in the Appendix A.

Two conclusions can be drawn from Table 4. First, the best-performing models are stacking ensembles. Second, the best ensembles consisting of between three and seven models include some or all of our proposed methods. The complete results for various combinations of ensembles can be seen in Table A3.

To test the statistical significance of the obtained results, two tests were used:

• A one-way ANOVA, testing a null hypothesis: “There is no difference among group means”. $\alpha =0.05$
• A rank-based Kruskal–Wallis, testing a null hypothesis: “There is no difference among medians of populations from which the groups are taken”. $\alpha =0.05$

To perform these tests, the accuracies obtained for each fold, random seed, and dataset for the ensembles and other methods presented in Table 4 were collected, amounting to a total of 100 values per system for a total of 11 systems. Then, the necessary metrics were calculated based on those values.

• For the ANOVA test: $f=116.36$, $p=1.73\times {10}^{-145}$, which is significantly below the accepted threshold. Therefore, the null hypothesis can be rejected.
• For the Kruskal–Wallis test: $h=476.42$, $p=4.80\times {10}^{-96}$, which is also significantly below the accepted threshold. Therefore, the null hypothesis can be rejected.

This proves that the tested systems differ and that the differences in the results are statistically significant.

The conclusion is that the stacking ensemble increases the accuracy of the system. Adding the ConvNeXt network had a bigger impact on prediction accuracy than further adding other networks, which might be a result of the individual accuracy of those networks being lower than that of ConvNeXt. Only for the Small dataset, including low-accuracy models (ConvNeXt, Vision Transformer-ViT) in the ensemble significantly increases the obtained accuracy.

The results indicate that the system is effective in recognizing spoofing attempts while simultaneously displaying high accuracy in correctly identifying authentic faces. The diversity of datasets used to evaluate the system’s performance shows that it is effective in various cases, with different backgrounds and appearances of the people in front of the camera, as well as that it performs significantly better than other systems when using a small dataset for its training.

As part of further testing, all possible ensemble combinations of the seven methods (Bezel, Smartphone Detection CNN, Context, Image Analysis CNN, EfficientNetV2, ConvNeXt, and ViT) were evaluated in an attempt to find the optimal one, separately for stacking and voting ensembles.

Unfortunately, it was not possible to use the Depth Prediction and MobileNet implementations as parts of the ensemble, as the way these models were implemented made it hard to obtain separate results for each input image, rather than aggregate results for the entire datasets, which was needed to test the ensembles. The full results can be viewed in Table A3 in the Appendix A. As can be seen from the results, the best results were obtained for ensembles encompassing more methods, and removing methods from the ensemble significantly affects the variation of the obtained results in cross-validation. The results also show that stacking ensembles are more effective than voting ensembles.

To test the impact of the selected threshold on the obtained results, multiple values were tested, in increments of $0.05$, and the key metrics of accuracy, precision, recall, and F1-score were noted for the final ensemble (consisting of four implemented methods, ConvNeXt, EfficientNetV2, and Vision Transformer). The results are presented in Table A7, Table A8, Table A9 and Table A10. These results were obtained for a different random seed value; therefore, they differ from the results presented in earlier contexts. The metrics have been calculated for the spoofed faces (positive values).

The obtained results for all metrics are very similar for different threshold values. This is especially noticeable for the larger datasets, where the differences between values are within 1%. This signals that the outputs obtained from the final neural network are often close to $0.00$ or $1.00$.

We have applied Platt scaling to the outputs of the final neural network for the ensemble consisting only of our own methods (Bezel, Smartphone Detection CNN, Context, and Image Analysis CNN). This was performed in an attempt to calibrate the final outputs based on expected results. The implementation used a logistic regression model that was trained on the outputs of the final neural network and the expected classes. However, the obtained results did not improve the classification performance, showing slightly lower or slightly higher accuracy. Since the obtained differences in accuracy are inconsistent and minimal, the technique does not provide a clear benefit. The obtained results are presented in

Table 5. Comparison of accuracy [%] obtained for the stacking ensemble consisting of our own methods (B,S,C,N) with and without Platt scaling with constant parameters and static random seed.

	Small	Spoofing	Crowd	CelebA
Baseline	90.64 ± 2.77	95.37 ± 1.10	96.83 ± 0.28	98.59 ± 0.38
Platt scaling	86.35 ± 2.13	94.49 ± 1.79	97.00 ± 0.66	98.64 ± 0.15

.

An additional commercial system was tested—“A Personalized Benchmark for Face Anti-spoofing” (D. Belli et al.) [44]. However, it did not offer the possibility to train the model using the provided data. Its accuracy was not satisfying, as the system frequently was not able to detect faces in the provided images, and incorrectly marked most authentic images as spoofed. This system was tested for all images from the first, small dataset (N = 477). The obtained results are in

Table 6. Obtained results for the commercial system “A Personalized Benchmark for Face Anti-spoofing” for the small dataset (N = 477).

Outcome	Authentic Faces	Spoof Faces
Authentic	8	0
Spoof	230	144
Not detected	5	90

.

Another tested commercial system is the Doubango 3D Passive face liveness detection [45]. However, it was only possible to test a few photos without paying for the system. The system performed perfectly when we showed our face or its photo to a laptop camera. However, for the photos from our work time registration system, the model predicted most authentic faces as spoofed, and even marked some as “deepfake”, with a very high probability. Unfortunately, it was not possible to test the system with more data to obtain more reliable metrics.

5. Conclusions

We proposed a passive face spoofing detection method designed to operate in a work time registration system under real-world constraints: variable lighting, low image quality (6 kB webp format), and no possibility to invest in additional hardware (e.g., infrared). Our approach effectively balances performance, flexibility, and computational efficiency, making it a viable solution for real-world employee authentication systems.

The presented method is based on a stacking committee, where each member model evaluates different aspects of the photo. Experimental evaluation showed that the use of a stacking committee improves the accuracy of the entire system. We experimentally evaluated 240 different committees and different threshold parameters for the final classifier in the committee.

Additionally, the use of a centering step in the CNN analysis improves the accuracy of that module, but not implementing that step only slightly affects the total accuracy of the system. The use of a committee should also increase the adaptability of the system to real-world conditions, as different methods can detect different types of spoofing attacks.

The modular nature of our design supports future expansion by adding additional models. This will be the subject of our future work. Specifically, we plan to examine the possibility of adding new models to the committee, to evaluate grayscale vs. RGB photos as CNN-based network input, and to train the final neural network with different parameters. Moreover, the acquisition and generation of new data specific to the industrial conditions of the employee registration environment are important aspects to further improve the system.