Writer-Independent Offline Signature Verification Using Local and Global Feature Fusion

Abstract

In offline signature verification, extracting effective features and enhancing identification accuracy remain critical challenges. Traditional feature extraction modules struggle to capture comprehensive, detailed characteristics from signatures alone and often suffer from overfitting issues. This paper introduces ResT, a novel approach that integrates the Residual Network (ResNet) and Transformer architectures for multi-scale feature extraction. The proposed method comprises two components: ResNet blocks and Transformer blocks, designed to extract local and global signature characteristics, respectively. Within the ResNet blocks, we integrate two modules—Spatial-to-Depth Convolution (SPD-Conv) and Spatial and Channel Reconstruction Convolution (SCConv)—to emphasize stroke-level features, thereby improving local feature extraction. For the Transformer blocks, Vision Transformer (ViT) is employed to analyze the signature’s overall shape and stroke trends, capturing global features. To evaluate performance, we implement a writer-independent (WI) verification system and conduct extensive experiments on four public datasets: GPDS, CEDAR, BHSig-Bengali, and BHSig-Hindi. Results demonstrate that the proposed ResT model effectively distinguishes genuine from forged signatures and achieves competitive performance compared to existing methods.

1. Introduction

Handwritten signatures, among the oldest and most widely adopted biometric identification methods [1,2], play an irreplaceable role in law enforcement, finance, and business. As a dynamic behavioral biometric, signatures not only carry legal validity but also reflect the unique motor patterns of the writer. Unlike static biometrics (e.g., fingerprints or irises), signature verification requires analyzing detailed behavioral traits such as stroke dynamics and writing rhythm.

From a technical perspective, handwritten signatures fall into two categories: offline and online. Offline signatures, captured via digitizers or scanned paper documents, exist as static pixel images. Online signatures, conversely, record rich temporal data including coordinate trajectories, pressure variations, timestamps, and azimuth angles. Given the prevalence of paper-based signatures in real-world applications, offline verification remains more practically relevant.

Signature verification fundamentally involves distinguishing intra-class variability (natural differences in a single writer’s signatures across time/conditions) from inter-class variability (differences between writers’ signatures) [3,4]. Forensic experts categorize verification outcomes into four scenarios [3,5]: (1) genuine signatures, (2) skilled forgeries (imitators with knowledge of the target style), (3) simple forgeries (imitators without prior exposure), and (4) random forgeries (unrelated signatures).

A critical challenge in offline signature verification systems (OSV) is the requirement for multiple genuine samples per user to ensure reliability. However, privacy concerns and fraud prevention policies often limit users to providing minimal training data, hindering traditional data-hungry methods. Early approaches relied on writer-dependent (WD) models [5,6,7], where classifiers were trained individually for each user. While effective in small-scale settings, these methods lacked scalability for large user bases. In contrast, writer-independent (WI) models [3,4,8,9,10] employ a unified architecture to process all users’ signatures, offering superior practicality and scalability.

Recent advancements leverage the Siamese Network [11,12], a dual-branch architecture with shared weights that compares feature similarities between input pairs. This design excels in similarity-based tasks but faces two key limitations in OSV: (1) treating signatures as generic images overlooks their stroke-specific dynamics, and (2) imbalanced class distributions (genuine vs. forged) bias models toward majority classes [13]. To address these issues, researchers have introduced innovations such as spatial transformation networks to emphasize stroke features and focal loss functions to balance training data [13].

To enhance accuracy and robustness, modern approaches integrate multi-branch architectures with feature pyramids [14]. These designs extract multi-scale features and capture global context through cross-attention mechanisms. Notably, CrossViT [15], which combines Convolutional Neural Networks (CNNs) and Vision Transformers, effectively handles long-range dependencies in signature data.

Distance measurement is further optimized via adaptive thresholding, where thresholds are dynamically adjusted based on statistical analysis of positive/negative sample distances [16]. This improves adaptability in complex real-world scenarios. Signature verification fundamentally aims to authenticate test signatures by comparing them to reference samples. Two core challenges persist in offline signature verification: information sparsity and style fragility. Information sparsity refers to the fact that signature strokes, which are critical for determining verification outcomes, occupy only a minimal portion of the image space due to their inherently thin and compact nature. Style fragility arises from the variability in writing styles exhibited by the same individual across different contexts—such as changes in writing speed, pressure, or emotional state—while skilled forgers can deliberately replicate genuine signature characteristics to create convincing imitations. Both challenges demand robust feature extraction strategies that can effectively capture and differentiate subtle stroke-level patterns, ensuring reliable discrimination between authentic and forged signatures even under conditions of data scarcity and adversarial mimicry.

This paper proposes ResT, a hybrid architecture that integrates ResNet and Vision Transformer (ViT) to synergistically capture both local and global signature characteristics. ResNet is employed to extract fine-grained stroke-level details, while ViT models high-level contextual relationships across the entire signature. The architecture incorporates three core components: Spatial-to-Depth Convolution (SPD-Conv), which enhances stroke feature learning by expanding spatial dimensions through depth-wise operations; Spatial and Channel Reconstruction Convolution (SCConv), designed to improve generalization by reducing reliance on common features through adaptive spatial-channel interactions; and general convolution layers, which aggregate and refine localized features to form a robust final representation. ResT employs a parallel dual-branch structure: the Transformer branch captures overall signature shapes and stroke trends, while the ResNet module integrates SPD-Conv and SCConv to focus on micro-deformation features of signature strokes. Combined with a lightweight ResNet-18 backbone, these redundancy-reduction techniques achieve superior processing efficiency over traditional hybrid models and outperform the translation-invariant assumptions of general CNNs. Operating within a writer-independent framework, the model eliminates the need for writer-specific training data, enabling scalable deployment across diverse scenarios. Comprehensive experiments conducted under WI evaluation metrics demonstrate that ResT achieves good performance in distinguishing genuine signatures from skilled forgeries across a range of OSV tasks, and has demonstrated competitive performance and robustness compared to existing methods.

2. Related Work

A complete OSV system is typically composed of three main stages: image preprocessing, feature extraction, and classifier training. Among these, the feature extraction stage plays the most crucial role in the construction of the entire verification system. Feature extraction is the core component of OSV systems, directly determining whether the system can obtain key information from signature images that effectively distinguishes genuine signatures from forgeries. To achieve efficient and accurate feature extraction, researchers have explored and applied various techniques, which can be roughly divided into two major categories: those relying on handcrafted features and those based on learning. In addition, strategies such as metric learning, transfer learning, multimodal fusion, and data augmentation are also widely used in the field of OSV, aiming to further enhance the performance and accuracy of signature verification.

2.1. Handcrafted Feature Method

The handcrafted feature method focuses on capturing signature characteristics such as shape, direction, and other attributes through manually designed features. Common approaches include geometric/directional features, mathematical transformations, texture descriptors, and keypoint detection techniques [17,18,19,20,21,22,23,24,25,26,27]. Early research emphasized macro-level form analysis and local directional information, extracting geometric features like signature height, width [28], area, and centroid position [29], alongside directional features such as the Histogram of Oriented Gradients (HOG) [30] and Scale-Invariant Feature Transform (SIFT) [31] to describe overall layout and writing style. Additionally, grid-based statistical features were employed to analyze signature details by dividing images into grids to calculate regional pixel density [32] and using Local Binary Pattern (LBP) [33] statistics to capture distribution patterns. To enhance feature robustness, mathematical transformations such as wavelet transforms [34] were applied, enabling multi-scale decomposition to extract frequency-domain features that address variations in stroke thickness. Local feature detection techniques further identified critical signature points (e.g., start/end points, turning points), with algorithms like SIFT, SURF, BRISK, KAZE, and FREAK [35,36,37,38,39] extracting spatial positions and descriptors for local similarity matching between signatures. While these methods generate handcrafted descriptors tailored to signature images, they struggle to detect subtle imitations in “skilled forged signatures.”

2.2. Learning-Based Methods

Learning-based methods address the limitations of handcrafted features by automatically extracting discriminative representations directly from signature images. These approaches fall into two main categories: dictionary learning/encoding techniques [40,41,42,43] and deep neural network models [44,45,46,47,48,49]. Dictionary learning methods represent signatures as sparse codes or compact feature vectors by constructing a learned dictionary—i.e., a set of basis elements that capture typical signature characteristics. For instance, discriminative dictionaries are often trained using pairs or triplets of genuine and forged signatures, enabling sparse encoding that highlights unique stylistic patterns. In recent years, deep learning has become the dominant paradigm in learning-based offline signature verification (OSV). CNNs, with their ability to model local structures and hierarchically abstract features, have emerged as the de facto standard. A notable example is Siamese Network for Signature Verification (SigNet), the first CNN architecture specifically designed for OSV, which learns robust feature representations by training on both genuine signatures and skilled forgeries. Subsequent advances include the Composite Backbone Capsule Network (CBCapsNet) [50] and the Multi-Scale Dense Network (MSDN) [51], which improve feature extraction through capsule routing and dense inter-signature connections, respectively. Beyond CNNs, other deep architectures have also been explored. Generative Adversarial Networks (GANs) [52] leverage their discriminator components to extract signature features, while Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTMs) [21] incorporate temporal modeling—though their applicability to offline signatures remains limited due to the absence of dynamic stroke data. Finally, variants of ResNet mitigate the degradation problem in very deep networks via residual connections and have demonstrated strong performance in OSV feature extraction tasks.

2.2.1. Metric Learning Methods

Metric learning [53] enhances signature verification performance by learning an appropriate distance metric that captures the relative relationships among signatures. This is typically achieved by optimizing the feature space using either signature pairs or triplets (comprising an anchor, a positive sample—i.e., a genuine signature from the same user—and a negative sample—a forgery or signature from another user). The objective is to minimize the distance between anchor–positive pairs while maximizing the distance between anchor–negative pairs, thereby improving class separability. For instance, methods based on Canonical Correlation Analysis (CCA) or Wasserstein Distance (WD) learn such metrics by training on labeled similar/dissimilar signature pairs. However, these approaches often assume knowledge of the test-set data distribution, which can limit their generalization to unseen users or domains. A key challenge of metric learning in offline signature verification is its reliance on large quantities of labeled pairs or triplets—particularly those involving skilled forgeries, which are difficult and costly to collect. Consequently, the performance of such methods is highly sensitive to both data scale and forgery quality.

2.2.2. Transfer Learning Methods

To mitigate the scarcity of real-world signature data, researchers have adopted transfer learning strategies that leverage auxiliary domains—such as handwritten text—to pre-train models before fine-tuning them for offline signature verification (OSV). For instance, the approach in [53] constructs an auxiliary task using handwritten text images, designs a tailored preprocessing pipeline to generate signature-like inputs, and employs contrastive loss to train a self-supervised feature extractor. This enables the model to learn generalizable signature representations without requiring skilled forgeries.

Beyond conventional transfer learning, self-supervised learning (SSL) [54] and knowledge distillation (KD) [55] have also been successfully adapted to OSV. In particular, KD-based methods utilize a pre-trained “teacher” model (e.g., SigNet) to guide a lightweight “student” model. Knowledge is transferred through handwritten text images, allowing the student to acquire discriminative capabilities while bypassing the need for real or synthetic signature data altogether.

2.2.3. Multimodal Fusion and Data Augmentation Strategies

To further enhance model performance, researchers have proposed multimodal feature fusion and data augmentation strategies to address the limitations of single-modality approaches. Multimodal feature fusion integrates deep learning-derived features—such as local descriptors extracted by CNNs—with handcrafted features like SIFT (for texture patterns) or LBP (for spatial structures), leveraging multi-view representation learning frameworks to align the relationships between these heterogeneous feature spaces. For instance, a hybrid architecture proposed in [56] combines Transformers (for capturing global signature patterns) and CNNs (for extracting local stroke shapes), employing spatial-to-depth blocks and channel reconstruction modules to enhance sensitivity to minor deformations. This design achieves global-local feature complementarity by integrating the global perspective of ViT with the local specificity of CNN decoders.

Data augmentation and synthetic generation techniques tackle the scarcity of real-world signature samples by artificially expanding training datasets. Cognitive-inspired algorithms, such as the GPDS synthetic dataset generator, simulate human writing behaviors to produce both genuine and skilled forgery samples. Image translation models like CycleGAN [57] further address this challenge by translating online signatures (with temporal dynamics) into offline counterparts or generating new real-like signatures through image-to-image translation. Additionally, feature space enhancement methods, such as the artificial immune system framework [58], synthesize features using multi-directional local difference operators, while Gaussian noise injection improves classifier robustness by perturbing feature vectors during training. These strategies collectively aim to expand data diversity, strengthen model generalization, and mitigate the limitations of limited real-world signature data.

Offline handwritten signature verification has made significant progress in developing feature extraction techniques and verification methodologies. Nevertheless, critical challenges persist, including the difficulty in distinguishing skilled forgeries, limited availability of real-world data, and suboptimal generalization across diverse user populations. To address these limitations, future research should prioritize the development of novel feature representation techniques, advanced model architectures, and optimized data augmentation strategies. These efforts aim to enhance both the accuracy and practical applicability of offline signature verification systems in real-world deployment scenarios.

3. Model

This section presents the framework of the proposed offline handwritten signature verification system through four structured components. First, a unified preprocessing methodology for signature images is introduced. This preprocessing stage ensures standardized and normalized input data, creating optimal conditions for subsequent feature extraction and enhancing the accuracy and stability of the overall pipeline. Second, the proposed architecture is named ResT, a designation that encapsulates the integration of residual learning principles and transformer-based mechanisms. This nomenclature provides immediate insight into the network’s design philosophy and technical foundation. Third, the feature extraction network is systematically analyzed, comprising four core modules: (1) a transformer module for global pattern modeling, (2) an SPD-Conv module for spatial feature aggregation, (3) an SCConv module for multi-scale context extraction, and (4) a ResNet module for hierarchical representation learning. Detailed discussions of these components will clarify their individual roles and synergistic interactions during the feature extraction process. Finally, the training strategy and performance validation protocol for the ResT network are described. Rigorous training procedures and comprehensive evaluation metrics are implemented to ensure the model meets predefined performance benchmarks in the offline signature verification task.

3.1. Preprocessing

Variations in acquisition environments often introduce noise, positional shifts, and size inconsistencies in signature images. To address these challenges, standardized preprocessing is essential. The workflow begins with Gaussian filtering to suppress random noise artifacts, followed by the application of the Otsu algorithm to isolate the signature region through thresholding. Subsequently, the geometric center of the signature is computed and aligned to the center of a predefined H × W canvas to ensure spatial consistency. Finally, the image is uniformly scaled to a fixed size of 256 × 256 pixels to standardize input dimensions. Once completed, the preprocessed images are compiled into a standardized dataset for subsequent analysis.

3.2. Model Structure

As illustrated in Figure 1, the proposed model processes two signature images after preprocessing: one genuine (original) signature and a candidate signature (either genuine or forged). During the feature extraction phase, a parallel architecture is employed, combining a Transformer-based module for global feature extraction and a ResNet-based module for local feature extraction. The Transformer module effectively captures global structural patterns in the signature image, generating a global feature vector $f_g$, while the ResNet module focuses on extracting detailed local characteristics, producing a local feature vector $f_l$. To enhance the model’s capacity to distinguish subtle local variations—critical for achieving fine-grained discrimination in the ResT framework—additional modules, including SPD-Conv and SCConv, are integrated to refine the local feature representation. Finally, the global ($f_g$) and local ($f_l$) features are concatenated to form a multi-scale feature vector $F$ ($F_r$ represents the reference signature feature, and $F_q$ represents the query signature feature), which comprehensively integrates both global structural information and local discriminative details. This fusion strategy provides a robust and diverse feature basis for subsequent verification tasks, enabling the model to make more accurate and reliable authenticity judgments.

Figure 1. WI verification system based on the Siamese Network framework.

3.3. Learning Global Features Through Transformer-Based Modules

To enable effective learning of global structural patterns in signature images, the ResT model incorporates a ViT [59] architecture. As illustrated in Figure 2, a convolutional preprocessing module is introduced prior to the Transformer encoder to enhance generalization and prevent overfitting. However, standard convolution operations inherently risk losing global contextual information due to downsampling; To mitigate this limitation, a residual connection is explicitly designed to preserve spatial coherence. This connection ensures that the global structure of the input signature is retained when fed into the ViT framework, enabling the self-attention mechanism to capture holistic representations. The implementation proceeds by partitioning the signature image into non-overlapping patches of uniform size, with each patch encoding a localized region of the signature. These patches are converted into token vectors and augmented with positional embeddings to retain spatial relationships between patches. The resulting sequence is then processed through a multi-layer Transformer encoder, which integrates multi-head attention mechanisms and feedforward networks. By analyzing cross-patch dependencies, the model learns global feature representations that emphasize contextual interactions across the entire signature. This hierarchical design allows the ResT to balance local detail preservation with global pattern recognition, ultimately improving discriminative capacity for signature verification tasks.

Figure 2. The architecture of the proposed ResT.

3.4. Learning Local Features Through ResNet-Based Modules

Although Transformer-based modules excel at capturing global features of signature images, local characteristics—such as stroke shapes and structural forms—remain critical for accurate verification. To address this limitation, we introduce a novel composite residual neural network module designed to enhance the extraction of fine-grained local features. As depicted in Figure 2, the proposed architecture integrates three components for local representation learning: SPD-Conv, SCConv, and a residual convolution module. These modules work synergistically to refine spatial details, with their underlying principles and functional frameworks systematically incorporated into the hybrid model to ensure robust feature discrimination.

3.4.1. SPD-Conv Module

The SPD-Conv module integrates a spatial-to-depth (SPD) layer with a non-strided convolutional layer [60]. The SPD operation achieves single-step downsampling by slicing the input or feature map. For an intermediate feature map X of arbitrary dimensions S × S × C₁, the SPD layer partitions it into a sequence of sub-feature maps through spatial slicing. This process reduces the spatial resolution while preserving channel-wise information, enabling efficient feature compression without significant loss of critical details.

$$\begin{array}{c} \begin{array}{c}{f}_{\mathrm{0,0}}=X\left[0:2:scale,0:2:scale\right],f_{\mathrm{1,0}}=X\left[1:2:scale,0:2:scale\right];\\ f_{\mathrm{0,1}}=X\left[0:2:scale,1:2:scale\right],f_{\mathrm{1,1}}=X\left[1:2:scale,1:2:scale\right];\end{array}\end{array}$$

(1)

As illustrated in Equation (1), the stride = 2 parameter in the SPD layer corresponds to a pixel-wise segmentation process where every two adjacent pixels form a single cycle. As illustrated in Figure 3, this results in S/2 cycles along both the X and Y axes, where each cycle contains only the odd or even rows and odd or even columns of the original map, dividing the original feature layer into S²/4 sub-feature maps. Each sub-feature map is further partitioned into four components: $f_{\mathrm{0,0}},f_{\mathrm{1,0}},f_{\mathrm{1,0}},f_{\mathrm{1,1}}$. Subsequently, feature maps sharing the same indices are recombined to generate 2² new feature maps. These are then concatenated to produce a consolidated feature map of dimensions (S/2, S/2, 4C). A non-strided convolutional operation is applied to this output to refine the feature representation.

Figure 3. The architecture of SPD-Conv Module.

Following the SPD feature transformation layer, the authors introduced a non-strided (stride = 1) convolutional layer to further refine the feature maps from X′(S/2, S/2, 4C) to X″(S/2, S/2, C). The rationale for employing a non-strided convolution lies in its ability to preserve all discriminative feature information without loss. In contrast, larger strides introduce sampling artifacts: for instance, a 3 × 3 filter with stride = 3 reduces the feature map size but samples each pixel only once, while a stride of 2 leads to asymmetric sampling, causing uneven sampling frequencies for even and odd rows/columns. Generally, any stride greater than 1 risks discarding non-discriminative yet structurally important information during the transformation X(S, S, C) → X″(S/2, S/2, 4C), bypassing the intermediate X′ representation entirely.

3.4.2. SCConv Module

In CNNs, the SCConv (Spatial and Channel Reconstruction Convolution) module is critical for addressing spatial and channel redundancies in feature maps, thereby reducing model size while improving computational efficiency. As illustrated in Figure 4, this module comprises two core components: the Spatial Reconstruction Unit (SRU) and the Channel Reconstruction Unit (CRU) [61].

Figure 4. The architecture of SCConv Module. The opacity of a feature map corresponds directly to its weight, with a higher weight magnitude resulting in a more opaque visualization.

The SRU employs a separation-reconstruction strategy to mitigate spatial redundancy in feature maps. In CNNs, spatial redundancy often manifests as repetitive or unnecessary patterns between adjacent regions or pixels. By partitioning the feature maps into distinct components and reconstructing them through spatial alignment, the SRU eliminates redundant correlations, resulting in more compact and spatially efficient feature representations. Conversely, the CRU utilizes a segmentation-transformation-fusion approach to address channel redundancy. Channel redundancy arises when different channels encode similar or overlapping information. The CRU first divides the feature maps along the channel dimension, applies a transformation to reshape their representations, and then fuses the processed components to remove inter-channel redundancies, enhancing the distinctiveness of channel-specific features.

By integrating the SRU and CRU, the SCConv module synergistically reduces both spatial and channel redundancies in CNNs, enabling more efficient feature learning. This dual-reduction mechanism offers two critical advantages. First, it mitigates overfitting by forcing the model to prioritize essential, generalizable features rather than relying on superfluous patterns in the training data. Second, it strengthens the ResT’s ability to distinguish subtle variations in signature verification tasks. Since authentic and forged signatures often differ in fine-grained details, reducing redundancy allows the model to focus on critical discriminative features, thereby improving its robustness and generalization across diverse signature datasets. This optimization not only enhances feature extraction efficiency but also ensures the model retains high discrimination accuracy for complex signature analysis scenarios.

The SRU in the SCConv module implements a separation-reconstruction strategy to optimize spatial feature representation. The separation step aims to distinguish feature maps with high informational density from those with low informational density, aligning with the spatial distribution of salient features. To quantify this distinction, the scaling factor in Group Normalization is leveraged as a metric for evaluating the informational content across different feature maps. The procedure for deriving the weight matrix W involves three stages: first, computing the scaling factors to assess feature map significance; second, applying thresholding to classify feature maps into high- and low-information categories; and third, reconstructing the refined feature maps by aggregating the high-information components while discarding redundant low-information ones. The entire process of obtaining W is as follows:

$${\mathit{X}}_{\mathit{o}\mathit{u}\mathit{t}}=GN\left(\mathit{X}\right)=\gamma {\displaystyle \frac{\mathit{X}-\mu }{\sqrt{{\sigma }^2+\epsilon }}}+\beta$$

(2)

$${\mathit{W}}_{\gamma }=\left\{{\omega }_i\right\}={\displaystyle \frac{{\gamma }_i}{{\sum }_{j=1}^C{\gamma }_j}},i,j=\mathrm{1,2},\cdots ,C$$

(3)

$$\mathit{W}=Gate\left(Sigmoid\right({\mathit{W}}_{\gamma }\left(GN\right(\mathit{X}\left)\right)\left)\right)$$

(4)

The parameters μ and σ denote the mean and standard deviation of the input distribution, respectively, while γ and β are learnable scaling and shifting coefficients. A small constant ε is introduced to prevent numerical instability during normalization. An increased value of γ correlates with heightened pixel-wise variation, suggesting enhanced richness in spatial feature representation.

The reconfiguration operation combines high-information and low-information features to produce higher-information features while reducing spatial complexity. This process, termed cross-reconstruction, involves weighting and combining distinct information components to derive ${\mathit{X}}^{w1}$ and ${\mathit{X}}^{w2}$, which are then concatenated to generate the spatially optimized feature map ${\mathit{X}}^w$. The workflow proceeds as follows:

$$\left\{\begin{array}{c}{\mathit{X}}_1^w={\mathit{W}}_1\otimes \mathit{X}\\ {\mathit{X}}_2^w={\mathit{W}}_2\otimes \mathit{X}\\ \begin{array}{c}{\mathit{X}}^{w1}={\mathit{X}}_{11}^w\oplus {\mathit{X}}_{12}^w\\ {\mathit{X}}^{w2}={\mathit{X}}_{21}^w\oplus {\mathit{X}}_{12}^w\\ {\mathit{X}}^w={\mathit{X}}^{w1}\cup {\mathit{X}}^{w2}\end{array}\end{array}\right.$$

(5)

The symbols ⊗, ⊕, and ∪ denote element-wise multiplication, summation, and union operations, respectively. Following SRU processing, high-information features are separated from low-information counterparts, which in turn reduces spatial redundancy by eliminating superfluous feature correlations.

Following the SRU module, the CRU module is integrated to further mitigate channel redundancy in the spatially refined feature map ${\mathit{X}}^w$. The CRU employs a segmentation-transformation-fusion framework. In the segmentation phase, ${\mathit{X}}^w$ is partitioned into two subsets: one containing αC channels and the other comprising (1 − α)C channels, where α denotes a hyperparameter within the range 0 ≤ α ≤ 1. In this paper, α is set to 0.5, meaning the feature channels are equally split. In most cases, α = 0.5 serves as a stable choice. Subsequently, both subsets undergo channel compression via 1 × 1 convolution kernels, generating ${\mathit{X}}_{up}$ (high-channel subset) and ${\mathit{X}}_{low}$ (low-channel subset).

The transformation phase processes ${\mathit{X}}_{up}$ for “rich feature extraction,” applying both global weight convolution (GWC) and pointwise convolution (PWC), followed by summation to produce ${\mathit{Y}}_1$. Meanwhile, ${\mathit{X}}_{low}$ serves as a supplementary input for “rich feature extraction,” undergoing PWC alone and being merged with the original input to yield ${\mathit{Y}}_2$.

The fusion phase adapts a streamlined SKNet mechanism to combine ${\mathit{Y}}_1$ and ${\mathit{Y}}_2$. First, global average pooling is applied to extract global spatial and channel statistics, producing pooled vectors S₁ and S₂. Softmax normalization is then applied to these vectors, generating feature weight vectors β₁ and β₂. Finally, the fused output is computed as:

$$\mathit{Y}={\beta }_1{\mathit{Y}}_1+{\beta }_2{\mathit{Y}}_2$$

(6)

where $\mathit{Y}$ represents the channel-refined feature map. In summary, SCConv maintains the dimensions of the feature maps while assigning weights to them based on their information content, thereby focusing on more critical features.

3.4.3. Final Local Features

Following the SCConv module, we introduce a residual convolution module to extract the final local features. Experimental results indicate that SPDConv alone cannot fully capture the subtle deformation features of signature strokes in the offline signature verification (OSV) task. While the SCConv module enhances model generalization through abstract feature map transformations, it still exhibits limitations in capturing local details. To address this, we integrate the ResNet convolutional layer architecture, leveraging its well-established capability for local feature extraction. The residual connection design in ResNet enables more effective capture of rich local detail information within signature images. This module incorporates multiple processing layers—convolutional layers, max pooling layers, and fully connected layers—where convolutional and fully connected layers contain learnable parameters optimized during training. After sequential processing, the resulting 512-dimensional local feature vector not only retains stroke-level detail information but also achieves more discriminative feature representations through ResNet’s deep network structure, providing a robust foundation for subsequent signature verification tasks.

3.5. Training Process

During the training phase of ResT, a subset of the publicly available dataset is used to train the model. The training samples include both authentic and skillfully forged signatures. To optimize model performance, we adopt the contrastive loss function proposed in [11], which calculates the Euclidean distance between feature representations extracted from the Siamese Network to measure similarity. This loss function is widely applied in Siamese Network architectures [62], with its mathematical formulation defined as follows:

$$L\left(s_1,s_2,y\right)=\alpha y{D}_w^2+\beta (1-y)max{(0,m-D_w)}^2$$

(7)

In this framework, $s_1$ and $s_2$ denote two signature image samples, $y$ is a binary label indicating whether the pair shares the same type (1 for same-type pairs, 0 for different-type pairs), and α and β are constant parameters (with margin parameter $m$ = 1 in this study). The Euclidean distance in the embedded feature space is calculated as $D_w={‖f\left(s_1;w_1\right)-f\left(s_2;w_2\right)‖}_2$, where $f$ represents the embedding function that maps input signatures through a CNN to a real-valued vector space, and $w_1$, $w_2$ are the learnable weights of the underlying network’s specific layers. Unlike traditional methods that assign binary similarity labels directly to input pairs, the Siamese network approach leverages geometric relationships in the feature space to determine similarity: for same-type pairs (e.g., genuine signatures from the same writer), the network learns to minimize the distance between their feature vectors; for different-type pairs (e.g., genuine and forged signatures), it maximizes this distance. The two network branches function as independent feature embedders, mapping inputs to distinct feature spaces. The contrastive loss function (Equation (7)) ensures a critical property in this space—distances between same-type signatures are significantly smaller than those between different-type signatures (e.g., genuine or forged signatures from different writers). A distance threshold is then applied to the output of the Euclidean distance layer to classify the input pair (e.g., “ genuine-genuine “ same-type or “ genuine-forged” different-type). This mechanism enables the model to effectively learn discriminative features that distinguish genuine and forged signatures through explicit spatial.

3.6. Verification System

To assess the method’s effectiveness, we trained a classifier based on WI principles following feature learning, thereby constructing a complete validation system. This WI-based system employs a distance-based discrimination approach to determine whether a signature pair $(i,j)$ belongs to the same class or different classes. The process begins by selecting m reference samples (genuine signatures) and computing their Euclidean distances to all other signatures, yielding the maximum $d_{max}$ and minimum $d_{min}$ distances. Using a sliding step, a sequence of thresholds $D=\left\{d_{min},d_{min}+step,d_{min}+2step,\dots ,d_{max}\right\}$ is generated, where n denotes the total number of signature pairs. For any threshold $d\in D$, the system evaluates a reference signature ${\mathit{x}}_{\mathit{i}}$ and a query signature ${\mathit{x}}_{\mathit{j}}$ using the following decision rule:

$$\left({\mathit{x}}_{\mathit{i}},{\mathit{x}}_{\mathit{j}}\right)=\left\{\begin{array}{c}0 if dist\left({\mathit{x}}_{\mathit{i}},{\mathit{x}}_{\mathit{j}}\right)>d (\mathrm{D}\mathrm{e}\mathrm{t}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{s} \mathrm{a} \mathrm{f}\mathrm{o}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{d} \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\mathrm{a}\mathrm{t}\mathrm{u}\mathrm{r}\mathrm{e})\\ 1 if dist\left({\mathit{x}}_{\mathit{i}},{\mathit{x}}_{\mathit{j}}\right)\le d (\mathrm{D}\mathrm{e}\mathrm{t}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{s} \mathrm{a} \mathrm{g}\mathrm{e}\mathrm{n}\mathrm{u}\mathrm{i}\mathrm{n}\mathrm{e} \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\mathrm{a}\mathrm{t}\mathrm{u}\mathrm{r}\mathrm{e})\end{array}\right.$$

(8)

Here, dist(⋅) denotes the Euclidean distance between two signatures. The optimal threshold $d_{opt}$, which maximizes system performance, is determined by evaluating all candidate values in the threshold set D. This threshold is subsequently adopted as the decision criterion for the WI verification system.

4. Experiment

This section presents a systematic evaluation of ResT through comprehensive experiments. We first outline the experimental design and implementation procedures to ensure reproducibility and comparability. Next, we analyze the verification performance of the WI approach across multiple signature datasets, highlighting the model’s adaptability to diverse linguistic contexts. To further validate its effectiveness, we conduct comparative experiments with state-of-the-art signature verification systems, quantitatively demonstrating ResT’s superiority in key metrics such as accuracy. Additionally, ablation studies are performed by sequentially removing or replacing critical components, enabling us to isolate the contribution of each module and clarify the role of innovative design choices in enhancing discriminative capability. These experiments collectively establish the method’s effectiveness and practicality from multiple perspectives.

4.1. Experimental Protocol

This section outlines the experimental setup. Initially, the dataset and data preprocessing steps are described. Subsequently, the methodology is detailed, including data division, training procedures, and validation protocols. Finally, the performance evaluation metrics and experimental conditions specific to ResT are presented.

To evaluate the method’s verification performance, experiments were conducted on four multilingual datasets: GPDS [63], Cedar [64], BHSig-B, and BHSig-H [65]. GPDS dataset contains 4000 distinct writers, with 24 genuine and 30 forged signatures per writer. Of these, 3200 writers’ samples were used for training, and 800 for testing. Cedar dataset contains 55 writers, with 24 genuine and 24 forged signatures per writer. 50 writers’ samples were used for training, and 5 for testing. BHSig-B and BHSig-H are subsets of the BHSig collection: BHSig-B contains 100 writers, with 24 genuine and 30 forged signatures per writer. 80 writers’ samples were used for training, and 20 for testing; BHSig-H contains 160 writers, with 24 genuine and 30 forged signatures per writer. 100 writers’ samples were used for training, and 60 for testing.

Specifically, we applied 5-fold cross-validation on the training set to maximize data utilization. After completing all five folds, we selected the hyperparameter configuration yielding the best average performance across folds. Finally, we trained the final model on the entire training set using these optimized hyperparameters and strictly reserved the test set for the final evaluation to ensure unbiased performance assessment. Throughout all random processes, the random seeds were kept fixed to ensure reproducibility.

To ensure balanced data distribution for each writer, positive (genuine-genuine pairs) and negative (genuine-forged pairs) samples were equated in number. Given that the number of negative samples exceeds that of positive samples, we randomly select negative samples equal to the number of positive samples. For example, in CEDAR, each writer had $C_{24}^2=276$ positive samples and 24 × 24 = 576 negative samples. To balance the dataset, 276 negative samples were randomly selected for each writer. The same approach is applied to the remaining datasets.

Table 1 illustrates genuine signatures and skilled forgery examples from the four datasets. To accommodate the ResT’s requirement for fixed-size inputs, preprocessing steps were implemented due to variations in original signature dimensions, backgrounds, and noise levels. Initially, the Otsu [66] algorithm was applied to remove background pixels, setting them to white while preserving foreground pixels as grayscale. Subsequently, the signature image underwent inversion by subtracting each pixel’s value from the maximum brightness, resulting in a zero-valued background. Finally, all signatures were uniformly resized to 256 × 256 size.

Table 1. Experimental parameter settings for each dataset. Train set represents the number of writers in the training set, and test set represents the number of writers in the test set.

Sample		Dataset	Language	Train Set	Test Set
Genuine	Forge
		CEDAR	English	50	5
		BHSig-B	Bengali	80	20
		BHSig-H	Hindi	100	60
		GPDS	Spanish	3200	800

4.1.1. Experimental Settings

It should be noted that during model training, the majority of the data is reserved for training, with the remaining portion allocated to testing. The loss function incorporates three parameters—α, β, and m—which were uniformly set to 1 to ensure consistent optimization. All experiments were executed on an NVIDIA 3080 Ti GPU using a batch size of 32. For data augmentation, the preprocessing included standard random rotation and horizontal image flipping, followed by cropping 224 × 224 size from the 256 × 256 signature images randomly, resulting in a 224 × 224 size for the model. Data augmentation techniques are applied only to the training set. For the validation and test sets, images are directly resized to 224 × 224. The weights of the Transformer encoder were initialized with the results of pre-training on the ImageNet-21k dataset, and the proposed model was trained using the ADAM optimizer with a learning rate of 5 × 10⁻⁵.

To evaluate the ResT’s performance, we employed diverse data splitting strategies and conducted ten independent training sessions, with results averaged across all iterations. In the WI scenario, the distance-based verification method described in Section 3.5 was applied to assess the model’s effectiveness.

4.1.2. Evaluation Metrics

To assess performance, the following metrics are considered: accuracy, the area under the ROC curve (AUC), false rejection rate (FRR), false acceptance rate (FAR), and equal error rate (EER). Accuracy measures the proportion of correctly classified samples. AUC quantifies the model’s ability to distinguish between genuine and forged signatures across all threshold values. FRR represents the proportion of genuine signatures incorrectly classified as forgeries, while FAR indicates the proportion of skilled forgeries mistakenly accepted as genuine. EER is defined as the point where FAR equals FRR, representing the threshold at which misclassification rates for both classes balance.

4.2. WI-Based Verification System Results

As the proposed ResT integrates ResNet and ViT architectures, ResNet and ViT are selected as baseline models for comparison. All models were trained under identical experimental settings, and their performance results are presented in Table 2. The results demonstrate that ResT outperforms both ResNet and ViT across most evaluation metrics.

Table 2. Signature verification comparison on CEDAR dataset.

Model	ACC	FRR	FAR	AUC	EER
Baseline (ResNet)	89.96 (0.41)	10.07	10.00	96.37	12.04 (0.35)
Baseline (ViT)	84.09 (0.36)	13.19	26.56	87.62	19.87 (0.39)
ResT	92.88 (0.34)	7.26	6.97	97.75	7.12 (0.22)

The feature distance distributions of the baseline models (ResNet and Vision Transformer) and the proposed ResT were compared, with visualization results presented in Figure 5. In Figure 5a,b, genuine and forged signatures exhibit significant overlap, with no clear separation boundary. However, Figure 5c reveals a distinct separation between genuine and forged signatures, accompanied by a marked reduction in overlapping regions. Analysis of the mean differences further highlights these distinctions: in Figure 5a, genuine and forged signatures cluster separately despite minimal mean variation, indicating that ResNet effectively groups similar signatures through local feature learning but struggles to differentiate categories. In contrast, Figure 5b shows a pronounced mean difference between genuine and forged signatures, yet the forged signature distribution remains scattered, suggesting that ViT excels at category discrimination via global features but lacks consistency in grouping similar samples. The proposed ResT in Figure 5c outperforms both baselines, achieving larger mean distances between classes while maintaining tighter clustering within each category. This dual capability—effective inter-class discrimination and intra-class convergence—enables robust separation of genuine and forged signatures.

Figure 5. Feature distance comparisons across models. (a) ResNet. (b) ViT. (c) ResT.

This study employs the distance-based methodology outlined in Section 3.5 to develop a WI signature verification system. For each dataset, we implement a stratified approach where dedicated feature extractors are trained on partitioned subsets: one portion for model development and the remaining data reserved for performance evaluation. Table 1 systematically characterizes the datasets by documenting signature styles, dataset nomenclature, language specifications, and writer’s counts for both training and testing.

Table 3 presents the performance metrics of various systems, with missing values in the original work indicated by “--”. The results show that the proposed ResT consistently achieves superior accuracy (ACC) and equal error rate (EER) metrics across all evaluated datasets, with complementary performance indicators showing statistically significant improvements. These outcomes validate the architectural effectiveness of the network. The model’s success stems primarily from its multi-scale feature fusion strategy combined with specialized modules that enhance generalization capabilities. Notably, the system’s core architecture comprises only two fundamental components—a feature extraction network and a distance-based discriminator—representing a deliberately simplified design framework. The fact that this streamlined configuration still outperforms existing methods underscores the inherent advantages of the proposed framework, particularly in capturing discriminative signature characteristics through its hybrid ResT architecture.

Table 3. Signature verification comparison on four datasets. The data marked with “*” were reproduced results from the cited reference. The optimal values in the indicators are displayed in bold.

Dataset	Systems	ACC	FRR	FAR	EER
CEDAR	SigNet [11]	83.41 *	10.14 *	22.39 *	16.27 *
	Morphology [67]	88.19	12.39	11.23	11.59
	MMDANet [68]	91.67	8.33	8.33	--
	ResT	92.88 (0.34)	7.26	6.97	7.12 (0.22)
Bengali	Top [69]	88.2	--	--	11.2
	SigNet [11]	86.11	13.89	13.89	--
	2C2S [70]	90.68	8.66	9.98	9.32
	SURDS [71]	89.50	12.01	8.98	--
	Correlated Feature [35]	--	14.43	15.78	--
	DeepHSV [72]	88.08	--	--	11.92
	ResT	93.52 (0.26)	6.26	6.83	6.54 (0.19)
Hindi	Top [69]	83.8	--	--	15.3
	SigNet [11]	84.64	15.36	15.36	--
	LwR [69]	92.6	--	--	8.8
	Correlated Feature [35]	--	15.05	13.10	--
	FHTF [32]	--	11.46	10.36	--
	DeepHSV [72]	86.66	--	--	13.34
	ResT	93.98 (0.31)	6.35	6.72	6.48 (0.24)
GPDS	CBCapsNet [50]	90.87	9.45	8.81	--
	TransOSV [73]	--	10.64	10.64	10.64
	SigNet [11]	77.76	22.24	22.24	--
	ResT	91.73 (0.41)	7.69	8.84	8.47 (0.26)

4.3. Ablation Study

The proposed ResT integrates modules derived from ViT and ResNet, with each module incorporating multiple subcomponents. To evaluate the contribution of each module, ablation experiments were conducted. As illustrated in Figure 2, the ResNet-based module includes SPD-Conv, SCConv, and residual convolutional blocks. These components serve as critical elements in the ResT architecture; consequently, the ablation analysis focuses on their individual impacts. The results of these experiments are presented in Table 4.

Table 4. Result of different combinations of modules on CEDAR dataset.

Type	Vision Transformer Block	ResNet-Based Block			EER (%)
		SPD-Conv	SCConv	ResNet18
Proposed	√	√	√	√	7.12 (0.22)
1	√		√	√	8.35 (0.18)
2	√	√		√	9.88 (0.24)
3	√			√	11.84 (0.30)
4	√				19.87 (0.34)
5				√	12.04 (0.28)
6		√	√	√	10.56 (0.25)

As shown in Table 4, the proposed method consistently achieves the highest validation performance across all experimental configurations. When only ViT (Table 4, entry 4) was employed, the model demonstrated the weakest performance due to its exclusive focus on global signature features, which limited its ability to capture subtle discriminative differences. In contrast, the combination of ViT and ResNet18 (Table 4, entry 3) significantly outperformed configurations using standalone networks (entries 4 and 5), demonstrating that integrating global and local features yields superior results. Notably, models relying solely on local features (entry 5) showed marginally better performance than those using only global features (entry 4), emphasizing the critical role of local feature extraction. Furthermore, entry 6 achieved strong results despite excluding global features entirely, reinforcing the effectiveness of our residual network enhancements.

The experimental findings conclusively demonstrate that the SPD-Conv and SCConv modules improve model generalization while mitigating overfitting risks. These modules enhance performance by directing the network’s attention toward distinctive stroke patterns and key local regions, thereby strengthening discriminative capability and reducing vulnerability to overfitting through more robust feature learning mechanisms.

5. Conclusions

This paper presents a novel hybrid framework combining residual neural networks and Transformers to learn discriminative representations for OSV tasks. The approach introduces multi-scale feature extraction from handwritten signatures and integrates both local and global signature characteristics to improve verification accuracy. A key innovation lies in the composite convolution block design, which enhances feature representation by capturing distinctive and generalizable local patterns across multiple scales. This architecture addresses limitations of single-depth feature generalization through its multi-scale processing capability, demonstrating strong adaptability across multilingual datasets. The framework incorporates SPD-Conv and SCConv modules to strengthen feature learning by emphasizing stroke-level details and micro-deformation patterns while minimizing reliance on common model features in OSV tasks. For training, we utilize genuine and skilled forged signatures alongside feature distance-based losses to optimize the ResT architecture. Following feature learning, we implement a WI-based verification system to evaluate performance. Comprehensive experimental analysis confirms that the proposed ResT outperforms baseline methods and achieves competitive results against state-of-the-art approaches in the field.