A Lightweight Deep Learning Framework for Parallax-Tolerant Image Stitching

Abstract

Image stitching aims to construct wide field-of-view scenes from multiple narrow-FoV images, yet existing deep learning-based approaches may introduce substantial computational and parameter overhead, limiting their applicability in efficiency-sensitive scenarios. To address this issue, we propose a lightweight deep stitching framework that integrates multi-scale feature fusion with attention-enhanced matching. Specifically, a transformer-based channel attention (TCA) block improves the discriminative capability of fused features in low-texture regions and enhances global consistency. A coordinate-aware correlation module (CACM) combines correlation-based matching with position-sensitive coordinate attention to support registration under parallax, while GhostNet serves as the shared backbone. On UDIS-D, the complete model achieves a 25.19 dB peak signal-to-noise ratio (PSNR) and a structural similarity index (SSIM) of 0.833 under the overlap-region protocol, with a reported full-system complexity of 19.28 giga multiply-accumulate operations (GMACs) and 52.86 M parameters. These results demonstrate a favorable accuracy–efficiency trade-off under the stated GMAC, runtime, and memory protocol and the potential of the proposed framework for resource-conscious image stitching applications.

1. Introduction

Image stitching is a fundamental technique for constructing wide field-of-view (FoV) scenes from multiple narrow-FoV images. It supports applications including autonomous driving [1], medical imaging [2], and video surveillance [3]. Early approaches have relied on hand-crafted geometric features, beginning with the scale-invariant feature transform (SIFT) [4] for keypoint detection and matching. Later studies incorporated line segments [5,6], large-scale edges [7], depth maps [8], and semantic planar regions [9] to improve alignment and structure preservation. Nevertheless, low-texture or repetitive-pattern regions remain difficult, and increasingly complex geometric processing can limit efficiency-sensitive applications.

Deep learning provides an alternative in which feature extraction and geometric alignment can be learned jointly. Convolutional neural network (CNN)-based stitching frameworks [10,11,12,13,14,15,16,17] have improved alignment in several challenging settings, but their computational and parameter costs vary substantially. Balancing alignment quality with model complexity therefore remains an important design objective, particularly when an application has limited compute or memory budgets.

Parallax caused by depth variation or camera translation is another persistent challenge because a single homography cannot generally align all scene planes. Local or multi-grid warps [18,19,20,21,22] increase deformation flexibility, but they may depend on sufficiently reliable correspondences and can introduce additional computation or shape distortion. These trade-offs motivate a coarse-to-fine formulation that combines a global transformation with a regularized local warp.

To address these considerations, we develop a lightweight stitching framework that combines multi-scale feature fusion, channel reweighting, correlation-based matching, and coarse-to-fine geometric estimation. The transformer-based channel attention (TCA) block is placed after multi-scale concatenation to model interactions between global-average and global-maximum channel descriptors and to reweight the fused channels.

The coordinate-aware correlation module (CACM) constructs a normalized patch-correlation volume and applies coordinate attention [23] to retain directional spatial information before motion regression. This design is intended to provide position-aware matching cues without claiming that occlusion, illumination variation, or every large-parallax case is explicitly resolved.

GhostNet [24] is used as the shared feature backbone and depthwise separable convolutions [25] are used in the regression branches. The resulting architecture is designed to reduce computational and parameter costs while retaining the two-stage homography-plus-thin-plate-spline (TPS) alignment structure.

The experiments reported in this manuscript evaluate the method on UDIS-D using peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) under the current protocol. The reported results show competitive alignment metrics together with 19.28 giga multiply-accumulate operations (GMACs) and 52.86 M parameters under the stated full-pipeline counting protocol; they do not establish real-time operation or deployment performance on resource-constrained hardware.

The main contributions of this work are summarized as follows:

(1)

A TCA block is used for hierarchical multi-scale feature fusion and pooled-descriptor channel-dependency modeling.

(2)

A CACM combines normalized patch correlation with coordinate attention before homography and TPS control-point regression.

(3)

GhostNet and lightweight operations are used to balance alignment performance and model complexity under the current UDIS-D evaluation protocol.

2. Related Work

In this section, we briefly review traditional image stitching methods and deep image stitching methods.

2.1. Traditional Image Stitching

Traditional image stitching methods are based on manually designed features. Brown and Lowe [4] established their invariant-feature-based panoramic stitching method as a baseline framework, leveraging SIFT features and random sample consensus (RANSAC) for global homography estimation. Subsequent studies explored alternative feature descriptors (e.g., SURF [26], FAST [27], ORB [28]) to optimize keypoint detection and matching. However, global homography-based methods often introduced misalignment artifacts in non-planar scenes, prompting a shift toward local warping strategies.

Gao et al. [18] proposed the DHW algorithm, which separated foreground and background alignment using distinct homographies, combined with seam blending for seamless stitching. Lin et al. [19] further advanced this concept with SVA stitching to mitigate transition boundary artifacts. Zaragoza et al. [20] introduced the APAP algorithm, partitioning images into grids for localized homography estimation. While APAP improved alignment flexibility, it suffered from projection distortion and poor performance in feature-sparse regions. To address these limitations, APAP+CI [29] introduced adaptive correspondence insertion to improve alignment in feature-sparse regions.

To balance alignment precision and shape preservation, hybrid deformation models emerged. Chang et al. [30] proposed SPHP warps, blending projective and similarity transformations to suppress perspective distortion in non-overlapping regions. Lin et al. [31] introduced AANAP stitching, combining local homographies with global similarity constraints. Chen et al. [32] strengthened this paradigm with the GSP algorithm, enforcing similarity transformations through optimized sparse systems to minimize structural distortion.

Recent advancements have focused on geometric feature fusion for robust alignment. Xiang et al. [5] integrated line-guided constraints with global similarity to enhance low-texture alignment. Jia et al. [6] proposed line-point consistence stitching, leveraging coplanar subregions and collinear constraints for structure preservation. Du et al. [7] introduced GES-GSP warping, utilizing large-scale edge constraints and adaptive similarity transitions. Concurrently, depth-aware methods [8] exploited depth maps and epipolar geometry for parallax-tolerant stitching.

Despite progress, traditional methods face inherent limitations: Complex geometric constraints struggle in texture-deficient scenarios, while computational inefficiency hinders practicality.

2.2. Deep Image Stitching

Recent advances in deep learning have revolutionized image stitching by shifting from handcrafted geometric features to data-driven semantic feature learning.

DeTone et al. [10] introduced HomographyNet for direct corner-displacement regression, and Nguyen et al. [11] developed an unsupervised photometric formulation. Content-aware masking [12], UDIS and UDIS2 [13,22], coplanarity-aware adversarial learning [14], pixel-wise warping [15], implicit neural reconstruction [16], and attention-based homography estimation [17] subsequently broadened the design space. Recent work has further examined residual homography–TPS warps with boundary constraints [33], optical-flow-based alignment and hole-aware composition [34], local patch alignment integrated with seam cutting [35], and the domain-shift behavior of deep homography estimators [36]. Collectively, these studies show that alignment flexibility, composition quality, generalization, and computational cost must be considered separately rather than inferred from a single overlap metric.

2.3. Lightweight Feature Extraction and Attention

GhostNet [24], MobileNetV3-Large [37], and ShuffleNetV2 × 1.0 [38] represent different compact feature-extraction strategies. Their classifier-level efficiency does not directly determine end-to-end stitching speed, so the present comparison retains identical TCA, CACM, regression, geometry, loss, preprocessing, and evaluation components and changes only the backbone plus the minimum 1 × 1 channel adapters required by the downstream interface. Coordinate attention [23] is established prior art; CACM’s contribution is its application to the target-indexed global correlation representation rather than invention of the attention operator itself.

Recent work includes pixel-wise alignment [39], progressive alignment with interwoven composition [40], hierarchical grid-constrained fusion [41], structure-preserving bidirectional pixel warps [42], segmentation-guided multi-homography warping [43], and depth-based stitching [44]. These methods highlight that overlap alignment, shape preservation, seam quality, and generalization require separate evidence. Related preliminary work introduced attention-guided multi-scale fusion [45]; the present study develops this direction through the integrated GhostNet–TCA–CACM coarse-to-fine architecture and adds controlled component, backbone, efficiency, and failure analyses.

Although recent systems increase deformation flexibility or explicitly optimize composition quality, their alignment, reconstruction, and computational objectives are not interchangeable. The present study therefore addresses a narrower accuracy–complexity trade-off on UDIS-D through a lightweight backbone, pooled-descriptor channel attention, and coordinate-aware correlation. Accordingly, the conclusions are drawn within the scope of the UDIS-D benchmark, overlap-region reconstruction metrics, and computational characteristics measured under the stated evaluation protocol.

3. Methodology

Figure 1 illustrates the local TPS transformation stage: the network predicts a control-point mesh relative to its initialized grid, and TPS solving transforms the input images. The complete registration framework is summarized in Figure 2. The network receives a reference image I_r and target image I_t, each with shape B × 3 × 512 × 512. A shared lightweight backbone extracts three feature scales. TCA-based fusion combines adjacent scales, and CACM constructs position-aware global correlation representations for two regression stages. The first stage predicts four two-dimensional corner offsets and solves a global homography. The second predicts offsets for a 13 × 13 control-point mesh initialized from the homography-warped regular grid. The resulting TPS warp is then used for target-image transformation, after which the reference and warped target are placed on a dynamic union canvas and average-fused in valid overlap. This ordering is coarse-to-fine; it does not guarantee seam-free composition or globally invertible TPS deformation.

Figure 1. Local thin-plate spline (TPS) transformation: the network predicts dynamic control points, TPS solving relates them to the initialized control points, and the resulting mapping transforms the input images.

Figure 2. Overall registration architecture: shared GhostNet reference and target branches feed contextual-correlation and motion-prediction stages for four-corner homography estimation and 169-point TPS-offset regression, followed by the corresponding warp operations.

3.1. Warp Parameterization

Let an arbitrary target-image location be p = (x, y)^T. The regular image domain is divided into U = 12 by V = 12 cells, yielding N = (U + 1)(V + 1) = 169 grid vertices. Denote the regular grid by P0, the homography-initialized grid by P, the network-predicted local offsets by $\Delta P$, and the final TPS control coordinates by $P^{\prime }$. Equation (1) expresses the implemented update $P^{\prime }=P+\Delta P$; thus, P in this equation is the homography-initialized mesh rather than the unwarped regular grid $P0$.

$$P^{\prime }=P+∆P,$$

(1)

By minimizing an energy function consisting of a data term and a distortion term [46], the TPS transformation can be parameterized as follows:

$$f\left(p\right)=C+Mp+{\displaystyle \sum _{i=1}^N}{w}_{i}O\left({‖p-p_i‖}_2\right),$$

(2)

In Equation (2), f(p) ∈ R² is the mapped coordinate, $C\in R^2$ is the translation term, $M\in R^{2\times 2}$ is the affine matrix, $w_{\mathrm{i}}\in R^2$ is the nonlinear weight associated with control point $p_{\mathrm{i}}$, and N = 169. The implementation uses the radial basis O(d_i) = d_i² log(d_i² + 10⁻⁶), where $d_{\mathrm{i}}={\left|\left|p-p_{\mathrm{i}}\right|\right|}^2$. For each batch item, the TPS linear system is assembled from the supplied normalized control-point pairs and its inverse is computed in double precision. The solve matrix is therefore batched rather than fixed, because the predicted source mesh varies across samples.

The global homography and local TPS are used sequentially. Four predicted corner offsets define the homography; the regular 13 × 13 grid is transformed by that homography; and the network then adds 169 local two-dimensional offsets before the TPS sampler is evaluated. The regularizer discourages excessive deformation but does not prove that every predicted mesh is invertible.

3.2. Network Structure

The overall registration architecture is shown in Figure 2. The reference and target branches share the GhostNet backbone; their multi-scale features feed contextual-correlation and motion-prediction stages that estimate a four-corner homography and 169 local TPS control-point offsets. The multi-scale representation allows larger-scale features to emphasize global structure and overall layout, whereas smaller-scale features retain texture and edge details. Their collaboration helps stabilize feature matching under large parallax and complex scene structure and provides a robust basis for estimating the two geometric transformations.

To reduce computational overhead while retaining multi-scale features, the network uses a shared GhostNet [24] backbone for the reference and target images. For a 512 × 512 input, the extracted features have 40 channels at $64 \times 64$ ($1/8$ scale), 112 channels at $32 \times 32$ ($1/16$ scale), and 960 channels at 16 × 16 ($1/32$ scale). These tensors provide progressively coarser semantic and spatial information for the two fusion and regression stages.

Fusion proceeds hierarchically. The $16 \times 16$ feature is first upsampled and concatenated with the $32 \times 32$ feature; the result is channel-reweighted and compressed. That tensor is then upsampled and concatenated with the 64 × 64 feature, followed by a second TCA block and channel compression. This ordering preserves the actual coarse-to-fine data flow used by the implementation.

The first regression stage uses the 32 × 32 fused features to predict eight values representing four two-dimensional corner offsets. A direct linear transform constructs the global homography, which is used to coarsely warp the target feature. The second stage uses the $64 \times 64$ fused features to predict 338 values, corresponding to two offsets for each of 169 TPS control points. The final control mesh is the homography-initialized regular mesh plus these local offsets, so the TPS refinement follows rather than replaces the global homography estimate.

3.2.1. Transformer-Based Channel Attention Module

The transformer-based channel attention (TCA) block is applied after concatenating two spatially aligned feature maps (Figure 3). In the first fusion stage, the 960-channel 16 × 16 tensor is bilinearly upsampled with align_corners = True and concatenated with the 112-channel 32 × 32 tensor, producing 1072 channels; a $1 \times 1$ convolution later compresses the reweighted result to 512 channels. In the second stage, that tensor is bilinearly upsampled and concatenated with the 40-channel 64 × 64 tensor, producing 552 channels before TCA and 256 channels after the subsequent 1 × 1 convolution.

Figure 3. Multi-scale fusion with transformer-based channel attention (TCA): the upsampled smaller-scale feature is concatenated with the larger-scale feature; average- and max-pooled descriptors are processed by a transformer encoder and multilayer perceptron (MLP) to generate multiplicative channel weights.

Let the TCA input $F$ have shape B × C × H × W. Global average pooling (GAP) and global max pooling (GMP) each reduce the spatial dimensions, producing $zavg$ and $zmax$, each with shape $B \times C$. GAP summarizes the average channel response, whereas GMP emphasizes strong activations. Their combination provides complementary channel descriptors for the subsequent attention operation. The descriptors are computed as follows:

$$z_{avg}=\mathrm{G}\mathrm{A}\mathrm{P}\left(F\right),z_{max}=\mathrm{G}\mathrm{M}\mathrm{P}\left(F\right),$$

(3)

where $zavg$ and $zmax$ are $B \times C$ channel descriptors obtained from the same fused tensor $F$.

The two descriptors are stacked as a two-token sequence. The intermediate tensor has shape $B \times 2 \times C$ and is transposed to 2 × B × C for the PyTorch transformer encoder:

$$Z=\left[\begin{array}{c}{z}_{avg}\\ z_{max}\end{array}\right],$$

(4)

The sequence $Z$ is processed by one transformer encoder layer with embedding dimension $C$, eight attention heads, the PyTorch-default feed-forward dimension of 2048, ReLU activation, dropout of 0.1, LayerNorm, and the encoder layer’s internal residual connections. The sequence length is two: the attention operates between the GAP and GMP descriptors and across their channel embeddings, not over all H × W spatial positions. Denoting the encoded sequence by $Z$′ = T($Z$), the tensor is returned to shape B × 2 × $C$ for channel-weight prediction.

The encoded two-token tensor is flattened to $B \times 2C$ and passed through a channel MLP with dimensions $2C \to C/8 \to C$. The hidden layer uses ReLU, and a sigmoid maps the output to channel weights:

$$w=\sigma \left(\mathrm{M}\mathrm{L}\mathrm{P}\left(Z^{\prime }\right)\right),$$

(5)

where $w$ has shape $B \times C \times 1 \times 1$ and contains the sigmoid-normalized channel weights.

The weights are broadcast over H and W and multiplied elementwise with the original fused tensor, yielding $Fout = F \odot w$. There is no additional outer residual term of the form $F + F \odot w$; residual connections occur only inside the standard transformer encoder layer. Combining average and maximum descriptors is intended to retain complementary global and salient responses, but this interpretation is not presented as an independently validated causal result.

3.2.2. Coordinate-Aware Correlation Module

After multi-scale fusion, the registration stages must represent correspondences between the reference and target features. The coordinate-aware correlation module (CACM) combines normalized patch correlation with coordinate attention before motion regression (Figure 4). Its purpose is to provide position-sensitive matching features for geometric regression.

Figure 4. Coordinate-aware correlation module (CACM) conceptual pipeline: target-patch correlations are evaluated exactly through pixel-wise all-pairs correlation and aligned 3 × 3 offset aggregation; coordinate attention reweights the resulting volume, and convolution produces the feature flow.

Conceptually, the target patches can be interpreted as dynamic filters applied to the reference feature map. CACM then treats target positions as correlation channels and reweights the correlation volume along the two reference spatial axes using coordinate attention.

Let $Fr$ and $Ft$ denote the reference and target features, each with shape $B \times C \times H \times W$. Both tensors are L2-normalized along the channel dimension. CACM extracts a K × K target patch at every spatial position with $K = 3$, stride 1, and zero padding 1, producing $B \times H \times W \times 3 \times 3 \times C$ patches. In the conceptual formulation, these patches define B × HW dynamic filters of size C × $3 \times 3$ applied to the corresponding reference feature map. For a target-centered patch $Pi,j$, the correlation is summarized by Equation (6).

$$V_{\left(i,j\right)}\left(x,y\right)=\sum _{u,v,c}{P}_{i,j}\left(u,v,c\right)F_r\left(x+u,y+v,c\right),$$

(6)

In Equation (6), $\left(i,j\right)$ indexes a target position, (x, y) indexes a reference position, $\left(u,v\right)$ indexes the 3 × 3 patch support, and c indexes the $C$ normalized feature channels.

$$D_{\left(a,b\right),\left(x,y\right)}={\displaystyle \sum _{c=1}^C}{\widehat{F}}_t\left(a,b,c\right){\widehat{F}}_r\left(x,y,c\right),$$

(6a)

$$V_{i,j}\left(x,y\right)=\sum _{\left(u,v\right)\in {-\mathrm{1,0},1}^2}{D}_{\left(i+u,j+v\right),\left(x+u,y+v\right)},$$

(6b)

Here, ${\widehat{F}}_t$ and ${\widehat{F}}_r$ denote feature vectors L2-normalized along the channel dimension at each spatial location. Out-of-range indices are treated as zero under the same padding convention as Equation (6). Under this dense nine-offset evaluation, the factorization requires $N^{2}C$ MACs and $8N^2$ scalar additions for $N=HW$, without changing the output correlation volume.

Equation (6) defines the patch-correlation operation conceptually. In the implementation, it is evaluated exactly by first computing pixel-wise all-pairs correlation between the normalized target and reference features, followed by aggregation over the nine aligned 3 × 3 offsets. This is an algebraic factorization rather than a low-rank or local-window approximation and produces the same correlation volume up to floating-point rounding.

Evaluating Equation (6) for every target and reference location yields a correlation volume $V$ with shape B × HW × $H \times W$. The second dimension enumerates all $H \times W$ target centers, while the last two dimensions enumerate reference locations. Thus, the module uses local 3 × 3 descriptors but compares each target-centered descriptor over the full reference feature grid; it is not a $K \times K$ displacement-only volume. No softmax normalization is applied to $V$ in the available implementation.

Coordinate attention [23] receives V with HW target positions treated as channels. Horizontal pooling averages the width axis and produces B × HW × H × 1, whereas vertical pooling averages the height axis and produces B × HW × 1 × W. After a shared $1 \times 1$ projection, batch normalization, and ReLU, separate $1 \times 1$ projections and sigmoids generate horizontal and vertical attention maps. Their broadcast product reweights V elementwise while retaining the reference coordinate axes.

The enhanced volume is passed through two $3 \times 3$, stride-1, padding-1 convolutions with channel dimensions $HW \to 64 \to 2$ and a ReLU between them, producing a feature flow of shape $B \times 2 \times H \times W$. At the $32 \times 32$ stage, $HW = 1024$ and this flow is used by the first regression branch to predict four corner offsets. At the 64 × 64 stage, HW = 4096 and the second branch predicts the 169 two-dimensional TPS control-point offsets.

3.3. Objective Function

Let I_r and I_t denote the reference and target images, respectively; W(X; θ) denote bilinear warping; and Mθ = W(J; θ) denote the warped validity field obtained from an all-one tensor J. The implemented color offsets and alignment loss are written explicitly in Equation (7a–d):

δ_H = [Σ W(I_t; H) − Σ(I_r ⊙ M_H)]/ΣM_H;   δ_H⁻¹ = [Σ W(I_r; H⁻¹) − Σ(I_t ⊙ M_H⁻¹)]/ΣM_H⁻¹.

(7a)

δ_TPS = [Σ W(I_t; TPS) − Σ(I_r ⊙ M_TPS)]/ΣM_TPS.

(7b)

L_H = ‖(I_r + δ_H) ⊙ M_H − W(I_t; H)‖₁ + ‖(I_t + δ_H⁻¹) ⊙ M_H⁻¹ − W(I_r; H⁻¹)‖₁.

(7c)

L_TPS = ‖(I_r + δ_TPS) ⊙ M_TPS − W(I_t; TPS)‖₁;   L_alignment = 3L_H + L_TPS.

(7d)

Equation (7c) is bidirectional: it compares the target warped toward the reference and the reference warped by H⁻¹ toward the target. The per-channel offsets in Equation (7a,b) are normalized by the sums of their corresponding warped validity fields. Equation (7d) gives the forward TPS term and the implemented 3:1 weighting between homography and TPS alignment. The validity fields exclude out-of-domain samples but do not explicitly model occlusion or non-corresponding regions. The per-channel offsets compensate for simple global photometric differences but do not account for spatially varying illumination changes.

The mesh-shape regularizer contains an intra-mesh term that limits excessive positive horizontal and vertical edge components and an inter-mesh term that penalizes angular changes between successive edges in regions marked as non-overlapping. These penalties discourage excessive local deformation but do not mathematically guarantee global injectivity or prevent every possible fold.

For a mesh with U = V = 12 cells, horizontal edges are differences between adjacent columns and vertical edges are differences between adjacent rows. The intra-mesh term applies ReLU to horizontal x components above $2W/V$ and vertical y components above $2H/U$. With the implemented $W = H = 512$, both thresholds equal 85.33 pixels. The loss averages each component-wise penalty over its full horizontal or vertical edge tensor and sums the two means, as represented by Equation (8).

$$\begin{array}{c}{\mathcal{L}}_{intra}={\displaystyle \frac{1}{\left(U+1\right)\times V}}{\displaystyle \sum _{\left\{{\overrightarrow{e}}_{hor}\right\}}}\sigma \left(⟨\overrightarrow{e},\overrightarrow{i}⟩-{\displaystyle \frac{2W}{V}}\right)\\ +{\displaystyle \frac{1}{U\times \left(V+1\right)}}{\displaystyle \sum _{\left\{{\overrightarrow{e}}_{ver}\right\}}}\sigma \left(⟨\overrightarrow{e},\overrightarrow{j}⟩-{\displaystyle \frac{2H}{U}}\right)\end{array},$$

(8)

In Equation (8), i and j denote the horizontal and vertical unit directions, respectively, and σ(·) is ReLU. The available implementation constrains the corresponding signed edge components; it does not directly penalize every Euclidean edge length or every reversed edge.

The inter-mesh term measures 1 minus the cosine similarity of successive horizontal or vertical edge vectors. Neighboring angular errors are combined at the cell level and multiplied by a derived non-overlap indicator before averaging. Let $es1$ and $es2$ denote successive edges in the same direction; Equation (9) summarizes this angular penalty.

$${\mathcal{L}}_{inter}={\displaystyle \frac{1}{Q}}\sum _{\left\{{\overrightarrow{e}}_{s1},{\overrightarrow{e}}_{s2}\right\}}{S}_{s1,s2}\cdot \left(1-{\displaystyle \frac{⟨{\overrightarrow{e}}_{s1},{\overrightarrow{e}}_{s2}⟩}{‖{\overrightarrow{e}}_{s1}‖\cdot ‖{\overrightarrow{e}}_{s2}‖}}\right),$$

(9)

Here, $Q$ denotes the set of candidate successive-edge pairs and $Ss1,s2$ is the derived indicator. In the code, the warped validity mask is averaged within each grid cell and a cell is marked 1 when that mean is below 0.9; despite the variable name overlap, this value functions as a non-overlap indicator. Transitions between differently marked cells are suppressed, and torch.mean averages the masked arrays including zero-valued inactive positions rather than normalizing only by the number of active pairs.

The two mesh terms therefore act as soft regularizers: the intra-mesh term discourages excessive directed expansion, and the inter-mesh term discourages bending across selected non-overlap cells. They reduce the objective assigned to these deformations but do not establish a strict no-folding guarantee:

$${\mathcal{L}}_{shape}={\mathcal{L}}_{intra}+{\mathcal{L}}_{inter},$$

(10)

The final overall objective function of the network can be expressed as follows:

$$L=\lambda L_H+{\omega }_1{L}_{TPS}+{\omega }_2\left(L_{intra}+L_{inter}\right),$$

(11)

Equation (11) defines the total objective as L = λL_H + ω₁L_TPS + ω₂(L_intra + L_inter), where λ = 3, ω₁ = 1, and ω₂ = 10. Substitution gives the executed objective L = 3L_H + L_TPS + 10L_intra + 10L_inter. These implemented values were fixed before the final evaluation; Section 4.7 subsequently reports a one-factor-at-a-time sensitivity analysis around this configuration. The results in the sensitivity analysis indicate that the model performance remains reasonably stable under moderate variations of these loss weights around the default configuration, supporting the robustness of the selected weighting scheme within the evaluated ranges.

4. Experiments and Results

4.1. Dataset and Evaluation Criterion

The experiments use the UDIS-D dataset [13], which contains image pairs derived from variable-motion videos. The UDIS-D training split used in this study contains 10,440 paired samples, and the test split contains 1106 pairs. Every image is resized once to 512 × 512 pixels by the dataset loader.

Dataset information summary.

Field	Setting	Verification Source
Dataset	UDIS-D	Manuscript and local dataset directories
Source	Image pairs derived from variable-motion videos	UDIS/UDIS-D description [13] and manuscript
Training pairs	10,440	Count verified in training/input1 and training/input2
Testing pairs	1106	Count matched in archived testing/input1 and testing/input2
Input resolution	512 × 512 pixels	dataset.py resize operation
Scene/source description	Variable-motion video frame pairs with real-scene overlap and parallax	Dataset documentation and [13]
Usage in this study	Unsupervised training and UDIS-D overlap-region evaluation	train.py, dataset.py, and manuscript

Peak signal-to-noise ratio (PSNR) and the structural similarity index measure (SSIM) are used as overlap-region alignment metrics. They compare the reference image and the TPS-warped target over the warped valid-field region, as summarized in Equations (12) and (13). These metrics primarily assess overlap alignment and do not fully characterize seam quality or distortion in non-overlapping panorama regions.

$${\mathrm{P}\mathrm{S}\mathrm{N}\mathrm{R}}_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{l}\mathrm{a}\mathrm{p}}=\mathcal{P}\mathcal{S}\mathcal{N}\mathcal{R}\left(I_r·\phi \left(J,\mathcal{T}\mathcal{P}\mathcal{S}\right),\phi \left(I_t,\mathcal{T}\mathcal{P}\mathcal{S}\right)\right),$$

(12)

$${\mathrm{S}\mathrm{S}\mathrm{I}\mathrm{M}}_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{l}\mathrm{a}\mathrm{p}}=\mathcal{S}\mathcal{S}\mathcal{I}\mathcal{M}\left(I_r·\phi \left(J,\mathcal{T}\mathcal{P}\mathcal{S}\right),\phi \left(I_t,\mathcal{T}\mathcal{P}\mathcal{S}\right)\right),$$

(13)

PSNR and SSIM are calculated over the warped valid field of view and primarily quantify overlap-region alignment. For each separate metric, the 1106 test pairs are sorted into a top 300, a middle 506, or a bottom 300 score group, with the term ‘all’ denoting all 1106 pairs. These score partitions are not fixed scene difficulty labels based on parallax, texture, or semantics. Unless common image IDs are explicitly recorded, the groups must not be interpreted as identical difficulty subsets across methods.

4.2. Experiments Details

The network is implemented in PyTorch and trained on UDIS-D for a pre-specified 100 epochs using AdamW, with an initial learning rate of 1 × 10⁻⁴ and an exponential decay factor of 0.97 per epoch. Training uses a single NVIDIA GeForce RTX 4090 GPU, batch size 4, and 512 × 512 inputs. The checkpoint from the final training epoch (epoch 100) is used for evaluation according to the fixed training schedule and is not selected based on test-set performance.

Implementation configuration summary.

Field	Setting	Verification Source
Framework	PyTorch	train.py
Python version	3.8.5	environment.yml
PyTorch version	1.7.1	environment.yml
GPU	Single NVIDIA GeForce RTX 4090	Experimental setup record
Input resolution	512 × 512	dataset.py
Batch size	4	train.py default
Epochs	100	train.py default
Optimizer	AdamW, β = (0.9, 0.999), ε = 10−8	train.py
Initial learning rate	10−4	train.py
Learning rate schedule	ExponentialLR, γ = 0.97 per epoch	train.py
Effective batch size	4 (one process/GPU; no accumulation found)	train.py
Random seed	Random seed: not explicitly fixed in the implementation	No seed-setting call in the training script
Data augmentation	Random input-order swap; independent brightness and color multipliers sampled in [0.7, 1.3]	dataset.py
Training/testing split	10,440 training pairs/1106 testing pairs	Local dataset directory counts
Checkpoint selection	Final checkpoint at epoch 100	Run record and epoch100_model.pth
Mesh size U × V	12 × 12 cells; 13 × 13 control points	grid_res.py and new_network.py
CACM patch size K	3 × 3; stride 1; zero padding 1	new_network.py
Transformer layers	1	new_network.py
Attention heads	8	new_network.py
Transformer feed-forward dimension	2048	PyTorch 1.7.1 default used by new_network.py
TCA channel MLP	2C → C/8 → C	new_network.py

All controlled experiments use the same UDIS-D split, preprocessing, 100-epoch training schedule, and evaluation code. The primary configuration—including the 12 × 12 mesh, 3 × 3 CACM patch, one transformer layer with eight attention heads, and loss weights (λ = 3, ω₁ = 1, and ω₂ = 10)—was fixed before the final test-set evaluation. Following the fixed training schedule, the checkpoint from epoch 100 is used for final evaluation. The 1106-pair test set is used for result reporting rather than for architecture, hyperparameter, or checkpoint selection. Section 4.7 provides a supplementary one-factor-at-a-time sensitivity analysis around the fixed loss-weight configuration.

Rationale for architectural hyperparameters. All architectural configurations were determined based on preliminary experiments on a held-out subset of the training split and established engineering conventions, with no access to the 1106-pair test set at any stage.

12 × 12 control mesh. We evaluated 8 × 8, 12 × 12, and 16 × 16 mesh configurations during preliminary training. The 8 × 8 mesh lacked sufficient deformation flexibility for parallax-rich scenes, while the 16 × 16 mesh sharply increased regression dimensionality and caused overfitting to local texture noise. The 12 × 12 mesh (169 control points) achieves the best trade-off between alignment capacity and computational stability, consistent with common settings in TPS-based image stitching literature.

3 × 3 CACM patch size. A 3 × 3 local support is a standard receptive field for dense feature matching. A 1 × 1 patch lacks spatial context and degrades matching robustness. Increasing the patch to 5 × 5 does not change the dimensions of the output correlation volume, which are determined by the target and reference feature-map sizes; it instead increases the aligned-offset aggregation from 9 to 25 terms per correlation entry, thereby increasing computation and memory traffic. The 3 × 3 setting balances matching discriminability and efficiency.

Single Transformer layer in TCA. The TCA module operates on only two pooled channel tokens (GAP and GMP). Adding more encoder layers did not yield measurable accuracy gains in preliminary experiments and increased overfitting risk. One layer is sufficient to model the interaction between the two channel descriptors while maintaining an extremely light weight.

Eight attention heads. The channel dimensions at both fusion stages (1072 and 552) are divisible by 8, making eight heads compatible with the standard PyTorch multi-head attention implementation. Preliminary tests with four heads showed no consistent accuracy improvement. A 16-head configuration was not evaluated because the 552-dimensional second-stage embedding is not divisible by 16 under the standard implementation. We therefore retained eight heads as the standard configuration.

None of these hyperparameters were tuned or selected based on test-set performance; the test partition is used exclusively for final result reporting.

4.3. Comparative Experiments

Table 1 presents the quantitative comparison with the learning-based methods UDIS, UDIS2, and AbHE under the UDIS-D overlap-region evaluation protocol. Traditional feature-matching methods, including SIFT+RANSAC, ORB+RANSAC, and APAP, are evaluated separately in the failure-aware analysis, where both all-sample and valid-only results are reported to account explicitly for geometric-estimation failures.

Table 1. Learning-based quantitative comparison on UDIS-D using peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM).

Method	PSNR				SSIM
	Top 300	Middle 506	Bottom 300	All 1106	Top 300	Middle 506	Bottom 300	All 1106
UDIS	25.16	20.96	18.36	21.17	0.83	0.67	0.50	0.65
UDIS2	30.21	25.81	21.57	25.42	0.93	0.88	0.74	0.84
AbHE	27.49	23.75	20.18	23.43	0.89	0.82	0.66	0.78
Ours	29.85	25.57	21.45	25.19	0.93	0.87	0.72	0.83

On the 1106-pair test set, geometric estimation fails on 84 pairs for SIFT+RANSAC (7.59%), 121 pairs for ORB+RANSAC (10.94%), and 33 pairs for APAP (2.98%). For the all-sample evaluation, failed cases are retained using an identity warp, while the valid-only evaluation includes only successfully estimated transformations.

As shown in Table 1, the proposed method achieves an average PSNR of 25.19 dB and an average SSIM of 0.833, demonstrating competitive alignment performance among the evaluated learning-based methods. The comparison is interpreted descriptively based on the reported PSNR and SSIM values, as no paired statistical significance test is performed.

Computational complexity for GhostNet/ours is measured with the project’s custom PyTorch 1.7.1 full-pipeline MAC profiler for one input image pair (batch size 1; two 512 × 512 images). One multiply-accumulate operation is counted as one MAC. The profiling scope covers the shared backbone, TCA, the factorized CACM correlation path, coordinate attention, regression heads, geometric operations, dynamic canvas construction, and fusion. Custom counting is used for the matmul/bmm correlation path; the 3 × 3 aligned-offset aggregation is counted separately as scalar additions, and the implementation does not use einsum. Non-MAC elementwise operations are excluded from the GMAC total.

At the 32 × 32 and 64 × 64 stages, the factorized all-pairs term N²C contributes 0.537 and 4.295 GMACs, respectively (4.832 GMACs in total), while the two 3 × 3 HW → 64 → 2 convolution pairs contribute 10.274 GMACs. With the configured reduction ratio of 8, the coordinate attention 1 × 1 projections contribute 0.554 GMACs. The exact itemized CACM MAC subtotal is 15.659041 GMACs and is reported as 15.659 GMACs after rounding the exact subtotal to three decimals. Summing the individually rounded components (4.832 + 10.274 + 0.554) would yield 15.660 GMACs; this 0.001-GMAC difference is solely a rounding-order effect. Coordinate pooling, batch normalization, activations, and elementwise attention reweighting are non-MAC operations and are excluded. The nine-offset aggregation requires approximately 0.143 G scalar additions; these additions are reported separately and are explicitly excluded from the 19.28 GMAC total. The complete GhostNet/ours pipeline requires 19.28 GMACs and contains 52.86 M parameters for one input image pair. The full-pipeline accuracy–efficiency trade-offs of the three lightweight backbones are summarized in Figure 5.

Figure 5. Full-pipeline accuracy and efficiency comparison of lightweight backbones under the stated giga multiply-accumulate operation (GMAC), runtime, and memory protocol.

For the controlled backbone comparison, all three variants use the same complete end-to-end pipeline and the full-pipeline MAC protocol described above. The reported parameter and MAC counts are computed under the same scope, covering the backbone, TCA, factorized CACM correlation, coordinate attention, regression heads, TPS sampling, dynamic canvas construction, and fusion. Accordingly, Table 2 summarizes the source-reported model size and computation at 512 × 512 input resolution; for GhostNet/Ours, the primary full-system complexity is consistent with that used in the controlled-backbone comparison. The PSNR and SSIM values in the controlled-backbone comparison are obtained from independently trained backbone control runs and are therefore reported separately from the primary benchmark result in Table 1.

Table 2. Source-reported model size and computation at 512 × 512 input resolution.

Method	Input	Params (M)	Source-Reported Computation
UDIS	512 × 512	188.78 *	176.03 GFLOPs *
UDIS2	512 × 512	78.00 *	71.50 GFLOPs *
AbHE	512 × 512	105.01 *	5689.83 GFLOPs *
GhostNet/Ours	512 × 512	52.86	19.28 GMACs †

* Values are taken from the respective source implementations and may follow different profiling scopes and MAC/FLOP conventions. ^† Our result is measured as GMACs under the protocol described above. These values are not directly comparable as absolute arithmetic counts.

Figure 6 presents four selected UDIS-D examples covering outdoor planar structure, indoor low-texture regions, repeated vertical structures, and staircase geometry. Every method is shown for the same input pair in each row; black regions denote invalid canvas areas after warping.

Figure 6. Qualitative stitching comparison on selected UDIS-D test pairs.

4.4. Ablation Studies

Table 3 presents the individual and joint ablation results for TCA and CACM under the same evaluation protocol. Removing TCA decreases PSNR from 25.19 to 24.13 dB and SSIM from 0.833 to 0.810, while removing CACM reduces PSNR to 24.09 dB and SSIM to 0.800. When both TCA and CACM are removed, the performance further decreases to 23.38 dB PSNR and 0.782 SSIM. Compared with this joint baseline, the complete model improves PSNR by 1.81 dB and SSIM by 0.051. These results demonstrate the benefit of combining TCA and CACM, while the individual ablations quantify the contribution of each module when the other is retained.

Table 3. Module-level and joint module ablation on UDIS-D under the unified protocol.

Variant	PSNR (dB)	SSIM
w/o TCA	24.13	0.810
w/o CACM	24.09	0.800
w/o TCA + CACM	23.38	0.782
Ours	25.19	0.833

4.5. Component and Lightweight-Backbone Analysis

Table 4 presents fine-grained component ablations, with the complete model (25.19 dB PSNR and 0.833 SSIM) serving as the reference. Replacing the dual GAP+GMP token construction with GAP alone decreases PSNR by 0.33 dB and SSIM by 0.008. Removing coordinate attention from CACM results in a larger reduction of 0.58 dB in PSNR and 0.021 in SSIM. Removing the mesh-shape regularization terms decreases PSNR by 0.44 dB and SSIM by 0.014. Section 4.7 further examines the individual contributions of the alignment and regularization terms.

Table 4. Fine-grained component ablation on UDIS-D.

Component Setting	Controlled Comparison	PSNR (dB)	SSIM
TCA-GAP-only	Single pooled token	24.86	0.825
Full TCA	GAP + GMP tokens	25.19	0.833
Correlation without CA	Global correlation; CA removed	24.61	0.812
Full CACM	Global correlation + CA	25.19	0.833
w/o mesh loss	Alignment loss only	24.75	0.819
Full loss	Alignment + mesh loss	25.19	0.833

Under the unified full-pipeline comparison, GhostNet/ours achieves 25.16 dB PSNR and 0.831 SSIM, compared with 24.86/0.822 for MobileNetV3-Large and 24.54/0.811 for ShuffleNetV2 × 1.0. GhostNet/ours has intermediate parameter and MAC counts of 52.86 M and 19.28 GMACs, respectively, while providing the highest throughput at 13.36 FPS and the lowest peak GPU memory usage of 2108 MB. The accuracy values in Table 5 are obtained from independently trained backbone control runs, whereas the complexity values follow the same complete-pipeline profiling scope as Table 2.

Table 5. Full-pipeline comparison of lightweight backbones at 512 × 512 input resolution under the unified GMAC, runtime, and memory protocol.

Backbone	PSNR (dB)	SSIM	Params (M)	GMACs	FPS	Peak GPU Memory (MB)
MobileNetV3-Large	24.86	0.822	53.24	21.07	12.13	2312
ShuffleNetV2 ×1.0	24.54	0.811	52.09	19.15	12.65	2242
GhostNet/Ours	25.16	0.831	52.86	19.28	13.36	2108

4.6. Runtime and Failure Analysis

Runtime and memory measurements for the complete backbone variants are presented in Table 5 under a unified GPU evaluation protocol. Traditional feature-matching baselines are evaluated separately in Table 6, where failed geometric estimates are retained in the all-sample results using identity-warp replacement and valid-only metrics are also provided. Table 7, Table 8, Table 9 and Table 10 present additional analyses of loss-term ablations, hyperparameter sensitivity, geometric-stage effectiveness, and mesh-cell orientation behavior.

Table 6. Failure statistics and reconstruction metrics for traditional feature-matching methods on 1106 UDIS-D test pairs. Failed estimates use an identity warp in the all-sample aggregate.

Method	Failures	Failure Rate	All PSNR	All SSIM	Valid PSNR	Valid SSIM	Replacement Rule
SIFT+RANSAC	84	7.59%	20.08	0.605	21.66	0.653	Identity warp on failure
ORB+RANSAC	121	10.94%	18.82	0.559	21.07	0.626	Identity warp on failure
APAP	33	2.98%	22.75	0.708	23.41	0.728	Identity warp on failure

Table 7. Fine-grained loss-term ablation on UDIS-D.

Variant	PSNR (dB)	SSIM	PSNR Drop (dB)
Full ours (complete loss)	25.19	0.833	—
w/o Lintra	25.00	0.827	0.19
w/o Linter	24.94	0.825	0.25
w/o mesh loss	24.75	0.819	0.44
w/o LTPS	24.12	0.803	1.07
w/o LH	24.47	0.812	0.72

Table 8. One-factor-at-a-time sensitivity analysis of loss weights on UDIS-D.

Parameter	Value	PSNR (dB)	SSIM
λ (ω1 = 1, ω2 = 10)	2	24.97	0.826
λ (ω1 = 1, ω2 = 10)	3 (default)	25.19	0.833
λ (ω1 = 1, ω2 = 10)	4	25.04	0.828
ω1 (λ = 3, ω2 = 10)	0.5	24.89	0.824
ω1 (λ = 3, ω2 = 10)	1 (default)	25.19	0.833
ω1 (λ = 3, ω2 = 10)	1.5	25.02	0.827
ω2 (λ = 3, ω1 = 1)	5	25.01	0.828
ω2 (λ = 3, ω1 = 1)	10 (default)	25.19	0.833
ω2 (λ = 3, ω1 = 1)	15	24.95	0.825

Table 9. Geometric-stage ablation on UDIS-D.

Variant	PSNR (dB)	SSIM
Homography-only	24.25	0.807
Full ours (homography + TPS)	25.19	0.833

Table 10. Two-triangle orientation-reversal/degeneration flag audit on the 1106-pair UDIS-D test set.

Metric	Value
Test images	1106
Total mesh cells	159,264
Images containing at least one flagged cell	1106/1106
Flagged cells	11,586/159,264 (7.275%)
Mean flagged cells per image	10.476
Maximum per image	37

The observed failure-rate ordering is ORB+RANSAC, SIFT+RANSAC, and APAP from highest to lowest. For every method, valid-only PSNR/SSIM exceeds the all-sample result because failed estimates are retained through identity-warp replacement. APAP provides the best traditional valid-only result (23.41 dB, 0.728), but it remains below the complete model by 1.78 dB PSNR and 0.105 SSIM. This comparison documents the limitations of hand-crafted matching under the stated protocol; it does not constitute a paired significance test.

4.7. Additional Ablation, Sensitivity, and Mesh-Cell Orientation Analysis

Table 7 decomposes the objective under the complete architecture. Removing L_intra and L_inter individually decreases PSNR by 0.19 dB and 0.25 dB, respectively, while removing both mesh regularizers results in a 0.44 dB decrease. The alignment terms have larger effects: removing L_TPS decreases PSNR by 1.07 dB, whereas removing L_H decreases PSNR by 0.72 dB. These results indicate that both global homography and local TPS supervision contribute substantially to the training objective, with L_TPS showing the larger performance impact in this loss-term ablation.

Table 8 varies one weight at a time around the implemented setting. The default configuration (λ = 3, ω₁ = 1, ω₂ = 10) yields the highest PSNR and SSIM among the tested values in all three sweeps. Across these tested local variations, PSNR changes by no more than 0.30 dB and SSIM by no more than 0.009, indicating that the model performance remains reasonably stable under moderate perturbations of the loss weights. These results support the local robustness of the selected configuration within the evaluated parameter ranges.

The homography-only baseline in Table 9 was trained independently from scratch with exactly the same training protocol as the full model, including the identical UDIS-D training split, data augmentation, AdamW optimizer, exponential learning rate decay, batch size of 4, and 100-epoch schedule. The TPS control-point regression branch and all three TPS-related loss terms ($L_{\mathrm{TPS}}$, $L_{\mathrm{intra}}$, $L_{\mathrm{inter}}$) were removed from the training objective. The model predicts only the four-corner homography offsets and uses the solved global homography as the final warp transformation, with no subsequent local refinement. The final checkpoint at epoch 100 is selected for evaluation, following the same fixed schedule as the complete model.

Table 9 evaluates the contribution of the second geometric stage. The homography-only baseline achieves 24.25 dB PSNR and 0.807 SSIM, whereas introducing TPS refinement improves the performance to 25.19 dB and 0.833, corresponding to gains of 0.94 dB in PSNR and 0.026 in SSIM. These results demonstrate the benefit of local TPS refinement following global homography initialization. The homography-only result is slightly higher than that of the w/o L_TPS variant because the two experiments modify different components: the former removes the TPS stage entirely, whereas the latter retains TPS deformation but removes its direct alignment supervision. Therefore, the two results are not directly equivalent ablations.

Mathematical criterion for flagging locally reversed or degenerate control-mesh triangles. For each cell of the 12 × 12 mesh, let the following vectors denote its top-left, top-right, bottom-left, and bottom-right vertices, respectively:

$${\mathbf{p}}_{00}={\mathbf{p}}_{u,v},{\mathbf{p}}_{01}={\mathbf{p}}_{u,v+1},{\mathbf{p}}_{10}={\mathbf{p}}_{u+1,v},{\mathbf{p}}_{11}={\mathbf{p}}_{u+1,v+1}$$

Using upper-left-origin image coordinates with the vertical axis pointing downward, the two-dimensional cross product is defined as follows:

$$\mathbf{a}\times \mathbf{b}=a_x{b}_y-a_y{b}_x$$

The implementation splits the quadrilateral along the diagonal connecting the top-right and bottom-left vertices and evaluates the two consistently oriented triangle signs:

$$\begin{gathered} s_{uv}^{\left(1\right)}=\left({\mathbf{p}}_{01}-{\mathbf{p}}_{00}\right)\times \left({\mathbf{p}}_{10}-{\mathbf{p}}_{00}\right) \\ {s}_{uv}^{\left(2\right)}=\left({\mathbf{p}}_{11}-{\mathbf{p}}_{01}\right)\times \left({\mathbf{p}}_{10}-{\mathbf{p}}_{01}\right) \end{gathered}$$

Both quantities are positive for the regular initial grid. A cell is flagged when at least one triangle is locally orientation-reversed or degenerate:

$${\mathrm{F}\mathrm{l}\mathrm{a}\mathrm{g}}_{uv}=\mathbf{1}\left[s_{uv}^{\left(1\right)}\le 0\vee s_{uv}^{\left(2\right)}\le 0\right]$$

The non-strict threshold matches the implemented audit: a negative triangle sign indicates orientation reversal, whereas a zero sign indicates triangle degeneration. This is a local control-mesh vertex audit and does not constitute a continuous Jacobian-sign or higher-order self-intersection audit of the TPS transformation.

For overlap-region classification, let $M_{\mathrm{TPS}}$ denote the validity mask obtained by bilinearly warping an all-one tensor into the reference-image coordinate domain, with values in $\left[\mathrm{0,1}\right]$. Let $C_{uv}$ denote the fixed cell-aligned raster block produced by the non-overlapping unfold operation. Its mean validity is as follows:

$$m_{uv}={\displaystyle \frac{1}{\left|C_{uv}\right|}}\sum _{p\in C_{uv}}{M}_{\mathrm{TPS}}\left(p\right)$$

A cell is classified as belonging to the overlap region when we find the following:

$${\mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{l}\mathrm{a}\mathrm{p}}_{uv}=\mathbf{1}\left[m_{uv}\ge 0.9\right]$$

The implementation uses non-overlapping 42 × 42 blocks with stride 42. Consequently, the 12 × 12 blocks cover the upper-left 504 × 504 region of each 512 × 512 mask; the final eight rows and final eight columns are not included in any block mean. The overlap criterion is defined for reproducibility, but we do not report a flag–overlap cross-tabulation because the retained audit output does not preserve the required cell-wise joint labels.

Using the above two-triangle criterion, all 1106 test images contain at least one flagged cell. In total, 11,586 of the 159,264 evaluated control-mesh cells are flagged (7.275%), corresponding to a mean of 10.476 flagged cells per image and a maximum of 37 in one image. The retained audit does not distinguish strict orientation reversals (s < 0) from zero-area degeneracies (s = 0); therefore, 11,586 denotes flagged cells and must not be interpreted as a count of folded cells. Because flags occur in every test image and in 7.275% of cells, the incidence of local orientation or area violations under this criterion is non-negligible. Such flags may indicate locally inverted or collapsed control-mesh triangles and potential geometric distortion. However, the number of true fold-overs and their direct effect on stitched-image quality cannot be determined without separate s < 0 and s = 0 counts or a continuous TPS-Jacobian audit. Future audits should retain the two categories and cell-wise spatial labels separately.

5. Conclusions

This work presents a coarse-to-fine image-stitching framework that integrates a shared GhostNet backbone, TCA-based multi-scale feature fusion, CACM-based correlation enhancement, global homography estimation, and local TPS refinement. TCA combines global-average and global-maximum channel descriptors to enhance feature representation, while CACM incorporates local correlation support and coordinate-aware reweighting to improve spatial correspondence modeling.

On UDIS-D, the complete model achieves 25.19 dB PSNR and 0.833 SSIM. Removing both TCA and CACM reduces the performance to 23.38 dB and 0.782, respectively, demonstrating the contribution of the proposed feature-enhancement modules. Loss-term ablations further show that both homography and TPS supervision are important to alignment accuracy, while the homography-only comparison confirms the additional benefit of local TPS refinement. The one-factor-at-a-time sensitivity analysis indicates stable performance within the evaluated loss-weight ranges, and the two-triangle orientation-reversal/degeneration flag audit identifies a non-negligible incidence of local control-mesh orientation or area violations under the stated criterion.

In the controlled backbone comparison, the GhostNet-based model achieves the highest reconstruction accuracy, the highest throughput of 13.36 FPS, and the lowest peak GPU memory usage of 2108 MB among the evaluated backbone variants, while maintaining moderate parameter and computational complexity under the stated GMAC, runtime, and memory protocol. The traditional feature-matching baselines also exhibit non-negligible geometric-estimation failures, highlighting the practical robustness advantage of the learning-based framework under the evaluated protocol.

Several limitations remain. The current evaluation is restricted to a single dataset and relies primarily on overlap-region PSNR and SSIM. Cross-dataset generalization, CPU and embedded-device profiling, perceptual seam-quality evaluation, and paired statistical analysis have not yet been investigated. In addition, the audit does not separate strict reversals (s < 0) from zero-area degeneracies (s = 0) and is not a continuous TPS-Jacobian analysis; it therefore cannot determine the number of true fold-overs, their direct image-level impact, or global TPS invertibility. Future work will therefore focus on broader cross-domain validation, deployment-oriented efficiency evaluation, perceptual stitching quality, separate orientation/degeneration reporting, and stronger geometric constraints for locally adaptive warping.