Multi-Scale Fractional-Order Image Fusion Algorithm Based on Polarization Spectral Images

Abstract

With the continuous advancement of polarization spectral sensing technology, multi-band polarization image fusion has emerged as a novel approach to image fusion. By integrating spectral and polarization information, this method overcomes the limitations of relying on a single information source and significantly improves overall image quality. To address this, this paper proposes a new polarization spectral fusion algorithm. First, feature matching is employed to achieve pixel-level spatial alignment of multi-band polarization images. Then, a fusion strategy based on multi-scale decomposition and singular value decomposition is adopted to preserve structural information and fine details. Subsequently, fractional-order processing and guided filtering are applied to enhance details and suppress noise. Finally, a progressive reconstruction from low to high scales is performed to ensure hierarchical consistency and information integrity throughout the fusion process. In addition, spectral information is utilized for color restoration, enabling the final image to achieve high spatial resolution while maintaining natural and rich color representation.Experimental results demonstrate that the proposed method effectively integrates features from different spectral bands and polarization information while preserving maximum similarity, leading to significant improvements in both image quality and detail representation.

1. Introduction

In the field of target detection and recognition, the current challenges are compounded by complex and changing environments. Under conditions involving concealed targets amidst backgrounds, weak radiation objects, and strong interference backgrounds, traditional methods relying on light intensity, spatial, or single-spectral information often prove ineffective for discrimination. Consequently, applications in areas such as military reconnaissance, environmental monitoring, and precision agriculture face limitations [1,2]. Polarization spectral imaging extends traditional three-dimensional information to seven dimensions, simultaneously capturing multi-dimensional data including intensity, polarization, spectral characteristics, and spatial information of the target. This technology enables precise description of surface roughness, material composition, and other attributes across different wavelengths, revealing various aspects of the target’s inherent state, thereby paving a new way to overcome the bottlenecks of conventional optical imaging techniques [3,4]. The rapid advancement of photoelectric detection devices and signal processing methods has significantly enhanced the ability to extract and fuse polarization spectral information from targets, making this a current hotspot in optical imaging research [5,6].

High-quality acquisition of target polarization spectral images is a prerequisite for effective fusion and precise identification, with the core challenge being the collaborative modulation of polarization-spectral information. Existing research methods include Acousto-Optic Tunable Filters (AOTF), Liquid Crystal Tunable Filters (LCTF), and integrated detectors [7,8,9]. The U.S. Army Research Laboratory (ARL) developed a polarization spectral imaging system utilizing AOTF and Liquid Crystal Variable Retarders (LCVR), operating within the 0.4–11.5 µm range. This system employs electronic control technology to enhance imaging timeliness, but suffers from a low signal-to-noise ratio due to the influence of acousto-optic sensor noise [10]. Domestically, Lu Fang from Xidian University adopted the Polarization-Difference Channel Imaging Spectropolarimetry (PDCISP) technique, achieving simultaneous acquisition of three orthogonal polarization modulation modes. This expands the effective optical path difference range and effectively suppresses inter-channel crosstalk [11]. The team led by Zhang Fei from the Chinese Academy of Sciences proposed a novel configurable dual-mode detection method, integrating it with a miniaturized shared-aperture meta-optical system. This enables simultaneous operation in the 8–14 µm band with a large 70-degree field of view, facilitating multi-mode, multi-target detection and laying the foundation for the miniaturization and practical application of multifunctional optical systems [12]. The compressed full-Stokes hyperspectral imaging technique proposed by Fan et al. employs a combined modulation of quarter-wave plate, linear polarizer, and LCTF, coupled with numerical reconstruction algorithms. This reduces reconstruction time by 83.73%, ultimately achieving high-precision and rapid calculation of Stokes parameters [13]. However, current technology still faces challenges such as the difficulty in balancing broadband operation with high resolution, imaging lag in dynamic scenes, and low integration levels, making it difficult to meet the demands of mobile and real-time detection.

Polarization-spectral image fusion is a method for complementary advantages of multi-source information. It effectively enhances target contours, reduces background noise, and improves image discriminability, serving as a crucial step connecting data acquisition and object recognition [14,15,16,17]. Early fusion techniques primarily employed multi-scale transforms such as wavelet transform and non-subsampled contourlet transform, as well as sparse representation. Zhu Pan et al. proposed a novel fusion algorithm based on Dual-Tree Complex Wavelet Transform (DTCWT) and sparse representation, which balances the fusion of infrared polarization and intensity images while preserving details and eliminating noise [18]. Building on this foundation, some research has proposed a hyperspectral image color space transformation method based on color space mapping, which accurately fuses spectral information from specular reflection with polarization information from diffuse reflection, providing an effective technical approach for applications such as astronomical observation [19]; Consequently, this project plans to employ a weighting method based on regional energy and Local Binary Pattern (LBP) operators to jointly analyze fused feature images, spectral images, and polarization images, thereby improving the information completeness and robustness of the images [20]. Furthermore, a method combining principal component analysis and energy weighting can effectively fuse multi-dimensional features to enhance the detail and edge characteristics of objects [21]; On this basis, research has been conducted on an edge detail enhancement algorithm based on rolling guidance filter and sparse representation. First, bilateral filtering is applied to the Stokes parameter images within the polarization vector space to suppress noise while preserving polarization characteristics. Second, RGF is used to decompose the grayscale and low-noise polarization images into a base layer and a detail layer. Employing a multi-scale fusion strategy ultimately achieves a synergistic improvement in visual quality and brightness while maintaining complex details [22].

In recent years, with the advancement of deep learning, various deep fusion models have emerged. Generative Adversarial Networks (GANs) represent an approach that utilizes polarization hyperspectral data for deep feature extraction and weight fusion [23]; By efficiently utilizing Degree of Linear Polarization (DoLP) images, an encoder-decoder based dense convolutional network demonstrates strong fusion performance across three dimensions: grayscale, edge, and texture [24]; This research innovatively combines multi-dimensional self-attention (F-MSA) with entropy-based fusion techniques, playing a significant role in the fusion process and exhibiting high adaptability to multi-modal tasks such as complex environment monitoring and visible light fusion [25]. Furthermore, an unsupervised deep neural network, employing a customized framework and loss function, fuses polarization images without relying on ground-truth image information to enhance visual quality and quantitative metrics [26]; Based on the Non-subsampled Contourlet Transform (NSCT) and Convolutional Neural Networks (CNN), denoising and polarization parameters are fused to obtain polarization feature maps. Techniques including NSCT decomposition, feature extraction and normalization, block-based strategies for determining fusion coefficients, and inverse transform reconstruction are utilized, thereby improving target detection accuracy in polarization images [27]; Leveraging a semantic-guided dual-discriminator generative adversarial network (SGPF-GAN), the fusion process is guided by block-weighted Polarization Image Quality Discriminators (PIQD). By establishing an adversarial game between the generator and the two discriminators, the detection and segmentation performance for transparent and camouflaged concealed targets can be significantly enhanced [28].

Although research on multi-band polarization image fusion has begun, effective fusion algorithms remain scarce. This is primarily due to two reasons: first, traditional polarization spectrum acquisition devices feature complex structures and large volumes; second, conventional data fusion methods employ a relatively single fusion strategy when processing polarization spectrum information, leading to information loss during the fusion process. To address these issues, this paper constructs a time-division polarization spectrum system for multi-band polarization image data acquisition. Image feature matching is employed to achieve pixel-level spatial alignment, ensuring data consistency throughout the subsequent fusion process. Through a multi-scale decomposition and SVD-based fusion strategy, the intensity information and polarization information from different bands are respectively fused into novel Gaussian pyramids and Laplacian pyramids that encompass features from all bands. Fractional-order processing and guided filtering techniques not only eliminate noise components from the polarization information but also enhance texture details. Finally, a progressive recovery and weighted fusion rule from low scale to high scale is proposed, eliminating information loss caused by single fusion rules. Color rendering based on original spectral information ensures that the final high-resolution fused images simultaneously possess authentic and rich colors.

Compared with previous work, the main contributions of our proposed method are as follows:

1.

A time-division polarization spectrum system is proposed, which can acquire polarization spectrum images across different bands while ensuring spatial resolution. Through feature matching techniques, pixel-level spatial alignment of polarization images from different bands is guaranteed.

2.

Based on multi-scale decomposition and SVD, the spectral information and polarization information from different bands are fused at various scales, maximizing the retention of feature information from each band. By integrating fractional-order processing and guided filtering, noise components in the polarization information are eliminated while texture details are enhanced.

3.

Through a progressive recovery and weighted fusion strategy from low scale to high scale, information loss caused by single fusion rules is eliminated, ensuring the imaging quality and natural appearance of the fused images.

The remainder of this paper is organized as follows: Section 2 introduces fundamental concepts, including SVD, Stokes vectors, and fractional-order differentiation, and elaborates on the fusion rules proposed in this paper. Section 3 discusses the experimental validation of the theoretical model and presents the corresponding results. Section 4 presents subjective and objective analyses comparing our method with other fusion algorithms. Section 5 concludes the paper and discusses its limitations.

2. Correlation Theory

2.1. Singular Value Decomposition

Singular Value Decomposition (SVD) is an important matrix factorization method in linear algebra, used to decompose any matrix into the product of three simpler matrices. This method is applicable not only to square matrices but also to rectangular matrices of any shape, serving as a generalization of eigenvalue decomposition to non-square matrices. SVD decomposes a matrix into an orthogonal matrix, a diagonal matrix, and the transpose (or conjugate transpose) of another orthogonal matrix, where the diagonal elements of the diagonal matrix are called singular values.

For an $m\times n$ real matrix, its singular value decomposition can be expressed as:

$$A=U\Sigma V^{\top }$$

(1)

Here, U is an $m\times m$ orthogonal matrix satisfying $U^{\top }U=I$, and its column vectors are called left singular vectors. v is an $n\times n$ orthogonal matrix satisfying $V^{\top }V=I$, and its column vectors are called right singular vectors. ∑ is an $m\times n$ rectangular diagonal matrix whose diagonal elements are the singular values of matrix A. These singular values are typically arranged in descending order, while all off-diagonal elements are zero.

SVD has a wide range of applications in mathematics and engineering, including data dimensionality reduction and Principal Component Analysis (PCA), recommendation systems, image compression and denoising, and solving linear equations. Its core idea is to achieve a low-rank approximation of the original data by retaining the largest singular values and their corresponding singular vectors, thereby removing noise and extracting key features. SVD reveals the geometric essence of a matrix in linear transformations, namely the process of “rotation-stretching-rotation,” and provides complete information about the four fundamental subspaces of a matrix (column space, row space, null space, and left null space).

In the context of images, an image can be represented as a large matrix, where each element corresponds to a pixel value. Through SVD decomposition, the image can be expressed as a sum of rank-1 matrices (outer product terms). Since singular values typically decay exponentially, retaining only the largest singular values along with their corresponding singular vectors allows for reconstructing a visually high-quality image using minimal storage space, enabling efficient compression. Additionally, smaller singular values often correspond to high-frequency noise in the image; setting them to zero can achieve image denoising while preserving the main structure of the image.

2.2. Stokes Vector

The polarization of light can be described by several methods, including the electric field vector method, the Jones matrix method, the Poincaré sphere, and the Stokes vector method. In practical applications of polarization imaging, the most common approach is to obtain the required polarization characteristic parameters by acquiring the Stokes vector. Therefore, this paper focuses on the Stokes vector representation of polarized light.

In 1852, Stokes proposed using four parameters to describe the intensity and polarization state of light waves. Unlike the Jones vector, the Stokes vector can describe fully polarized light, partially polarized light, and completely unpolarized light, as well as both monochromatic and non-monochromatic light. These four Stokes parameters are time-averaged values of light intensity (averaged over a time interval sufficiently long for measurement) and together form a four-dimensional mathematical vector. They are defined as follows:

$$\begin{array}{cc}\hfill S_0& ={E_x}^2+{E_y}^2\hfill \\ \hfill S_1& ={E_x}^2-{E_y}^2\hfill \\ \hfill S_2& =2E_x{E}_y\cos \delta \hfill \\ \hfill S_3& =2E_x{E}_y\sin \delta \hfill \end{array}$$

(2)

In the Equation, $S_0$ represents the total intensity of light waves, which is always non-negative. $S_1$ denotes the difference between the horizontal and vertical linear polarization components, describing the preferential polarization degree of light in the 0° and 90° directions. $S_2$ represents the difference in intensity between the 45° and 135° linear polarization components, describing the preferential polarization degree of light in the ±45° directions. $S_3$ represents the difference in intensity between the right-handed circular polarization and left-handed circular polarization components, describing the degree of circular polarization of light. $S_3>0$ indicates that right-handed circular polarization is dominant, while $S_3<0$ indicates that left-handed circular polarization is dominant. In practical applications, Stokes can also be directly represented by image intensity [29,30,31]:

$$\begin{array}{cc}\hfill S_0& =I\left(0\right)+I\left(90\right)\hfill \\ \hfill S_1& =I\left(0\right)-I\left(90\right)\hfill \\ \hfill S_2& =I\left(45\right)-I\left(135\right)\hfill \\ \hfill S_3& =I\left(R\right)-I\left(L\right)\hfill \end{array}$$

(3)

In the Equation (3), $I\left(0\right)$, $I\left(45\right)$, $I\left(90\right)$, and $I\left(135\right)$ represent the four polarization images obtained by the sensor at directions of 0°, 45°, 90°, and 145°, respectively. $I\left(R\right)$ denotes the right-handed circular polarization image, and $I\left(L\right)$ denotes the left-handed circular polarization image. Typically, to combat noise interference, the value of $S_0$ is taken as half the sum of the four angles. In addition, in natural environments, the linear polarization component predominantly dominates the polarized light. In contrast, the intensity of the circular polarization component (i.e., Stokes parameter $S_3$ is typically only on the order of ${10}^{-4}$ to ${10}^{-3}$ of the total light intensity, significantly smaller than the linear polarization component. Therefore, from the perspective of energy contribution, the polarized light in nature can be considered dominated by linear polarization, with elliptically polarized light merely manifesting as a minor and negligible correction term. Based on this prior understanding, the $S_3$ component will not be considered in the subsequent analysis of this paper, simplifying the Stokes vector to the following form:

$$\begin{array}{cc}\hfill S_0& =I\left(0\right)+I\left(45\right)+I\left(90\right)+I\left(135\right)/2\hfill \\ \hfill S_1& =I\left(0\right)-I\left(90\right)\hfill \\ \hfill S_2& =I\left(45\right)-I\left(135\right)\hfill \end{array}$$

(4)

According to Malus’s theorem, the light intensity values for different polarization directions are given by Equation (5).

$$\begin{array}{cc}\hfill I_0& ={\displaystyle \frac{1}{2}}S(1-p)+Sp{\cos }^2(\theta -0°)\hfill \\ \hfill I_{45}& ={\displaystyle \frac{1}{2}}S(1-p)+Sp{\cos }^2(\theta -45°)\hfill \\ \hfill I_{90}& ={\displaystyle \frac{1}{2}}S(1-p)+Sp{\cos }^2(\theta -90°)\hfill \\ \hfill I_{135}& ={\displaystyle \frac{1}{2}}S(1-p)+Sp{\cos }^2(\theta -135°)\hfill \end{array}$$

(5)

Here, p is the degree of polarization (DOP) and $\theta$ is the angle of polarization (AOP). Using Malus’s theorem and the Stokes vector, the degree of polarization and polarization angle in the detected light can be determined, as shown in Equations (6) and (7).

$$p={\displaystyle \frac{\sqrt{{S_1}^2+{S_2}^2}}{S}}(0<p<1)$$

(6)

The value range of the degree of polarization is [0, 1]. The larger the value, the greater the proportion of fully polarized light. When DOP = 1, the light wave is fully polarized light; when DOP = 0, the light wave is natural light (completely non-polarized light); in other cases, the light wave is partially polarized light.

$$\begin{array}{cc}\hfill \theta =& {\displaystyle \frac{1}{2}}\arctan {\displaystyle \frac{S_2}{S_1}}\hfill \\ \hfill (S_2>0,& S_1>0)\theta \in \left(0,{\displaystyle \frac{\pi }{4}}\right)\hfill \\ \hfill (S_2>0,& S_1<0)\theta \in \left({\displaystyle \frac{\pi }{4}},{\displaystyle \frac{\pi }{2}}\right)\hfill \\ \hfill (S_2<0,& S_1<0)\theta \in \left({\displaystyle \frac{\pi }{2}},{\displaystyle \frac{3\pi }{4}}\right)\hfill \\ \hfill (S_2<0,& S_1>0)\theta \in \left({\displaystyle \frac{3\pi }{4}},\pi \right)\hfill \end{array}$$

(7)

From the definition, it can be seen that the range of values for the polarization angle is [0, $\pi$]. It should be noted that when performing actual calculations, the relative sizes of $S_2$ and $S_1$, as well as the quadrant range, must be considered. Otherwise, calculation errors may occur. The calculation formula for AOP can also be expressed as:

2.3. GL Fractional Order Differential

Fractional-order differentiation is a mathematical theory that extends the order of differentiation from integers to arbitrary real numbers or even complex numbers. In contrast to the limitation of traditional differential equations, which can only describe instantaneous changes, fractional-order differentiation possesses long-term memory and the ability to model complex processes with historical dependence. Currently, with the significant development of fractional-order differentiation theory, there are typically three definitions. The first is the Grünwald–Letnikov (GL) definition, which is best suited for numerical computation and discrete implementation. The second is the Riemann–Liouville (RL) definition, which is often used for theoretical derivations. The third is the Caputo definition, which is commonly applied in engineering and control systems. Given that images are in discrete form, this paper focuses on explaining the GL definition.

$$f^{\prime }\left(t\right)=\underset{h\to 0}{\lim }{\displaystyle \frac{f\left(t\right)-f(t-h)}{h}}$$

(8)

Furthermore, according to the theory of integer-order derivatives, the second-order derivative of a function can be derived as follows:

$$f^{″}\left(t\right)=\underset{h\to 0}{\lim }{\displaystyle \frac{f\left(t\right)-2f(t-h)+f(t-2h)}{h^2}}$$

(9)

By analogy, the definition of the nth-order derivative is obtained as follows:

$$f^{\left(n\right)}\left(t\right)=\underset{h\to 0}{\lim }{\displaystyle \frac{{\sum }_{r=0}^n{(-1)}^r\left(\genfrac{}{}{0pt}{}{n}{r}\right)f(t-rh)}{h^n}}$$

(10)

where r is an integer ranging from 0 to n, and $\left({\displaystyle \genfrac{}{}{0pt}{}{n}{r}}\right)={\displaystyle \frac{n!}{r!(n-r)!}}$ represents the standard binomial coefficient. By extending the integer n to any real number a (where $a>0$), the following formula can be obtained:

$$f^{\left(a\right)}\left(t\right)=\underset{h\to 0}{\lim }{\displaystyle \frac{{\sum }_{r=0}^{\left[\frac{t-a}{h}\right]}{(-1)}^r\left(\genfrac{}{}{0pt}{}{a}{r}\right)f(t-rh)}{h^a}}$$

(11)

$\left[{\displaystyle \frac{t-a}{h}}\right]$ represents the greatest integer not exceeding ${\displaystyle \frac{t-a}{h}}$, which ensures that only the interval from the starting point a to the current point t is covered. Then, we introduce the Gamma function to transform the binomial coefficient, originally representing integers, into a generalized binomial coefficient capable of representing fractions.

$$\left({\displaystyle \genfrac{}{}{0pt}{}{a}{r}}\right)={\displaystyle \frac{\Gamma (a+1)}{\Gamma (r+1)\Gamma (a-r+1)}}$$

(12)

Finally, the Grünwald-Letnikov fractional-order derivative formula is obtained.

$$D^{a}f\left(t\right)={\displaystyle \frac{1}{h^a}}\underset{h\to 0}{\lim }\sum _{r=0}^{\left[{\displaystyle \frac{t-a}{h}}\right]}{(-1)}^r{\displaystyle \frac{\Gamma (a+1)}{\Gamma (r+1)\Gamma (a-r+1)}}f(t-rh)$$

(13)

Since we are dealing with discretization operations based on images, in image processing, $h=1$, and the formula is simplified to

$$D^{a}f\left(t\right)=\sum _{r=0}^N{(-1)}^r{\displaystyle \frac{\Gamma (a+1)}{\Gamma (r+1)\Gamma (a-r+1)}}f(t-r)$$

(14)

N represents $\left[{\displaystyle \frac{t-a}{h}}\right]$, which also indicates that fractional-order derivatives require memory of the historical context, rather than only observing the current changes. In contrast to integer-order derivatives, which can only enhance edges but lead to the loss of weak textures, fractional-order derivatives enable both the enhancement of details and the preservation of textures. This lays a theoretical foundation for our subsequent image processing tasks.

2.4. Multi-Scale Fractional-Order Enhanced Fusion Model

In related image fusion algorithms, a fixed, single rule is typically employed for fusion, which results in insufficient clarity of image edges and texture details after processing. This issue is particularly pronounced when dealing with two entirely distinct types of information, such as spectral and polarization data. Therefore, we propose a novel fusion algorithm in this paper. By constructing a multi-scale pyramid model, information from different bands is fused at various scales. Additionally, fractional-order differentiation is utilized to extract texture and features from polarization information, enabling the progressive fusion of polarization and spectral data in a coarse-to-fine manner.

First, as shown in Equation (15), we perform scale decomposition on the polarization spectra of different bands to construct Gaussian pyramids and Laplacian pyramids for images from various bands. The construction process of the Gaussian pyramid can be represented recursively. Specifically, the image $G_n$ at the N-th layer is obtained by applying Gaussian low-pass filter convolution to the image $G_{n-1}$ from the previous layer, followed by downsampling, as shown in Equation (15).

$$G_n(x,y)=\sum _{m=-2}^2\sum _{n=-2}^{2}w(m,n)G_{n-1}(2x=m,2y+n)$$

(15)

$G_n(x,y)$ represents the pixel value of the pyramid image at layer n at coordinates $(X,Y)$. $w(m,n)$ is the Gaussian weight kernel. The Laplacian pyramid is constructed by recording the residuals between each layer of the Gaussian pyramid and its upper layer, as shown in Equation (16).

$$L_n=G_n-up\left(G_{n+1}\right)$$

(16)

In this context, $l-n$ represents the Laplacian image at the n-th layer, and $up\left(G_{n+1}\right)$ denotes the upsampling of the image from the $n+1$-th layer. Next, we apply multi-scale processing to the intensity image, degree of polarization image, and angle of polarization image for each spectral band, thereby obtaining Gaussian pyramids for different types of information across various bands, as well as Laplacian pyramids for each band.

Then, we process the different types of information separately. The n images from different bands at the same scale are expanded into one-dimensional vectors, $i_{\lambda }=vec{\left(I_{\lambda }\right)}^T$, which are then combined into an n-dimensional vector, $I=[i_{\lambda 1},i_{\lambda 2},\dots i_{\lambda n}]$. These vectors are decomposed using SVD, the largest eigenvalue is extracted, and the result is reconstructed and reshaped to obtain the fused information, as shown in Equation (17).

$$\begin{array}{cc}\hfill f& =U_1{\sigma }_1=I{v}_1\hfill \\ \hfill F& =reshape\left(f\right)\hfill \end{array}$$

(17)

Here, f represents the one-dimensional vector of the fused maximum feature, and F denotes the result after reshaping f back into an image matrix. Based on this, we can obtain the fused multi-scale pyramids for intensity, degree of polarization, and angle of polarization. Next, we extract polarization details by performing fractional-order detail extraction in the x-direction and y-direction of the polarization image, respectively, and calculate their modulus, as shown in Equation (18).

$$D^{a}I(x,y)=\sqrt{D_x^{a}I{(x,y)}^2+D_y^{a}I{(x,y)}^2}$$

(18)

As shown in Equation(18), $D^{a}I(x,y)$ represents the fractional-order detail image, $D_x^{a}I(x,y)$ denotes the detail image in the x-direction, and $D_y^{a}I(x,y)$ represents the detail image in the y-direction. Using Equation(18), we can solve for the final detail image, as shown in Equation (19).

$$\begin{array}{cc}\hfill G_n^F& ={\lambda }_1{D}^a{G}_n^{FS0}+{\mathsf{\mu }}_1{D}^a{G}_n^{FDOP}+{\varphi }_1{D}^a{G}_n^{FAOP}\hfill \\ \hfill L_n^F& ={\lambda }_2{D}^a{L}_n^{FS0}+{\mathsf{\mu }}_2{D}^a{L}_n^{FDOP}+{\varphi }_2{D}^a{L}_n^{FAOP}\hfill \end{array}$$

(19)

Among them, $G_n^F$ represents the fused Laplacian image of the n-th layer; $G_n^{FS0}$, $G_n^{FDOP}$, and $G_n^{FAOP}$ denote the Gaussian pyramid intensity image, degree of polarization image, and angle of polarization image of the n-th layer after SVD fusion, respectively; $L_n^{FS0},L_n^{FDOP},L_n^{FAOP}$ represents the Laplacian pyramid intensity image, degree of polarization image, and angle of polarization image of the inner layer after SVD fusion. Here, $\lambda ,\mathsf{\mu },\varphi$ are weighting coefficients.

Finally, we need to generate the final multi-band polarization information fusion image through gradual restoration from low scale to high scale, as shown in Equation (20).

$$H_{n-1}=a(up\left(H_n\right)+L_n^F)+b{G}_n^F+c{G}_n^{S0}$$

(20)

where $H_n$ represents the final fused image of the N-th layer, and $a,b,c$ are different weighting coefficients. Finally, through continuous recursive solution, we can obtain the final fused image.

To obtain more accurate weighting coefficients, we constructed an evaluation model based on information entropy [32], average gradient [33], and standard deviation [34], as shown in Equation (21). Meanwhile, to ensure the fusion effect, we set constraints: the sum of the weighting coefficients is 1, and the structural similarity (SSIM) between the fused image and the reference image is not less than a given threshold T. The weighting parameters $a,b,c$ that maximize this value are the weighting coefficients used in our fusion.

$$\begin{array}{cc}\hfill f_{guass}(\lambda ,\mathsf{\mu },\varphi )& =E\left(G_n^F\right)+AG\left(G_n^F\right)+STD\left(G_n^F\right)\hfill \\ \hfill f_{laplacian}(\lambda ,\mathsf{\mu },\varphi )& =E\left(L_n^F\right)+AG\left(L_n^F\right)+STD\left(L_n^F\right)\hfill \\ \hfill {\lambda }_1,{\mathsf{\mu }}_1,{\varphi }_1& =argmax\left(f_{g}uass\right)\hfill \\ \hfill {\lambda }_2,{\mathsf{\mu }}_2,{\varphi }_2& =argmax\left(f_{l}aplacian\right)\hfill \end{array}$$

(21)

The proposed evaluation model based on information entropy, average gradient, and standard deviation is sufficiently reasonable and correct in theory. Information entropy can effectively measure the amount and richness of image information, average gradient reflects the clarity of image details and textures, and standard deviation characterizes the dispersion of gray distribution and contrast. These three indicators comprehensively describe image quality from three complementary dimensions: information content, detail sharpness, and contrast difference, so the selection of indicators is typical and comprehensive. Meanwhile, constraining the sum of weighting coefficients to 1 ensures the rationality of normalized weight distribution, and setting SSIM to be no less than the threshold T guarantees that the fusion result maintains high structural consistency with the reference image, avoiding structural distortion or artifacts. Therefore, the weighting coefficients $\lambda$, $\mathsf{\mu }$ and $\varphi$ obtained by maximizing this comprehensive evaluation value achieve optimal allocation while taking into account information content, clarity, contrast, and structural similarity, making the model effective and practical for reliable weight optimization in image fusion.

3. Experiment and Results

As shown in Figure 1, the process begins with the acquisition of polarization images in different spectral bands using a time-division polarization spectroscopy system. Subsequently, the SIFT feature matching algorithm is employed for image registration, yielding multi-band polarization images with pixel-level spatial correspondence. The Stokes vectors are then used to compute the DOP and AOP. Following this, Gaussian and Laplacian pyramids are constructed for the intensity, DOP, and AOP images in each band. Image fusion is then performed using SVD, where images from all bands at each scale are combined into a single fused image containing the integrated information, thereby creating fused Gaussian and Laplacian pyramids. Fractional-order image processing is applied to the fused images at each scale to enhance image details. Subsequently, detail enhancement and noise removal are carried out using self-guided filtering. Finally, multi-scale fusion is performed to obtain the final fused and enhanced image, followed by color restoration.

3.1. System Construction and Calibration

To enable polarization spectral image data acquisition in multiple scenarios, this paper constructs a time-division polarization spectral system. As shown in Figure 2a, the camera used is an OR-250CNC-P polarization camera, which outputs images with a resolution of 2448 × 2048. The adjacent four pixels within the camera sensor are arranged in a 2 × 2 pattern, corresponding to the output polarization image information. As shown in Figure 2b, the lens is a 50 mm fixed-focal-length lens. An optical cage plate, illustrated in Figure 2c, is used to secure the filter in front of the lens, resulting in the time-division polarization spectral camera depicted in Figure 2d. The camera position is fixed using a tripod and a gimbal. The final time-division polarization spectral system, shown in Figure 2e, ensures maximum precision in data acquisition under relatively stable conditions.

The bands of the filters selected in this paper are within the visible light spectrum, specifically 450 nm, 467 nm, 488 nm, 500 nm, 514 nm, 532 nm, 540 nm, 562 nm, 568 nm, 590 nm, 625 nm, and 660 nm. Due to variations in the transmittance of different filters and the camera sensor’s response to light intensity across different bands, it is necessary to calibrate the time-division polarization spectral system to accurately restore the spectral information of the object. An integrating sphere is chosen as the light source for system calibration because of its uniform brightness and non-polarized light output. For each band, multiple images are captured at different exposure times and brightness levels, and their average values are calculated. The corresponding average values of multiple dark-field images are subtracted. Fitting is then performed using the formula shown in Equation (22).

$$S_{\lambda }={\displaystyle \frac{L-L_{Dark}}{K_{\lambda }t}}$$

(22)

where $S_{\lambda }$ represents the brightness of the band, L denotes the response of the camera sensor, $L_{Dark}$ represents the dark-field noise, $K_{\lambda }$ is the gain factor for different bands, and t is the exposure time. The value of $K_{\lambda }$ is determined by fitting using Equation (22). Finally, the $k_{\lambda }$ values for different bands are shown in

Table 1. Gain coefficient table for different wavebands.

Band	K Value	Standard Deviation	R-Squared
450 nm	1.05959	0.00939	0.98551
467 nm	0.87918	0.01022	0.97384
488 nm	1.40818	0.01365	0.98327
500 nm	1.64818	0.01622	0.98341
514 nm	1.71443	0.01396	0.98831
532 nm	1.88691	0.01580	0.98981
540 nm	1.83860	0.01640	0.98552
560 nm	2.40439	0.02136	0.98533
568 nm	2.04129	0.02060	0.98121
590 nm	1.89386	0.01276	0.99173
625 nm	2.37749	0.01688	0.99060
660 nm	2.55280	0.03391	0.98058

.

3.2. Data Preprocessing

During the data acquisition process, it is necessary to continuously switch the filters of the time-division polarization spectroscopy system to obtain polarization images in different spectral bands. Due to the inherent limitations of the time-division polarization spectroscopy system, multi-band information cannot be acquired simultaneously but is obtained at different times. As a result, some spatial misalignment inevitably occurs. Figure 3 shows the original data image acquired by the time-division polarization spectroscopy system.

During the acquisition process, spatial misalignment errors arise from scene motion caused by temporal factors and vibration errors induced by filter switching, while the polarization spectral camera itself remains stationary. Therefore, we employ a feature point-based image registration algorithm to eliminate spatial misalignment. In image processing, various methods exist for calculating image feature points. To enhance registration accuracy, this paper selects the SIFT algorithm for feature point matching. The SIFT algorithm is a local feature description algorithm in image processing, known for its excellent stability and invariance. It adapts well to rotations, scale scaling, and brightness variations, while demonstrating strong robustness against interference. Consequently, we utilize the SIFT algorithm to compute feature points for each spectral band image. These feature points are then used to calculate the homography matrix between adjacent band images, followed by affine transformation. Overlapping regions are cropped, and the process repeats: matching features between the cropped region and the next band image. Ultimately, this yields registered images with precise spatial correspondence, as shown in Figure 4.

After image registration, DOP and AOP for different spectral bands are calculated according to Equations (6) and (7), based on the arranged polarizing elements array. Subsequently, through normalization, the value range of DOP, [0.0, 1.0], is linearly mapped to the image grayscale interval [0.0, 1.0]; similarly, the value range of AOP, [0, $\pi$], is mapped to [0.0, 1.0]. As shown in Figure 5, these are the DOP images for different spectral bands. Figure 6 presents the AOP images corresponding to these bands.

Through observation of Figure 5 and Figure 6, it is found that the DOP and AOP vary across different spectral bands, revealing the polarization characteristics of the target in these bands. Therefore, it is necessary to effectively fuse the information from different bands using the algorithm proposed in this paper.

3.3. Polarization Multispectral Image Fusion

After obtaining fully registered polarization spectral image data, we proceed with the fusion of polarization multispectral images. Based on the theory in Section 2, we first perform multi-scale decomposition of the intensity information and polarization information of different bands using Equation (17). To ensure that the lowest scale still retains certain information, we set the minimum scale to no less than 100 × 100 pixels. Therefore, for the data in this paper, we selected a 4-layer Gaussian pyramid and a 3-layer Laplacian pyramid.As shown in Figure 7, Figure 7a is the Gaussian pyramid of light intensity, Figure 7b is the Gaussian pyramid of the degree of polarization, and Figure 7c is the Gaussian pyramid of the angle of polarization.

After obtaining the Gaussian pyramid, we can solve for the Laplacian pyramid of intensity information and polarization information under different bands according to Equation (17). As shown in Figure 8, Figure 8a is the Laplacian pyramid of the intensity image, Figure 8b is the Laplacian pyramid of the degree of polarization band, and Figure 8c is the Laplacian pyramid of the angle of polarization.

By performing multi-scale decomposition on the intensity information and polarization information of different bands, we obtained their Gaussian pyramids and Laplacian pyramids. Using Equation (17), we performed image fusion on the Gaussian pyramids and Laplacian pyramids of intensity, degree of polarization, and angle of polarization, respectively, to obtain the fused Gaussian pyramids of different information as shown in Figure 9.

Among them, Figure 9a is the fused Gaussian pyramid of intensity information, Figure 9b is the fused Gaussian pyramid of the degree of polarization, and Figure 9c is the fused Gaussian pyramid of the angle of polarization. To gain a more intuitive understanding of the image fusion effect, we evaluated it by calculating the contribution of the maximum features. As shown in

Table 2. The contribution degree of the maximum feature vector of the fused image in the Gaussian pyramid at the same scale.

	The First Layer	The Second Layer	The Third Layer	The Fourth Floor
S0	98.82%	98.92%	99.01%	99.10%
DOP	70.06%	84.69%	88.18%	89.58%
AOP	68.99%	60.82%	84.61%	83.71%

, the contribution of the maximum features is mostly greater than 80%, indicating that the fused image retains most of the feature information from all bands.

Next, we performed image fusion again on the Laplacian pyramids of intensity, degree of polarization, and angle of polarization across different bands, as shown in Figure 10. Among them, Figure 10a is the fused Laplacian pyramid of intensity information, Figure 10b is the fused Laplacian pyramid of the degree of polarization, and Figure 10c is the fused Laplacian pyramid of the angle of polarization.

Then, by calculating the contribution of the maximum features, as shown in

Table 3. The contribution degree of the maximum feature vector of the fused image in the Laplacian pyramid at the same scale.

	The First Layer	The Second Layer	The Third Layer
S0	84.46%	91.80%	92.61%
DOP	41.68%	68.14%	77.52%
AOP	48.63%	50.27%	64.35%

, we found that in the high-scale layers, the contribution rate of the maximum features of the polarization information did not exceed 50%, indicating that some high-frequency information was lost during the fusion process. Therefore, in the subsequent processing, we need to enhance these high-frequency components.

To further ensure the quality of image fusion, we solve for the fractional-order detail image using Equation (18). Fractional-order differentiation allows us to both extract the texture details of the image and enhance its high-frequency components, thereby addressing the loss of high-frequency information caused by the previous SVD fusion. To ensure computational accuracy and time efficiency, we select the first five terms of Equation (14) for the calculation. As illustrated in Figure 11 and Figure 12, different fractional orders a produce varying effects. When $a=0.1$, the texture details of the degree of polarization are significantly enhanced, while when $a=0.7$, the texture details of the angle of polarization are substantially improved. Therefore, in this paper, $a=0.1$ is chosen as the fractional-order differentiation factor for the degree of polarization, and $a=0.7$ is chosen as the fractional-order differentiation factor for the angle of polarization.

After determining the fractional-order differentiation factors, we perform image fusion between different types of information using Equation (19), and determine the specific coefficients ${\lambda }_1,{\mathsf{\mu }}_1,{\varphi }_1,{\lambda }_2,{\mathsf{\mu }}_2,{\varphi }_2$ through Equation (21). To ensure the structural similarity of the images, we set the threshold T to 0.85. Ultimately, we obtain the Gaussian pyramid fractional-order fusion image and the Laplacian fractional-order fusion image, as shown in Figure 13 and Figure 14, which lays the foundation for subsequent multi-scale fusion. It can be observed from the Figure that the texture details have been significantly enhanced.

However, during the fusion process, noise interference is inevitably introduced. Therefore, we adopt a corresponding filtering method to remove the noise. The filtering method employed in this paper is guided filtering, which injects the aforementioned detail maps into the intensity image. Compared with traditional filtering methods, guided filtering offers several advantages in image processing tasks such as denoising, detail enhancement, image fusion, HDR, and dehazing. It preserves edges effectively without causing edge blurring, while simultaneously enhancing textures and details. The Gaussian fractional-order filtered image is shown Figure 15 and Figure 16.

Finally, we need to perform multi-scale image fusion to integrate the enhanced images from different scales into a single image at the same scale. To achieve this, we adopt the method described in Equation (20) for multi-scale fusion. The images are progressively reconstructed from the lowest scale to higher scales. To ensure that the features of each image are preserved during fusion, Equation (21) is used to enable adaptive selection while maintaining the threshold $T>0.85$. Ultimately, the fused image is obtained, as shown in Figure 17.

3.4. Color Restoration

There are three types of cone photoreceptor cells in the human retina, which are sensitive to long waves (L, red), medium waves (M, green), and short waves (S, blue), respectively. The RGB sensors of color cameras are designed to simulate this response characteristic of the human eye. Therefore, based on the principles of color science and the response characteristics of typical imaging sensors, and since the three central wavelengths corresponding to the RGB color model are 450 nm (blue), 550 nm (green), and 650 nm (red), we selected 450 nm (blue), 562 nm, and 660 nm from our spectral bands. Finally, we reconstructed an image with color information, as shown in Figure 18.

At the same time, we can inject the luminance information from the grayscale version of the fused image into the color image while preserving the original colors of the color image. To achieve this, we convert the color image to the HSV color space, separate the channels, replace the V channel with the fused grayscale image, merge the channels, and then convert back to the RGB image. Finally, the resulting fused color image is obtained, as shown in Figure 19.

4. Result Analysis

To verify the error and accuracy of the algorithm proposed in this paper, we conducted histogram analysis and spectral analysis on Figure 18 and Figure 19 respectively. The histogram is used to compare the deviation in pixel distribution of the images before and after fusion, thus reflecting the fidelity of brightness and contrast. Spectral analysis is used to evaluate the preservation of details in different frequency bands and the potential introduction of spectral distortion. The analysis results are shown in Figure 20.

From the analysis of the gray-level histogram, it can be seen that Figure 20b has a more complete dynamic range and a more balanced distribution of light and dark areas, with both dark and bright information well preserved. Overall, it presents a more natural bimodal structure. On the other hand, the gray-level distribution of Figure 20a is noticeably shifted towards medium and high gray levels, with serious loss of dark information, resulting in a compressed overall dynamic range. Therefore, from the perspective of information integrity, Figure 20b is superior to Figure 20a.

Figure 20c,d are both centered Fourier transform amplitude spectra of the image. The core commonality between the two is the presence of a bright spot at the center representing the average brightness of the image, and both have cross shaped bright lines reflecting the horizontal and vertical structure and edge discontinuity of the image; The core difference lies in the fact that the cross shaped bright lines in Figure 20d are thinner and sharper, the background is cleaner, and the energy attenuation is faster, corresponding to a regular structure, lower noise, and higher information purity in the original image; The cross lines in Figure 20c are thicker, the energy diffusion range is larger, and the background is covered with dense noise textures, corresponding to higher frequency noise or richer microscopic details in the original image. However, the overall purity and structural regularity are weaker than those in the first image.

To further verify the effectiveness of the algorithm proposed in this paper, we compare it with other algorithms in different scenarios, as shown in Figure 21. Figure 21a presents the original spectral color images in different scenarios, while Figure 21b shows the fused color images in the corresponding scenarios. From Figure 21, it can be observed that in different scenarios, the texture details of the images fused by the algorithm proposed in this paper are significantly improved compared to the original images. The image quality is notably enhanced.

To further demonstrate the effectiveness of the algorithm proposed in this paper, it is also compared with commonly used image fusion algorithms, as shown in Figure 22. Figure 22a presents the PCA fused color images in different scenarios, Figure 22b shows the Poisson fusion color images in different scenarios, Figure 22c displays the NSCT fused color images in different scenarios, and Figure 22d presents the fused color images obtained by the proposed algorithm in different scenarios. From Figure 22, it can be observed that although the PCA fusion algorithm can process multispectral images efficiently, the texture details in the fused images are not prominent. The Poisson fusion algorithm, a gradient-domain-based method, performs image fusion based on luminance transformation; while details are improved, color distortion is severe. The NSCT algorithm, a multi-scale analysis method, achieves relatively high fusion quality but may introduce artifacts and blurring in certain scenarios. In contrast, the algorithm proposed in this paper not only preserves color fidelity but also significantly enhances texture details, greatly improving image quality.

To more objectively reflect image quality, we adopt four image evaluation metrics—information entropy [32], average gradient [33], image standard deviation [34], and SSIM—to assess image quality [35]. As shown in

Table 4. Objective evaluation table of different algorithms.

	Algorithm	E	AG	STD	SSIM
Scenario 1	Ours	7.85	26.88	70.96	0.95
	PCA	7.29	10.38	47.79	0.95
	Poisson	7.29	12.71	49.15	0.94
	NSCT	7.33	17.46	48.35	0.94
Scenario 2	Ours	7.32	31.04	60.39	0.92
	PCA	6.22	20.80	41.50	0.86
	Poisson	5.78	19.23	28.89	0.62
	NSCT	6.25	29.00	33.49	0.74
Scenario 3	Ours	6.58	25.51	31.92	0.88
	PCA	6.56	19.77	31.69	0.88
	Posisson	6.26	23.19	20.87	0.71
	poissson	6.60	37.87	31.61	0.83
Scenario 4	Ours	6.39	62.69	25.94	0.88
	PCA	6.29	50.84	22.70	0.86
	Poisson	5.98	59.63	31.87	0.73
	NSCT	6.33	60.85	26.80	0.81

, the proposed algorithm, while maintaining maximum image similarity, achieves the highest information entropy, average gradient, and standard deviation in most scenarios, further demonstrating its effectiveness.

5. Conclusions and Prospect

This paper proposes a multi-scale image fusion algorithm based on polarization spectral images, which achieves favorable fusion results even in multi-scene and multi-target backgrounds. First, through multi-scale decomposition, the polarization images of different bands are transformed into Gaussian pyramids and Laplacian pyramids. For the multi-band images at different pyramid scales, fusion is performed using SVD to obtain fused Gaussian and Laplacian pyramids. Fractional-order processing is then applied to extract detailed textures from the images, followed by guided filtering for noise filtering. Subsequently, through the fusion strategy proposed in this paper, the images are progressively fused and reconstructed back to the original image scale, yielding the final fused image. While ensuring maximum structural similarity, this method significantly enhances the fusion details and guarantees the quality of the fused image. Although the proposed algorithm achieves good fusion performance, there are still some limitations that require improvement in future work.

• The proposed method relies heavily on the accuracy of image registration. For moving objects, perfect registration cannot always be achieved, which may lead to artifacts and overlapping in the subsequent image fusion process, thereby degrading the fusion quality.
• The algorithm depends on multi-scale image processing, which is time-consuming. Its time complexity and space complexity are relatively high compared to other algorithms.
• The proposed algorithm is suitable for high-resolution images. For low-resolution images, continuous scale changes may lead to degradation in the quality of the fused image.