Image Fusion Algorithm Selection Based on Fusion Validity Distribution Combination of Difference Features

Abstract

Aiming at addressing the problem whereby existing image fusion models cannot reflect the demand of diverse attributes (e.g., type or amplitude) of difference features on algorithms, leading to poor or invalid fusion effect, this paper puts forward the construction and combination of difference features fusion validity distribution based on intuition-possible sets to deal with the selection of algorithms with better fusion effect in dual mode infrared images. Firstly, the distances of the amplitudes of difference features between fused images and source images are calculated. The distances can be divided into three levels according to the fusion result of each algorithm, which are regarded as intuition-possible sets of fusion validity of difference features, and a novel construction method of fusion validity distribution based on intuition-possible sets is proposed. Secondly, in view of multiple amplitude intervals of each difference feature, this paper proposes a distribution combination method based on intuition-possible set ordering. Difference feature score results are aggregated by a fuzzy operator. Joint drop shadows of difference feature score results are obtained. Finally, the experimental results indicate that our proposed method can achieve optimal selection of algorithms that has relatively better effect on the fusion of difference features according to the varied feature amplitudes.

1. Introduction

The fusion of dual mode Infrared images can synthesize their imaging advantages, which is conducive to effective storage of the detection images, significantly improving the imaging quality and detection accuracy of the detection system [1,2]. It has been widely used in military surveillance, infrared countermeasures, industrial monitoring and fault detection of lesions, etc. The infrared intensity and polarization images fusion of the site is an effective way to ensure the integrity of the information can be fully utilized, and the complementary advantages of image can improve the detection and recognition of target objects [3,4,5]. However, due to the differences in imaging mechanisms and the diversity of detection environments and targets, the difference characteristics of the two images in the same scene are complicated and varied. Different features have different fusion performance (fusion effectiveness) for each algorithm, so it is difficult to meet the different needs with static algorithms, which is only possible by selecting the appropriate algorithm according to the difference features, so as to meet the fusion requirements of complex infrared imaging scenes. At the moment, adjusting the fusion algorithm dynamically and optimally according to different attributes of difference features has been the key technology and a hot topic to improve pertinence and effectiveness of dual mode infrared image fusion [6].

At present, for dual mode infrared image fusion, through qualitative analysis of the relationship between some known difference feature types and multiple fusion algorithms, a better fusion algorithm is determined. By combining support value conversion (SVT) wherein the difference in brightness relationship with Top-Hat transform, [7] has improved the fused images contrast. Reference [8] analyzed the fusion performance of multi-resolution transform domain methods such as DWT, SWT, CVT, CT, DTCWT, NSCT, etc., established the corresponding relationship between these algorithms and difference features, and achieved good fusion results. Xiang [9] considered the influence of statistical difference features in the NSCT domain on the adaptive dual-channel unit link PCNN, which enriched the details of fused images. Meng [10] described the relationship between the brightness difference feature and the fusion algorithm with salient maps and points of interest, and reserved salient and bright targets for the fused image. In Reference [11] the mapping relationship between the edge details of visible light and infrared images and the fusion scheme based on compressed sensing was studied. The above algorithms have a good fusion effect under certain circumstances.

The above studies all use the crisp value to describe the fusion effect of difference features. It is recommended to apply fuzzy sets to overcome the situation where the crisp value cannot resolve the uncertain relationship between the difference feature and the fusion algorithm. An improved fuzzy set was used to fuse the low-frequency part of infrared and visible image fusion [12]. In addition, based on basic human judgment and membership functions, fuzzy logic inference was used to process multi-band image synchronization fusion [13]. Therefore, the fuzzy set theory is usually used to solve the image fusion problem by combining some multi-resolution transform domain methods instead of crisp values.

For actual target detection, however, another attribute of the difference features between the two kinds of images, called ‘amplitude’, in addition to the one known as ‘type’, equally has great effects on the fusion results. The changes of difference feature types and amplitudes are also random in practice, especially for a dynamic detection scene, making the changes of various attributes more complicated. The existing methods only consider the single attribute-type fusion effect on algorithms, which cannot reflect the impact of different attributes (such as type and amplitude) on algorithm selection, and is unable to quantify fusion validity change state of difference feature attributes, leading to poor or invalid fusion effects. Therefore, only when an algorithm with pertinence is selected optimally according to different attributes of difference features can the fusion quality of dual mode infrared image be improved.

For the fusion of infrared polarization and intensity images, the pre-selected fusion algorithm cannot always maintain better fusion performance when the amplitudes of different features are different, so the fusion algorithm effect is not fixed and changes dynamically along with the amplitudes of difference features. For the actual detection images, the fusion validity used to describe the degree of influence of the difference feature value is mostly predicted and estimated through the fusion results of existing, limited and similar scene images, so the measurement of fusion validity is predictable and possible. Therefore, a possibility distribution is needed to adopt the description of fusion validity change process [14,15,16]. However, the possibility distribution can express the dynamic changes of integration fusion validity, only describing double-sided properties, but it cannot describe the middle state of fusion validity of difference features [17,18,19], which has a great influence on fusion algorithm selection. In order to solve this problem, some researchers used intuitionistic fuzzy sets to model and manage uncertainty, which means that the process of word calculation is completed in the field of group decision-making. Tirupal studied a multi-modal medical image fusion model based on Yager’s intuitionistic fuzzy sets [20]. The initial image is converted to complementary intuition Yager fuzzy images and the best new objective function value, called intuitionistic fuzzy entropy, is used to obtain membership functions and non-membership function parameters. In Reference [21], a new infrared and visible image fusion method employing a non-subsampled contourlet transform (NSCT) with intuitionistic fuzzy sets is put forward, which outperformed the advanced fusion methods in terms of objective assessment and visual quality.

Therefore, this paper takes advantage of intuitionistic fuzzy set and possibility theory and proposes a novel construction method of fusion validity distribution based on intuition-possible sets to solve the algorithm selection problem. For multiple difference features, a distribution combination method based on intuition-possible set ordering is proposed. Fusion validity scores of the algorithms on multiple intervals of difference feature amplitude are calculated, and comprehensive values of the algorithms for different feature amplitudes are obtained. Based on this, for each difference feature, the algorithm with relative better effect is selected. The flow chart of the method is shown in Figure 1. The descriptions of some particular terms are listed in

Table 1. The descriptions of particular terms.

Particular Terms	Description
Difference feature	The difference information of infrared polarization and intensity image.
Diverse attribute	The type and amplitude of difference features.
The type of difference features	The brightness, the edge and detailed features, including gray mean, standard deviation, edge intensity and spatial frequency.
The amplitude of difference features	The absolute difference of the feature pixels intensity value of two types of image.
Fusion validity	It is used to measure effective degree of fusing the features in fused images and the source images for the specific fusion algorithm.
Fusion validity distribution	It can reflect fusion validity changing process of attributes of image difference features to algorithms.
Intuition possible sets	It can realize quantitative description of fusion effect changing process of difference feature of images to algorithms.
Relative better effect	From the comprehensive consideration of objective evaluation and subjective evaluation, this algorithm has the best fusion effect.
Distribution construction	The changing process based on the new method (intuition possible sets) is constructed.

.

The following main contributions of this paper can be highlighted:

(1)

Intuition-possible sets are built to model fusion validity of attributes of image difference features.

(2)

A novel construction method of fusion validity distribution based on intuition-possible sets is proposed, which can reflect the fusion validity change process of attributes of image difference features to algorithms.

(3)

This paper puts forward a distribution combination method based on intuition-possible set ordering to solve the optimal algorithm selection problem that has a relatively better effect on the fusion of the difference features according to the varied image feature attribute values, which provides the basis to algorithm classification and mimicry bionic fusion. The rest of this paper is organized as follows: Section 2 briefly analyses the type of difference feature for infrared polarization and intensity images. Section 3 determines intuition-possible set on fusion effect according to the distance of amplitudes of difference features, then proposes a fusion validity distribution construction method. A distribution combination method based on intuition possible set ordering is put forward. Experimental results and comparisons are given and analyzed. Conclusions are presented in Section 4.

2. Determination of the Type of Difference Feature

The main reason for the formation of the difference features is that there are differences between the types of imaging characteristics, including the radiation difference between the target and background, the atmospheric transmission difference and the imager response difference. It can be seen from eight groups of closely registered infrared polarization and intensity experimental images (with a uniform size of 256 × 256) in Figure 2 that the infrared polarization images have sharp edge feature details, etc., but lack sufficient brightness characteristics. Infrared intensity image information based on thermal radiation luminance characteristics is significant, but lacks sufficient edge and detailed features. Thus, the difference in the infrared polarization and intensity image, edge features and details are evident. The difference feature selected in this paper includes gray mean, standard deviation, edge intensity and spatial frequency. These are defined as follows:

• Gray mean: In the grayscale image, the brightness information changes continuously from dark to bright. The difference gray mean value represents the absolute value of mean difference of all pixel intensity values of two types of images in the dual-mode infrared images, thus it can effectively reflect the change of the difference of brightness characteristics of images.
• Edge intensity: The edge information is the contour structural feature of the human eye recognition feature information, and the distributions of the two types of images are very different. The difference edge intensity represents the absolute value of the difference in the edge amplitude intensity of the two types of images. This paper selects the Sobel operator based on the commonly used edge extraction operators to extract the edge amplitude intensity information to characterize the edge feature difference change of the images.
• Standard deviation: The difference standard deviation can reflect the discrete situation of the gray scale of the dual mode infrared image compared to the average gray scale. The larger the difference standard deviation is, the more discrete the gray level distribution is, indicating the greater the contrast between the two kinds of images, the more information available, that is, the better the fusion effect.
• Spatial frequency: Difference spatial frequency can reflect the sharpness of pixel gray value changes in dual-mode infrared images, can effectively represent image texture feature information, and reflect the image’s ability to describe the contrast of small details. The greater the difference spatial frequency, the clearer the fused image. The above can be well described image information, so we adopt four difference features in this paper, labelled as T₁, T₂, T₃ and T_4.

3. Construction of Fusion Validity of Difference Feature Amplitude Based on Intuition Possible Sets

3.1. Intuition Possible Sets

Let X be the universe of discourse, X is a primary variable taken from X, if there are two mappings: ${\pi }_{\tilde{A}}^1:X\to [0,1]$ and ${\pi }_{\tilde{A}}^2:X\to [0,1]$, satisfying

$$x\in X\mapsto {\pi }_{\tilde{A}}^1(x)\in [0,1]$$

(1)

and:

$$x\in X\mapsto {\pi }_{\tilde{A}}^2(x)\in [0,1]$$

(2)

and meeting the following condition:

$$0\le {\pi }_{\tilde{A}}^1(x)+{\pi }_{\tilde{A}}^2(x)\le 1$$

(3)

Then, $\tilde{A}$ is called an intuition possible set in the universe of discourse X, and it can be expressed as Equation (4):

$$\tilde{A}=\{<x,{\pi }_{\tilde{A}}^1(x),{\pi }_{\tilde{A}}^2(x)>|x\in X\}$$

(4)

Here, ${\pi }_{\tilde{A}}^1$ and ${\pi }_{\tilde{A}}^2$ are possibility distribution and non-possibility distribution of $\tilde{A}$ respectively, and ${\pi }_{\tilde{A}}^1(x)$ and ${\pi }_{\tilde{A}}^2(x)$ represent possibility degree and non-possibility degree of $x$ about $\tilde{A}$. ${\pi }_{\tilde{A}}^3(x)=1-{\pi }_{\tilde{A}}^1(x)-{\pi }_{\tilde{A}}^2(x)$ is hesitancy degree of $x$ about $\tilde{A}$. We can denote ${\pi }_{\tilde{A}}^1(x_j)$ and ${\pi }_{\tilde{A}}^2(x_j)$ of $x_j(j=1,2,\cdots ,n)$ as ${\pi }_j^1$ and ${\pi }_j^2$. $<{\pi }_j^1,{\pi }_j^2>$ is an ordered pair of possibility degree and non-possibility degree of $x_j$.

For infrared polarization and intensity image fusion, X represents different amplitude values of difference feature, and intuition possibility set $\tilde{A}$ is the set of high fusion effect of difference feature to algorithms. Then ${\pi }_{\tilde{A}}^1(x)$ represents the possibility degree of high fusion validity of difference feature amplitude $x$ to algorithms when we set the value of difference feature amplitude equal to $x$, and ${\pi }_{\tilde{A}}^2(x)$ is the non-possibility degree of high fusion validity of difference feature amplitude, that is, the possibility degree of low fusion validity of difference feature amplitude $x$. ${\pi }_{\tilde{A}}^3(x)$ is the hesitancy degree of medium fusion validity of difference feature amplitude $x$.

3.2. Calculation of the Distance of the Amplitudes of Difference Feature between Fused Image and Source Images

In this paper, the eight pairs of experimental images in Figure 2 are fused respectively by 12 classical fusion algorithms such as the Discrete Wavelet Transform [22,23], Quaternion Wavelet Transform [24,25], Dual Tree Complex Wavelet Transform [26,27], Wavelet Packet Transform [28], Non-Subsample Shearlet Transform [29,30,31], Laplacian Pyramid Transform [32,33], Gradient Pyramid Transform [34], KSVD [35,36], Weighted Average Fusion, Principal Component Analysis [37], Top-Hat Transform [38], and Guided Filter [39,40,41]. selected from multi-scale transform, sparse representation, variation, morphology, and pixel domain fusion algorithm classifications, etc., labelled as A₁, A₂, A₃, A₄, A₅, A₆, A₇, A₈, A₉, A₁₀, A₁₁ and A₁₂. Thus we can obtain fused images of different algorithms.

We first obtain a series of image blocks using 16 × 16 size and non-overlap block method to process fused image and source images respectively. Then the distances of the amplitudes of difference feature between fused image and source images are calculated by distance similarity as Equation (5).

$$\begin{array}{c}{D}_i^X=\underset{1\le i\le n}{\min }‖\left|{\overline{X}}_f^i-{\overline{X}}_P^i\right|,\left|{\overline{X}}_f^i-{\overline{X}}_I^i\right|‖\\ D^X=\{d_1^X,\cdots ,d_i^X,\cdots d_n^X\}\end{array}$$

(5)

where $D^X$ is n dimensional vector of the distances of the amplitudes of difference feature for all image blocks. ${\overline{X}}_f^i$ represents the mean of the amplitudes of difference feature for the ith image block in fused images. Also, ${\overline{X}}_P^i$ and ${\overline{X}}_I^i$ are, respectively, the mean of the amplitudes of difference feature for the i-th image block in infrared polarization and infrared intensity images. X can be difference feature T₁, T₂, T₃ or T₄.

Discrete points figure of the four types of difference feature can be obtained as shown in Figure 3. Here, the amplitudes of difference feature are assigned as abscissa axis, and the normalized distances of the amplitudes of difference feature between fused image (e.g., A₅) and source images are given as vertical axis.

3.3. Distribution Construction

For the infrared polarization and intensity images, the better fusion results(that is, higher fusion validity) of the fused images are shown as follows: (i) The brightness feature value of the fused image is closely related to that of the infrared intensity image. (ii) The edge and texture feature values of the fused images are also as close as possible to those of the infrared polarization images. Algorithm 1 summarizes the procedure of the distribution construction step by step in pseudo code format.

Algorithm 1: Distribution construction

Input: Infrared polarization image, infrared intensity image and fused image Output: Fusion validity distribution Step 1: The amplitudes of difference feature // T1, T2, T3 and T4. Step 2: Calculation of the distance of the amplitudes of difference feature using (5) Step 3: Building of Intuition possible sets on fusion effect Step 4: Construction of fusion validity distribution     a. Determine the number of image blocks $N^{X_k}$     b. Initialize $n_1^{X_k}$,$n_2^{X_k}$ and $n_3^{X_k}$     c. Update $n_1^{X_k}$, $n_2^{X_k}$ and $n_3^{X_k}$ using Equations (6)–(8)     d. Calculate fusion validity

According to algorithm fusion result to the amplitudes of difference feature , the distances can be divided into three intervals, as $[0,\left|{\overline{X}}_f^i-\frac{{\overline{X}}_P^i+{\overline{X}}_I^i}{2}\right|)$, $(\left|{\overline{X}}_f^i-\min ({\overline{X}}_P^i,{\overline{X}}_I^i)\right|,1]$ and $[\left|{\overline{X}}_f^i-\frac{{\overline{X}}_P^i+{\overline{X}}_I^i}{2}\right|,\left|{\overline{X}}_f^i-\min ({\overline{X}}_P^i,{\overline{X}}_I^i)\right|]$, where $\frac{{\overline{X}}_P^i+{\overline{X}}_I^i}{2}$ represents the mean of the amplitudes of difference feature for fused image using weighted average method. Three cases are discussed:

(1)

For image blocks, if the distances of the amplitudes of difference feature between fused image and source images are in the interval $[0,\left|{\overline{X}}_f^i-\frac{{\overline{X}}_P^i+{\overline{X}}_I^i}{2}\right|)$, then fused images in these image blocks have better effect than those based on weighted average method. So in this case, there is definite possibility for the set of high fusion effect of difference feature to algorithms.

(2)

When the distances of the amplitudes of difference feature are in the interval $(\left|{\overline{X}}_f^i-\min ({\overline{X}}_P^i,{\overline{X}}_I^i)\right|,1]$, difference features with high complementary have not been fused in fused image effectively, so there should be a definite possibility for the set of low fusion effect of difference feature to algorithms.

(3)

If the distance of the amplitudes of difference feature between fused image and source images are in the interval $[\left|{\overline{X}}_f^i-\frac{{\overline{X}}_P^i+{\overline{X}}_I^i}{2}\right|,\left|{\overline{X}}_f^i-\min ({\overline{X}}_P^i,{\overline{X}}_I^i)\right|]$, we cannot determine the fusion validity to be high or low under this circumstance, i.e., medium state.

The amplitudes of difference feature in Figure 3 can be divided into K intervals, and we can obtain the total number of image blocks $N^{X_k}$ included in each amplitude interval $X_k(k=1,2,\cdots ,K)$. Meanwhile, the number of image blocks $n_1^{X_k}$, $n_2^{X_k}$ and $n_3^{X_k}$ can be got included in three distance intervals for each amplitude interval. The mathematical formulation of fusion validity of intuition possible set is given as follows:

$${\pi }_{\tilde{A}}^{1,X_k}=\frac{n_1^{X_k}}{N^{X_k}}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}s.t\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le d^{X_k}<\left|{\overline{X}}_f^k-\frac{{\overline{X}}_p^k+{\overline{X}}_I^k}{2}\right|$$

(6)

$${\pi }_{\tilde{A}}^{2,X_k}=\frac{n_2^{X_k}}{N^{X_k}}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}s.t\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\left|{\overline{X}}_f^k-min({\overline{X}}_p^k,{\overline{X}}_I^k)\right|<d^{X_k}\le 1$$

(7)

$${\pi }_{\tilde{A}}^{3,X_k}=\frac{n_3^{X_k}}{N^{X_k}}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}s.t\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\left|{\overline{X}}_f^k-\frac{{\overline{X}}_p^k+{\overline{X}}_I^k}{2}\right|\le d^{X_k}\le \left|{\overline{X}}_f^k-\min ({\overline{X}}_p^k,{\overline{X}}_I^k)\right|$$

(8)

Here, ${\pi }_{\tilde{A}}^{1,X_k}+{\pi }_{\tilde{A}}^{2,X_k}+{\pi }_{\tilde{A}}^{3,X_k}=1$.

Then, fusion validity distribution of intuition possible set can be obtained as shown in Figure 4. Similarly, we can get the fusion validity distribution of each difference feature to A₁, A₂, A₃, A₄, A₅, A₆, A₇, A₈, A₉, A₁₀, A₁₁ and A₁₂. Take difference feature T₁ as an example, Figure 5 shows fusion validity distribution of difference feature T₁ to different algorithms.

3.4. Combination of Fusion Validity Distribution Based on Intuition Possible Set Ordering

According to the above method, we can obtain fusion validity matrix based on intuition possible set of difference feature to fusion algorithm $A_i$ ($i=1,2,\cdots m$). For difference feature T₁, fusion validity matrix F is given by:

$$F={\left(\langle {\pi }_{ik}^1,{\pi }_{ik}^2\rangle \right)}_{m\times L}=\begin{array}{cc}& \begin{array}{cccc}1 & 2 & \cdots & L\end{array}\\ \begin{array}{c}{A}_1\\ A_2\\ ⋮\\ A_m\end{array}& \left[\begin{array}{cccc}\langle {\pi }_{11}^1,{\pi }_{11}^2\rangle & \langle {\pi }_{12}^1,{\pi }_{12}^2\rangle & \cdots & \langle {\pi }_{1L}^1,{\pi }_{1L}^2\rangle \\ \langle {\pi }_{21}^1,{\pi }_{21}^2\rangle & \langle {\pi }_{22}^1,{\pi }_{22}^2\rangle & \cdots & \langle {\pi }_{2L}^1,{\pi }_{2L}^2\rangle \\ ⋮& ⋮& & ⋮\\ \langle {\pi }_{m1}^1,{\pi }_{m1}^2\rangle & \langle {\pi }_{m2}^1,{\pi }_{m2}^2\rangle & \cdots & \langle {\pi }_{mL}^1,{\pi }_{mL}^2\rangle \end{array}\right]\end{array}$$

Then, the total evaluation value $V_k(k=1,2,\cdots K)$ of the amplitude of difference feature T₁ to algorithm A_i is expressed as follows:

$$V_i=\frac{1}{L}{\displaystyle \sum _{k=1}^L<{\pi }_{ik}^1,{\pi }_{ik}^2>=<{\pi }_i^1,{\pi }_i^2>}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}(i=1,2,\cdots ,m)$$

(9)

Obviously, $V_i(i=1,2,\cdots ,m)$ is also an intuition-possible set.

Let $\tilde{A}=<{\pi }_{ik}^1,{\pi }_{ik}^2>$ be an intuition-possible set, the score function and accuracy function of $\tilde{A}$ is defined as [42,43]:

$$\begin{array}{l}m(\tilde{A})={\displaystyle \sum _{i=1}^m{\displaystyle \sum _{k=1}^L\frac{{\pi }_{ik}^1-{\pi }_{ik}^2}{1-({\pi }_{ik}^1+{\pi }_{ik}^2)(1-{\pi }_{ik}^1-{\pi }_{ik}^2)}}}\\ \Delta (\tilde{A})={\displaystyle \sum _{i=1}^m{\displaystyle \sum _{k=1}^L\frac{{\pi }_{ik}^1+{\pi }_{ik}^2}{1-({\pi }_{ik}^1+{\pi }_{ik}^2)(1-{\pi }_{ik}^1-{\pi }_{ik}^2)}}}\end{array}$$

(10)

For two intuition possible sets ${\tilde{A}}_i$ and ${\tilde{A}}_j$ [44,45,46], then:

1.

If $m({\tilde{A}}_i)>m({\tilde{A}}_j)$, then ${\tilde{A}}_i$ is superior to ${\tilde{A}}_j$, denoted by ${\tilde{A}}_i>{\tilde{A}}_j$;

2.

If $m({\tilde{A}}_i)<m({\tilde{A}}_j)$, then ${\tilde{A}}_j$ is superior to ${\tilde{A}}_i$, denoted by ${\tilde{A}}_i<{\tilde{A}}_j$;

3.

If $m({\tilde{A}}_i)=m({\tilde{A}}_j)$, then,

(1)

If $\Delta ({\tilde{A}}_i)>\Delta ({\tilde{A}}_j)$, then ${\tilde{A}}_i$ is superior to ${\tilde{A}}_j$, denoted by ${\tilde{A}}_i>{\tilde{A}}_j$;

(2)

If $\Delta ({\tilde{A}}_i)=\Delta ({\tilde{A}}_j)$, then ${\tilde{A}}_j$ is equivalent to ${\tilde{A}}_i$, denoted by ${\tilde{A}}_i\text{~}{\tilde{A}}_j$

(3)

If $\Delta ({\tilde{A}}_i)<\Delta ({\tilde{A}}_j)$, then ${\tilde{A}}_j$ is superior to ${\tilde{A}}_i$, denoted by ${\tilde{A}}_i<{\tilde{A}}_j$.

We can calculate score results ranging from the 1st amplitude interval of difference feature T₁ to the 10th interval regarding fusion algorithm $A_i$, as shown in

Table 2. Score results in the range of the 1st of difference feature amplitude interval through the 10th interval.

Fusion Algorithm	Difference Feature Amplitude Interval
	1	2	3	4	5	6	7	8	9	10
A1	0.2143	0.4742	0.6535	0.6526	0.8095	0.9182	0.8829	0.8962	0.9394	0.9167
A2	0.3571	0.3918	0.2970	0.2211	0.2143	0.1818	0.0450	0.0189	0.0101	0.0167
A3	0.0429	0.0206	−0.0198	0.1474	0.1270	0.1091	−0.0270	−0.0094	−0.0101	−0.0500
A4	−0.6286	−0.4845	−0.4356	−0.2737	−0.0635	0.0727	0	0	−0.0101	−0.0333
A5	0.3571	0.5464	0.6139	0.7053	0.7540	0.8273	0.8378	0.7264	0.6465	0.6167
A6	0.3857	0.5670	0.6733	0.6632	0.6190	0.7818	0.8108	0.6887	0.7374	0.6167
A7	0.2429	0.4124	0.5050	0.4947	0.5159	0.5818	0.6306	0.4906	0.5253	0.3833
A8	0.7000	0.7010	0.7228	0.5368	0.5794	0.5091	0.4865	0.5000	0.5859	0.5333
A9	1.0000	0.0515	0	0	0	0	0	0	0	0
A10	0.7571	0.6392	0.3762	0.2842	0.2778	0.1455	0.0721	0.0943	0.0606	0.0500
A11	0.9143	0.1856	0.0198	0.0105	0	0	0	0	0	0
A12	0.1286	0.5670	0.7129	0.6737	0.8254	0.9182	0.8739	0.7642	0.8081	0.8167

.

Table 3. Score results in the range of the 11st of difference feature amplitude interval through the 20th interval.

Fusion Algorithm	Difference Feature Amplitude Interval
	11	12	13	14	15	16	17	18	19	20
A1	0.9310	0.9286	0.9245	0.9535	1.0000	0.9231	0.9167	1.0000	0	0
A2	0	−0.1000	−0.0755	−0.1395	−0.0870	−0.1538	−0.1667	0	0	−1.0000
A3	−0.0115	−0.0429	−0.0943	−0.0233	0.2609	0.2308	−0.3333	−0.3333	0	−1.0000
A4	0	−0.0143	−0.0377	0	0	0	−0.2500	−0.3333	0	−1.0000
A5	0.7241	0.8286	0.7736	0.8372	0.8696	0.9231	1.0000	1.0000	0	0
A6	0.8161	0.8000	0.8491	0.8837	0.8696	0.9231	0.9167	1.0000	0	−1.0000
A7	0.4943	0.6000	0.5660	0.6744	0.4783	0.3846	0.3333	0.3333	0	−1.0000
A8	0.3678	0.0714	0.2453	0.0465	−0.4783	−0.5385	−0.3333	−0.3333	0	1.0000
A9	0	0	−0.9623	−1.0000	−1.0000	−1.0000	−1.0000	−1.0000	0	−1.0000
A10	0.0460	0.0714	0.0189	0.0930	−0.0870	−0.1538	−0.3333	−0.3333	0	−1.0000
A11	0	−0.0429	−0.7736	−1.0000	−1.0000	−1.0000	−1.0000	−1.0000	0	−1.0000
A12	0.8966	0.9429	0.9057	0.9767	1.0000	0.9231	1.0000	1.0000	0	1.0000

is score results ranging from the 11th amplitude interval of difference feature T₁ to the 20th interval.

According to Table 2 and Table 3, for each amplitude interval of difference feature T₁, the nearer the score of algorithm approximates to 1, the better fusion effect this algorithm will have. The nearer the score of algorithm approximates to −1, the worse fusion effect the algorithm will have. Also, we can compute scores of each amplitude interval of the other three classes of difference features based on intuition possible set ordering. The advantage of such approach is that we can know the mapping between the amplitude of difference feature and fusion algorithms clearly. When the amplitude of difference feature changes, the fusion validity of each algorithm will also change accordingly. The total evaluation values of the amplitudes of difference features to algorithm $A_i$ are presented in

Table 4. The total evaluation values of difference features.

Fusion Algorithm	T1	T2	T3	T4
A1	0.7467	0.4579	0.3354	0.3260
A2	0.1738	0.1923	0.1901	0.2168
A3	0.1447	0.2418	0.1036	0.2181
A4	0.1819	0.1115	0.0817	0.1921
A5	0.6794	0.4127	0.2383	0.1781
A6	0.7301	0.5755	0.4967	0.4929
A7	0.4823	0.1355	0.1871	0.2020
A8	0.4635	0.3241	0.0651	0.0303
A9	0.4007	0.0912	0.2193	0.3080
A10	0.2447	0.2817	0.1657	0.1079
A11	0.3973	0.1899	0.1811	0.2653
A12	0.7867	0.5007	0.2744	0.3739

and pictorially depicted in Figure 6.

According to Table 4 and Figure 6, we can easily observe that, for difference feature T₁, the A₁₂ algorithm outperforms the other algorithms when source images contain T₂, T₃, and T₄ with the proposed intuition-possible set ordering method in this study. For difference feature T₂, T₃, and T₄, the A₆ algorithm has obvious advantages in the fidelity of salient information (including contrast, edge feature and texture) and human visual effect.

The score results of difference features use the disjunctive fuzzy operator to select the optimal fusion algorithm with the largest score. The aggregation of four difference feature score results of source images are presented in Figure 7. We can also obtain joint drop shadows of aggregation of four difference feature score results, shown in Figure 8.

By counting the numbers of each fusion algorithm appears in the joint drop shadow graph and the corresponding score values, the proportion f_i of the fusion algorithm A_i in the area and the average score value $\overline{E_i}$ are calculated. Because the values are both related to the performance of algorithm A_i, the fusion index E_i is constructed, as shown in Equation (11). The fusion index of different fusion algorithms synthesis of all the difference feature amplitudes is summed, and the fusion algorithm corresponding to the highest value is the optimal fusion algorithm:

$$E_i=f_i\ast \overline{E_i}$$

(11)

3.5. Experimental Results and Comparisons

To test the proposed fusion validity distribution construction and combination method in this study, we select two groups of infrared polarization and intensity images with different scenes (as shown in Figure 9) to experimental verification. Figure 10 and Figure 11 are fused images with the above fusion algorithms for each group of verified images respectively. The fusion index E_i for each group are shown in

Table 5. E_i of each fusion algorithms of the first group.

Difference Features	Index	Fusion Algorithm
		A1	A2	A3	A4	A5	A6	A7	A8	A9	A10	A11	A12
T1, T2	$f_i$	0.0207	0	0.0059	0	0.0237	0.6657	0.2219	0	0	0.0148	0.0059	0.0059
	$\overline{E_i}$	0.2140	0	0.2031	0	0.3813	0.3439	0.2101	0	0	0.2226	0.2031	0.5980
	Ei	0.0044	0	0.0012	0	0	0.2289	0.0466	0	0	0.0033	0.0012	0.0035
T1, T3	$f_i$	0.1030	0	0	0	0.0258	0.4052	0.0585	0.1288	0.0081	0.0094	0	0.2646
	$\overline{E_i}$	0.3341	0	0	0	0.3125	0.3118	0.1430	0.4267	0.1210	0.2688	0	0.3333
	Ei	0.0344	0	0	0	0.0081	0.1263	0.0084	0.0550	0.0010	0.0025	0	0.0882
T1, T4	$f_i$	0.1592	0.0100	0	0.0149	0.1194	0.3383	0.1741	0.0647	0	0.0697	0	0
	$\overline{E_i}$	0.1818	0.2264	0	0.2814	0.5393	0.3561	0.1917	0.2691	0	0.2918	0	0
	Ei	0.0289	0.0023	0	0.0042	0.0644	0.1205	0.0334	0.0174	0	0.0203	0	0
T2, T3	$f_i$	0.1633	0	0	0	0.0854	0.4296	0.1985	0.0578	0	0.0226	0	0.0302
	$\overline{E_i}$	0.3727	0	0	0	0.5650	0.5005	0.2120	0.3981	0	0.6550	0	0.5400
	Ei	0.0609	0	0	0	0.0483	0.2151	0.0421	0.0230	0	0.0148	0	0.0163
T2, T4	$f_i$	0.2111	0	0	0	0.0427	0.2563	0.1231	0	0	0.0427	0	0.2739
	$\overline{E_i}$	0.2440	0	0	0	0.5755	0.4892	0.2553	0	0	0.6599	0	0.4480
	Ei	0.0515	0	0	0	0.0246	0.1254	0.0314	0	0	0.0282	0	0.1227
T3, T4	$f_i$	0.2601	0	0	0	0.0934	0.1086	0.1010	0.0985	0	0.0253	0	0.3131
	$\overline{E_i}$	0.3210	0	0	0	0.6009	0.5951	0.1894	0.3213	0	0.7471	0	0.4510
	Ei	0.0835	0	0	0	0.0561	0.0646	0.0191	0.0316	0	0.0189	0	0.1412
sum		0.2636	0.0023	0.0012	0.0042	0.2105	0.8808	0.1810	0.1270	0.0010	0.0880	0.0012	0.3719
rank		3	10	11	9	4	1	5	6	12	7	11	2

and

Table 6. E_i of each fusion algorithms of the second group.

Difference Features	Index	Fusion Algorithm
		A1	A2	A3	A4	A5	A6	A7	A8	A9	A10	A11	A12
T1, T2	$f_i$	0	0.2553	0.0184	0	0.3289	0.0474	0.1263	0.0184	0.0145	0.0094	0	0.1158
	$\overline{E_i}$	0	0.6891	0.8492	0	0.6361	0.8124	0.7805	0.8492	0.6549	0.0851	0	0.6876
	Ei	0	0.1759	0.0156	0	0.2092	0.0385	0.0986	0.0156	0.0095	0.0008	0	0.0796
T1, T3	$f_i$	0.0797	0.1087	0	0	0.5966	0.1522	0	0	0	0	0.0338	0.0072
	$\overline{E_i}$	0.7975	0.6974	0	0	0.6790	0.7858	0	0	0	0	0.7048	0.7717
	Ei	0.0636	0.0758	0	0	0.4051	0.1196	0	0	0	0	0.0238	0.0056
T1, T4	$f_i$	0	0.2746	0.0405	0	0.2197	0.1069	0.1012	0.0405	0.0825	0	0	0.2572
	$\overline{E_i}$	0	0.6857	0.6456	0	0.5714	0.8249	0.7746	0.6456	0.2839	0	0	0.6969
	Ei	0	0.1883	0.0261	0	0.1255	0.0882	0.0784	0.0261	0.0234	0	0	0.1793
T2, T3	$f_i$	0.0150	0	0	0	0.6675	0.1925	0.0425	0	0.1457	0.0246	0	0
	$\overline{E_i}$	0.5056	0	0	0	0.2652	0.4916	0.4146	0	0.4015	0.0285	0	0
	Ei	0.0076	0	0	0	0.1770	0.0946	0.0176	0	0.0585	0.0007	0	0
T2, T4	$f_i$	0	0	0	0.0025	0.4347	0.0779	0.1382	0	0.0023	0	0.0553	0.1457
	$\overline{E_i}$	0	0	0	0.3074	0.1668	0.4844	0.4525	0	0.6105	0	0.3465	0.2850
	Ei	0	0	0	0.0008	0.0725	0.0377	0.0625	0	0.0014	0	0.0192	0.0415
T3, T4	$f_i$	0.0090	0	0	0.0045	0.6014	0.2545	0.0383	0	0	0	0.0180	0.0721
	$\overline{E_i}$	0.7382	0	0	0.2611	0.1995	0.4321	0.3799	0	0	0	0.3426	0.3236
	Ei	0.0067	0	0	0.0012	0.1200	0.1100	0.0145	0	0	0	0.0062	0.0233
sum		0.0778	0.4400	0.0418	0.0019	1.1094	0.4886	0.2716	0.0049	0.1705	0.0015	0.0492	0.3293
rank		7	3	9	11	1	2	5	10	6	12	8	4

.

We adopt both subjective and objective assessment to analyse the fused results with the algorithm selected by the proposed fusion validity distribution construction and combination method. It can be observed from Figure 10 that the fusion results of A₁, A₆ and A₁₂ algorithms can retain the brightness of infrared intensity image, while the other fused images are dark as a whole. Under the premise of preserving the fused images with high brightness and contrast, algorithm A₆ has better texture detail and edge contour (particularly the box region of Figure 10) preservation performance, also it has better visual effects and resolution. For the box region with great difference of source image in Figure 11, algorithms A₅ and A₆ have better visual brightness, contrast, texture details and edge profile edge contour preservation performance than the other algorithms, particularly the A₅ algorithm, which preserves the texture details completely. The other fused images are dark, leading to poor human visual effect.

It can be observed from Table 5 and Table 6 and Figure 12, for the first group, in the five combination forms (T₁ and T₂, T₁ and T₃, T₁ and T₄, T₂ and T₃, T₂ and T₄), the fusion indicators of fusion algorithm A₆ are higher than other algorithms, and it is obvious that A₆ is much larger than other algorithms on the final calculation, so it is concluded that A₆ is the optimal fusion algorithm for the first group. In the same way, A₅ is the optimal fusion algorithm for the second group. In order to prove the effectiveness and advantages of this method in this study, based on the first set of images, a comparative analysis was made with the existing fusion effectiveness based on fuzzy theory. The fusion validity as follows:

$${\mu }_i^{}=\frac{{\overline{X}}_{f,j}^i\cdot max({\overline{X}}_{f,j}^i)}{\sqrt{{({\overline{X}}_{f,j}^i)}^2+\max {({\overline{X}}_P^i,{\overline{X}}_I^i)}^2}}$$

(12)

Thus, we can obtain fusion validity scatterplot of the four difference features, as shown in Figure 13.

The difference feature information and fuzzy operators in the above cases are used to rank the fusion algorithm. With fusion validity based on fuzzy theory, the order of fusion algorithms, namely, A₆ > A₁₀ > A₁₁ > A₁₂ > A₁ > A₇ >A₅ >A₉ > A₃ > A₂ > A₈ > A₄.

The rankings order compared with the result obtained by this paper is slightly different, but the best algorithm of the two methods is the same, that is A₆. It shows that the method proposed in this paper is effective.

To further demonstrate the correctness of the sort order, we compare the values of X_j=1:8 (information entropy, Q₀, Q_w, Q_E, VIFF, SSIM, mutual information, and average gradient). The fusion results of the 12 fusion algorithms corresponding to the two groups of source images are evaluated, we employ grade score R_i (as shown in Formula (12)) to fully consider all evaluation indexes evaluate:

$$R_i={\displaystyle \sum _{j=1}^8{r}_{.j},i=1,2,\cdots ,12}$$

(13)

According to the ranking $r_{.j}$ of each fusion algorithm about the value, the optimal fusion algorithm is selected and compared with the selected fusion algorithm of proposed method. The results are shown in

Table 7. The evaluation index of different fusion algorithms for two group images.

Group	Method	Evaluation Index								R	Rank
		X1	X2	X3	X4	X5	X6	X7	X8
1	A1	0.5791	0.4356	0.7113	0.3051	0.5886	0.4116	3.1011	0.0534	36	2
	A2	0.3698	0.3807	0.5591	0.2022	0.6174	0.3445	4.3507	0.0637	64	10
	A3	0.5734	0.4118	0.7060	0.2878	0.4635	0.3886	2.8213	0.0556	58	8
	A4	0.4056	0.3777	0.5651	0.1905	0.4327	0.3432	3.1173	0.0376	82	12
	A5	0.5587	0.4005	0.6697	0.2745	0.5100	0.3693	3.0407	0.0485	55	7
	A6	0.5873	0.4663	0.6353	0.2743	0.8514	0.4385	7.1886	0.0686	21	1
	A7	0.4721	0.3851	0.5955	0.2101	0.6797	0.3510	6.3275	0.0597	51	6
	A8	0.3651	0.3095	0.6639	0.2452	0.4863	0.2724	3.3568	0.0487	72	11
	A9	0.3621	0.4446	0.6110	0.2366	0.5029	0.4141	3.4465	0.0313	62	9
	A10	0.4007	0.4531	0.5648	0.2173	0.7542	0.4633	4.0854	0.0418	50	5
	A11	0.4587	0.4214	0.6708	0.2751	0.6502	0.3978	3.0044	0.0668	41	4
	A12	0.4511	0.5241	0.5441	0.2582	0.5451	0.5197	7.4121	0.0558	40	3
2	A1	0.4536	0.3853	0.5386	0.3719	0.1691	4.5003	0.0605	0.1259	53	7
	A2	0.4849	0.3779	0.657	0.374	0.31	1.9077	0.0538	0.2986	37	3
	A3	0.4895	0.3549	0.6488	0.3458	0.1678	1.5654	0.0459	0.2517	64	10
	A4	0.4319	0.3421	0.5639	0.3305	0.159	2.7579	0.043	0.1568	77	11
	A5	0.4921	0.4368	0.6406	0.4851	0.2989	8.0634	0.0657	0.3351	17	1
	A6	0.4663	0.3883	0.6753	0.3824	0.268	1.9236	0.066	0.2915	32	2
	A7	0.4377	0.3666	0.6813	0.3546	0.2708	1.6754	0.0659	0.2613	50	5
	A8	0.4398	0.3846	0.5538	0.3676	0.1829	4.4382	0.0654	0.1175	57	9
	A9	0.4477	0.377	0.5654	0.3211	0.2021	5.2334	0.0489	0.322	51	6
	A10	0.2843	0.0524	0.0652	0.0244	0.0255	4.6624	0.0354	0.0203	88	12
	A11	0.4442	0.3704	0.6375	0.3619	0.2823	2.2638	0.0504	0.227	54	8
	A12	0.5028	0.4665	0.5359	0.4711	0.1699	6.3297	0.048	0.1505	43	4

.

The following can be observed from the analysis of the data in the Table 7, the R of algorithm A₆ in the first group is significantly smaller than other fusion algorithms, which indicates that A₆ is the optimal fusion algorithm of the first group, with the strongest overall performance. With regard to the suboptimal algorithm A₁, the X₂ and X₃ values are higher than the others, and the X₁ value is second highest, and the other values rank ahead. It is reasonable that A₁ is the suboptimal algorithm. Therefore, we can use the proposed method based on the intuitive possible sets to select the best fusion algorithm in dual mode infrared images.

In the second group, the R of the A₅ algorithm is significantly better. For other fusion algorithms, it is consistent with the experimental results in Table 5 and Table 6, which verifies the correctness and effectiveness of the method in this paper. The algorithm with better fusion result by subjective and objective analysis are selected by the proposed fusion validity distribution construction and combination method, which shows that based on intuition possible set, it is feasible and effective to select the best fusion algorithm. According to multi-attributes (e.g., type and amplitude) of difference feature, the proposed method can select relatively better fusion algorithm specifically.

4. Conclusions

This paper proposes a novel fusion validity distribution construction method based on intuition-possible sets. We have considered the dynamic changes of difference feature attributes (including type and amplitude) to select algorithm and established the mappings of the attributes of difference feature and fusion algorithm. The proposed distribution can realize quantitative description of fusion effect changing process of difference feature of images to algorithms.

For multiple amplitude intervals of difference features, this paper put forward a distribution combination method based on intuition possible set ordering. Through calculating the score function and accuracy function of intuition possible sets on fusion validity, we can select fusion algorithm with relatively better result. It provides new way to algorithm classification and mimicry bionic fusion.

In the process of approaching a solution, future research is formulated with the following questions in mind:

(1)

In this article, four difference features are used to select the best one of the twelve fusion algorithms. In further research, we should consider the recently published solutions for image fusion to choose the algorithm with better fusion effects.

(2)

Although the method proposed in this paper has great advantages in selecting the optimal fusion algorithm, there is still some room for improvement. We utilize a fuzzy operator to aggregate difference feature score results in this paper. It would be very interesting to apply some fuzzy weighted averaging operators to cope with fusion validity distribution combination of difference features, such as fuzzy ordered weighted averaging operator, and Pythagorean fuzzy averaging and geometric averaging operators, etc.