An Improved Pansharpening Method for Misaligned Panchromatic and Multispectral Data

Numerous pansharpening methods were proposed in recent decades for fusing low-spatial-resolution multispectral (MS) images with high-spatial-resolution (HSR) panchromatic (PAN) bands to produce fused HSR MS images, which are widely used in various remote sensing tasks. The effect of misregistration between MS and PAN bands on quality of fused products has gained much attention in recent years. An improved method for misaligned MS and PAN imagery is proposed, through two improvements made on a previously published method named RMI (reduce misalignment impact). The performance of the proposed method was assessed by comparing with some outstanding fusion methods, such as adaptive Gram-Schmidt and generalized Laplacian pyramid. Experimental results show that the improved version can reduce spectral distortions of fused dark pixels and sharpen boundaries between different image objects, as well as obtain similar quality indexes with the original RMI method. In addition, the proposed method was evaluated with respect to its sensitivity to misalignments between MS and PAN bands. It is certified that the proposed method is more robust to misalignments between MS and PAN bands than the other methods.


Introduction
Due to the physical constraints of remote sensing imaging and limited bandwidth of satellite transmission, a large number of currently operating satellites, such as SPOT, IKONOS (IK), QuickBird (QB), and WorldView-2/3, provide both a single relative high-spatial-resolution (HSR) panchromatic (PAN) band and several low-spatial-resolution (LSR) MS bands. Fusion of PAN and MS images is also referred to as pansharpening [1]. It is an important pre-processing step for generating high quality HSR MS image in various remote sensing tasks [2,3], such as land cover classification, object recognition [4,5], water-bodies mapping [6][7][8], and shadow detection [9]. In recent years, the requirement for fused HSR MS imagery is unceasingly growing, due to abundant remote sensing data sources [10].
Research on pansharpening has been done for decades and numerous algorithms have been developed to produce HSR MS imagery. The major problem encountered by current pansharpening methods is to reduce spectral and spatial distortions. Researchers have proposed several categorizations to group existing pansharpening methods [11][12][13][14][15][16]. One of the categorizations classifies current pansharpening methods into two categories: the component-substitution (CS) methods and the methods based on multiresolution analysis (MRA) [17][18][19][20]. The CS methods can provide fused products with good spatial quality in most cases, but they sometimes suffer from spectral distortions. The MRA-based methods can provide fused products with good spectral quality. However, their fused products may suffer spatial distortions [21]. Moreover, this disadvantage may be emphasized for misaligned MS and PAN data, especially for MSA-based methods using transformations that are not shift-invariant to achieve multiresolution analysis [21,22]. In contrast, the performances of CS methods

The RMI Method
The RMI method is a ratio-based approach considering haze caused by atmospheric path radiances. Let H P and H i denote image haze values determined for PAN and ith MS bands, respectively, I i denotes the ith band of the up-sampled MS image, and P and P S are an original PAN band and a synthetic version of the original PAN band, respectively. The fused ith MS band F i generated by the RMI method can be expressed using Equation (1): Equation (1) can also be rewritten as (2): in which P − P S represents spatial details derived from the original PAN band, and I i −H i P S −H p is the space-varying injection coefficients for the ith band. P S is obtained by a weighted sum the up-sampled MS bands, as in (3): in which a i is the weight coefficient of the ith band, and b is a constant. The optimal values of a i and b can be estimated by the least-squares approach using Equation (4): where P L is a degraded version of the original PAN band, and can be obtained using an averaging approach; I L i is the ith original MS band. For misaligned MS and PAN bands, the optimal regression coefficients, representing the maximum multiple correlations between misaligned PAN with each MS band, can discount the effect of misalignment on quality of fused images. Another factor contributing to the good performance of the RMI method is that image haze caused by atmospheric path radiance is considered by the method. However, haze values of PAN and MS bands give an important effect on spectral vector directions of fused MS pixels and thus the spectral quality of the whole fusion product [32].

The Improved Version of the RMI Method
In this work, two improvements are made based on the original RMI method. For the first improvement, boundary pixels of the PAN band are identified using edge detection operators, and then fused using larger injection coefficients than the other pixels. Fused images generated by commonly used methods show blurred boundaries between different objects, due to several reasons, such as misalignments, aliasing, and the effect of mixed pixels. As this improvement aims to add more spatial details into edge pixels, it helps to obtain fused products with sharpened boundaries between different image objects. For an edge pixel t, the fusion version of t is calculated using (5): in which k is a value determining the sharpness boundaries between different objects in pansharpened products. The value can be set by users, according to different application purposes. For the second improvement, relative dark pixels in the image, which mainly correspond to dark objects such as water-bodies and shadows, are identified according to a certain threshold and then fused using relative low haze values. As a high haze value may result in abnormal high injection coefficients for dark pixels, this may result in over-enhancement of dark pixels. This improvement is helpful for reducing spectral distortions of water-bodies and shadow covered regions in fused products. In this work, a threshold T, used to justify dark pixels, were automatically determined using the product of the standard deviation of the PAN band δ P and a scale factor S.
For a pixel with a gray value higher than T in PAN, the fused version F i can be produced using (2). In contrast, for a pixel with a gray value lower than T in PAN, relative low haze values for MS and PAN bands, which are denoted as H L i and H L P , respectively, are employed to generate fused pixels. In this situation, the fused version F i can be produced using (6): in which H L i is determined using (7): where p is a scale constant be lower 1. A value of 0.75 was assigned to p in this work. The values of H L P are obtained with respect to coefficients obtained using (4), as in (8): The proposed method is implemented in MATLAB. The pseudocode of the proposed method is also reported, as Algorithm 1, to facility other people to implement and use the method in remote sensing tasks. Algorithm 1: Improved_RMI input: upsampled MS bands I, PAN band P, band number N output: fused MS bands F Let P L be a PAN band at MS scale, generating from P using an averaging approach Let a i and b are the coefficients generated using (4) by the least-squares approach

Test Data
Three image scenes recorded by WorldView-2 (WV2), IK, and QB, respectively, were used in the experiment to evaluate the performances of the proposed method. A subset with a size of 512 × 512 pixels for MS bands, and a size of 2048 × 2048 pixels for PAN bands, were selected from each scene. The MS image from WV2 has eight bands, whereas those of IK and QB have four bands. The spatial resolution ratio R for all the three datasets is 4. The MS images of three datasets are presented in Figure 1.

Fusion Methods for Comparison and Evaluation Criteria
The proposed method was compared with adaptive Gram-Schmidt (GSA) [5,33], and Generalized Laplacian Pyramid (GLP) with Gaussian-shaped filter adjusted by modulation transfer function using spectral distortion minimizing model [34]. Actually, image haze can also be considered by some other methods using similar injection models. In order to achieve impartial comparison, the GLP method considering image haze was also achieved and included in the comparisons. It is noted as GLP-H henceforth. The GSA method with image haze was also considered, but it was not included in this work because it yielded the same performance as the original GSA method. Actually, it can be inferred that the considering of image haze could not improve or decrease the performance of GSA.
Besides the original versions of the three datasets, the degraded versions of the original datasets were also considered, as fused images generated at the degraded scale can be assessed using the original MS images as reference. Quality of fused products of the degraded scale was assessed using several widely used quality indexes including the relative average spectral error (RASE) [35], dimensionless global relative error of synthesis (ERGAS) [36], spectral angle mapper (SAM) [37], Q2 n [38][39][40], and spatial correlation coefficient (SCC) [41]. An index named quality with no reference index (QNR) [42], which is increasingly used in recent studies, was employed to evaluate quality of pansharpened images produced using the original datasets. The QNR index is a comprehensive index consisting of a spectral distortion indicator Dλ and a spatial distortion indicator DS [42]. In addition, visual inspection was also performed to comparing the quality of fused products. In order to avoid misalignments between MS and PAN bands, an aligned version of an original PAN band was determined using the similar approach employed in [43], and then used to replace the original PAN bands in this experiment. The degraded MS and PAN images were generated by a box filter with a size of 4 × 4, according to the resolution ratio of the three datasets, in order to avoid misalignment introduced during the decimation process [30]. Table 1 reports the quality indexes generate from the fused products obtained at the two scales. The improved methods with different values for k used the same haze values as those employed by

Fusion Methods for Comparison and Evaluation Criteria
The proposed method was compared with adaptive Gram-Schmidt (GSA) [5,33], and Generalized Laplacian Pyramid (GLP) with Gaussian-shaped filter adjusted by modulation transfer function using spectral distortion minimizing model [34]. Actually, image haze can also be considered by some other methods using similar injection models. In order to achieve impartial comparison, the GLP method considering image haze was also achieved and included in the comparisons. It is noted as GLP-H henceforth. The GSA method with image haze was also considered, but it was not included in this work because it yielded the same performance as the original GSA method. Actually, it can be inferred that the considering of image haze could not improve or decrease the performance of GSA.
Besides the original versions of the three datasets, the degraded versions of the original datasets were also considered, as fused images generated at the degraded scale can be assessed using the original MS images as reference. Quality of fused products of the degraded scale was assessed using several widely used quality indexes including the relative average spectral error (RASE) [35], dimensionless global relative error of synthesis (ERGAS) [36], spectral angle mapper (SAM) [37], Q2 n [38][39][40], and spatial correlation coefficient (SCC) [41]. An index named quality with no reference index (QNR) [42], which is increasingly used in recent studies, was employed to evaluate quality of pansharpened images produced using the original datasets. The QNR index is a comprehensive index consisting of a spectral distortion indicator D λ and a spatial distortion indicator D S [42]. In addition, visual inspection was also performed to comparing the quality of fused products. In order to avoid misalignments between MS and PAN bands, an aligned version of an original PAN band was determined using the similar approach employed in [43], and then used to replace the original PAN bands in this experiment. The degraded MS and PAN images were generated by a box filter with a size of 4 × 4, according to the resolution ratio of the three datasets, in order to avoid misalignment introduced during the decimation process [30]. Table 1 reports the quality indexes generate from the fused products obtained at the two scales. The improved methods with different values for k used the same haze values as those employed by the original RMI method, for each test image. In this table, RMI (k = n) denotes the fused images generated by the RMI method with k = n, where n is an integer ranging from 0 to 4. In order to highlight the performances of the comparing methods on the fusion of dark pixels, a SAM value between fused and reference dark pixels is calculated from each fused image generated at the degraded scale. Version and the original version of dark pixels, which are identified during the implement of the proposed method. This index is also listed in Table 1, and it is denoted as SAM d in the table. It can be observed from Table 1 that RMI (k = 0) gives slightly better or similar performances with the original RMI method. The improved and original RMI methods provide higher Q2 n and QNR values than other methods. An exception is that GSA and GLP-H yield lower D λ and D s values, and higher QNR values than other methods for the original QB dataset. For the degraded WV2 dataset, RMI (k = 0) offers slightly lower RASE and ERGAS values, and slightly higher Q2 n and SCC values than the original RMI method. For the original WV2 dataset, RMI (k = 0) provides a slightly lower D s value and a slightly higher QNR value than the original RMI method. For the degraded IK dataset, RMI (k = 0) provides slightly lower Q2 n and RASE values than the original RMI method, and the same ERGAS and SAM values as the original RMI. The former offers a slightly higher QNR value and a slightly lower D s value than the latter, for the original IK dataset. For the degraded QB dataset, RMI (k = 0) provides slightly lower RASE, ERGAS, SAM values, and higher Q2n and SCC values than the original RMI. For the original QB dataset, RMI (k = 0) also yields a lower D λ and a higher QNR than the original RMI. The improved RMI methods offer lower SAM d than the original RMI for all the three datasets, indicating that the proposed method is effective for reducing spectral distortions of fused dark pixels. The GSA method yields the highest SAM d values for the three degraded datasets, indicating it gives the poorest performances for dark pixels. As the three test images have different numbers of dark pixels, it is reasonable that the proposed approach gives slightly different performances for these images.

Results and Analysis
The Q2 n and QNR values of fused products generated by the improved RMI methods descend along with the increasing of k. However, the fused products generated by RMI (k = 4) also offer higher Q2 n and QNR values than those generated by the GSA, GLP, and GLP-H methods for the WV-2 and IK datasets. The GLP-H method offers higher Q2 n and QNR values than GSA and GLP for the three datasets, due to the considering of image haze. The GLP method offers lower Q2 n and QNR values than GSA for the three datasets. This may due to the fact that misalignments between MS and PAN bands occur during an down-sampling progress for obtaining a LSR PAN band at MS scale, and an    Although pansharpened images generated by these methods show similar tone in each figure, fused images produced by the improved and original RMI methods, and GLP-H show more texture details in vegetation covered regions. This is obvious in Figure 2. This is due to the considering of image hazes by these methods. Some artefacts can be seen from the fused images generated by the original RMI method, as shown in Figures 2f and 4f. The artefacts occur in some regions covered by shadows or water-bodies. In contrast, these artefacts are absent from the pansharpened images generated by RMI (k = 0), RMI (k = 2), and RMI (k = 4), as shown in Figures 2 and 4. Although GSA yields the highest QNR value for the original QB dataset, the GSA-fused images show obvious spectral distortions. This can be observed from the yellow rectangle in Figure 2g, and the red and yellow rectangles in Figure 4g. In addition, the GSA-fused WV2 image show very few texture details for vegetation covered regions. This is because a very small amount of spatial details is injected into the two near-infrared (NIR) bands of the WV2 dataset, due to relative low correlations between PAN and the two NIR bands. The fused images generated by GLP and GLP-H also show noticeable spectral distortions, as shown in yellow rectangles in Figures 2-4. Moreover, aliasing effects can be observed in the fused images generated by GLP and GLP-H, as shown in yellow rectangles in Figures 3 and 4. Moreover, the fused products produced by RMI (k = 2), and RMI (k = 4) offer more sharpened boundaries between different objects than the original RMI, GSA, GLP, and GLP-H. This Although pansharpened images generated by these methods show similar tone in each figure, fused images produced by the improved and original RMI methods, and GLP-H show more texture details in vegetation covered regions. This is obvious in Figure 2. This is due to the considering of image hazes by these methods. Some artefacts can be seen from the fused images generated by the original RMI method, as shown in Figures 2f and 4f. The artefacts occur in some regions covered by shadows or water-bodies. In contrast, these artefacts are absent from the pansharpened images generated by RMI (k = 0), RMI (k = 2), and RMI (k = 4), as shown in Figures 2 and 4. Although GSA yields the highest QNR value for the original QB dataset, the GSA-fused images show obvious spectral distortions. This can be observed from the yellow rectangle in Figure 2g, and the red and yellow rectangles in Figure 4g. In addition, the GSA-fused WV2 image show very few texture details for vegetation covered regions. This is because a very small amount of spatial details is injected into the two near-infrared (NIR) bands of the WV2 dataset, due to relative low correlations between PAN and the two NIR bands. The fused images generated by GLP and GLP-H also show noticeable spectral distortions, as shown in yellow rectangles in Figures 2-4. Moreover, aliasing effects can be observed in the fused images generated by GLP and GLP-H, as shown in yellow rectangles in Figures 3  and 4. Moreover, the fused products produced by RMI (k = 2), and RMI (k = 4) offer more sharpened boundaries between different objects than the original RMI, GSA, GLP, and GLP-H. This can be seen from the fused subsets in yellow rectangles shown in Figures 2-4. The fused images generated by RMI (k = 2), and RMI (k = 4) provide more sharpened boundaries than those produced by RMI (k = 0), due to more spatial details are injected into edge pixels. Consequently, the improved approach can reduce spectral distortions of fused dark pixels and sharpen some boundaries in fused products, as well as obtain similar quality indexes with the original RMI method. can be seen from the fused subsets in yellow rectangles shown in Figures 2-4. The fused images generated by RMI (k = 2), and RMI (k = 4) provide more sharpened boundaries than those produced by RMI (k = 0), due to more spatial details are injected into edge pixels. Consequently, the improved approach can reduce spectral distortions of fused dark pixels and sharpen some boundaries in fused products, as well as obtain similar quality indexes with the original RMI method.

Sensitivity to Misalignments between MS and PAN Bands
The proposed method is also evaluated with respect to its sensitivity to misalignments between MS and PAN bands using another experiment, with the degraded IK dataset. The up-sampled version of the degraded MS image was shifted by zero to four PAN pixels on row and column, respectively, to yield several shifted versions of the up-sampled MS image. A shift of r pixels on row, along with c pixels on column, is denoted as a couple (r, c). Several couples including (0, 1), (1, 1), (2, 1), (2,2), (3,2), (3,3), (4,3), and (4,4) were employed to generated shifted MS images. These shifted up-sampled MS images were then fused with the degraded 4-m PAN band using the proposed method, GSA, GLP, and GLP-H, to produce 4-m fused products.

Sensitivity to Misalignments between MS and PAN Bands
The proposed method is also evaluated with respect to its sensitivity to misalignments between MS and PAN bands using another experiment, with the degraded IK dataset. The up-sampled version of the degraded MS image was shifted by zero to four PAN pixels on row and column, respectively, to yield several shifted versions of the up-sampled MS image. A shift of r pixels on row, along with c pixels on column, is denoted as a couple (r, c). Several couples including (0, 1), (1, 1), (2, 1), (2,2), (3,2), (3,3), (4,3), and (4,4) were employed to generated shifted MS images. These shifted up-sampled MS images were then fused with the degraded 4-m PAN band using the proposed method, GSA, GLP, and GLP-H, to produce 4-m fused products.
The quality of these fused products was also assessed using ERGAS, SAM, Q4, and SCC, which are shown in Figure 5a-d, respectively. As the improved RMI methods with different k values give very clustered indexes, only the cases including k = 0, k = 2, k = 4 are shown in the figure for better visual inspection. As illustrated in those figures, the original RMI, RMI (k = 0), and RMI (k = 2) offer very similar performances, in terms of the four indexes. Actually, RMI (k = 0) offers slightly lower ERGAS and SAM values, and slightly higher Q4 and SCC values, than the original RMI method. RMI (k = 2) also gives lower SAM values than the original RMI method. The quality of these fused products was also assessed using ERGAS, SAM, Q4, and SCC, which are shown in Figures 5a-d, respectively. As the improved RMI methods with different k values give very clustered indexes, only the cases including k = 0, k = 2, k = 4 are shown in the figure for better visual inspection. As illustrated in those figures, the original RMI, RMI (k = 0), and RMI (k = 2) offer very similar performances, in terms of the four indexes. Actually, RMI (k = 0) offers slightly lower ERGAS and SAM values, and slightly higher Q4 and SCC values, than the original RMI method. RMI (k = 2) also gives lower SAM values than the original RMI method. The pansharpened images generated by the original RMI, RMI (k = 0), and RMI (k = 2) offer the lowest ERGAS and SAM values, and the highest Q4 and SCC values, in most cases. Exceptions occur when the misalignments are (4, 3) and (4,4), the GSA method yields the highest Q4 values, and higher SCC values than the RMI (k = 2). In addition, RMI (k = 4) offers lower SCC values than the GSA and GLP-H methods in some cases, although the former offers lower ERGAS and SAM values for all the cases. This indicates that RMI (k = 0), and RMI (k = 2) are the best choices, when misalignments between MS and PAN bands are no more than three PAN pixels on row and column.
The GSA method gives better performances than the two GLP methods when the misalignment is larger than two pixels. However, when the misalignment is no more than one pixel, the GSA method offers higher ERGAS values and lower Q4 values than the GLP-H method. The pansharpened images generated by the original RMI, RMI (k = 0), and RMI (k = 2) offer the lowest ERGAS and SAM values, and the highest Q4 and SCC values, in most cases. Exceptions occur when the misalignments are (4, 3) and (4,4), the GSA method yields the highest Q4 values, and higher SCC values than the RMI (k = 2). In addition, RMI (k = 4) offers lower SCC values than the GSA and GLP-H methods in some cases, although the former offers lower ERGAS and SAM values for all the cases. This indicates that RMI (k = 0), and RMI (k = 2) are the best choices, when misalignments between MS and PAN bands are no more than three PAN pixels on row and column.
The GSA method gives better performances than the two GLP methods when the misalignment is larger than two pixels. However, when the misalignment is no more than one pixel, the GSA method offers higher ERGAS values and lower Q4 values than the GLP-H method.
The GLP-H method yields lower ERGAS and SAM values, and higher SCC values than the GLP method for all the cases. However, it offers higher Q4 values than the GLP and GSA methods when the misalignment is not more than two pixels, lower Q4 values than the latter two when the misalignment is larger. This indicates that the GLP-H gives better performances than GLP and GSA when the misalignment is not more than two pixels.
It can also be observed that the four quality indexes decline at different rates, as the misalignment increases. The four quality indexes of the original RMI, RMI (k = 0), RMI (k = 2), RMI (k = 4), and GSA decrease the more slowly than GLP and GLP-H. This is due to the fact that the MRA methods are more sensitive to misalignments between MS and PAN bands than the CS methods. Although the curves of GSA are even gentler than those of the original RMI, RMI (k = 0), RMI (k = 2), and RMI (k = 4), GSA offers poor performances than the latter in terms of quality indexes in most cases. In conclusion, the proposed method is also robust to misalignments between MS and PAN bands, besides offering comparable quality indexes to other methods.
The original and fused images of a 300 × 300 subset of the original IK dataset with a shift of (2, 1) are shown in Figure 6. All the fused MS images were stretched to the same histogram as the corresponding original MS bands. It can be observed that the three fused products generated by RMI (k = 0), RMI (k = 2), RMI (k = 4), shown in Figure 6c-e, yield the best visual quality, followed by those produced by the original RMI and GSA. It can be observed from the yellow rectangle in Figure 6g that the GSA-fused image shows noticeable spectral distortion. In addition, aliasing effects can be observed from the fused images generated by GLP and GLP-H, as shown in the red and yellow rectangles in Figure 6h,i. Hence, visual inspection also indicates that the improved RMI is effective for reducing the impacts of misalignments between PAN and MS bands on quality of pansharpened products. The GLP-H method yields lower ERGAS and SAM values, and higher SCC values than the GLP method for all the cases. However, it offers higher Q4 values than the GLP and GSA methods when the misalignment is not more than two pixels, lower Q4 values than the latter two when the misalignment is larger. This indicates that the GLP-H gives better performances than GLP and GSA when the misalignment is not more than two pixels.
It can also be observed that the four quality indexes decline at different rates, as the misalignment increases. The four quality indexes of the original RMI, RMI (k = 0), RMI (k = 2), RMI (k = 4), and GSA decrease the more slowly than GLP and GLP-H. This is due to the fact that the MRA methods are more sensitive to misalignments between MS and PAN bands than the CS methods. Although the curves of GSA are even gentler than those of the original RMI, RMI (k = 0), RMI (k = 2), and RMI (k = 4), GSA offers poor performances than the latter in terms of quality indexes in most cases. In conclusion, the proposed method is also robust to misalignments between MS and PAN bands, besides offering comparable quality indexes to other methods. The original and fused images of a 300 × 300 subset of the original IK dataset with a shift of (2, 1) are shown in Figure 6. All the fused MS images were stretched to the same histogram as the

Discussion
The effects of misalignment on pansharpened RS images draw an increasing concern in recent years. Although the RMI method is been proposed for several years, the advantage of this method for fusion of misaligned MS and PAN bands is not explored yet. In addition, image haze is a non-negligible factor for ratio-based fusion methods, especially when image fusion is performed on remote sensing images in digital number. The determination of haze values gives an important impact on spectral distortion of fused products [32,43,44], especially for fused pixels corresponding vegetation and dark objects, such as shadows and water-bodies. This work proposed an improved version of the RMI method though two improvements. The quality indexes for the proposed method, which aims at reducing spectral distortions of fused dark pixels, did not show significant improvement in the experiment, due to a limited amount of dark pixels included by the test datasets. However, the proposed method shows consistent improvements in terms of SAM d , which is an SAM index calculated using only dark pixels. It was shown that the improved RMI method is also robust to misalignment between MS and PAN bands, as well as allowing to produce pansharpened images with sharped boundaries. This is very useful for producing high-resolution MS images covering urban regions. It is also helpful for local region mapping, image interpretation, and applications related to water-bodies and shadows. As the proposed method yield rich texture details in vegetation covered regions, it can provide comparable performances for remote sensing images covering agricultural areas and forest areas. The proposed method also can be used to medium resolution images, i.e., ASTER, Landsat 7/8, and Sentinel 2A. However, it shows more advantages for high-resolution images, as boundary information and texture details are richer in high-resolution images than in medium resolution images.
A major difference between the proposed method and other methods is that it allows producing fused products with more sharpened boundaries, through the determination of different values for k. In order to give some guidelines on how to choose the optimal value for this parameter, a fusion experiment was performed to test values ranging from 0 to 10 for k, using the degraded QB dataset. The quality indexes calculated from the fused images generated by the proposed method are reported in Table 2. It can be seen that the values of the Q2 n index descend with the increase of k. This is because more spatial details injected into the up-sampled MS bands may results in larger spectral distortion of fused products. As RMI (k = 4) provides comparable indexes to other methods (as shown in Table 1), such as GSA and GLP, we suggest that the maximum value for k can be set to 4, for the proposed method. We think this value can yield a better balance between spectral and spatial quality of fused pixels corresponding to edge PAN pixels. As introduced in Section 2.2, a threshold used to justify dark pixels were automatically determined using the product of the standard deviation of the PAN band and a scale factor S. We used a value of 0.3 for S in the experiments in this work. In order to evaluate the impact of this parameter on performances of the proposed method, values ranging from 0.1 to 1 with a step of 0.1 were tested for S in another experiment. The degraded QB dataset was used in this experiment, and the value of k is set to 0. The quality indexes obtained from fused images generated by the proposed method are shown in Table 3. The proposed method with S = 0.2 yields the highest Q2 n value. It can be seen that the performance of the proposed method gets poor along with the increases of S from 0.2. However, the changes are not significant. The fused images generated using S values ranging from 0.1 to 0.4 offer very similar Q2 n values. We suggest that the value of S can be set to 0.2, or 0.3, according to the results of this experiment. In this work, edge pixels of the PAN band were automatically identified using the CANNY detector with sensitivity thresholds that are automatically chosen. The proposed method may yield better performances if an optimal sensitivity threshold is selected for each dataset.

Conclusions
An improved pansharpening method considering image haze caused by atmospheric path radiance is proposed in this study, in order to further reduce spectral distortion of fused dark pixels and sharpen boundaries between different image objects. The improved method was compared with the original RMI, GSA, GLP, and GLP-H. The main conclusions derived from the experimental results are as follows: (1) The improved approach can reduce spectral distortions of fused dark pixels, thus the proposed method is a good choice for producing high-resolution MS images in applications related to water-bodies and shadows. (2) The improved approach can be used to obtain fused images with sharpened boundaries between different objects, through choosing a reasonable value for k. This is very useful for fused products covering urban regions, and fused products used for local region mapping or image interpretation. (3) The experiment used to evaluate the sensitivities of these method to misalignments between MS and PAN bands showed that the proposed method is more robust to misalignments between the MS and PAN bands than the other methods. These conclusions indicate that the improved method is very promising to be widely used in practical remote sensing applications.
Author Contributions: Hui Li was responsible for the research design, experiments and the analyses, wrote the manuscript. Linhai Jing provided technical guide for the proposed method, the research design, and the analyses, and reviewed the manuscript. Yunwei Tang and Haifeng Ding took part in the processing of the IK and WV-2 datasets.

Conflicts of Interest:
The authors declare no conflict of interest.