Pansharpening with the Multidirection Tree Ridgelet Dictionary

. In this work, we propose a novel pansharpening method based on the multidirection tree ridgelet dictionary. A pansharpened image has a wide-ranging application area, such as object detection, image segmentation, feature extraction, and so on. Remote sensing (RS) imagery contains more abundant information on surface features. In order to represent different object information, we use three main classes of different dictionaries, which can reveal the latent structure of RS image. First, RS imagery is divided into several blocks. Each block is classified as smooth, irregular, or multidirection categories. Different categories are sparsely represented in different dictionaries. Second, the smooth blocks are sparsely represented in the discrete cosine transform (DCT) dictionary. The irregular and the multidirection blocks are sparsely represented in the KSVD and multidirection tree ridgelet (MDTR) dictionary, respectively. Finally, we can obtain the fusion image by reconstructing those blocks. Some experiments are taken on three different datasets acquired by QuickBird, GeoEye, and IKONOS satellites. Experimental results show that the proposed method can reduce spectral distortion and enhance spatial information. Meanwhile, numerical guidelines outperform some related methods.


Introduction
Pansharpening refers to the fusion of a panchromatic (PAN) image and a multispectral (MS) or hyperspectral (HS) image.PAN has high spatial resolution but low spectral resolution.MS or HS has high spectral resolution but low spatial resolution.e fused imagery, which is unachievable by a single sensor, has not only high spatial resolution but also high spectral resolution, and the fused image can protect more detailed information for the downstream object detection, image segmentation, and feature extraction.Many fusion measures have been proposed.Generally, these measures can be divided into four main categories [1].
In the DM category, image fusion is the degradation problem.ese methods require additional priors to regularize the solution space.Li and Yang [7] proposed a new fusion method based on compressed sensing, which used the MS and PAN images through a KSVD [22] dictionary.Zhu and Bamler proposed the SparseFI method [8].In SparseFI method, the atoms of coupling dictionaries came from the PAN image.In Ref. [9], the PN-TSSC method was proposed followed by SparseFI, and the PN-TSSC method was a twostep sparse coding method.In the study in Ref. [10], the reconstruction of the fusion imagery was achieved by using the identical sparse representation coe cients on the coupling dictionaries.Wang et al. [11] proposed a novel hybrid dictionary to fuse the RS imagery.However, those above methods had ignored the internal structural diversity information of RS imagery.Li et al. [12] proposed a novel pansharpening method with NSCT and HSAE, in which the detailed information was hierarchically injected into the MS image.In Ref. [13], Zhang et al. put forward a new RS fusion method, in which multiscale convolution sparse decomposition was used to extract more subtle feature.e similarity is important information in image processing.us, pansharpening based on similarity was proposed [14][15][16].Li et al. [14] proposed a local geometrical similarity-based method to capture detailed information.In Ref. [15], the similarity was obtained by a local adaptive sparse representation metric.Zhang et al. [16] proposed a spatial weighted neighborhood embedding method sharing the similar manifold structure.Nonnegative matrix factorization is a noteworthy method in RS image fusion.Nonnegativity is also used to reduce the ill-posedness of the spectral and spatial degradation models.In Ref. [17], semi-NMF based pansharpening was proposed to improve the image effect.DM methods enhance the image quality; however, the complexity is high for these methods.
Recently, the DNN methods are arousing more attention.In Ref. [18], the salient features can be extracted through two branches DNNs.e image feature, extracted from the convolutional layers, yielded the fusion result.In Ref. [19], the Pan-GAN model was proposed, and this method did not rely on the ground truth.Zhang et al. [20] proposed an SSE network-based pansharpening.In this paper, AFFMs were used to merge image features through information content.NLRNet was proposed for RS image fusion [21].e author proposed the ENLA mechanism and ReZero technology to spread the signal easily, and the SpecAM was used to adjust the spectral information.However, DNN methods show that the training time is long and the sample size is large.
RS image contains various types of ground objects.Multiscale analysis tools have been used to capture image orientation information, and researchers have also demonstrated their excellent performance in processing geometric information.Based on this, we propose a novel pansharpening with a multidirection tree ridgelet dictionary to represent the diverse information of RS image.First, RS imagery is divided into several blocks.Each block is classified as smooth, irregular, or multidirection categories.Different categories are sparsely represented in different dictionaries.Second, we construct the discrete cosine transform (DCT), KSVD, and multidirection tree ridgelet (MDTR) dictionaries.Smooth and irregular blocks are sparsely represented in DCT and KSVD dictionaries, respectively, and the direction blocks are sparsely represented in the MDTR dictionary.e contribution of this work is as follows: (1) the multidirection ridgelet dictionary can be constructed via discretized parameters of ridgelet function; (2) spatial details can be captured by sparsely encoding patches in DCT, KSVD, or MDTR dictionaries.e proposed approach has performed three different datasets acquired by the QuickBird, GeoEye, and IKONOS.e experimental results show that the proposed MDTR method can outperform its counterparts.e remainder of this paper is structured as follows.e construction of the MDTR dictionary is depicted in Section 2. Section 3 presents the pansharpening method by taking the MDTR dictionary.Some experiments on different datasets are performed in Section 4. Finally, Section 5 concisely reviews conclusions.

Construction of the Multidirection Tree Ridgelet Dictionary
One of the major problems in sparse representation is the construction of dictionary.In view of the inherent diversity of RS image and other characteristics, in our paper, we construct three different dictionaries, namely, DCT [12], KSVD [22], and MDTR dictionaries.e DCT dictionary can represent the smooth region.e KSVD dictionary is used to learn the information of the irregular blocks adaptively.e MDTR dictionary is used to sparsely represent direction blocks.Ridgelet scale function is 8 .e prototype of the ridgelet dictionary is where x � a(x , a is the scale parameter, b is the location parameter, and θ is the direction parameter. We can obtain the ridgelet dictionary via discretization of a, b, and θ. Figure 1 shows the ridgelet function with different parameters.Figure 1 e ridgelet dictionary can be obtained by discretized three parameters (a, b, and θ).Lin et al. [23] proposed the collaborative compressed sensing reconstruction method.Authors proposed that the natural images were reconstructed by using the overcomplete ridgelet dictionary.It can be seen that the overcomplete ridgelet dictionary shows its advantages in image processing, which can maintain the structure and edge information of the image.However, the large-scale overcomplete ridgelet dictionary increases the complexity of the algorithm.In our work, we propose a novel fusion method with the multidirection tree ridgelet dictionary.e multidirection tree ridgelet dictionary shows superior fusion results than the global ridgelet dictionary and requires less time complexity.Figure 2 shows ridgelet dictionaries.Figure 2

Image Fusion Using the Multidirection Tree Ridgelet Dictionary
In order to represent the structural diversity, image blocks are classified into different categories, e.g., smooth category, irregular category, or multidirection category.Figure 3 shows the classification results of blocks.3.1.Smooth Category.Set the threshold, Thres 1 .Calculate the variance, std, of each block.If the variance is less than the threshold (std < Thres 1 ), then the block is classi ed into the smooth category.For smooth blocks, the DCT dictionary, D DCT , is used for sparse representation.

Irregular Category.
For each pixel of a block, we calculate the gradient in x and y direction.en the gradient can be de ned as en, the gradient matrix is decomposed by singular value decomposition: R can be calculated as Set the threshold, Thres 2 .If R < Thres 2 , then the block is classified as the irregular category.Otherwise, the block can be considered multidirection category.For irregular blocks, the KSVD dictionary (D KSVD ) is used for sparse reconstruction.Mathematical Problems in Engineering

Multidirection Category.
In equation ( 4), the first column (v 1 ) of the matrix V is named as the first principal direction.e second column (v 2 ) of the matrix V is the second principal direction.e angle of the block is determined as e MDTR dictionary is represented as D n Rig , where n(n � 1, . . ., 18) is the number of D n Rig .e angle range is set as [0 0 , 180 0 ]; that is to say, there are 18 groups ridgelet dictionaries, and the interval of each group is 10 0 .e angle of D 1  Rig is from 0 0 to 10 0 .e angle of D 2 Rig is from 11 0 to 20 0 .e angle of D 3  Rig is from 21 0 to 30 0 and so forth.For multidirection blocks, D n Rig is used for sparse reconstruction.e fusion method is summarized in Algorithm 1.
e best results is highlighted in bold.Table 1 shows the quality evaluation indexes.
4.1.QuickBird Dataset.QuickBird has been acquired in Xi'an, China.e resolution is 0.61 m for the PAN image and 2.44 m for the MS image.Figure 4 shows all the fused results.Figures 4(a)-4(c) are MS, PAN, and reference MS images, respectively.Figures 4(d)-4(j) are the results obtained by employing the seven different fusion methods.In Figures 4(d)-4(j), the first row is the fused image, and the second row is the difference image between the fused image and reference image.In addition, we display the magnified area for each fused image.
We can see that the spatial information is improved for all the fused images.e fusion result of the PCA method is darkness.For the difference image, we can see that the result of CT is better than the result of GIHS.For the magnified  Mathematical Problems in Engineering  Mathematical Problems in Engineering (       area of the PN-TSSC method, the color information is lost seriously.In the SparseFI, NMF, and proposed method, the spatial information is close to the ground truth.However, the spectral improvements achievable can be easily remarked for our method.Table 2 shows the numerical values of fusion results.We can see that the best Q 4 is obtained from the NMF method, and the MDTR method provides the best CC, UIQI, RMSE, SAM, and ERGAS.

GeoEye Dataset.
e fused results of GeoEye dataset is exhibited in this portion.e PAN image is 0.5 m resolution.
e MS image is 2.0 m resolution.Figure 5 shows the result of different measures.Figures 5(a)-5(b) are MS and PAN images, respectively.Figure 5(c) is the reference MS image.In addition, we analyze the difference image.In Figure 5(d), we can see that the spatial details preserve well, but the result is darkness.In Figures 5(e) and 5(f ), there are slight spectral distortion and ringing artifacts.In Figure 5(g), the color information is poor.In Figures 5(h) and 5(i), the spatial details are enhanced although the spectral information is slight distortion.From the visual effect, the result from the MDTR method is close to the ground truth, and the missing information is less.Meanwhile, we can see that the difference image information of MDTR method is less than that of other compared methods.
Table 3 shows the results of each assessment indexes.e best UIQI, SAM, and ERGAS are given by the MDTR method.For CC and RMSE, the best values are produced by SparseFI, and the best Q 4 performs the NMF method.

IKONOS Dataset.
In this section, IKONOS dataset are tested.IKONOS dataset is 1 m resolution and 4 m resolution in PAN and MS images, respectively.e fused result is shown in Figures 6(d)-6(j).e enlarged area in the red rectangles appears in the fused result, and the difference image is illustrated in the second row.All the fusion images provide enhanced spatial information.But the result is darkness for the PCA method.For GIHS, CT, PN-TSSC, and NMF methods, fused images has inferior performance in terms of the spatial information.For the proposed method, spatial details are maintained well, and the fused image is close to the reference image.For the enlarged region, the fused image of the MDTR method is better than the other fused images.From Table 4, the best SAM is from the NMF method.Other best indexes are from the MDTR method.

Conclusions
In this paper, we propose a new pansharpening with the multidirection tree ridgelet dictionary.Our method is assessed on three datasets.e contribution of our work is to construct the multidirection tree ridgelet dictionary, which can capture the different directional information of blocks.
e spatial and spectral quality of the fused image is evaluated by six different indexes.Experimental results show that our proposed method can supress the color distortions in the fused image.Meanwhile, our method can produce satisfactory performance in both visual comparison and numerical evaluations.e future work will perform DNN and tensor-based methods to explore the spatial information and maintain image spatial relationships.

Mathematical Problems in Engineering
(a) is the ridgelet function.And Figures 1(b)-1(e) are the result of different parameters.

Table 1 :
e quality evaluation indexes.