Nonsubsampled Contourlet transform (NSCT) has properties such as multiscale, localization, multidirection, and shift invariance, but only limits the signal analysis to the time frequency domain. Fractional Fourier transform (FRFT) develops the signal analysis to fractional domain, has many super performances, but is unable to attribute the signal partial characteristic. A novel image fusion algorithm based on FRFT and NSCT is proposed and demonstrated in this paper. Firstly, take FRFT on the two source images to obtain fractional domain matrices. Secondly, the NSCT is performed on the aforementioned matrices to acquire multiscale and multidirection images. Thirdly, take fusion rule for low-frequency subband coefficients and directional bandpass subband coefficients to get the fused coefficients. Finally, the fused image is obtained by performing the inverse NSCT and inverse FRFT on the combined coefficients. Three modes images and three fusion rules are demonstrated in the proposed algorithm test. The simulation results show that the proposed fusion approach is better than the methods based on NSCT at the same parameters.
Image fusion is synthesizing two or more images of the same object, which come from different sensors, into a new image. The new image can describe the object more accurately and more comprehensively. Image fusion has been widely used in military, remote sensing, robot vision, medical image processing, and other areas. Along with the developing of mathematical tools and fusion rules, the image fusion methods are continually renewing. Recently, various fusion methods based on multiscale transforms (MSTs) have been proposed and some satisfactory results have been obtained (i.e., Pyramid, Wavelet, etc.). These multiscale methods can decompose the image into low-frequency subbands and high-frequency subbands, detailed and coarse features remain in the two types of subbands; and then process separately in different subbands for different demands [
Do and Vetterli proposed a multidirection and multiresolution image expression method, namely, Contourlet transform in 2002 [
The fractional Fourier transform (FRFT) is a new transformation which develops the signal analysis into fractional time-frequency domain. It is a revolving operation of the signal Wigner distribution, and it revolves the Fourier transform or the angle Fourier transform. The introduction of the parameter
Nonsubsampled Contourlet transform (NSCT) has properties such as multiscale, localization, multi-direction, and shift invariance, which, however, only limits the signal analysis to the time frequency domain. Fractional Fourier transform (FRFT) develops the signal analysis to fractional domain and has many super performances but is unable to attribute the signal partial characteristic. Combining the merits of NSCT and FRFT to meet the high demands, a novel kind of image fusion algorithm based on FRFT-NSCT is proposed in this paper. The related theories and the flow charts of the fusion algorithm are introduced in Section
The NSCT is a shift invariant version of the Contourlet transform. The Contourlet transform employs Laplacian pyramids (LPs) for multiscale decomposition and directional filter banks (DFBs) for directional decomposition. To achieve shift invariance, the NSCT is built upon nonsubsampled pyramids (NSPs) and nonsubsampled directional filter banks (NSDFBs). The NSP is a two-channel nonsubsampled filter bank and has no downsampling or upsampling, and hence it is shift invariant. The multiscale decomposition is achieved by iterating using the nonsubsampled filter banks. Such expansion has
NSP framework.
The equivalent filters of a
The NSDFB is a shift-invariant version of the critically sampled DFB in the Contourlet transform. The building block of a NSDFB is also a two-channel nonsubsampled filter bank. To obtain finer directional decomposition, the NSDFBs are iterated. For the next level, all filters are upsampled by a quincunx matrix given by
NSDFB framework.
The equivalent filter in each channel is given by
The NSCT is obtained by combining the 2D NSP and the NSDFB as shown in Figure
Decomposition framwork of the NSCT.
The NSCT is very suitable for image fusion because it has such important properties as multiresolution, localization, shift invariance, and multi-direction. Usually the process of image fusion based on NSCT includes the following: first, get the low-frequency and high-frequency components of all scales by using multiscale and multi-direction NSCT decomposition to the image A and image B separately, and then fuse them via different fusion rules to get the combined coefficients of the fused image, finally, the fused image can be obtained by using inverse NSCT.
If
The data after performing FRFT contains both the time domain information and the frequency domain information. When
Mendlovic et al. define the fractional wavelet transform (FRWT): performing a FRFT with the fractional order
According to this idea a new image fusion method based on FRFT-NSCT is proposed. First perform fractional Fourier transformation on two source images to obtain the fractional field transformation result, and then take nonsubsampled Contourlet transform (NSCT) on it to decompose to different frequency bands and directions, obtain the fused coefficients according to some fusion rules, and finally obtain the fused image through inverse nonsubsampled Contourlet transform (INSCT) and inverse fractional Fourier transform (IFRFT). The flowchart of the image fusion method based on FRFT-NSCT is illustrated in Figure
Flowchart of the FRFT-NSCT image fusion.
Human visual perception can help judge the effects of fusion results. However, it is easily influenced by visual psychological factors. The effect of image fusion should be based on subjective vision and objective quantitative evaluation criteria. Some objective evaluation merits, such as entropy, average gradient, and standard deviation, and so forth, are employed to describe the information contained in the fused images [
(1) Information entropy (IE): the IE of the image is an important index to measure the abound degree of the image information. Based on the principle of Shannon information theory, the IE of the image is defined as
(2) Average gradient (AG): AG is the index to reflect the expression ability of the little detail contrast and texture variation, and the definition of the image. It can be expressed as
Without loss of generality, three groups of different pattern images, with the same size of 512 × 512, are employed in the following experiments.
(1)
Fusion results of the multifocus images.
Focus on the right
Focus on the left
NSCT & rule 1#
FRFT-NSCT & rule 1#
NSCT & rule 2#
FRFT-NSCT & rule 2#
NSCT & rule 3#
FRFT-NSCT & rule 3#
(2)
Fusion results of multispectrum images.
Visible image
Infrared image
NSCT & rule 1#
FRFT-NSCT & rule 1#
NSCT & rule 2#
FRFT-NSCT & rule 2#
NSCT & rule 3#
FRFT-NSCT & rule 3#
(3)
Fusion results of multimodality medical images.
CT
SPECT
NSCT & rule 1#
FRFT-NSCT & rule 1#
NSCT & rule 2#
FRFT-NSCT & rule 2#
NSCT & rule 3#
FRFT-NSCT & rule 3#
Supplementary information exists separately in the two images of every group; therefore image fusion is very suitable for processing these images.
In the process of image fusion, the choice of fusion rules is very important because it can influence the fusion results. The common fusion rules include weighted average, max select, gradient, regional energy and regional variance, and so forth. In this paper our purpose is to compare the FRFT-NSCT fusion method with NSCT fusion method under the same parameters condition, so three simple fusion rules are chosen in the experiment here. Fusion rule 1#: both the low-frequency and the high-frequency coefficients follow the average value rule. Fusion rule 2#: the low-frequency coefficients follow the average value and the high-frequency coefficients follow the largest absolute value rule. Fusion rule 3#: both the low-frequency and the high-frequency coefficients follow the largest absolute value rule.
In this section the proposed fusion algorithm based on FRFT-NSCT is compared with NSCT fusion algorithm. The parameters are set the same in the experiments. According to Section
Figures
Figures
Figures
The above is human visual perception. However, it is easily influenced by visual psychological factors. The objective evaluation criteria such as entropy, mean value and average gradient, and so forth can also judge the fusion results. Table
Evaluation criteria of NSCT-based fusion method and FRFT-NSCT-based fusion method.
Image | Fusion method | Fusion rules | Entropy | Average gradient | Fusion time (s) |
---|---|---|---|---|---|
Multifocus images | NSCT | 1# | 7.2875 | 2.8112 | 200.274324 |
FRFR-NSCT | 1# | 7.4792 | 2.9483 | 431.291303 | |
NSCT | 2# | 7.3651 | 4.0919 | 201.992649 | |
FRFR-NSCT | 2# | 7.5294 | 4.1447 | 435.006276 | |
NSCT | 3# | 7.3654 | 4.1382 | 201.088658 | |
FRFR-NSCT | 3# | 7.5316 | 4.1912 | 424.331975 | |
Visible light & infrared images | NSCT | 1# | 6.4158 | 2.0073 | 202.955161 |
FRFR-NSCT | 1# | 6.4222 | 2.5573 | 413.585567 | |
NSCT | 2# | 6.4608 | 2.7797 | 200.966261 | |
FRFR-NSCT | 2# | 6.8802 | 3.3953 | 413.735101 | |
NSCT | 3# | 6.9314 | 3.0062 | 202.404612 | |
FRFR-NSCT | 3# |
|
3.4081 | 416.433884 | |
CT&SPECT medical images | NSCT | 1# | 3.6556 | 1.1601 | 207.066354 |
FRFR-NSCT | 1# | 4.8169 | 2.2516 | 431.697082 | |
NSCT | 2# | 4.5866 | 2.0947 | 208.800631 | |
FRFR-NSCT | 2# | 4.6902 | 2.4386 | 431.882050 | |
NSCT | 3# | 3.7528 | 1.6275 | 208.549554 | |
FRFR-NSCT | 3# | 4.6902 | 2.4386 | 436.366335 |
Entropy can weigh image information abundance; the larger entropy is, the more image information contained in the fused image. Average gradient may indicate the distinct degree of an image. The larger the average gradient, the clearer fusion image is. The fusion time can measure the complexity of the algorithm. From Table
In image fusion study, the multiscale algorithm has been the main trend. In this paper, a novel fusion method based on FRFT-NSCT is proposed. The nonsubsampled Contourlet transform (NSCT) has properties such as multiscale, multi-direction, and shift invariance. The fractional Fourier transform (FRFT) develops the signal analysis into fractional domain and can reflect the signal information in the time domain and the frequency domain simultaneously. FRFT-NSCT combines the merits of the FRFT with that of the NSCT. For testing the effect of the proposed method, three groups of different pattern images are introduced as the source images, and three fusion rules are chosen. Fused images based on FRFT-NSCT can get more outline information and detail information of the source images than NSCT. The fused results demonstrate that the proposed algorithm has validity and feasibility. Further problems, such as parameter optimization, fusion rules improving, color image processing, and quick-algorithm will be discussed in the follow-up research.
This work is supported by National Natural Science Foundation of China (no. 11271106), the Plan of Scientific Research of Hebei Education Department (no. 2010218), and Open Foundation of Biomedical Multidisciplinary Research Center of Hebei University (no. BM201103).