This paper presents a kind of image fusion method based on fuzzy integral, integrated spectral information, and 2 single factor indexes of spatial resolution in order to greatly retain spectral information and spatial resolution information in fusion of multispectral and high-resolution remote sensing images. Firstly, wavelet decomposition is carried out to two images, respectively, to obtain wavelet decomposition coefficients of the two image and keep coefficient of low frequency of multispectral image, and then optimized fusion is carried out to high frequency part of the two images based on weighting coefficient to generate new fusion image. Finally, evaluation is carried out to the image after fusion with introduction of evaluation indexes of correlation coefficient, mean value of image, standard deviation, distortion degree, information entropy, and so forth. The test results show that this method integrated multispectral information and space high-resolution information in a better way, and it is an effective fusion method of remote sensing image.
The starting points for researching remote sensing image fusion at present can be divided into two categories: one category is specific to the detailed application purpose, and the other category is aimed at the integrated quality for optimizing image. The former is based on statistical theory and assorted with the understanding of ground object features on ground sample area, as well as the a priori knowledge like cognition, so as to generate fusion model with the criterion of specific application purpose and reach the purpose of distinctly identifying some ground object features. The latter mainly extract obvious feature information in image with the mathematical tools like spectral analysis, wavelet transform, and so forth, according to image imaging mechanism and its feature analysis and set up the relationship among image data feature information in different scale space, so as to form the fusion model with the purpose of optimizing image information content and comprehensive distinguishing feature. However, regardless of the remote sensing image carried out with any kind of purposes, currently adopted main method is greatly based on the fusion on pixel layer. Therefore, the process of remote sensing image fusion should be firstly realized with high accuracy geometric registration among different image pixels and then with pixel spectrum information fusion [
Different types of remote sensing images have different spatial resolution, spectral resolution, and time phase resolution. Fusion of remote sensing information is to combine their respective resolution advantages to compensate for the deficiency of some resolution on single image. At present, the information fusion methods of remote sensing image mainly are principal component analysis, mineral or vegetation index or ratio, Tasseled Cap, Multiplication Transform, Ratio Transform, IHS based Transform, and so forth. All the above methods have the problem of partial loss of image spectral information with original resolution due to its limitation, while wavelet transform can carry out image information fusion to multiple wave bands, which not only can utilize the image spatial information of high resolution, but also can keep the maximum integrality of the image spectral information with low resolution. It is also the main purpose for studying fusion technique of current remote sensing image.
Wavelet transform has sound frequency division property in transform domain, and statistical characteristics of wavelet sparse reflected the obvious features of remote sensing image like edge, line, and domain, and so forth. Coefficient of low frequency of multispectral image is kept to carry out optimized fusion to the high frequency part of two images in accordance with weighting coefficient, so as to create new fusion image. The test carried out with remote sensing image data proved the effectiveness of this algorithm. The method for integrating the two single factors of spectral information and spatial resolution and the determination of the optimal point during the course of iterative optimization are still the problems demanding prompt solution.
This paper presented an image fusion method based on fuzzy integral specific to multispectral and high-resolution image fusion problem. This method effectively integrated two single factor indexes of spectral information and spatial resolution and carried out pixel-level optimal fusion and introduced fuzzy integral method, which can conveniently and efficiently determine the optimal weight number.
The detailed steps of image fusion method based on fuzzy integral are as follows [ respectively, register high resolution image spot into three wave bands of multispectral image dmtm; extract the three wave bands of multispectral image dmtm; respectively, carry out 3 layers of wavelet transform to extract its coefficient of low frequency; carry out 3 layers of wavelet transform to high resolution image spot, so as to extract its coefficients of high and low frequency; carry out infusion to images in various wave bands in accordance with fusion, and image after fusion can be obtained after wavelet transform; rectify weighting coefficients efficiently determine the optimal value of fusion image can be obtained after integrating the fusion image of the three wave bands.
The fusion rules of multispectral image dmtm and high resolution image spot after 3 layers of wavelet decomposition are as follows: extract the 3rd layer of coefficient of low frequency of multispectral image; determine the fusion value of high frequency coefficient according to the following equation, so as to carry out pixel-level fusion:
carry out the 3rd layer of wavelet inverse transform according to the 3rd coefficient of low frequency of multispectral image and the high frequency coefficient after being processed; take the image value of the 3rd layer of wavelet after inverse transform as the coefficient of low frequency of the 2nd layer of wavelet transform, and the high frequency coefficient is calculated according to ( carry out the 1st layer of wavelet inverse transform according to Steps
wherein
Correlation degree of fusion image and multispectral image is used to identify the evaluation index of spectral information. Set
Corresponding correlation degree of fusion image between high-frequency component of gray scale and high-frequency component of high-resolution image is used to identify index under spatial resolution [
The key point of comprehensive evaluation with the utilization of fuzzy integral is the definition of fuzzy measure
Sequencing is carried out to
According to the definition of fuzzy integral, it can obtain
There are the following values. When When When
During above inferring process, the operation with the utilization of “
MATLAB software is selected as a test tool in this test. Dmtm image and spot image are selected for the test to satisfy that the weighting coefficient of the optimal objective function is the intersection point of
According to the two items of feature evaluation indexes defined in (
The detailed test process is as follows: respectively, register high-resolution image in the three wave bands of multispectral image dmtm; respectively, carry out wavelet decomposition to the three wave bands of dmtm, so as to extract its coefficient of high and low frequency; carry out wavelet decomposition to the registered spot image, so as to extract its coefficient of high and low frequency; determine the optimal high-frequency coefficient with fuzzy integral; carry out wavelet reverse transform layer by layer; integrate the fusion images of the three wave bands.
The determination process of the maximum value of the fuzzy integral is as follows: firstly, calculate value
In order to prove the effectiveness of remote sensing image fusion method based on fuzzy integral, this paper selects fusion test with a 1024
Original TM (multispectral) image.
Original SPOT (high resolution) image.
Fusion image based on fussy integral.
Image after fusion of combinatorial algorithm based on energy selection and wavelet transform.
Image after fusion of combinatorial algorithm.
Image after fusion based on PCA.
Image after fusion based on IHS transform.
During the process of determining fuzzy measure value, the values of fuzzy measure
|
|
|
|
|
---|---|---|---|---|
The 1st wave band | 0.457 | 0.9519 | 0.9519 | 0.9520 |
The 2nd wave band | 0.459 | 0.9511 | 0.9511 | 0.9514 |
The 3rd wave band | 0.487 | 0.9379 | 0.9380 | 0.9379 |
|
|
|
|
|
---|---|---|---|---|
The 1st wave band | 0.456 | 0.9518 | 0.9518 | 0.9523 |
The 2nd wave band | 0.459 | 0.9511 | 0.9511 | 0.9514 |
The 3rd wave band | 0.487 | 0.9379 | 0.9380 | 0.9379 |
|
|
|
|
|
---|---|---|---|---|
The 1st wave band | 0.46 | 0.9510 | 0.9524 | 0.9510 |
The 2nd wave band | 0.459 | 0.9511 | 0.9511 | 0.9514 |
The 3rd wave band | 0.487 | 0.9379 | 0.9380 | 0.9379 |
It can be seen from above tables that the optimal weighting coefficients determined in the 2nd and 3rd wave bands determined by different
The performance parameters mainly can be divided into three categories: Category I means spectral preservation degree, such as distortion degree, deviation index, and correlation coefficient; Category II reflects expressive ability of spatial detail information, such as variance, information entropy, cross entropy, and definition; Category III reflects image brightness information, such as mean value [
As for some image,
Mean value of multispectral image and fusion image.
Mean value | Multispectral image | Fusion image |
---|---|---|
The 1st wave band | 48.0060 | 53.8250 |
The 2nd wave band | 62.4486 | 67.1111 |
The 3rd wave band | 83.7475 | 102.0046 |
Variance of multispectral image and fusion image.
Variance | The 1st wave band | The 2nd wave band | The 3rd wave band |
---|---|---|---|
Multispectral image | 26.2903 | 29.7145 | 24.4353 |
Image after fusion | 27.5264 | 30. 6260 | 24.4566 |
Energy selection method | 24.5099 | 27.5120 | 22.4653 |
Combined method of maximum and consistency detection of absolute value | 21.9932 | 24.0895 | 19.4255 |
Combined method of PCA and wavelet transform | 26.2368 | 31.3601 | 25.0627 |
It can be seen from the table that the standard deviation of fusion image is higher than that of multispectral image, which shows that its contained information volume is more than that of multispectral image.
Comparing this test method with the combined method of maximum and consistency detection of absolute value, the variances of all the wave bands are larger than those of this method, while the larger standard deviation means that the probabilities of occurrence of all the grey levels in images more and more tend to be same, and the included information volume more and more tends to be the maximum; therefore, the information contained in the image after fusion in this test is more than the information volume of the fusion method based on the maximum and consistency detection of absolute value. Similarly, the variance data of this test is also larger than that of the energy selection method, and the contained information volume is also more than this method.
Comparing this test method with combined method of PCA and wavelet transform, the variance is slightly less than the value obtained in this method, which shows that the contained information volume is also slightly less than that of the image obtained in this method.
Correlation coefficient in this test.
Correlation coefficient | Multispectral image and fusion image | High-resolution image and fusion image |
---|---|---|
The 1st wave band | 0.9519 | 0.9125 |
The 2nd wave band | 0.9511 | 0.9089 |
The 3rd wave band | 0.9380 | 0.7421 |
It can be seen from Table
Compared with the data in Tables
Combined method of maximum and consistency detection of absolute value.
Correlation coefficient | Multispectral image and fusion image | High-resolution image and fusion image |
---|---|---|
The 1st wave band | 0.8877 | 0.9416 |
The 2nd wave band | 0.8785 | 0.9381 |
The 3rd wave band | 0.8500 | 0.7913 |
Combined method based on energy selection and wavelet transform.
Correlation coefficient | Multispectral image and fusion image | High-resolution image and fusion image |
---|---|---|
The 1st wave band | 0.8593 | 0.9592 |
The 2nd wave band | 0.8553 | 0.9566 |
The 3rd wave band | 0.8023 | 0.8159 |
Spectral distortion degree.
Spectral distortion degree | The 1st wave band | The 2nd wave band | The 3rd wave band |
---|---|---|---|
Combination of fuzzy integral and wavelet transform | 7.7653 | 7.7870 | 18.6638 |
Combination of maximum absolute value and consistency detection | 13.3177 | 18.7888 | 18.9686 |
Combination of PCA and wavelet transform | 28.7838 | 16.2149 | 8.1825 |
It can be seen from Table
Compared with the combined method of PCA and wavelet transform, firstly, the distortion of the 2nd wave band is small, which shows that the spectral information is kept well, while that of the 3rd wave band is obviously larger than that of this method, which shows that the 3rd wave band is slightly distorted.
It is easy to know
Information entropy.
Information entropy | The 1st wave band | The 2nd wave band | The 3rd wave band |
---|---|---|---|
Based on fuzzy integral and wavelet transform | 4.5291 | 4.7003 | 4.5535 |
Combination of energy selection and wavelet transform | 4.4169 | 4.6055 | 4.4781 |
Combination of maximum absolute value and consistency detection | 4.3177 | 4.4789 | 4.3436 |
Combination of PCA and wavelet transform | 3.9297 | 4.5087 | 4.3084 |
It can be seen from Table
The following conclusions can be obtained through research and test in this paper. The optimal image fusion method based on fuzzy integral presented in this paper combined the features of fuzzy integral and wavelet transform to carry out pixel-level optimal fusion to high-frequency coefficient, introduced fuzzy integral method to effectively integrate the spectral information and the 2 single factor indexes of spatial resolution, conveniently and efficiently determined the optimal weight, so as to enable the image after fusion to reach the maximum spatial resolution, reduced color distortion in maximum degree, which balanced the two feature indexes of spatial detail information and spectral information in fusion effect, and effectively improved the spectral information index of fusion image. When the optimal fusion of multi-index is considered, the utilization of fuzzy integral can effectively integrate the multi-index factors. The feature for fuzzy integral of integrated multifactor can be used in fusion problem of multispectral and high-resolution images, and the results will be more suitable for subjective feeling of people to fusion image. It is presented in this paper that the utilization of fuzzy integral can efficiently determine the optimal weight, which enables calculated amount of the algorithm to be less, and the fusion effect can satisfy the requirements in most occasions with certain use value.
In addition, in the methods presented in this paper, high-frequency coefficient is determined by optimal weight, which is based on pixel-level fusion method. The optimal fusion method for remote sensing image based on wavelet statistical property can be considered: in IHS space, wavelet fusion method is used for the high-frequency part with intensity component I to carry out high-frequency detailed feature fusion of high-frequency subband. Fuzzy integral is used for the low-frequency part to carry out fusion. Such kind of detailed processing to high-frequency part should receive better effect. Wavelet transform method is an effective algorithm in current fusion field. There are questions of how to determine the order of wavelet transform and carry out fusion of coefficients within wavelet domain in a most effective way. These contents are crucial to image fusion, which should be further researched.
The authors declare that there is no conflict of interests regarding the publication of this paper.