A hybrid multiscale and multilevel image fusion algorithm for green fluorescent protein (GFP) image and phase contrast image of Arabidopsis cell is proposed in this paper. Combining intensity-hue-saturation (IHS) transform and sharp frequency localization Contourlet transform (SFL-CT), this algorithm uses different fusion strategies for different detailed subbands, which include neighborhood consistency measurement (NCM) that can adaptively find balance between color background and gray structure. Also two kinds of neighborhood classes based on empirical model are taken into consideration. Visual information fidelity (VIF) as an objective criterion is introduced to evaluate the fusion image. The experimental results of 117 groups of Arabidopsis cell image from John Innes Center show that the new algorithm cannot only make the details of original images well preserved but also improve the visibility of the fusion image, which shows the superiority of the novel method to traditional ones.

The purpose of image fusion is to integrate complementary and redundant information from multiple images of the same scene to create a single composite that contains all the important features of the original images [

Arabidopsis cell images.

GFP image

Phase contrast image

The outline of this paper is as follows. In Section

In 2005, Do and Vetterli [

The block diagram of the Contourlet transform with two levels of multiscale decomposition is shown in Figure

Block diagram of Contourlet transform with 2 levels of multiscale decomposition.

Dual iterative form

Equivalent parallel form

Frequency division

Due to the periodicity of 2D frequency spectrums for discrete signals and intrinsic paradox between critical sample and perfect reconstruction of DFB, it means that we cannot get perfect reconstruction and frequency domain localization simultaneously by a critically sampled filter bank with the frequency partitioning of the DFB. When the DFB is combined with a multiscale decomposition as in the Contourlet transform, the aliasing problem becomes a serious issue. For instance, Figure

Frequency support of one channel for Contourlet transform and desired scheme.

Contourlet

Desired

In order to overcome the aliasing disadvantage of Contourlet transform, Lu proposed a new construction scheme which employed a new pyramidal structure for the multiscale decomposition as the replacement of LPT [

Block diagram of SFL-CT.

In the diagram,

Figure

Comparison of basis image.

Frequency support of Contourlet

Frequency support of SFL-Contourlet

Spatial basis image of Contourlet

Spatial basis image of SFL-Contourlet

The intensity-hue-saturation (IHS) transform substitutes the gray image for the intensity component of the color image and thus handles the fusion of the gray and color images [

There are various algorithms that can transform image from RGB to IHS space, common transformation model including sphere transformation, cylinder transformation, triangle transform, and single six cones [

From RGB to IHS space (forward transform),

The reverse transform

From Figure

Schematic diagram of the proposed image fusion algorithm, where subscripts

When GFP image and phase contrast image are decomposed by the SFL-CT, the coefficients of the coarsest subband represent the approximation component of the input images. Considering approximate information of fused image is constructed by the two kinds of approximation subband coefficients; maximum region energy rule (MRE) is a good choice for the fused approximation subband coefficients.

MRE rule is defined as follows:

After decomposing the input images using SFL-CT, the image details are contained in the directional subbands in SFL-CT domain. The directional subband coefficients with larger absolute values, especially for subband coefficients at the finest scale, generally correspond to pixels with sharper brightness in the image and thus to the salient features such as edges, lines, and regions boundaries. Therefore, we can use the maximum absolute value (MAV) scheme to make a decision on the selection of coefficients at the finest detailed subbands.

MAV fusion rule is defined as follows:

Let

The NCM is defined as a threshold for directional coefficients based on one region mentioned above. Let

It is not hard to see that the NCM is smaller than 1. In fact, NCM indicates whether the neighborhood is homogenous. Bigger NCM means being more homogenous.

Taking the number of directions in each detailed subband into consideration, we classify neighborhood into two classes: Nhd I and Nhd II which are shown in Figures

Neighborhood coefficients of SFL-CT.

Nhd I

Nhd II

Empirical distribution model for neighborhood selection.

8 directions

16 directions

We define a threshold

If

If

The fusion process, accompanied with the proposed fusion rule, is carried out as in the following steps.

Define the register original images: GFP image as image

Make IHS transform for image

Decompose

Combine transform coefficients according to the selection rule: coefficients of approximation subbands are based on MRE rule; coefficients of the finest detailed subbands are based on MAV rule; coefficients of other detailed subbands are based on NCM. We can get the approximation subband

Reconstruct the intensity of the fused image

Reconstruct the fused image

All images used in this experiment come from the GFP database of John Innes Center [

For the proposed method, the practical windows (Ω) in NCM rule are usually chosen to be of size

We compare the proposed fusion rule with the traditional methods or rules. They include traditional IHS fusion method (T-IHS) [

The fusion results, shown in Figure

Fused images using different methods.

GFP image

Phase contrast image

T-IHS

IHS + MRE and MAV

NSCT + PCNN

Hybrid NCM

Considering the difference in function orientation of the two kinds of images, especially the corresponding relationship between the fluorescence area in GFP images and the protein distribution in cells, the fluorescence area is firstly extracted from the original two images, then the VIF between fused image and phase contrast image is calculate, and thirdly the VIF between fused image and fluorescence image is calculated too. Fused image should keep high similarity with both phase contrast image and fluorescence image in fluorescence area. However, in the other area, only the similarity between it and the phase contrast image is considered. Therefore, this paper first segments both fused image and source image into fluorescence area and nonfluorescence area with Otsu method [

VIF computing result.

Fusion methods | VI |
VI |
VI |
---|---|---|---|

T-IHS | 0.4368 | 0.2759 | 0.4975 |

IHS + MRE and MAV | 0.3119 | 0.8318 | 0.8318 |

NSCT + PCNN | 0.3188 | 0.8992 | 0.8992 |

Hybrid NCM | 0.3112 | 0.9299 | 0.9299 |

VIF algorithm flow chart.

In the table, superscript fl refers to the fluorescent area while nfl refers to nonfluorescence area,

From Table

VIF distribution histogram of 117 groups of Arabidopsis cell fusion image is shown in Figure

117 groups of VIF distribution histogram of Arabidopsis cell fusion image.

This paper proposes a hybrid multiscale and multilevel image fusion method based on IHS transform and SFL-CT to balance the gray structural information and molecular distribution information for the fusion of GFP image and phase contrast image. In manner of SFL-CT’s advantage of directional and excellent detailed expression ability, we use SFL-CT to decompose the intensity components of both GFP image and phase contrast image, and different fusion rules are utilized for coefficients of different subbands in order to keep the localization information in GFP image and detailed high-resolution information in phase contrast image. Visual information fidelity (VIF) is introduced to assess the fusion result objectively which quantifies the similarity inside and outside the fluorescent area between the fused image and original images. The experiment fusion results of 117 groups of Arabidopsis cell images from John Innes Center demonstrate that the new algorithm can both make the details of original images well preserved and improve the visibility of the fusion image and also show the superiority of the novel method to traditional methods. Although the results of the proposed method and NSCT + PCNN look similar, the former is much better in line with the image of fused image similarity degree which means that this algorithm has made full use of the advantages of SFL-CT to keep the structural information of the phase contrast image effectively. The complexity of the algorithm is obviously lower than the latter and more advantageous to the actual application.

It is also needed to point out that from the experiment we find that

This paper is partially supported by the Fundamental Research Funds for the Central Universities (no. CDJZR11120006).