Estimation of Large Scalings in Images Based on Multilayer Pseudopolar Fractional Fourier Transform

Accurate estimation of the Fourier transform in log-polar coordinates is a major challenge for phase-correlation based motion estimation. To acquire better image registration accuracy, a method is proposed to estimate the log-polar coordinates coefficients using multilayer pseudopolar fractional Fourier transform (MPFFT). The MPFFT approach encompasses pseudopolar and multilayer techniques and provides a grid which is geometrically similar to the log-polar grid. At low coordinates coefficients the multilayer pseudopolar grid is dense, and at high coordinates coefficients the grid is sparse. As a result, large scalings in images can be estimated, and better image registration accuracy can be achieved. Experimental results demonstrate the effectiveness of the presented method.


Introduction
Motion estimation is of great importance to many image processing and computer vision applications.It plays a vital role in image mosaic, image compression, video enhancement, scene representation, and so forth.[1][2][3][4][5][6].Numerous image registration techniques have been developed throughout the years [1,[7][8][9].These methods can be loosely divided into the following groups: algorithms that use image pixel values directly, for example, correlation methods [10]; algorithms that use the frequency domain, for example, Fourier-based methods [11][12][13][14]; algorithms that use low-level features such as edges and corners, for example, feature-based methods [15]; and algorithms that use high-level features such as identified (parts of) objects or relations between features [16], for example, graph-theoretic methods [17].Fourier-based methods, which take advantage of fast Fourier transform (FFT), are among the most popular.
The theory background of Fourier-based image registration is described in [18,19].Based on the shift property of Fourier transform, the translation can be robust-estimated by phase correlation.In order to account for rotations and scaling, the image is transformed into a uniform polar or logpolar Fourier representation.As a result, rotations and scalings are reduced to translations, which can be estimated by phase correlation again.It has been shown that Fourier-based methods are robust to noise and time varying illumination disturbances [20,21].
In the Fourier-based method, the largest challenge is to evaluate the log-polar Fourier transform efficiently and accurately.FFT is commonly defined on the Cartesian grid.To calculate the log-polar coordinates coefficients, one might calculate the FFT coefficients on the Cartesian grid and then interpolate them to the log-polar grid (see [19] for more details).Unfortunately, such an interpolation process is not accurate enough and results in significant errors.Based on the technique of pseudopolar fractional Fourier transform (PPFFT) [11], a method is developed in [20] to estimate large translations, rotations, and scaling in images.It is demonstrated in [20] that a more accurate estimation of registration parameters can be achieved by PPFFT compared to the method given in [19].However, the PPFFT method Mathematical Problems in Engineering [20] can only recover scales up to 4. Recently, Pan et al. [21] present a novel method called multilayer fractional Fourier transform (MLFFT) for the fast and accurate polar/log-polar Fourier transforms.The advantages of MLFFT over other fast log-polar Fourier transform methods have been proven theoretically and experimentally in [21].
Though PPFFT and MLFFT have tremendous improvements in the fast polar/log-polar Fourier transform, they are still unsatisfying in motion estimation because they are not suitable for the log-polar Fourier transform.PPFFT pays more attention to the reduction of angular differences between the constructed grids and the uniform log-polar grids, and the radial differences have been ignored.On the other hand, MLFFT pays more attention to the reduction of radial differences between the constructed grids and the uniform log-polar grids, and the angular differences have been ignored.
To acquire better image registration accuracy, a method is proposed to estimate the log-polar coordinates coefficients using multilayer pseudopolar fractional Fourier transform (MPFFT).It encompasses PPFFT and MLFFT and provides a grid which is geometrically similar to the log-polar grid.The multilayer pseudopolar grid is dense at low coordinates coefficients and sparse at high coordinates coefficients.We compare the approximation accuracy of the log-polar Fourier transform using the PPFFT, MLFFT, and MPFFT.It is shown that accumulated distances of MPFFT are lower than those of PPFFT and MLFFT.As a result, large scalings in images can be estimated, and better image registration accuracy can be achieved.Experimental results demonstrate the effectiveness of the presented method.
The rest of the paper is organized as follows.Prior techniques related to FFT-based image registration are given in Section 2. In Section 3, MPFFT is described, and the accuracy of MPFFT is also analyzed.Experimental results are discussed in Section 4, and final conclusions are given in Section 5.

Basic Theory of Fourier Domain
2.1.Phase Correlation.Let  1 and  2 be two images that differ only by a displacement ( 0 ,  0 ), that is, Their corresponding Fourier transforms  1 and  2 will be related by The cross-power spectrum of two images  1 and  2 with Fourier transforms  1 and  2 is defined as follows: where  * 2 is the complex conjugate of  2 ; the shift theorem guarantees that the phase of the cross-power spectrum is equivalent to the phase difference between the images.By taking inverse Fourier transform of the representation in the frequency domain, we will have a function that is an impulse; it is approximately zero everywhere except at the displacement that is needed to optimally register the two images [19].

Log-Polar Coordinates Translation.
Let  2 be a translated, rotated, and scaled replica of  1 : where  0 , , and ( 0 ,  0 ) are the rotation angle, scale factor, and translation parameters, respectively.The Fourier transform of (4) in polar coordinate systems is where   =  cos ,   =  sin .Let  1 and  2 be the magnitudes of Î1 and Î2 .Then, we have Letting  = log ,  = log , (7) can be written as Due to the fact that the Fourier spectrum is conjugate symmetric for real sequences, only two quadrants of the Fourier spectra of images are used to map them to the logpolar plane for estimation parameters.These parameters can be recovered by phase correlation using (8) [22].
Obviously, it is hard for us to calculate the discrete Fourier magnitudes of images in the log-polar coordinates.2D FFT is used in traditional algorithms to approximate the log-polar Fourier representation of images.This will introduce large interpolation errors.

Multilayer Pseudopolar Fractional Fourier Transform
3.1.Fractional Fourier Transform.The proposed algorithm is based on the fractional Fourier transform.As in [20], the centered fractional Fourier transform is employed in the proposed method.Given a vector  = (), −/2 ≤  ≤ /2, the fractional Fourier transform is defined as follows: When  = 1, the fractional Fourier transform is equal to the discrete Fourier transform; thus, we get the values of the  + 1 frequencies that are distributed uniformly in − ≤  ≤  in the frequency domain.When 0 <  < 1, the fractional Fourier transform returns the values of the +1 frequencies that are scattered uniformly in − ≤  ≤  in the frequency domain.
In this paper, we only discuss the case of  = 1/2 −1 , where  ∈  + .

Multilayer Pseudopolar Grid.
In this paper, we estimate motions of images using MPFFT.MPFFT is defined on multilayer pseudopolar grid.The set of multilayer pseudopolar grid points (MP  ) is defined as follows: where ,  ∈  + , ,  ∈ , and  = {1, 2, 3, . . ., } (e.g., see Figure 1(d)).If  = 1, this grid is the same as the pseudopolar grid.MP  can be viewed as a composite of pseudopolar grid and multilayer grid, but it is denser than them.Let log  be the set of log-polar grid points, PP the set of pseudopolar grid points, and ML  the set of -layer Cartesian grid points: , ,  ∈ } , where ,  ∈  + ,  = {1, 2, 3, . . ., }. Figure 1 shows examples of log , PP, and ML  .We note that if  = 1, this ML  grid is the same as the Cartesian grid.
From Figure 1, we observed that the multilayer pseudopolar grid is much closer to the log-polar grid than to the pseudopolar grid and the multilayer grid.

The Interpolation Error.
In this section, the interpolation errors of PPFFT, MLFFT, and MPFFT will be discussed.
For a certain image's spectrum magnitude , the interpolation error is defined as follows [20]: By calculating (13) with such a constant max {|∇(  ,   )|}, we are able to compare the max interpolation errors of different transform methods without considering the signal itself [21].
In the rest of this paper, we only consider two-layer MP (MP 2 ).Other MPs can be discussed similarly.
We denote  as the set of Cartesian grid points: Let ML 2 1 be the set of the first layer Cartesian grid points in ML 2 : Let ML 2 2 be the set of the second layer Cartesian grid points in ML 2 .In other words, ML 2  2 is the complement set of ML 2 1 in ML 2 .We denote MP 2 1 as the set of first-layer pseudopolar grid points in MP 2 : Furthermore, we denote MP 2 2 as the set of second-layer pseudopolar grid points in MP 2 .In other words, MP 2  2 is the complement set of MP 2  1 in MP 2 .The distance between a point on log  and the closest point on PP can be expressed as follows [20]: where ( closest PP (, ),  closest PP (, )) in pseudopolar grid is the closest point of ( real log  (, ),  real log  (, )) in logpolar grid.In pseudopolar grid it is derived in [20] that where Δ PP is the angle between two neighboring polar radii, and the Δ PP is the distance between two points on neighboring polar radii in pseudopolar grid.Furthermore, it is shown in [20] that where  Grid PP is the distance between PP and log ,  Grid  is the distance between  and log .
The distance between ML 2 and log  can be expressed as In ML 2 1 , we note that Δ = Δ = 1/2.Therefore, The points in ML 2 2 coincide with the points in .It follows that The distance between MP 2 and log  can be expressed as In MP 2 (see Figure 2), we notice that where Δ MP 2 is the angle between two neighboring polar radii in MP 2 , Δ MP 2 2 is the distance between two points on one polar radius in MP 2  2 , and Δ MP 2 1 is the distance between two points on one polar radius in MP 2  1 .As a consequence, The points in MP 2 2 coincide with the points in PP, and from (19) we reach the following conclusion: MP 2  1 is denser than ML 2 1 , and if we magnified the ML 2 1 and MP 2  1 two times, they should be the same as the  and PP.Then, we obtain From ( 26) and ( 27), we get It follows from ( 19), ( 22), (25), and (28) that the distance between the log-polar grid points and two-layer pseudopolar grid points is the shortest one.We calculate the average distances between the log-polar grid points and other grid points.The results which are listed in Table 1 demonstrate the foregoing discussion.

Experiments and Results
The proposed registration algorithm was tested using a large set of test images which were rotated, scaled, and translated.

A Framework for Image Registration with MPFFT.
Let  1 and  2 be the target images.The algorithm operates as follows.
(i) Input images  1 and  2 .Apply a Blackman window function to them if it is necessary, as shown in Figure 3.
(ii) Choose the proper layer ().In this paper, we use the two-layer pseudopolar grid.
(iii) Calculate the MPFFT magnitudes of  1 and  2 .We get MP  1 and MP  2 .Then apply a high-pass filter to them.(iv) Calculate the log-polar Fourier transform magnitude spectrums of  1 and  2 by bilinear interpolation on the multilayer pseudopolar grid.
(v) Calculate the scale factor and the rotation angle by the phase correlation technique on the log-polar Fourier transform spectrums of both images.
(vi) Apply the scale factor and the rotation angle to the reference image and use the phase correlation again to detect the translation.

Registration for Arbitrary Rotation and Large Scaling
Images.Registration experiments of the MPFFT algorithm were provided by comparing with PPFFT and MLFFT in this section.All the experiments were tested on the 256 × 256 images which were rotated  0 and scaled 1/ as shown in Figure 3.All the test images were preprocessed by a highpass filter to reduce the scaling overlapping [23].Some of the test images were preprocessed by a Blackman window to decrease the rotation aliasing [24].We presented the numeric registration results for these images in  transform method by testing the Cameraman image with the Blackman window, the Lena image, and the Westconcordortho image without the Blackman window.

Registration for Noisy Images.
All the experiments in this section were tested on the 256 × 256 images which were rotated and scaled as shown in Figure 4.It is well known that Fourier-based methods are robust to noise.MPFFT algorithm was verified to be also robust to noise by experiments.The images of Figure 4(b) are target images with noise.The registration results of noisy images by PPFFT, MLFFT, and MPFFT are the images in Figures 4(c), 4(d), and 4(e).We present the numeric registration results for these images in Table 3. 5(a) and 5(b) were used to test the estimation of real images.In this section, MPFFT was tested on the real images which were rotated and scaled, as shown in Figure 5, similar to previous experiments.The images of Figure 5(a) are reference images, and the images of Figure 5(b) are target images.The registration results of campus images by PPFFT, MLFFT, and MPFFT are the images in Figures 5(c), 5(d), and 5(e).It was shown from the experiments that PPFFT method could not find the rotation factor and the scale factor.MLFFT method could find the scale factor but could not find the rotation factor while all the correct factors could be found by MPFFT.

Conclusions
Based on PPFFT and MLFFT, a new image registration algorithm called MPFFT was presented for recovering large translations, rotations, and scaling between images.The multilayer pseudopolar grid was utilized to approximate logpolar grid to improve the accuracy of the PPFFT and MLFFT in the proposed algorithm.There are some problems that deserve further studies.For example, although our algorithm is more accurate than PPFFT and MLFFT, the computing cost is twice (layer number) as that of the PPFFT algorithm and    twice as that of the MLFFT algorithm.Further works will focus on these problems.

Figure 4 :
Figure 4: Registration results of noisy images.(a) Reference images.(b) Target images with noise.(c) Registration results by PPFFT.(d) Registration results by MLFFT.(e) Registration results by MPFFT.

Table 1 :
Average distances between log-polar grid and other grids.

Table 2 .
Twolayer pseudopolar fractional Fourier transform method was shown to be more accurate than pseudopolar fractional Fourier transform method and two-layer fractional Fourier